Develop Reference

Bottle server POST options from listbox to SQL database

I currently have a button, that when clicked, it creates two variables. These hold the selected options from a list box. Then I am attempting to POST these into my database via the bottle script, however it doesnt work. Can someone explain to me why?
The HTML/JQuery code for the button is this:
$("#btnStartEvent").bind("click", function () {
var selectedStudents = $('#lstBox2 option:selected');
var selectedEvent = $('#event_options option:selected');
alert(selectedEvent);
$.post( "/send_data", { eventIDPost: selectedEvent},function(data ) {
alert( data );
});
And my bottle code that handles this POST is:
#post('/send_data')
def send_data():
postdata = request.body.read()
events_id = request.forms.get("eventIDPost")
sql = "INSERT INTO tblResults VALUES('{}')".format(events_id)
run_sql(sql)

I am not sure if bottle #post is broken, but referencing to
http://bottlepy.org/docs/dev/api.html#bottle.Bottle.route
and updating the api method to below:
#app.route('/send_data', method=['POST'])
def send_data():
started listening to the API calls.Otherwise it was 405 error
Also please refer https://api.jquery.com/jquery.post/
$.post("/send_data", { "eventIDPost": 1 })
.done( function (data) {console.log(data)});
Hope this helps.

I can assure you that the #post is not broken. I use it every single day.
from bottle import post, request
def merge_dicts(*args):
result = {}
for dictionary in args:
result.update(dictionary)
return result
#post('/send_data')
def send_data():
return merge_dicts(dict(request.forms), dict(request.query.decode()))

Interact with Jupyter Notebooks via API

The problem: I want to interact with Jupyter from another application via Jupyter API, in particular I want to run my notebooks from the app at least (Perfect variant for me is to edit some paragraphs before running it). I've read the API documentation but haven't found what I need.
I've used for that purpose Apache Zeppelin which have the same structure (Notebooks and paragraphs).
Does anybody used Jupyter for the purpose I've just described?

Ignoring if the use of Jupyter API is the best solution for the problem (not clearly described in the question), the code below does what you have asked for: it will execute remotely a Jupyter notebook over http and get some results. It is not production ready, it more an example of how it can be done. Did not test it with cells that generate lots of output - think it will need adjustments.
You can also change/edit the code programmatically by altering the code array.
You will need to change the notebook_path, base and headers according to your configuration, see code for details.
import json
import requests
import datetime
import uuid
from pprint import pprint
from websocket import create_connection
# The token is written on stdout when you start the notebook
notebook_path = '/Untitled.ipynb'
base = 'http://localhost:9999'
headers = {'Authorization': 'Token 4a72cb6f71e0f05a6aa931a5e0ec70109099ed0c35f1d840'}
url = base + '/api/kernels'
response = requests.post(url,headers=headers)
kernel = json.loads(response.text)
# Load the notebook and get the code of each cell
url = base + '/api/contents' + notebook_path
response = requests.get(url,headers=headers)
file = json.loads(response.text)
code = [ c['source'] for c in file['content']['cells'] if len(c['source'])>0 ]
# Execution request/reply is done on websockets channels
ws = create_connection("ws://localhost:9999/api/kernels/"+kernel["id"]+"/channels",
header=headers)
def send_execute_request(code):
msg_type = 'execute_request';
content = { 'code' : code, 'silent':False }
hdr = { 'msg_id' : uuid.uuid1().hex,
'username': 'test',
'session': uuid.uuid1().hex,
'data': datetime.datetime.now().isoformat(),
'msg_type': msg_type,
'version' : '5.0' }
msg = { 'header': hdr, 'parent_header': hdr,
'metadata': {},
'content': content }
return msg
for c in code:
ws.send(json.dumps(send_execute_request(c)))
# We ignore all the other messages, we just get the code execution output
# (this needs to be improved for production to take into account errors, large cell output, images, etc.)
for i in range(0, len(code)):
msg_type = '';
while msg_type != "stream":
rsp = json.loads(ws.recv())
msg_type = rsp["msg_type"]
print(rsp["content"]["text"])
ws.close()
Useful links based on which this code is made (that I recommend reading if you want more info):
https://jupyter-client.readthedocs.io/en/latest/messaging.html#python-api
https://github.com/jupyter/jupyter/wiki/Jupyter-Notebook-Server-API
Note that there is also https://jupyter-client.readthedocs.io/en/stable/index.html, but as far as I could tell it does not support HTTP as a transport.
For reference this works with notebook-5.7.4, not sure about other versions.

Extending the code by #vladmihaisima
from websocket import create_connection, WebSocketTimeoutException
while msg_type != "stream":
try:
rsp = json.loads(ws.recv())
print(rsp["msg_type"])
print(rsp)
msg_type = rsp["msg_type"]
if msg_type == "error":
raise Exception(rsp['content']['traceback'][0])
except WebSocketTimeoutException as _e:
print("No output")
return

I believe that using of remote Jupyter Notebook is over-engineering in your case.
I see good way is pass necessary parameters to python program with well logging.

SoftLayer API Nessus Scan Status / Report via python

I want to use python client to create a Nessus Security Scanner and check the status by getStatus and get the result by getReport method. While, I have read these helps by php(SoftLayer API Nessus Scan Status / Report via PHP). But how can i use these by python client?
When I call setInitParameter(scan_id) in by python, the exception as flows:
SoftLayerAPIError(Client): Function ("setInitParameter") is not a valid method for this service

i recomend you to read documentation of the client first:
https://github.com/softlayer/softlayer-python
https://softlayer-api-python-client.readthedocs.io/en/latest/
the init parameters are set like this:
clientService.getObject(id=myInitParameter)
here you can find more examples using the client:
https://softlayer.github.io/python/
Here you can find additional documentation:
http://sldn.softlayer.com/blog
And renember that with the Softlayer's python client unlike the php client the data are sending in json format so the request:
$client = SoftLayer_SoapClient::getClient('SoftLayer_Account', null, $apiUsername, $apiKey);
$accountInfo = $client->getObject();
$hardware = $client->getHardware();
foreach ($hardware as $server){
$scanclient = SoftLayer_SoapClient::getClient('SoftLayer_Network_Security_Scanner_Request', '', $apiUsername, $apiKey)
$scantemplate = new stdClass();
$scantemplate->accountId = $accountInfo->id;
$scantemplate->hardwareId = $server->id;
$scantemplate->ipAddress = $server->primaryIpAddress;
try{
// Successfully creates new scan
$scan = $scanclient->createObject($scantemplate);
} catch (Exception $e){
echo $e->getMessage() . "\n\r";
}
would be like this:
clientAccount = client['SoftLayer_Account']
accountInfo = clientAccount.getObject() #for this case we do not need init parameter
hardware = clientAccount.getHardware() #for this case we do not need init parameter
for server in hardware:
scanclient = client['SoftLayer_Network_Security_Scanner_Request']
scantemplate = {
"accountId": accountInfo["id"],
"hardwareId": server["id"],
"ipAddress": server["primaryIpAddress"]
}
scanclient.createObject(scantemplate)

Long Polling in Python with Flask

I'm trying to do long polling with JQuery and Python under the Flask Framework.
Having done long polling before in PHP, I've tried to go about it in the same way:
A script/function that has a while(true) loop, checking for changes periodically eg.every 0,5 seconds in the database, and returns some data when a change occurs.
So in my ini.py I've created an app.route to /poll for JQuery to call. JQuery gives it some information about the client's current state, and the poll() function compares this with what's currently in the database. The loop is ended and returns information when a change is observed.
Here's the python code:
#app.route('/poll')
def poll():
client_state = request.args.get("state")
#remove html encoding + whitesapce from client state
html_parser = HTMLParser.HTMLParser()
client_state = html_parser.unescape(client_state)
client_state = "".join(client_state.split())
#poll the database
while True:
time.sleep(0.5)
data = get_data()
json_state = to_json(data)
json_state = "".join(data) #remove whitespace
if json_state != client_state:
return "CHANGE"
The problem is that, when the code above starts polling, the server appears to be overloaded and other Ajax calls, and other requests like loading a "loading" image to the html using JQuery are unresponsive and timeout.
For completion's sake I've included the JQuery here:
function poll() {
queryString = "state="+JSON.stringify(currentState);
$.ajax({
url:"/poll",
data: queryString,
timeout: 60000,
success: function(data) {
console.log(data);
if(currentState == null) {
currentState = JSON.parse(data);
}
else {
console.log("A change has occurred");
}
poll();
},
error: function(jqXHR, textStatus, errorThrown) {
console.log(jqXHR.status + "," + textStatus + ", " + errorThrown);
poll();
}
});
}
Does this need to multi-threaded or something? Or does anyone have any idea why I'm experiencing this behavior?
Thanks in advance!! :)

Just as the link #Robᵩ mentioned, you flask app is just overload. That's because a flask app is in single threading mode by default when running with app.run(), so it can only serve one request per time.
You can start multi threading with:
if __name__ == '__main__':
app.run(threaded=True)
Or using a WSGI server like gunicorn or uwsgi to serve flask with multi processing:
gunicorn -w 4 myapp:app
Hopes you're enjoying with Python and Flask!

Stream a song from SoundCloud using the Python API

I writing a little program that should stream a song from soundcloud..
my code is:
import soundcloud
cid="==="
cs="==="
un="==="
pw="==="
client = soundcloud.Client(
client_id=cid,
client_secret=cs,
username=un,
password=pw
)
print "Your username is " + client.get('/me').username
# fetch track to stream
track = client.get('/tracks/293')
# get the tracks streaming URL
stream_url = client.get(track.stream_url, allow_redirects=False)
# print the tracks stream URL
print stream_url.location
It just printing the usernsame, and the track URL
It prints something like this:
Your username is '==='
https://ec-media.soundcloud.com/cWHNerOLlkUq.128.mp3?f8f78g6njdj.....
Then, i want to play the MP3 from the URL. I can download it using urllib, but if it is a big file, it would take a lot of time.
What is the best way to stream the MP3?
Thanks!!

Before using the solution I suggest here, you should be aware of the fact that you must credit SoundCloud somewhere in your application and possibly in your audio player that users will be seeing that it is served through SoundCloud. Doing the opposite will be unfair and probably violate their terms of usage.
track.stream_url is not the end point URL associated with the mp3 file.
All the associated audio is only served 'on demand' when you send an http request with track.stream_url. Upon sending the http request you are being redirected to the actual mp3 stream (which is created just for you and will expire in next 15 mins).
So if you want to point the audio source you should at first get the redirect_url for the stream:
Below is the C# code which does what I am talking and it will give you the main idea - just convert it to Python code;
public void Run()
{
if (!string.IsNullOrEmpty(track.stream_url))
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(track.stream_url + ".json?client_id=YOUR_CLIENT_ID");
request.Method = "HEAD";
request.AllowReadStreamBuffering = true;
request.AllowAutoRedirect = true;
request.BeginGetResponse(new AsyncCallback(ReadWebRequestCallback), request);
}
}
private void ReadWebRequestCallback(IAsyncResult callbackResult)
{
HttpWebRequest myRequest = (HttpWebRequest)callbackResult.AsyncState;
HttpWebResponse myResponse = (HttpWebResponse)myRequest.EndGetResponse(callbackResult);
using (StreamReader httpwebStreamReader = new StreamReader(myResponse.GetResponseStream()))
{
this.AudioStreamEndPointUrl = myResponse.ResponseUri.AbsoluteUri;
this.SearchCompleted(this);
}
myResponse.Close();
}