I have a web desktop app which writes and reads to local file system using XPCOM filepicker and it works flawlessly in firefox12. However later firefox versions (particularly current one v17) completely disables use of xpcom file functions.
I thought of passing file requests to python's tkinter on a server on the local machine. I can open the tkinter filepicker in IDLE and from a .py or .cgi file, but how to get the file dialog to appear back in the calling html page of the app? I need that interactivity without leaving the app's page. Any ideas much appreciated.
Depending on the complexity and frequency of I/O, you could just post the parameters to a python CGI script and return the result in a JSON object
First, in JS:
function doIO (data) {
var request = new XMLHttpRequest();
request.onreadystatechange = function () {
if (request.readyStatus == 4) {
if (request.status == 200) {
alert('Success!');
var data = JSON.parse(xmlhttp.responseText);
// do something with response data
}
else { alert('Failure'!); }
}
};
request.open("POST", python_cgi, true);
request.send(data);
}
In python you would need to implement a CGI script to parse the data, figure out the requested I/O and perform it. Your python script could be something like the following:
import cgi, json, sys
form = cgi.FieldStorage()
command = form.getFirst('command')
if command == "filePicker":
# run tkinter file picker
# build json response object
elif commoand == "myIoCommand":
# do some I/O
# build json response object
print "Status: 200"
print "Content-Type: application/json"
json.dump(response, sys.stdout)
See for example Google Maps' JSON responses if you need some inspiration forming your json response object.
If you need more frequent/complex I/O, maybe what you would have to do is set up a Python server with mirrored state to your application via smaller, more frequent AJAX calls. You can use a framework to make a RESTful app, or you can roll your own by subclassing from BaseHTTPServer.
Related
I have been using Google Cloud Functions for over a week now and they have been great. I used a simple python 3.9 function to print a string to my terminal in my Next.js app (for testing purposes) and it was working great. Here is my sample Google Cloud Function.
def hello_world(request):
"""Responds to any HTTP request.
Args:
request (flask.Request): HTTP request object.
Returns:
The response text or any set of values that can be turned into a
Response object using
`make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
"""
request_json = request.get_json()
if request.args and 'message' in request.args:
return request.args.get('message')
elif request_json and 'message' in request_json:
return request_json['message']
else:
return f'Function ran'
And here is my Next.js code that calls the function:
// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import type { NextApiRequest, NextApiResponse } from "next"
import { GoogleAuth } from "google-auth-library"
export default async function handler(req: NextApiRequest, res: NextApiResponse<any>) {
const url = process.env.FUNCTION_URL as string
//Example with the key file, not recommended on GCP environment.
const auth = new GoogleAuth({ keyFilename: process.env.KEYSTORE_PATH })
//Create your client with an Identity token.
const client = await auth.getIdTokenClient(url)
const result = await client.request({ url })
console.log(result.data)
res.json({ data: result.data })
}
I wrote another function to do the same thing and now every function just prints out raw html to the console? When I open the text in an index.html file it looks like this.
I rewrote the original cloud function exactly and even that doesn't work anymore. It prints that same html to the console. What is going on? My code is exactly the same and it breaks now...?
It was a simple "needle in the haystack" type of fix. I was getting the function url from within the cloud function listing view, clicking the Actions Menu dropdown on the desired function, and selecting "Copy Function." This took me to the configuration display page where it had the function url in the "Trigger" section.
Notice what I circled in red. For some reason this page appends a -1 to the function url. This is what made it fail for me. In my code I deleted the -1 from the url stored in the process.env.FUNCTION_URL variable. That made it work!
I'm working on a web interface for a project and need to plot a realtime chart with some data I'll get from an API I'm building. The API runs on http://SERVER_IP:5000/signal and my web interface on http://SERVER_IP:80/, both of them with Flask.
I was having CORS issues at the very beginning and then found out about Flask-CORS. I tried to implement that and now I'm getting no errors on the browser console, but no messages either! I've added some console messages for debugging, but it doesn't seem alright.
On my dashboard, I try to reach the API with the following code:
const source = new EventSource("http://SERVER_IP:5000/signal", {withCredentials: true});
console.log ("Things started!!!");
source.onmessage = function (event) {
console.log ("An event's just happened");
// parse data and do stuff
}
and in my API, I set Flask-CORS like this:
self.__cors = CORS(self.__app, supports_credentials = True)
and the route like this:
#self.__app.route('/signal')
def get_signal():
import json
def get_data():
while True:
json_data = json.dumps(...)
yield "{}\n\n".format(json_data)
time.sleep(1)
return Response(get_data(), mimetype='text/event-stream')
Then, when I open my web browser and open console, I can see the "Things started!!!" message, but no "An event's just happened" and no data on the chart.
I was malforming my response. As I found out here, my "get_data" method should yield "data: ...", so after I added the "data: " after my data, things started to work pretty well.
Im using Python bottle framework to upload a file..The controller receives the file and transmits to another system using Java rest service..
Everything works fine when the file is small but if the file is huge (it takes 5 mints) the application returns a blank page instead of a html page that its supposed to return.
#app.route('/DataLoads',method='GET')
def pptriv():
return template('dataload',
footer_html=FOOTER_HTML,feedback_html=FEEDBACK_HTML)
#app.route('/DataLoads',method='POST')
def pptriv():
username = request.forms.get('username')
password = request.forms.get('password')
uploadfile = request.files.get('File Upload')
....
..use Python requests module to transmit the files.....
...
print 'r.status_code = ', r.status_code
return template('dataload',
footer_html=FOOTER_HTML,feedback_html=FEEDBACK_HTML)
I see the print stmt r.status_code but the dataload html page, if its small file everything looks good..
Have you tried increasing MEMFILE_MAX?
import bottle
bottle.BaseRequest.MEMFILE_MAX = 1024 * 1024 * 200
Python bottle module causes "Error: 413 Request Entity Too Large"
Also: are you using a browser or a command-line client to upload the file? If browser, then I'd suggest using curl to see exactly what (if anything) is returned from your server.
I have a REST frontend written using Python/Bottle which handles file uploads, usually large ones. The API is wirtten in such a way that:
The client sends PUT with the file as a payload. Among other things, it sends Date and Authorization headers. This is a security measure against replay attacks -- the request is singed with a temporary key, using target url, the date and several other things
Now the problem. The server accepts the request if the supplied date is in given datetime window of 15 minutes. If the upload takes long enough time, it will be longer than the allowed time delta. Now, the request authorization handling is done using decorator on bottle view method. However, bottle won't start the dispatch process unless the upload is finished, so the validation fails on longer uploads.
My question is: is there a way to explain to bottle or WSGI to handle the request immediately and stream the upload as it goes? This would be useful for me for other reasons as well. Or any other solutions? As I am writing this, WSGI middleware comes to mind, but still, I'd like external insight.
I would be willing to switch to Flask, or even other Python frameworks, as the REST frontend is quite lightweight.
Thank you
I recommend splitting the incoming file into smaller-sized chunks on the frontend. I'm doing this to implement a pause/resume function for large file uploads in a Flask application.
Using Sebastian Tschan's jquery plugin, you can implement chunking by specifying a maxChunkSize when initializing the plugin, as in:
$('#file-select').fileupload({
url: '/uploads/',
sequentialUploads: true,
done: function (e, data) {
console.log("uploaded: " + data.files[0].name)
},
maxChunkSize: 1000000 // 1 MB
});
Now the client will send multiple requests when uploading large files. And your server-side code can use the Content-Range header to patch the original large file back together. For a Flask application, the view might look something like:
# Upload files
#app.route('/uploads/', methods=['POST'])
def results():
files = request.files
# assuming only one file is passed in the request
key = files.keys()[0]
value = files[key] # this is a Werkzeug FileStorage object
filename = value.filename
if 'Content-Range' in request.headers:
# extract starting byte from Content-Range header string
range_str = request.headers['Content-Range']
start_bytes = int(range_str.split(' ')[1].split('-')[0])
# append chunk to the file on disk, or create new
with open(filename, 'a') as f:
f.seek(start_bytes)
f.write(value.stream.read())
else:
# this is not a chunked request, so just save the whole file
value.save(filename)
# send response with appropriate mime type header
return jsonify({"name": value.filename,
"size": os.path.getsize(filename),
"url": 'uploads/' + value.filename,
"thumbnail_url": None,
"delete_url": None,
"delete_type": None,})
For your particular application, you will just have to make sure that the correct auth headers are still sent with each request.
Hope this helps! I was struggling with this problem for a while ;)
When using plupload solution might be like this one:
$("#uploader").plupload({
// General settings
runtimes : 'html5,flash,silverlight,html4',
url : "/uploads/",
// Maximum file size
max_file_size : '20mb',
chunk_size: '128kb',
// Specify what files to browse for
filters : [
{title : "Image files", extensions : "jpg,gif,png"},
],
// Enable ability to drag'n'drop files onto the widget (currently only HTML5 supports that)
dragdrop: true,
// Views to activate
views: {
list: true,
thumbs: true, // Show thumbs
active: 'thumbs'
},
// Flash settings
flash_swf_url : '/static/js/plupload-2.1.2/js/plupload/js/Moxie.swf',
// Silverlight settings
silverlight_xap_url : '/static/js/plupload-2.1.2/js/plupload/js/Moxie.xap'
});
And your flask-python code in such case would be similar to this:
from werkzeug import secure_filename
# Upload files
#app.route('/uploads/', methods=['POST'])
def results():
content = request.files['file'].read()
filename = secure_filename(request.values['name'])
with open(filename, 'ab+') as fp:
fp.write(content)
# send response with appropriate mime type header
return jsonify({
"name": filename,
"size": os.path.getsize(filename),
"url": 'uploads/' + filename,})
Plupload always sends chunks in exactly same order, from first to last, so you do not have to bother with seek or anything like that.
BrowserStack is a powerful platform for testing web sites against the most
current and modern browser. So far so good.
BrowserStack also provides an API
The API has the concept of a worker representing a specific browser (version) loading a particular URL.
What useful things can I do with such a worker instance?
How would one integrate such a worker with Selenium tests?
How would one integrate such a worker with unittests (Python)?
How would one use such a worker e.g. for testing if a particular website with a video player would actually load and play a video (e.g. for cross-browser video testing)?
Current API opens your provided url in all platform/browser combinations.
So, if you open an HTML page with lot of JS tests, you need to be using tool like yeti/testswarm/js-test-driver which automatically fetch results from browser.
Another example of using BrowserStack API is http://ryanseddon.github.com/bunyip/
Sample integration with Jenkins: http://github.com/jquery/testswarm/wiki/Automated-Distributed-Continuous-Integration-for-JavaScript
For local JS testing, you will need to use tools like localtunnel to get a public url for your local servers.
One of the most useful capabilities of the current BrowserStack API is to allow you to mark a session as a failed test.
Like any Selenium hub/node system, BrowserStack doesn't know why you're sending commands to the browser. It just runs the commands you request. Consequently, it has no way to know when a test fails.
But you can use the API to tell it that a test failed, so that the session gets marked as failed in the BrowserStack UI. Then you can filter on just the failed sessions to investigate them.
This is in Java, not Python, but here's some sample code that shows how to update sessions to reflect that they represent failed tests. You just pass in the Selenium session IDs (which you need to save as you run the test in question) and the exception you got when the test failed.
import com.mashape.unirest.http.Unirest;
import com.mashape.unirest.http.exceptions.UnirestException;
import com.unblu.automation.support.settings.Prop;
import com.unblu.automation.support.settings.Settings;
import org.openqa.selenium.remote.SessionId;
public class BrowserStackUpdater {
private void markSessionAsFailed(SessionId sessionId, Throwable e) {
var url = "https://api.browserstack.com/automate/sessions/" + sessionId + ".json";
try {
var userName = "BROWSERSTACKUSERNAMEHERE";
var key = "BROWSERSTACKKEYHERE";
var result = Unirest.put(url)
.basicAuth(userName, key)
.field("status", "failed")
.field("reason", e.toString())
.asString();
System.out.println("Marking test failed; reply from BrowserStack: " +
result.getStatus() + " : " + result.getBody());
}
catch (UnirestException ue) { ue.printStackTrace(); }
}
public void markTestFailedInBrowserStack(Iterable<SessionId> sessionIds, Throwable e) {
var env = Settings.getString(Prop.runEnvironment);
if (env.equals("BrowserStack")) {
for (var sid : sessionIds) {
markSessionAsFailed(sid, e);
}
}
}
}