I am trying to develop a web site where one can upload a file, which is then analyzed in the back to generate interactive figures (a scatter plot and a histogram), which is then return to the browser for the user to manipulate. (Imagine putting Excel online so that one can upload a file, get a figure and manipulate the figure.) I considered various options and decided to go with Bokeh for plotting. I wrote a python script and an html page to upload a file. Using Bokeh, I was able to create an output file (e.g. "plot.html"). This works well.
Separately, I have installed Tornado so that I can upload and dynamically read a simple file (e.g. "test.txt"), and simply return the content of the file in output.html. So this works well.
However, when I modify the script written for use with Tornado so that it displays the earlier, Bokeh generated plot.html, this doesn't work. Is there anything about a Bokeh generated html (containing plot objects) that cannot be properly rendered by Tornado? For example, I read that an entry from a database search can contain elements that cannot be serialized, and non-serializable elements need to be removed before the search result can be displayed. I am wondering if something like this may be going on.
Here're some relevant scripts:
display.py: for reading and returning the content of a file using
Tornado. This works and returns the content of test.txt, as
expected.
import os.path
import random
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=80, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
class OutputHandler(tornado.web.RequestHandler):
def post(self):
fname = self.get_argument('upfile')
f = open(fname, 'r')
lines = f.readlines()
self.render('plot.html', content=lines)
if __name__ == '__main__':
tornado.options.parse_command_line()
app = tornado.web.Application(
handlers=[(r'/', IndexHandler), (r'/content', OutputHandler)],
template_path=os.path.join(os.path.dirname(__file__), "templates"),
static_path=os.path.join(os.path.dirname(__file__), "static"),
debug=True
)
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
plot.py: for reading and generating a plot using Bokeh. This works
and creates plot.html containing the generated plots.
from bokeh.plotting import *
output_file("plot.html")
filename = 'test.dat'
f=open(filename,'r')
#####
# deleted lines for manipulating data
#####
scatter(ndarray[:1000,0], ndarray[:1000,1], color='red', alpha=0.7)
quad(top=hist, bottom=np.zeros(len(hist)), left=edges[:-1], right=edges[1:],
fill_color="#036564", line_color="#033649")
display_plot.py: for displaying a Bokeh generated plot.html through Tornado. This does not work.
import os.path
import random
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=80, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
def get(self):
self.render('plot.html')
# the rest the same as in display.py for now.
This last script is what led me to think that displaying a Bokeh generated html containing plot objects is not the same as displaying other html files containing text and figures only, which can be rendered correctly. Is this true and what can I do to display plot.html using Tornado (or any web service for that matter)?
Thanks for your help.
RequestHandler.render interprets a file as a Tornado template. This means that some character sequences (especially double curly braces {{ }}) have a special meaning. If you just want to serve a file as-is, use write instead of render:
with open('plot.html') as f:
self.write(f.read())
Related
I have seen a tutorial here Which is demonstrating the data analysis in the jupyter notebook cell, I am looking for the solution that how can i show the output of autoplotter which is python library in the django templates. Below is the code snippet of autoplotter which i have taken from its official website:
from autoplotter import run_app # Importing the autoplotter for GUI Based EDA
import pandas as pd # Importing Pandas to read csv
df = pd.read_csv("https://raw.githubusercontent.com/ersaurabhverma/autoplotter/master/Data/data.csv") # Reading data
run_app(df,mode = "inline", host="127.0.0.1", port=5000) # Calling the autoplotter.run_app in inline mode
run_app(df,mode = "external", host="127.0.0.1", port=5000) # Calling the autoplotter.run_app in external mode
I am looking for that what is the output format of this command
run_app (dataframe, host, port....)
how can I display its output in my django templates? so that a person could interact with his data through my launched website? Looking for any smart solution. Thank you
You can't run it under Django project because autoplotter uses Flask to serve the data and Flask can work only in the main thread of the main interpreter.
However, you can solve your problem using Docker. You will have a separate service that serves the autoplotter app and Django can have an iframe in HTML template that shows the content of the service.
UPD:
For the Docker - you can start with this guide. The only difference will be in your case is that app.py will contain only the call for a plotter:
if __name == '__main__':
run_app(df, mode="external", host="127.0.0.1", port=5000)
And requirements.txt with autoplotter
I can't run bokeh server within flask beind apache so now I'm trying to serve bokeh within flask locally. The figure does not render.
Here is the flask code:
from flask import Flask, render_template
app = Flask(__name__)
from bokeh.embed import server_document
#app.route("/")
def techblog():
try:
tag = server_document(url=r'/bokeh', relative_urls=True)
return render_template('techblog.html', tag=tag)
except Exception as e:
return str(e)
if __name__ == '__main__':
app.run(debug=True)
Here is the bokeh code:
from numpy.random import random
from bokeh.io import curdoc
from bokeh.plotting import figure
from bokeh.layouts import column, widgetbox
from bokeh.models import Button, ColumnDataSource
from bokeh.server.server import Server
def run(doc):
fig = figure(title='random data', width=400, height=200, tools='pan,box_zoom,reset,save')
source = ColumnDataSource(data={'x': [], 'y': []})
fig.line('x', 'y', source=source)
def click(n=100):
source.data = {'x': range(n), 'y': random(n)}
button = Button(label='update', button_type='success')
button.on_click(click)
layout = column(widgetbox(button), fig)
doc.add_root(layout)
click()
# configure and run bokeh server
kws = {'port': 5000, 'prefix':'/bokeh','allow_websocket_origin': ['127.0.0.1']}
server = Server(run, **kws)
server.start()
# if __name__ == '__main__':
server.io_loop.add_callback(server.show, '/')
server.io_loop.start()
Here is my html template:
<h1 style='color:blue'>Hello There!</h1>
</br>
{{ tag|safe }}
</br>
{{ tag }}
I'm running flask app via python. And on a separate command processor I run bokeh app via,
bokeh serve --allow-websocket-origin=localhost:5000 filename.py
I only get the tag without "safe" as
<script src="/bokeh/autoload.js?bokeh-autoload-element=1001&bokeh-app-path=/bokeh" id="1001"></script>
And I have this message on flask console. It is a standart 404:
"GET /bokeh/autoload.js?bokeh-autoload-element=1000&bokeh-app-path=/bokeh HTTP/1.1" 404 -
That'a all. No figure or button is rendered. What should I change to see the figure?
Edit: I've specified the port and the prefix in the bokeh code. The outcome has not changed.
Edit 2: I've add the console msj for 404 error on flask console.
There are a few things wrong:
You have not configured a port, so the Bokeh server will use its default port of 5006 (not 5000)
You have not configured an app path, so the Bokeh server will serve the app from its default location of / (not /bokeh)
It's also worth mentioning that if you whitelist localhost as an allow websocket origin, then it literally must be localhost in the URL bar (i.e. not 127.0.0.1, they are not interchangeable in this context)
Lastly, it's a lot of extra work to put the Bokeh app code in a plain python script that calls Server manually, etc. You could just put the contents of run in a file and call bokeh serve --port=5000 app.py and then the app will be available at localhost:5000/app
I add some details to #bigreddot's answer.
Code didn't work for me too. I even found exactly this code in some tutorial and main problem was that it was using nginx which was converting /bokeh to http://127.0.0.1/bokeh. But on local computer without nginx I have to change all urls.
EDIT: I found tutorial with this code https://rbtechblog.com/blog/deploy_bokeh_app
I start changing code and reduce it to create minimal code which works. I made changes similar to changes mentioned by bigreddot.
Bokeh
I put code directly in file without def and without Server
filename.py
from numpy.random import random
from bokeh.io import curdoc
from bokeh.plotting import figure
from bokeh.layouts import column, widgetbox
from bokeh.models import Button, ColumnDataSource
def click(n=100):
source.data = {'x': range(n), 'y': random(n)}
fig = figure(title='random data', width=800, height=400, tools='pan,box_zoom,reset,save')
source = ColumnDataSource(data={'x': [], 'y': []}) # place for data
fig.line('x', 'y', source=source) # draw plot
button = Button(label='update', button_type='success') # create button
button.on_click(click) # assign function to button
layout = column(fig, widgetbox(button)) # create layout
curdoc().add_root(layout) # add all to document
click() # generate random data at start
Now I can run it in console
bokeh serve filename.py
and I can see it in web browser using url
http://localhost:5006/filename
(bokeh should display this url in console after start - if you will use different file or options then you may see different url)
At this moment I don't need any other options but later I will need --allow-websocket-origin but I will describe it later.
BTW: I not use name bokeh.py because it can make problem to import original bokeh.
Flask
Because I don't use nginx which could convert /bokeh to http://localhost:5006/filename so I have to use full url in serve_document
For test I used render_template_string instead of render_template so I don't have to create templates/index.html so it will easier to copy and test code.
I removed try/except to get more details if there will be error.
app.py
from flask import Flask, render_template, render_template_string
from bokeh.embed import server_document
app = Flask(__name__)
#app.route("/")
def index():
tag = server_document(url='http://localhost:5006/filename')
#return render_template('index.html', tag=tag)
return render_template_string('''<div>{{ tag|safe }}</div>''', tag=tag)
if __name__ == '__main__':
app.run(debug=True)
Now I can run it
python app.py
and I can open page in web browser using standard url
http://localhost:5000/
but I will not see plot and bokeh will display
Refusing websocket connection from Origin 'http://127.0.0.1:5000';
use --allow-websocket-origin=127.0.0.1:5000
or set BOKEH_ALLOW_WS_ORIGIN=127.0.0.1:5000 to permit this;
currently we allow origins {'localhost:5006'}
so I have to restart bokeh with this option
bokeh serve filename.py --allow-websocket-origin=127.0.0.1:5000
(as bigreddot mentioned it has to be 127.0.0.1, not localhost)
And now flask should display plot.
BTW: if I use template without any HTML tag
render_template_string('''{{ tag|safe }}''', tag=tag)
then browser may treat all code (<script ...></scrip>) as part of <head></head> and it will not display it because browser never display elements which are in <head></head> even if there are correct images or plots.
I'm working on some code that pulls course info from Canvas. As pure python, it works fine. If I try to incorporate it with Flask, I get the following error
requests.exceptions.MissingSchema: Invalid URL 'run/api/v1/courses/1234567': No schema supplied. Perhaps you meant http://run/api/v1/courses/1234567?
This is the code in question:
Canvas file
import sys
from canvasapi import Canvas
def getinfo():
canvasurl = "https://canvas.instructure.com/";
canvastoken = #Redacted for this example
try:
canvastoken = sys.argv[1];
canvasurl = sys.argv[2];
except:
print()
#Create a new canvas object passing in the newly aquired url and token
canvas = Canvas(canvasurl, canvastoken);
#print(canv)
# Create a new course oject -- passing in course number as a parameter
# Course number is currently hard coded
print(canvas.get_course(1234567))
Flask file code (the file that I'm trying to run):
from flask import Flask
import canvas
canvas.getinfo()
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
No schema provided usually means you haven't specified the http:// or https:// in the URL.
In the code you provided, I don't see any reference to a run/api/v1/courses/1234567. One possibility is if you are using the url_for method from requests anywhere in your code, try setting _external=True:
url = url_for('relativeURL', _external=True)
This allows Flask to construct an absolute URL (i.e., a URL with domain included).
If you aren't using url_for, check other places in your code where you might be omitting the http or https from the URL.
If you update your question to include the part that refers to the offending URL, we might be able to provide more specific help.
I'm trying to write a web application and am using Tornado Web for the json xhr calls. But I'm trying to serve a static index.html which is to serve the main app.
How can I serve a simple page and still have requesthandlers for the rest of my application?
Here's what I tried so far:
import tornado.ioloop
import tornado.web
import json
import os
games = [...]
class HomeHandler(tornado.web.RequestHandler):
def get(self):
self.render('index.html')
class MatchHandler(tornado.web.RequestHandler):
def get(self):
self.write(json.dumps(games))
path = os.path.join(os.getcwd(), 'app')
if __name__ == "__main__":
application = tornado.web.Application(
[
(r'/', HomeHandler),
(r'/games', MatchHandler),
(r'/*.*', tornado.web.StaticFileHandler, {'path': path})
],
template_path=os.path.join(os.path.dirname(__file__), 'app')
)
application.listen(16001)
tornado.ioloop.IOLoop.current().start()
Thanks in advance!
The StaticFileHandler regex needs to A) contain a capturing group and B) use regex syntax instead of glob syntax:
(r'/(.*\..*)', tornado.web.StaticFileHandler, {'path': path})
This will match any paths containing a dot and send it to the StaticFileHandler.
Your code looks correct to me. Put a file named "index.html" in the "app" subdirectory of your current working directory when you run the app, and the contents of that "index.html" will be the response when you visit http://localhost:16001/
Your code should work fine, as #a-jesse-jiryu-davis answered. To expand a bit on it, you could use tornado.web.StaticFileHandler if you just need to serve your static file. This will make it more flexible, and also take advantage of server-side caching etc.
I'm want to test my web service (built on Tornado) using tornado.testing.AsyncHTTPTestCase. It says here that using POST for AsyncHttpClients should look like the following.
from tornado.testing import AsyncHTTPTestCase
from urllib import urlencode
class ApplicationTestCase(AsyncHTTPTestCase):
def get_app(self):
return app.Application()
def test_file_uploading(self):
url = '/'
filepath = 'uploading_file.zip' # Binary file
data = ??????? # Read from "filepath" and put the generated something into "data"
self.http_client.fetch(self.get_url(url),
self.stop,
method="POST",
data=urlencode(data))
response = self.wait()
self.assertEqual(response.code, 302) # Do assertion
if __name__ == '__main__':
unittest.main()
The problem is that I've no idea what to write at ???????. Are there any utility functions built in Tornado, or is it better to use alternative libraries like Requests?
P.S.
... actually, I've tried using Requests, but my test stopped working because probably I didn't do good for asynchronous tasking
def test_file_uploading(self):
url = '/'
filepath = 'uploading_file.zip' # Binary file
files = {'file':open(filepath,'rb')}
r = requests.post(self.get_url(url),files=files) # Freezes here
self.assertEqual(response.code, 302) # Do assertion
You need to construct a multipart/form-data request body. This is officially defined in the HTML spec. Tornado does not currently have any helper functions for generating a multipart body. However, you can use the MultipartEncoder class from the requests_toolbelt package. Just use the to_string() method instead of passing the encoder object directly to fetch().