I have a project where I save a lot of data in the sqlite3 database. I want to create a simple application in Flask - displaying appropriate data from the database.
New records are added to the database every few minutes - I want this application to refresh automatically. I prepared some information about it and found information that it can be done somehow using socketio Can flask framework send real-time data from server to client browser? or using AJAX. I tried to do something like this: or Python Flask date update real-time but I didn't manage to do it.
here's the code I'm using right now:
from flask import Flask, request, render_template, session, redirect
import numpy as np
import pandas as pd
import sqlite3
con = sqlite3.connect(r'path')
app = Flask(__name__)
df = pd.read_sql('select * from table1', con)
df1 = pd.read_sql('select * from table2', con)
#app.route('/', methods=("POST", "GET"))
def html_table():
return render_template('app.html',
tables=[df.to_html(classes='data1'),df1.to_html(classes='data2')],
titles=df.columns.values)
if __name__ == '__main__':
app.run()
Template:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Title</title>
</head>
<body>
{% for table in tables %}
{{titles[loop.index]}}
{{ table|safe }}
{% endfor %}
</body>
</html>
in this case, everything is displayed, but it doesn't work in real time.
I'm asking for some instructions or some advice on how to do it? Can I use the links I posted here somehow? This is my first application in Flask so I don't quite know how to achieve such a thing.
If i got your question what you want to do is load the data without loading the page. As you mentioned AJAX and socket.io, AJAX helps in getting the data on the same page but it requires reloading.It get different data on the same route without going to another route.Socketio will help you in achieving that.As it is used commonly in real time chat apps that is why we dont need to refresh the page to see the messages.In flask, use flask_socketio library to use it and you also need Javascript and by incorporating these two, you can do what you want.
Related
I have a script which takes uploaded data, munges it together, turns it into a plot (using Bokeh) and then exports it to a directory as JSON.
At some point in the future, a user can hit the right URL and the appropriate plot should be displayed to the user as part of the HTML template.
I can generate the plot. I can save it as JSON. I can get the URL to retrieve it as JSON, but I cannot get the JSON plot to render within the template.
I've had a dig around the Bokeh documentation and examples, but they all seem to use a flask app to serve the pages.
I think I'm on the right track, using views.py to find and return JSON as part of a render() response, and then have Bokeh.embed.embed_items() do the work in the template to make it look right, but it's not working out - everything but the plot is displayed.
1) Create the plot and puts it in the directory for later use (app/results/1)
create plot.py
import os
import json
from django.conf import settings
from bokeh.embed import json_item
from bokeh.plotting import figure
x=[1,2,3,4,5]
y=[0,-1,-2,3,4]
p=figure(title="test_example")
p.line(x, y)
#json_export = json_item(p, "result")
json_export = json_item(p)
with open(os.path.join(settings.RESULTS_DIR,"1", "test.json"), 'w') as fp:
fp.write(json.dumps(json_export))
2) Set up the url
urls.py
urlpatterns=[
path('result/<int:pk>', views.resultdetailview, name='result-detail'),
]
3) Take the request, use the pk to find the plot json and render it all in the appropriate template.
views.py
def resultdetailview(request, pk):
results=str(pk)
with open(os.path.join(settings.RESULTS_DIR, results, "test.json"), 'r') as fp:
#data=json.load(fp)
data=json.loads(fp.read())
#context={'result':data}
#return render(request, 'app/sandbox_detail.html', context)
return render(request=request,
context={'json_object':data, 'resources':CDN.render()})
NB: If I instead use return JsonResponse(data, safe=False) then the url returns the json successfully ...
I think therefore that the issue is in the template.
4) Show the wonderous plot to the user
sandbox_detail.html
<header>
<link rel="stylesheet" href="http://cdn.bokeh.org./bokeh/release/bokeh-0.11.1.min.css" type="text/css" >
<script type="text/javascript" src="http://cdn.bokeh.org./bokeh/release/bokeh-0.11.1.min.js"> </script>
</header>
<h1> Title: Test </h1>
<div>Test</div>
<body>
<div id="result"></div>
<script>
Bokeh.embed.embed_item({{json_object}}, "result");
</script>
</body>
This template renders everything but the 'result' div.
What have I missed?
This is what I see so far:
FIRST: You are mixing 2 methods for injecting plot json data into the page.
According to documentation you can do it using either of these two methods:
1) specify the div directly:
Python: json_data = json.dumps(json_item(p, "myplot"))
JavaScript: Bokeh.embed.embed_item(item);
2) specify the div in embed_item function:
Python: json_data = json.dumps(json_item(p))
JavaScript: Bokeh.embed.embed_item(item, "myplot");
But not both of them at the same time. Could this be the problem?
SECOND: Preferably don't insert Bokeh resources by hand: rather use CDN.render() or INLINE.render() to automatically include all that your script needs:
import json
from bokeh.resources import CDN
return render(request = request,
template_name = 'app/sandbox_detail.html',
context = { json_object = json.loads(json_string),
resources = CDN.render() } )
sandbox_detail.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
{{ resources }}
</head>
<body>
<div id="result"></div>
<script>
Bokeh.embed.embed_item({{ json_object }}, "result");
</script>
</body>
</html>
THIRD: Make sure what you embed in the page is json object not a string (see variable naming above)
It helps when you debug your rendered template in your browser's debug tool.
I tried a very similar approach and found one large flaw: My browser noted that it could not find the None object. The reason here is that python stores the empty value as None, while JavaScript expects a null object.
The solution? Python already translates None to null, when you run json.dumps. To keep it that way, read the json string as a string. So instead of your data=json.loads(fp.read()) use data=fp.read().
The idea is that user push the button and Flask should add some content on the page.
In my case python and Flask is not an aleatory choice, because Python make a lot of backdoor work (that for example js can't).
And perfectly it should be just one page without redirection on another page.
So my question is:
How to get the moment when user push the button and run the function I want
How to make this function add some generated content on the page without reloading.
As one of the commenters said, you are missing some key concepts in the web development stack you are working with. Lemme try to explain.
Flask is a backend framework/server. It receives requests from your browser and generates and sends a response. Your browser can make a request and load the response as a new page.
JavaScript runs in the browser (or to use a more generic term frontend). It can make changes to the way the current page looks and behaves without reloading. It can make requests to the backend, process the results it receives and act on them.
There are two basic ways browser makes a request to the backend. When you load a page. And when your Javascript code makes an XHR request.
There are three basic ways your frontend does things in response to user actions.
Run some javascript and do something without making any requests. For example, highlight a button user is hovering his mouse over, autocomplete what user is typing from some pre-defined list, switch UI tabs, hide/show elements etc.
Make a request to the backend and load the results as a new page. For example, follow a link to a new page on the site, submit a form, reload the page etc. This is how most web applications used to work 10-15 years ago.
Run some javascript and use it to make a request to the backend in the background. Use javascript to process the results and update the page dynamically without reloading. This is how most web applications work now.
Basically, there are two ways to do what you want, the old way, where you have your Flask app generate the whole page with your new content and send it to the browser. This will make the browser reload the page. And the new way, where your Flask app just provides the data you want to change and the javascript in teh browser makes an XHR request in the background and uses the data to update the page. This can happen without a reload and this is the way most modern web apps work.
I hope this clears things up for you a bit.
app.py
from flask import Flask,render_template,request
app = Flask(__name__)
#app.route("/add_item")
def add_item():
#in reality you are doing something else i imagine
return "<li>%s</li>"%(request.args.get('name'))
#app.route('/')
def hello_world():
return render_template('index.html',items=["Item 1","Item 2","Item 3"])
if __name__ == '__main__':
app.run()
templates/index.html
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport"
content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Document</title>
<script src="https://code.jquery.com/jquery-3.2.1.js"></script>
</head>
<body>
<ul id="items_holder">
{% for item in items %}
<li>{{ item }}</li>
{% endfor %}
</ul>
<button id="do_something">Do Something</button>
</body>
<script src="/static/app.js"></script>
</html>
static/app.js
$(function(){
$("#do_something").click(function(){
var item_name = prompt("enter item name")
$.get("/add_item?name="+item_name,function(result_data){
$('#items_holder').append(result_data)
})
})
})
I'm new to django and bokeh.
I was trying to render a simple bokeh plot supported by a few select options that essentially allow me to tweak my plot's content in a django web application.
The plots are rendered when the script and div elements obtained from the bokeh.embed.components() are passed to the template as context variables.
The same didn't work when i had a widget and a plot held in a bokeh.io.vform object.
I get the output right when i perform a bokeh.io.show(), by specifying the bokeh.plotting.output_file(), but I'm trying to get this running in my web application.
Am I missing anything? Or is there any other approach that serves my intent?
my code to just render a bokeh widget is as follows:
views.py
#django imports
from bokeh.embed import components
from bokeh.plotting import figure
from bokeh.io import vform
from bokeh.models.widgets import Select
def test(request):
s = Select(title="test", value="a", options=['a','b','c'])
script,div = components(s)
return render(request,'test.html',RequestContext(request,{'script':script,'div':div}))
test.html
<html>
<head>
<link href="http://cdn.bokeh.org/bokeh/release/bokeh-0.11.1.min.css" rel="stylesheet" type="text/css">
<script src="http://cdn.bokeh.org/bokeh/release/bokeh-0.11.1.min.js"></script>
</head>
{% load staticfiles %}
<body>
{{ div | safe }}
{{ script | safe }}
</body>
</html>
I would expect a select form element to be rendered when test.html is launched from test() in django. but doesn't happen.
BokehJS was recently split up into separate pieces to provide more flexible options for usage depending on what parts of the library are actually being used. Because loading resources is normally automatically handled in many cases, it was unintentioinally neglected to mention this split prominently in the docs. However, it is important to know about for embedding. There is an issue to update the docs, but what you need to know is that if you are using widgets, you now also need to load additional scripts from CDN:
http://cdn.bokeh.org/bokeh/release/bokeh-widgets-0.11.1.min.js
http://cdn.bokeh.org/bokeh/release/bokeh-widgets-0.11.1.min.css
I'm using Google Books API to integrate with an E-commerce site that I'm developing. The idea is to let the users search for books using a search bar, and then calling the Google API to output the number of books corresponding to the search keyword.
However, I get a 403 Forbidden error after I click submit after entering my query in the form. This is strange, because this never happened when I was testing my application on the localhost. Here's the code for my application:
main.py
class SearchHandler(Handler):
def get(self):
self.render("search.html")
def post(self):
keey = self.request.get('keey')
finaal = "https://www.googleapis.com/books/v1/volumes?q=" + keey + "&key=MY_APP_KEY"
f = urllib2.urlopen(finaal).read()
self.render("jsony.html", finaal = finaal)
app = webapp2.WSGIApplication(('/search', SearchHandler)], debug=True)
search.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
Book Name:<input type = "text" name = "keey">
<input type = "submit">
</form>
</body>
</html>
jsony.html
<html>
<head>
<title>Web Mining</title>
</head>
<body>
<form method = "post">
{{finaal}}
</form>
</body>
Now, the jsony.html is still incomplete. All I'm doing now is displaying the URL which contains the outputted json in it's raw, unprocessed form.
What seems to be causing this 403 error to arise after I deploy my application ?
EDIT 1:
The problem resolves when I remove the following line from my main python file:
f = urllib2.urlopen(finaal).read()
However, I would be needing my API's URL in order to extract data from its source code. What's happening ?
Try adding &country=US to the URL query string.
I have data in csv format and I want to create a webpage with charts or tables. I'm using the Pyramid Framework, chameleon, and deform_bootstrap.
I'm new to web development and there doesn't seem to be any tutorials for this out there. Can anyone point me in the right direction?
Without knowing more details its difficult to say. However, basically you will need a route registered in your config (in the root __init__.py file), a view callable (this can be just a method) to read the file and pass the data to a renderer and a chameleon template to render the data.
First set a route in your configuration. For example, to add a route for the table and one for the chart you could do something like this in your __init__.py file.
config.add_route('show_table', '/table')
config.add_route('show_chart', '/chart')
The choice of names and paths is up to you of course.
Then, you need to implement a view callable for each route. These would read the file and return a dictionary containing the data. They also tie the data to a particular renderer, in your case a chameleon template. Something like this might be right for your case where both routes need the same data.
from pyramid.view import view_config
def read_file():
"""read the file and return the data in a suitable format"""
return [1,4,2,4,56,7,45,3]
#view_config(route_name='show_table', renderer='templates/table.pt')
def table_view(request):
data = read_file()
return {'data': data}
#view_config(route_name='show_chart', renderer='templates/chart.pt')
def chart_view(request):
data = read_file()
return {'data': data}
Finally, you will need to implement the template files. These will be different depending on what you need.
<!DOCTYPE html>
<html xmlns:tal="http://xml.zope.org/namespaces/tal">
<head>
</head>
<body>
<table>
<tr><th>Data</th></tr>
<tr tal:repeat="datum data"><td>${datum}</td></tr>
</table>
</body>
</html>
To make charts I use d3.js but this is another question I think. Here is a simple example based on the first steps in this tutorial. First your template needs to make the data available to javascript. One way is to write the data into a javascript variable. This will work but is a bit messy - see this question for alternative approaches. In a real app you might want to use ajax so you would be writing the url to access the data here.
<!DOCTYPE html>
<html xmlns:tal="http://xml.zope.org/namespaces/tal">
<head>
<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>
</head>
<body>
<div class="chart"></div>
<script type="text/javascript">
var data = ${data};
var x = d3.scale.linear()
.domain([0, d3.max(data)])
.range([0, 420]);
d3.select(".chart")
.selectAll("div").data(data)
.enter().append("div")
.style("width", function(d) { return x(d) + "px"; })
.text(function(d) { return d; });
</script>
</body>
</html>
This should work but is untested, if you have any problems let me know and I will update it when I have a moment.