This question already has answers here:
Are global variables thread-safe in Flask? How do I share data between requests?
(4 answers)
Closed 11 months ago.
I run thin code.
And I want that after 20 seconds that the variable msg will get the value "hello msg2".
And if I refresh the page I see there "hello msg2" instead of "msg1".
from flask import Flask, render_template
import time
mag = "msg1"
app = Flask(__name__)
#app.route("/")
def home():
return render_template("home.html")
#app.route("/index.html")
def user():
return render_template("index.html", msg=msg)
if __name__ == "__main__":
app.run(port=7654, debug=True)
The index.html
<!doctype html>
<html>
<head>
<title>Home page</title>
</head>
<body>
<h1>Home Page!5 {{ msg }} </h1>
</body>
</html>
It is possible? Because I could not run any more commands in Python while the site was running.
You can use threading to spawn up a background thread to take care of updating value of msg. Use time.sleep() to add a delay to the execution.
from flask import Flask, render_template
import time
import threading
msg = "msg1"
# utility to change msg variable
def change_msg(delay=20):
time.sleep(delay)
global msg
msg = "msg2"
app = Flask(__name__)
#app.route("/")
def home():
return render_template("home.html")
#app.route("/index.html")
def user():
return render_template("index.html", msg=msg)
if __name__ == "__main__":
# spawn a thread that changes value of msg after a delay
t = threading.Thread(target=change_msg)
t.start()
app.run(port=7654, debug=True)
You'll need some Javascript. The process you're looking at is
Load the page which makes a call to your server (python)
Check for the presence of a cookie which tells you the page has been loaded before. If this cookie is not present, you return "msg1" and set a cookie letting you know you've returned "msg1". If the cookie is present, then you return "hello msg2" and you don't have to set the cookie again.
This part is where Javascript comes in - After 20 seconds, make another call to your server. You can do this asynchronously so that your page is not reloaded. The cookie will be sent along and step 2 above comes into play.
The easiest way to do what you asked for is a Timer. In your case the content of some_function() should be in your handler function user().
from threading import Timer
from time import sleep
msg = "msg1"
def setMessage():
global msg
msg = "msg2"
print("Message has changed")
def some_function():
global msg
timer = Timer(1.0, setMessage)
timer.start()
print(msg)
some_function()
sleep(3)
some_function()
Expected output:
msg1
Message has changed
msg2
Message has changed
Note: setMessage() is called twice here, but you could check if the message is already msg2 before starting the timer to prevent this from happening.
If what you actually want is concurrency or parallelism you should have a look at Python Threading or for asynchronous programming at asyncio.
Related
Using Tornado, I have a POST request that takes a long time as it makes many requests to another API service and processes the data. This can take minutes to fully complete. I don't want this to block the entire web server from responding to other requests, which it currently does.
I looked at multiple threads here on SO, but they are often 8 years old and the code does not work anylonger as tornado removed the "engine" component from tornado.gen.
Is there an easy way to kick off this long get call and not have it block the entire web server in the process? Is there anything I can put in the code to say.. "submit the POST response and work on this one function without blocking any concurrent server requests from getting an immediate response"?
Example:
main.py
def make_app():
return tornado.web.Application([
(r"/v1", MainHandler),
(r"/v1/addfile", AddHandler, dict(folderpaths = folderpaths)),
(r"/v1/getfiles", GetHandler, dict(folderpaths = folderpaths)),
(r"/v1/getfile", GetFileHandler, dict(folderpaths = folderpaths)),
])
if __name__ == "__main__":
app = make_app()
sockets = tornado.netutil.bind_sockets(8888)
tornado.process.fork_processes(0)
tornado.process.task_id()
server = tornado.httpserver.HTTPServer(app)
server.add_sockets(sockets)
tornado.ioloop.IOLoop.current().start()
addHandler.py
class AddHandler(tornado.web.RequestHandler):
def initialize(self, folderpaths):
self.folderpaths = folderpaths
def blockingFunction(self):
time.sleep(320)
post("AWAKE")
def post(self):
user = self.get_argument('user')
folderpath = self.get_argument('inpath')
outpath = self.get_argument('outpath')
workflow_value = self.get_argument('workflow')
status_code, status_text = validateInFolder(folderpath)
if (status_code == 200):
logging.info("Status Code 200")
result = self.folderpaths.add_file(user, folderpath, outpath, workflow_value)
self.write(result)
self.finish()
#At this point the path is validated.
#POST response should be send out. Internal process should continue, new
#requests should not be blocked
self.blockingFunction()
Idea is that if input-parameters are validated the POST response should be sent out.
Then internal process (blockingFunction()) should be started, that should not block the Tornado Server from processing another API POST request.
I tried defining the (blockingFunction()) as async, which allows me to process multiple concurrent user requests - however there was a warning about missing "await" with async method.
Any help welcome. Thank you
class AddHandler(tornado.web.RequestHandler):
def initialize(self, folderpaths):
self.folderpaths = folderpaths
def blockingFunction(self):
time.sleep(320)
post("AWAKE")
async def post(self):
user = self.get_argument('user')
folderpath = self.get_argument('inpath')
outpath = self.get_argument('outpath')
workflow_value = self.get_argument('workflow')
status_code, status_text = validateInFolder(folderpath)
if (status_code == 200):
logging.info("Status Code 200")
result = self.folderpaths.add_file(user, folderpath, outpath, workflow_value)
self.write(result)
self.finish()
#At this point the path is validated.
#POST response should be send out. Internal process should continue, new
#requests should not be blocked
await loop.run_in_executor(None, self.blockingFunction)
#if this had multiple parameters it would be
#await loop.run_in_executor(None, self.blockingFunction, param1, param2)
Thank you #xyres
Further read: https://www.tornadoweb.org/en/stable/faq.html
I've been trying to get this flask server to update itself with data generated from a loop that runs on a a .py script when called for by the user via push button on webpage. I've been looking into recommended solutions and have seen websockets (sockets.io), ajax, nodejs come up. I understand that i need to implement some form of js in my project, and ajax looked to be the most simple (so i thought). I only have about 3 weeks of experience programming in python. Mainly i look for examples close to what i want, and then try to modify it to suit my needs, but haven't found any examples for what i'm looking for. Even then, my general newness to programming means that the more examples i "tack on" the more likely i am to degrade the overall structure of what i've already accomplished.
Goal
The goal is to update a value displayed on the page without a reload but instead have js update the value every second. The value is generated from a x=x+1 counter in my .py file. This will be replaced by sensor inputs gathered from my Rpi later.
Actual results
When i run the current code,
my html elements get double posted so i see what i've put into the index.html file twice although the second button elements don't actually respond to clicking,
I also get an endless stream of Posts in my terminal window.
Clicking on the button elements no longer execute my loop in the .py file and instead displays "Method not allowed"
What i've tried
I've tried to implement setTmeout in my html file as a way to call back to the python app and get an updated value (the x=x+1) every second. I've read posts around using setTimeout as a way to deal with issues using setInterval. Due to the variety of ways i've seen ajax calls employed and learning resources being primarily structured towards forms, databases, and chat apps, most of my searches aren't bringing up anything new for me to learn from that might help. I'm currently doing ajax tutorials hoping to come accross something i can use, any help would be greatly appreciated.
ajaxTest.py My python flask file
import threading
import time
from flask import Flask, render_template, jsonify, request
import RPi.GPIO as GPIO
import datetime
from datetime import datetime
from datetime import timedelta
app = Flask(__name__)
bioR_on = False
ledGrnSts = 0
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
air = 21
light = 20
waste = 16
feed = 12
water = 26
pinList = [21,20,16,12,26]
def pump(pin):
GPIO.output(pin, GPIO.LOW)
print(pin,'on')
time.sleep(1)
GPIO.output(pin, GPIO.HIGH)
print(pin, 'off')
time.sleep(1)
def on(pin):
GPIO.output(pin, GPIO.LOW)
#app.route("/")
def index():
templateData = {
'title' : 'Bioreactor output Status!',
'ledGrn' : ledGrnSts,
}
return render_template('index.html', **templateData)
#app.route('/<deviceName>/<action>', methods = ["POST"])
def start(deviceName, action):
# script for Pi Relays
def run():
if action == "on":
alarm = datetime.now() + timedelta(seconds =10)
global bioR_on
bioR_on = True
while bioR_on:
tday = datetime.now()
time.sleep(1)
#feed(tday, alarm)
x=x+1
return jsonify(x)
GPIO.setmode(GPIO.BCM)
for i in pinList:
GPIO.setup (i, GPIO.OUT)
GPIO.output(i, GPIO.HIGH)
on(air)
on(light)
print(tday)
if tday >= alarm:
print('alarm activated')
# run = False
pump(waste)
print('waste activated')
pump(feed)
print('feed on')
GPIO.cleanup()
alarm = datetime.now() + timedelta(seconds =10)
print('next feeding time ', alarm)
time.sleep(1)
if action == 'off':
bioR_on = False
#return "off"
GPIO.cleanup()
thread = threading.Thread(target=run)
thread.start()
templateData = {
'ledGrn' : ledGrnSts,
}
return render_template('index.html', **templateData)
if __name__ == "__main__":
app.run(host='0.0.0.0', port=80, debug=True, threaded=True)
My index.html file
<!DOCTYPE html>
<head>
<title>BioReactor Control</title>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<link rel="stylesheet" href='../static/style.css'/>
</head>
<body>
<h1>Actuators</h1>
<h2> Status </h2>
<h3> GRN LED ==> {{ ledGrn }}</h3>
<br>
<h2> Commands </h2>
<h3>
Activate Bioreactor Ctrl ==>
TURN ON
TURN OFF
</h3>
<h3>
Current Count
</h3>
<p id="demo"></p>
<script>
setTimeout($.ajax({
url: '/<deviceName>/<action>',
type: 'POST',
success: function(response) {
console.log(response);
$("#num").html(response);
},
error: function(error) {
console.log(error);
}
}), 1000);
</script>
<h1>Output</h1>
<h1 id="num"></h1>
</body>
</html>
Picture of results
I created minimal code which uses AJAX to get new value every 1 second.
I use setInterval to repeate it every 1 second. I also uses function(){ $.ajax ... } to create function which is not executed at once but setInterval will call it every 1 second. Without function(){...} code $.ajax was executed at start and its result was used as function executed every 1 second - but it returns nothing - so finally it updated value only once (at start) and later setInterval was running nothing.
I added current time to see if it is still running.
buttons run function '/<device>/<action>' which start and stop thread but AJAX uses /update to get current value.
I use render_template_string to have all code in one file - so other people can easily copy and test it.
I reduced HTML to minimal. To make sure I put <h1> before script which needs these tags.
I didn't tested it with global=True which may run it in new threads and it can make problem.
from flask import Flask, request, render_template_string, jsonify
import datetime
import time
import threading
app = Flask(__name__)
running = False # to control loop in thread
value = 0
def rpi_function():
global value
print('start of thread')
while running: # global variable to stop loop
value += 1
time.sleep(1)
print('stop of thread')
#app.route('/')
#app.route('/<device>/<action>')
def index(device=None, action=None):
global running
global value
if device:
if action == 'on':
if not running:
print('start')
running = True
threading.Thread(target=rpi_function).start()
else:
print('already running')
elif action == 'off':
if running:
print('stop')
running = False # it should stop thread
else:
print('not running')
return render_template_string('''<!DOCTYPE html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
</head>
<body>
TURN ON
TURN OFF
<h1 id="num"></h1>
<h1 id="time"></h1>
<script>
setInterval(function(){$.ajax({
url: '/update',
type: 'POST',
success: function(response) {
console.log(response);
$("#num").html(response["value"]);
$("#time").html(response["time"]);
},
error: function(error) {
console.log(error);
}
})}, 1000);
</script>
</body>
</html>
''')
#app.route('/update', methods=['POST'])
def update():
return jsonify({
'value': value,
'time': datetime.datetime.now().strftime("%H:%M:%S"),
})
app.run() #debug=True
I have 2 functions.
1st function stores the data received in a list and 2nd function writes the data into a csv file.
I'm using Flask. Whenever a web service has been called it will store the data and send response to it, as soon as it sends response it triggers the 2nd function.
My Code:
from flask import Flask, flash, request, redirect, url_for, session
import json
app = Flask(__name__)
arr = []
#app.route("/test", methods=['GET','POST'])
def check():
arr.append(request.form['a'])
arr.append(request.form['b'])
res = {'Status': True}
return json.dumps(res)
def trigger():
df = pd.DataFrame({'x': arr})
df.to_csv("docs/xyz.csv", index=False)
return
Obviously the 2nd function is not called.
Is there a way to achieve this?
P.S: My real life problem is different where trigger function is time consuming and I don't want user to wait for it to finish execution.
One solution would be to have a background thread that will watch a queue. You put your csv data in the queue and the background thread will consume it. You can start such a thread before first request:
import threading
from multiprocessing import Queue
class CSVWriterThread(threading.Thread):
def __init__(self, *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.input_queue = Queue()
def send(self, item):
self.input_queue.put(item)
def close(self):
self.input_queue.put(None)
self.input_queue.join()
def run(self):
while True:
csv_array = self.input_queue.get()
if csv_array is None:
break
# Do something here ...
df = pd.DataFrame({'x': csv_array})
df.to_csv("docs/xyz.csv", index=False)
self.input_queue.task_done()
time.sleep(1)
# Done
self.input_queue.task_done()
return
#app.before_first_request
def activate_job_monitor():
thread = CSVWriterThread()
app.csvwriter = thread
thread.start()
And in your code put the message in the queue before returning:
#app.route("/test", methods=['GET','POST'])
def check():
arr.append(request.form['a'])
arr.append(request.form['b'])
res = {'Status': True}
app.csvwriter.send(arr)
return json.dumps(res)
P.S: My real life problem is different where trigger function is time consuming and I don't want user to wait for it to finish execution.
Consider using celery which is made for the very problem you're trying to solve. From docs:
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system.
I recommend you integrate celery with your flask app as described here. your trigger method would then become a straightforward celery task that you can execute without having to worry about long response time.
Im actually working on another interesting case on my side where i pass the work off to a python worker that sends the job to a redis queue. There are some great blogs using redis with Flask , you basically need to ensure redis is running (able to connect on port 6379)
The worker would look something like this:
import os
import redis
from rq import Worker, Queue, Connection
listen = ['default']
redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
conn = redis.from_url(redis_url)
if __name__ == '__main__':
with Connection(conn):
worker = Worker(list(map(Queue, listen)))
worker.work()
In my example I have a function that queries a database for usage and since it might be a lengthy process i pass it off to the worker (running as a seperate script)
def post(self):
data = Task.parser.parse_args()
job = q.enqueue_call(
func=migrate_usage, args=(my_args),
result_ttl=5000
)
print("Job ID is: {}".format(job.get_id()))
job_key = job.get_id()
print(str(Job.fetch(job_key, connection=conn).result))
if job:
return {"message": "Job : {} added to queue".format(job_key)}, 201
Credit due to the following article:
https://realpython.com/flask-by-example-implementing-a-redis-task-queue/#install-requirements
You can try use streaming. See next example:
import time
from flask import Flask, Response
app = Flask(__name__)
#app.route('/')
def main():
return '''<div>start</div>
<script>
var xhr = new XMLHttpRequest();
xhr.open('GET', '/test', true);
xhr.onreadystatechange = function(e) {
var div = document.createElement('div');
div.innerHTML = '' + this.readyState + ':' + this.responseText;
document.body.appendChild(div);
};
xhr.send();
</script>
'''
#app.route('/test')
def test():
def generate():
app.logger.info('request started')
for i in range(5):
time.sleep(1)
yield str(i)
app.logger.info('request finished')
yield ''
return Response(generate(), mimetype='text/plain')
if __name__ == '__main__':
app.run('0.0.0.0', 8080, True)
All magic in this example in genarator where you can start response data, after do some staff and yield empty data to end your stream.
For details look at http://flask.pocoo.org/docs/patterns/streaming/.
You can defer route specific actions with limited context by combining after_this_request and response.call_on_close. Note that request and response context won't be available but the route function context remains available. So you'll need to copy any request/response data you'll need into local variables for deferred access.
I moved your array to a local var to show how the function context is preserved. You could change your csv write function to an append so you're not pushing data endlessly into memory.
from flask import Flask, flash, request, redirect, url_for, session
import json
app = Flask(__name__)
#app.route("/test", methods=['GET','POST'])
def check():
arr = []
arr.append(request.form['a'])
arr.append(request.form['b'])
res = {'Status': True}
#flask.after_this_request
def add_close_action(response):
#response.call_on_close
def process_after_request():
df = pd.DataFrame({'x': arr})
df.to_csv("docs/xyz.csv", index=False)
return response
return json.dumps(res)
I have a function that crawls the web for data and computes a score for the search. However, this can take a while and sometimes the webpage times out before finishing execution.
So I created a separate thread that executes the function and loading.html that tells the client that data is still being collected. Once the function ends in the thread, how do I reload the webpage to display output.html that displays the score.
This is a simpler version of what I have so far:
from flask import Flask
from flask import render_template
from threading import Thread
app = Flask(__name__)
#app.route("/")
def init():
return render_template('index.html')
#app.route("/", methods=['POST'])
def load():
th = Thread(target=something, args=())
th.start()
return render_template('loading.html')
def something():
#do some calculation and return the needed value
if __name__ == "__main__":
app.run()
How do I route my app to render_template('output.html', x=score) once something() inside the thread th finishes?
I am trying to avoid task queues like redis since I want to deploy this app on the web and I don't want to incur charges (this is more of an experiment and hobby).
A detailed answer with code would help a lot since I am new to flask and multithreading
An easy way is making cyclic Ajax requests to a thread_status endpoint that gives you information about the currently running task.
import time
from flask import Flask, jsonify
from flask import render_template
from threading import Thread
app = Flask(__name__)
th = Thread()
finished = False
#app.route("/")
def init():
return render_template('index.html')
#app.route("/", methods=['POST'])
def load():
global th
global finished
finished = False
th = Thread(target=something, args=())
th.start()
return render_template('loading.html')
def something():
""" The worker function """
global finished
time.sleep(5)
finished = True
#app.route('/result')
def result():
""" Just give back the result of your heavy work """
return 'Done'
#app.route('/status')
def thread_status():
""" Return the status of the worker thread """
return jsonify(dict(status=('finished' if finished else 'running')))
if __name__ == "__main__":
app.run(debug=True)
So in your loading.html just insert a cyclic Ajax get() request:
<html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
<script>
$(document).ready(function() {
var refresh_id = setInterval(function() {
$.get(
"{{ url_for('thread_status') }}",
function(data) {
console.log(data);
if (data.status == 'finished') {
window.location.replace("{{ url_for('result') }}");
}
}
)}
, 1000);
});
</script>
</head>
<body>
<p>Loading...</p>
</body>
</html>
You can even append this by a progress counter if you like. But you need to take care that you prevent the thread from being run multiple times.
I'm trying to make a simple web interface for my arduino using a raspberry pi. I want to click a link that i create in html and send the string "on" to the python program so it can tell the arduino to turn on.
Here's my python code
import serial
from flask import Flask, render_template, request
import datetime
app = Flask(__name__)
ser = serial.Serial('/dev/ttyACM0', 9600)
#app.route("/<action>")
def action(action):
print action
#command = ""
#while command != "done":
# command = raw_input("what do you want? ")
if action == "on":
ser.write('1')
elif action == "off":
ser.write('0')
return render_template('index.html', **templateData)
#app.route("/")
def display():
now = datetime.datetime.now()
timeString = now.strftime("%Y-%m-%d %H:%M")
templateData = {
'title' : 'arduino',
'time' : timeString
}
return render_template('index.html', **templateData)
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8080, debug=True)
and here's my html code
<!DOCTYPE html>
<head>
<title>{{title}}</title>
</head>
<body>
<p>The time at server is {{time}}</p>
<p>
The LED is currently off (turn on)
</p>
<body>
</html>
When someone clicks the link turn on I want the string on to be sent to the action method so that it can take it from there. Instead what it does is go to the /on directory which is not surprising. I've looked everywhere and can't find how to do this. This is my first time ever using Flask and I'm fairly new to python so please don't be too harsh if I'm completely off.
You could just redirect after you take the action
from flask import Flask, render_template, request, redirect
#app.route("/<action>")
def action(action):
...
return redirect(url_for('display'))