With eventlet, callback function invoked only after bg thread exit (python Flask) - python

I am using Flask and Socketio. Here's a simplification of my code:
from flask import Flask, url_for, render_template, send_from_directory, request, redirect
from flask_socketio import SocketIO, emit, send
app = Flask(__name__)
socketio = SocketIO(app, async_mode='threading')
#socketio = SocketIO(app) # using eventlet, don't write "async_mode='threading' or it will ignore eventlet
#socketio.on('xxx', namespace='/xxx')
def start_bg(cid):
# eventlet.spawn(bg_conversion, cid)
socketio.start_background_task(bg_conversion, cid) # Do some time-consuming things
print('test')
if __name__ == '__main__':
socketio.run(app, port=80)
It works fine without eventlet, but now I have the need to stop a thread so I guess eventlet would be good.
However, if I switch to the annotated code, the client side's callback function will only be invoked after the thread(bg_conversion) exits.
I added a print after the spawning line, and it prints immediately - so it is running in the background. But why isn't the callback function on the client's side invoked immediately? Doesn't the function return immediately after the print?

Related

Passing argument in thread without invoking - how to put Flask server in thread

How I can start flask server in a thread with custom ip (listening the network)?
This line doesn't block the main thread, but it doesn't listen connections from the network.
threading.Thread(target = app.run).start()
When this is used it waits to this thread to finish, and main thread is blocked.
#threading.Thread(target = app.run(host='192.168.1.42')).start()
I have tried to make game, Pygame is running in mainthread and flask server is used to host webpage, which offer joystick for players.
At the moment i can control it with local machine, but not via mobilephone. And if i config the flask with custom ip, the main thread will stop for to wait the server thread.
It's something to do with invoking and referring, but i don't know how to setup the thread with argument, but without invoking.
Whole pycharm project is in GitHUB
Here is the server.py
from flask import Flask,render_template,request
from flask_wtf import FlaskForm
from wtforms import StringField, SubmitField
import threading
def initServer(controlDataToPyGame): #argument is queue for transferring data to main thread
app = Flask(__name__)
app.debug= False
#app.route('/')
def index():
print("index")
return render_template('index.html')
#app.route('/play/')
def play():
print("play")
return render_template('controller.html')
#app.route("/Control/")
def UP():
x = request.args.get('joyX')
y = request.args.get('joyY')
controlDict = {"name":"ice", "x":x,"y":y}
controlDataToPyGame.put(controlDict)
return ("nothing")
##this doesn't block the main thread, but it doesn't listen connections from the network.
threading.Thread(target = app.run).start()
#when this is used it waits to this thread to finish, and main thread is blocked.
#threading.Thread(target = app.run(host='192.168.1.42')).start()
If you read documentation for Thread then you see args= and kwargs=
threading.Thread(target=app.run, kwargs={'host': '192.168.1.42'}).start()
Using
threading.Thread(target = app.run(host='192.168.1.42')).start()
you simply run app.run() before sending its result to Thread like
result = app.run(host='192.168.1.42')
Thread(target=result).start()
and it runs app.run() in main thread forever - and it never use Thread

Non blocking reads from stdout in greenlet

I have a flask app with SocketIO. Due to this, I choose to run the application using the eventlet library. Under the hood, eventlet uses green threads to achieve concurrency if I'm not mistaken.
In my app, I want to spawn a process and stream the output over web sockets. Below is a toy example
# eventlet==0.25.1
# flask==1.1.1
# flask-socketio==4.2.1
import eventlet
import subprocess
from flask import Flask, jsonify
from flask_socketio import SocketIO
app = Flask(__name__)
socketio = SocketIO(app)
socketio.init_app(app, cors_allowed_origins='*')
#app.route('/ping')
def start_ping():
eventlet.spawn_n(ping)
return jsonify(success=True)
#app.route('/hello')
def hello():
return jsonify(data='Hello World')
def ping():
proc = subprocess.Popen(
('ping', '-t', 'google.com'),
stdout=subprocess.PIPE,
)
for line in proc.stdout:
eventlet.sleep(0.5)
socketio.emit('log_receive', str(line))
if __name__ == '__main__':
socketio.run(app)
User hits the /ping endpoint.
The ping() function is executed in a green thread which runs the ping command in a child process
lines are read from the subprocess's stdout and emited via web sockets
eventlet.sleep(0.5) is used to give other parts of the application a chance to run.
Question:
for line in proc.stdout: is blocking. Until something comes through stdout, eventlet.sleep(0.5) will not execute and therefore the rest of the application isn't given a chance to run. Thus rendering the app, unresponsive.
I came across this question on how to do non-blocking reads from subprocess.PIPE and the suggestion is to essentially use a separate thread to do the reading.
Unfortunately, I can not use a separate thread because of the concurrent/coroutine programming model I'm restricted to because of eventlet/greenlet.
I could use fcntl() to do non-blocking reads but I'm on windows and so it isn't an option
What's an alternative to avoid having the application be at mercy of the subprocess's stdout

Python sockets with flask and vue for continious serial data stream

So I have an flask application which is extended by the flask_socketio package.
I currently have a vue front end, that connects to an socket.
Now the output of the socket connection in the following:
I also see an connection in the terminal of the flask application:
127.0.0.1 - - [23/Aug/2019 21:07:11] "GET /socket.io/?EIO=3&transport=polling&t=Mo_u1wR HTTP/1.1" 200
However, I have on my code that when the socket connects, there should start an thread with while true, to continious receive the data.
This is the code I have:
from flask import Flask, jsonify
from flask_socketio import SocketIO, send, emit
import threading
from threading import Lock
import time
import controllers.gpsController
# configuration
DEBUG = True
# instantiate the app
app = Flask(__name__)
app.config.from_object(__name__)
thread = None
thread_lock = Lock()
async_mode = None
# app.config['SECRET_KEY'] = 'secret!'
# enable CORS
socketio = SocketIO(app, cors_allowed_origins='*', async_mode=async_mode)
def activate_gps():
ser = controllers.gpsController.open_serial_connection()
while True:
data = controllers.gpsController.readGPS(ser)
if data != None:
socketio.emit('fetch_gps_data', data, broadcast=True)
socketio.sleep(0.1)
# Use sockets here
#socketio.on('connect')
def start_get_data_thread():
global thread
with thread_lock:
if thread is None:
thread = socketio.start_background_task(target=activate_gps)
if __name__ == '__main__':
socketio.run(app, host='127.0.0.1', port=12345)
Anyone an idea what I'm doing wrong here? If I have te code of activate_gps() in another file and call that, I get the wanted output.

Flask-SocketIO, eventlet, multiprocessing.Process not compatible

My code (Python 3.5 on Raspbian 9 - Stretch) is divided up into a number of separate processes, which are run from main.py. A simplified example of my code is below, which I believe is plain vanilla use of Flask, socketIO, eventlet with multiprocessing.Process. The problem is it hangs when I try to access the pipe that connects the different processes.
My understanding (which wouldn’t surprise me if I was completely wrong) is that this is a long standing issue related to eventlet and multiprocessing.Process and as of January 2018 has not been resolved. How to combine multiprocessing and eventlet
https://github.com/eventlet/eventlet/issues/147
My question is this. This seems like a common use case, but doesn’t work. So, what work around or different approach would you recommend?
--- in webprocess.py ---
#!/usr/bin/python3
def WebFunc(outfrompipe, intopipe):
global thread
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, async_mode=”eventlet”)
thread = None
thread_lock = Lock()
#app.route('/')
def index():
return render_template('index.html', async_mode=socketio.async_mode)
#socketio.on('my_event', namespace='/test')
def test_msg(msg):
# Receive a message from a web app
print(“Received message”, msg)
# Send this message to another process
# THIS IS WHERE IT HANGS!!
intopipe.send(msg)
socketio.run(app, debug=False, host='0.0.0.0')
--- in main.py ---
#!/usr/bin/python3
import webprocess as webproc
import multiprocessing
import time
if __name__ == '__main__':
multiprocessing.set_start_method('spawn')
outfrompipe, intopipe = multiprocessing.Pipe()
wf = multiprocessing.Process(name=”WebProc”, target=webproc.WebFunc,
args=(outfrompipe, intopipe))
wf.start()
while True:
message = outfrompipe.recv()
print(message)
time.sleep(1)
wf.join()

gevent-socketio send message from thread

I would like to use gevent-socketio to send messages from a worker thread and update all connected clients on the status of the job.
I tried this:
from flask import Flask, render_template
from flask.ext.socketio import SocketIO, send, emit
import threading
import time
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
#socketio.on('message')
def handle_message(message):
send(message, broadcast=True)
#app.route('/')
def index():
return render_template('index.html')
def ping_thread():
while True:
time.sleep(1)
print 'Pinging'
send('ping')
if __name__ == '__main__':
t = threading.Thread(target=ping_thread)
t.daemon = True
t.start()
socketio.run(app)
And it gives me this error:
RuntimeError: working outside of request context
How do I send messages from a function that doesn't have the #socketio.on() decorator? Can I use gevent directly to send messages to socketio?
From this section of the documentation:
Sometimes the server needs to be the originator of a message. This can be useful to send a notification to clients of an event that originated in the server. The socketio.send() and socketio.emit() methods can be used to broadcast to all connected clients:
def some_function():
socketio.emit('some event', {'data': 42})
This emit is not from from flask.ext.socketio import SocketIO, send, but instead called on your socketio variable from socketio = SocketIO(app). Had you done socketio_connection = SocketIO(app), then you'd be calling socketio_connection.emit() to broadcast your data.

Categories

Resources