I am using Flask and Socketio. Here's a simplification of my code:
from flask import Flask, url_for, render_template, send_from_directory, request, redirect
from flask_socketio import SocketIO, emit, send
app = Flask(__name__)
socketio = SocketIO(app, async_mode='threading')
#socketio = SocketIO(app) # using eventlet, don't write "async_mode='threading' or it will ignore eventlet
#socketio.on('xxx', namespace='/xxx')
def start_bg(cid):
# eventlet.spawn(bg_conversion, cid)
socketio.start_background_task(bg_conversion, cid) # Do some time-consuming things
print('test')
if __name__ == '__main__':
socketio.run(app, port=80)
It works fine without eventlet, but now I have the need to stop a thread so I guess eventlet would be good.
However, if I switch to the annotated code, the client side's callback function will only be invoked after the thread(bg_conversion) exits.
I added a print after the spawning line, and it prints immediately - so it is running in the background. But why isn't the callback function on the client's side invoked immediately? Doesn't the function return immediately after the print?
Related
How I can start flask server in a thread with custom ip (listening the network)?
This line doesn't block the main thread, but it doesn't listen connections from the network.
threading.Thread(target = app.run).start()
When this is used it waits to this thread to finish, and main thread is blocked.
#threading.Thread(target = app.run(host='192.168.1.42')).start()
I have tried to make game, Pygame is running in mainthread and flask server is used to host webpage, which offer joystick for players.
At the moment i can control it with local machine, but not via mobilephone. And if i config the flask with custom ip, the main thread will stop for to wait the server thread.
It's something to do with invoking and referring, but i don't know how to setup the thread with argument, but without invoking.
Whole pycharm project is in GitHUB
Here is the server.py
from flask import Flask,render_template,request
from flask_wtf import FlaskForm
from wtforms import StringField, SubmitField
import threading
def initServer(controlDataToPyGame): #argument is queue for transferring data to main thread
app = Flask(__name__)
app.debug= False
#app.route('/')
def index():
print("index")
return render_template('index.html')
#app.route('/play/')
def play():
print("play")
return render_template('controller.html')
#app.route("/Control/")
def UP():
x = request.args.get('joyX')
y = request.args.get('joyY')
controlDict = {"name":"ice", "x":x,"y":y}
controlDataToPyGame.put(controlDict)
return ("nothing")
##this doesn't block the main thread, but it doesn't listen connections from the network.
threading.Thread(target = app.run).start()
#when this is used it waits to this thread to finish, and main thread is blocked.
#threading.Thread(target = app.run(host='192.168.1.42')).start()
If you read documentation for Thread then you see args= and kwargs=
threading.Thread(target=app.run, kwargs={'host': '192.168.1.42'}).start()
Using
threading.Thread(target = app.run(host='192.168.1.42')).start()
you simply run app.run() before sending its result to Thread like
result = app.run(host='192.168.1.42')
Thread(target=result).start()
and it runs app.run() in main thread forever - and it never use Thread
I have a flask app with SocketIO. Due to this, I choose to run the application using the eventlet library. Under the hood, eventlet uses green threads to achieve concurrency if I'm not mistaken.
In my app, I want to spawn a process and stream the output over web sockets. Below is a toy example
# eventlet==0.25.1
# flask==1.1.1
# flask-socketio==4.2.1
import eventlet
import subprocess
from flask import Flask, jsonify
from flask_socketio import SocketIO
app = Flask(__name__)
socketio = SocketIO(app)
socketio.init_app(app, cors_allowed_origins='*')
#app.route('/ping')
def start_ping():
eventlet.spawn_n(ping)
return jsonify(success=True)
#app.route('/hello')
def hello():
return jsonify(data='Hello World')
def ping():
proc = subprocess.Popen(
('ping', '-t', 'google.com'),
stdout=subprocess.PIPE,
)
for line in proc.stdout:
eventlet.sleep(0.5)
socketio.emit('log_receive', str(line))
if __name__ == '__main__':
socketio.run(app)
User hits the /ping endpoint.
The ping() function is executed in a green thread which runs the ping command in a child process
lines are read from the subprocess's stdout and emited via web sockets
eventlet.sleep(0.5) is used to give other parts of the application a chance to run.
Question:
for line in proc.stdout: is blocking. Until something comes through stdout, eventlet.sleep(0.5) will not execute and therefore the rest of the application isn't given a chance to run. Thus rendering the app, unresponsive.
I came across this question on how to do non-blocking reads from subprocess.PIPE and the suggestion is to essentially use a separate thread to do the reading.
Unfortunately, I can not use a separate thread because of the concurrent/coroutine programming model I'm restricted to because of eventlet/greenlet.
I could use fcntl() to do non-blocking reads but I'm on windows and so it isn't an option
What's an alternative to avoid having the application be at mercy of the subprocess's stdout
So I have an flask application which is extended by the flask_socketio package.
I currently have a vue front end, that connects to an socket.
Now the output of the socket connection in the following:
I also see an connection in the terminal of the flask application:
127.0.0.1 - - [23/Aug/2019 21:07:11] "GET /socket.io/?EIO=3&transport=polling&t=Mo_u1wR HTTP/1.1" 200
However, I have on my code that when the socket connects, there should start an thread with while true, to continious receive the data.
This is the code I have:
from flask import Flask, jsonify
from flask_socketio import SocketIO, send, emit
import threading
from threading import Lock
import time
import controllers.gpsController
# configuration
DEBUG = True
# instantiate the app
app = Flask(__name__)
app.config.from_object(__name__)
thread = None
thread_lock = Lock()
async_mode = None
# app.config['SECRET_KEY'] = 'secret!'
# enable CORS
socketio = SocketIO(app, cors_allowed_origins='*', async_mode=async_mode)
def activate_gps():
ser = controllers.gpsController.open_serial_connection()
while True:
data = controllers.gpsController.readGPS(ser)
if data != None:
socketio.emit('fetch_gps_data', data, broadcast=True)
socketio.sleep(0.1)
# Use sockets here
#socketio.on('connect')
def start_get_data_thread():
global thread
with thread_lock:
if thread is None:
thread = socketio.start_background_task(target=activate_gps)
if __name__ == '__main__':
socketio.run(app, host='127.0.0.1', port=12345)
Anyone an idea what I'm doing wrong here? If I have te code of activate_gps() in another file and call that, I get the wanted output.
My code (Python 3.5 on Raspbian 9 - Stretch) is divided up into a number of separate processes, which are run from main.py. A simplified example of my code is below, which I believe is plain vanilla use of Flask, socketIO, eventlet with multiprocessing.Process. The problem is it hangs when I try to access the pipe that connects the different processes.
My understanding (which wouldn’t surprise me if I was completely wrong) is that this is a long standing issue related to eventlet and multiprocessing.Process and as of January 2018 has not been resolved. How to combine multiprocessing and eventlet
https://github.com/eventlet/eventlet/issues/147
My question is this. This seems like a common use case, but doesn’t work. So, what work around or different approach would you recommend?
--- in webprocess.py ---
#!/usr/bin/python3
def WebFunc(outfrompipe, intopipe):
global thread
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, async_mode=”eventlet”)
thread = None
thread_lock = Lock()
#app.route('/')
def index():
return render_template('index.html', async_mode=socketio.async_mode)
#socketio.on('my_event', namespace='/test')
def test_msg(msg):
# Receive a message from a web app
print(“Received message”, msg)
# Send this message to another process
# THIS IS WHERE IT HANGS!!
intopipe.send(msg)
socketio.run(app, debug=False, host='0.0.0.0')
--- in main.py ---
#!/usr/bin/python3
import webprocess as webproc
import multiprocessing
import time
if __name__ == '__main__':
multiprocessing.set_start_method('spawn')
outfrompipe, intopipe = multiprocessing.Pipe()
wf = multiprocessing.Process(name=”WebProc”, target=webproc.WebFunc,
args=(outfrompipe, intopipe))
wf.start()
while True:
message = outfrompipe.recv()
print(message)
time.sleep(1)
wf.join()
I would like to use gevent-socketio to send messages from a worker thread and update all connected clients on the status of the job.
I tried this:
from flask import Flask, render_template
from flask.ext.socketio import SocketIO, send, emit
import threading
import time
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
#socketio.on('message')
def handle_message(message):
send(message, broadcast=True)
#app.route('/')
def index():
return render_template('index.html')
def ping_thread():
while True:
time.sleep(1)
print 'Pinging'
send('ping')
if __name__ == '__main__':
t = threading.Thread(target=ping_thread)
t.daemon = True
t.start()
socketio.run(app)
And it gives me this error:
RuntimeError: working outside of request context
How do I send messages from a function that doesn't have the #socketio.on() decorator? Can I use gevent directly to send messages to socketio?
From this section of the documentation:
Sometimes the server needs to be the originator of a message. This can be useful to send a notification to clients of an event that originated in the server. The socketio.send() and socketio.emit() methods can be used to broadcast to all connected clients:
def some_function():
socketio.emit('some event', {'data': 42})
This emit is not from from flask.ext.socketio import SocketIO, send, but instead called on your socketio variable from socketio = SocketIO(app). Had you done socketio_connection = SocketIO(app), then you'd be calling socketio_connection.emit() to broadcast your data.