I'm investigating the possibility of using a Flask application as an interface to an embedded system. I've used flask before (I've written some very basic flask sites to poll external systems in response to a page load to populate a chart for example) but I'm not sure how I would go about pushing data into the Flask application and on to the user's browser(s).
I was planning on pushing data from a C++ application running on the embedded device into the flask application (also running on the embedded device) using ZeroMQ.
From what I've read, something like flask-socketIO would be a possibility to get things from Flask to the user's browser.
The one thing that's not clear to me is whether it's possible / how you would go about receiving data from ZeroMQ and pushing that out to the browser?
In case anyone else wants to do the same, this is the simplest example I could boil things down to based on the example by reptilicus...
Instructions
Set the code below laid out in the structure mentioned below.
Install the Python modules listed below
Run the server
Run the data source
Open a web browser and navigate to http://localhost:25000/
If everything worked you should see a very basic page along these lines:
Python / Modules
These were the versions my test implementation used. Others may work too.
Python v3.5.2
Pyzmq v15.2.0
gevent v1.2.0
karellen-geventws v1.0.1 (required over gevent-websocket for Python 3 support)
Flask v0.10.1
Flask-Sockets v0.2.1
You also need a copy of Reconnecting Websocket, available here.
Code layout
\ZmqFlaskForwarder
\static
\js
application.js
reconnecting-websocket.min.js
\templates
index.html
data_source.py
server.py
Server App (server.py)
import zmq.green as zmq
import json
import gevent
from flask_sockets import Sockets
from flask import Flask, render_template
import logging
from gevent import monkey
monkey.patch_all()
app = Flask(__name__)
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
sockets = Sockets(app)
context = zmq.Context()
ZMQ_LISTENING_PORT = 12000
#app.route('/')
def index():
logger.info('Rendering index page')
return render_template('index.html')
#sockets.route('/zeromq')
def send_data(ws):
logger.info('Got a websocket connection, sending up data from zmq')
socket = context.socket(zmq.SUB)
socket.connect('tcp://localhost:{PORT}'.format(PORT=ZMQ_LISTENING_PORT))
socket.setsockopt_string(zmq.SUBSCRIBE, "")
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
gevent.sleep()
received = 0
while True:
received += 1
# socks = dict(poller.poll())
# if socket in socks and socks[socket] == zmq.POLLIN:
data = socket.recv_json()
logger.info(str(received)+str(data))
ws.send(json.dumps(data))
gevent.sleep()
if __name__ == '__main__':
logger.info('Launching web server')
from gevent import pywsgi
from geventwebsocket.handler import WebSocketHandler
server = pywsgi.WSGIServer(('', 25000), app, handler_class=WebSocketHandler)
logger.info('Starting serving')
server.serve_forever()
Data Source (data_source.py)
import zmq
import random
import sys
import time
import json
port = "12000"
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:%s" % port)
while True:
first_data_element = random.randrange(2,20)
second_data_element = random.randrange(0,360)
message = json.dumps({'First Data':first_data_element, 'Second Data':second_data_element})
print(message)
socket.send_string(message)
time.sleep(0.5)
Client JavaScript (application.js)
ws = new ReconnectingWebSocket("ws://" + location.host + '/zeromq')
ws.onmessage = function(message) {
payload = JSON.parse(message.data);
$('#latest_data').html('<h2> Data: ' + message.data + '</h2>');
};
Template (index.html)
<!DOCTYPE html>
<html>
<head>
<title>Python Websockets ZeroMQ demo</title>
</head>
<body>
<div class="container">
<h2> Simple ZeroMQ data streaming via web sockets! </h2>
<div id="latest_data"></div>
</div>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<script type="text/javascript" src="static/js/reconnecting-websocket.min.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.2.6/d3.min.js"></script>
<script type="text/javascript" src="static/js/application.js"></script>
</body>
</html>
Related
I am trying to set up bidirectional communication between my Flutter android project and a server written in Python/Flask.
The code on the python side is basically the same as can be found at https://flask-sock.readthedocs.io/en/latest/quickstart.html. ie.
from flask import Flask, render_template
from flask_sock import Sock
app = Flask(__name__)
sock = Sock(app)
#sock.route('/echo')
def echo(ws):
while True:
data = ws.receive()
ws.send(data)
On the Dart side, I have:
final channel = WebSocketChannel.connect(
Uri.parse('wss://10.0.2.2:5000/echo'),
);
And the dependency in the pubspec.yaml file is:
web_socket_channel: ^2.1.0
Running it I get the bizarre error message:
127.0.0.1 - - [23/Jul/2022 17:49:09] code 400, message Bad request version ('\x18±-8ÛF\x18Ã\x9b(ôP')
There is nothing wrong with the flask side of things, my app processes the standard html requests fine. But setting up the websockets causes these runtime complaints.
I tried flask.socketio and got the similar errors.
I am trying to build an application that lets you schedule and execute multiple long running jobs in a background thread using APScheduler. To control the job schedules and view the (live) output of the jobs I want to send messages to the Flask application that runs in the same process (using Blinker) so I can stream them to a web client using Flask-SocketIO.
I came up with the following code but it seems send_log_update() is not being called at all. Please note that I have not yet added Flask-SocketIO to this example. I first wanted to make sure I could communicate to the Flask application before further complicating things.
Is this a sensible way to go about things? And if so: Am I doing something wrong here? I am not married to any of the used solutions specifically but I do need something like APScheduler to schedule jobs at specific times (instead of just intervals, like in this example).
I have considered the possibility of also using websockets to provide the communication between the background job and the rest of the application but that would be too unreliable. I have to process all output coming from the background process (to send to a log ingester) in addition to streaming it to a web client and I would like to keep the background job as agnostic of any databases and logging frameworks as possible.
# pip install flask apscheduler sqlalchemy blinker
from time import sleep
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
from apscheduler.schedulers.background import BackgroundScheduler
from blinker import signal
from flask import Flask
from pytz import utc
# initialize Flask+SocketIO
app = Flask(__name__)
# signal to communicate between background thread and Flask
logsignal = signal('log')
# handle signals coming from background thread and emit them
# over the websocket
#logsignal.connect_via(app)
def send_log_update(sender, log_line, context, **extra):
# eventually I want to send this to the web client using
# Flask-SocketIO
print('received signal: ' + log_line)
# Background job that will run in the scheduler thread
def background_job():
print('starting background job')
logsignal.send('starting job')
sleep(3)
logsignal.send('job done')
# configure APScheduler
jobstores = {
'default': SQLAlchemyJobStore(url='sqlite:///scheduler.sqlite')
}
job_defaults = {
'coalesce': False,
'max_instances': 1
}
# create and start scheduler
scheduler = BackgroundScheduler(
job_defaults=job_defaults, jobstores=jobstores, timezone=utc)
if __name__ == '__main__':
scheduler.add_job(background_job, 'interval', seconds=5,
replace_existing=True, id='sample_job',
args=[])
scheduler.start()
app.run()
The answer was quite simple, I was using #logsignal.connect_via(app) which restricts the send_log_update() handler to only respond to signals originating from the Flask app. After using the regular #logsignal.connect method the handler got executed. I made into a fully working example with a web interface that shows the log being streamed.
# Runs a scheduled job in a background thread using APScheduler and streams
# it's output to a web client using websockets. Communication between the Flask
# thread and APScheduler thread is being done through (blinker) signals.
#
# Install dependencies (preferably in your virtualenv)
# pip install flask apscheduler sqlalchemy blinker flask-socketio simple-websocket
# and then run with:
# python this_script.py
from time import sleep
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
from apscheduler.schedulers.background import BackgroundScheduler
from blinker import signal
from flask import Flask
from flask_socketio import SocketIO
from pytz import utc
# initialize Flask+SocketIO
app = Flask(__name__)
socketio = SocketIO(app)
# signal to communicate between background thread and Flask
logsignal = signal('log')
# handle signals coming from background thread and emit them
# over the websocket
#logsignal.connect
def send_log_update(log_line):
socketio.emit('logUpdate', log_line)
# Background job that will run in the scheduler thread
def background_job():
logsignal.send('starting job')
sleep(3)
logsignal.send('job done')
# configure APScheduler
jobstores = {
'default': SQLAlchemyJobStore(url='sqlite:///scheduler.sqlite')
}
job_defaults = {
'coalesce': False,
'max_instances': 1
}
# create and start scheduler
scheduler = BackgroundScheduler(
job_defaults=job_defaults, jobstores=jobstores, timezone=utc)
# simple websocket client for testing purposes
#app.route("/")
def info():
return """
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js" integrity="sha512-q/dWJ3kcmjBLU4Qc47E4A9kTB4m3wuTY7vkFJDTZKjTs8jhyGQnaUrxa0Ytd0ssMZhbNua9hE+E7Qv1j+DyZwA==" crossorigin="anonymous"></script>
</head>
<body>
<h1>Streaming log</h1>
<pre id="log"></pre>
<script type="text/javascript" charset="utf-8">
var socket = io();
socket.on('logUpdate', function(msg) {
let log = document.getElementById('log');
log.append(msg + '\\n');
});
</script>
</body>
</html>
"""
if __name__ == '__main__':
scheduler.add_job(background_job, 'interval', seconds=5,
replace_existing=True, id='sample_job',
args=[])
scheduler.start()
socketio.run(app)
I'm currently following a Udemy course that's having me use flask and socketio to use a neural network model to drive a simulated car. However, as he's explaining the basics of how flask and socketio work, he had us write this code:
import socketio
import eventlet
from flask import Flask
sio = socketio.Server()
app = Flask(__name__)
#sio.on('connect')
def connect(sid, environ):
print('Connected')
if __name__ == "__main__":
app = socketio.Middleware(sio, app)
eventlet.wsgi.server(eventlet.listen(('', 4567)), app)
Which is supposed to print "Connected!" to the console when we connect to the server. Now, I get this message when I run it, so I'm pretty sure I'm connected.
(7532) accepted ('127.0.0.1', 49374)
But it's refusing to print "Connected!" when I connect like it's supposed to, no matter what I try.
EDIT:
So, I'm still not sure what the root cause of this is, but I found out how to fix it.
conda install python-engineio==3.13.2
conda install python-socketio==4.6.1
You might need to run anaconda as an administrator. If so, search for "Anaconda Powershell Prompt" then run it as an admin.
socketio uses special protocol so you need special client to use it.
At least
import socketio
sio = socketio.Client()
con = sio.connect('http://0.0.0.0:4567')
sio.wait()
I will not work with web browser. With web browser you can see only accepted.
Browser have to load web page which uses special JavaScript module to use socketio.
You can find more details in socketio documentation: Client
EDIT:
And here server which you can test with web browser.
When you open http://0.0.0.0:4567 in browser then index() sends to browser HTML with JavaScript code which loads special library and uses socketio to send own event. And you should see Connected, my event. When you close page or browser then you should see Disconnected
It is based on example from documentation for flask-socketio
import socketio
import eventlet
from flask import Flask
sio = socketio.Server()
app = Flask(__name__)
#app.route('/')
def index():
return """
Hello World!
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js" integrity="sha512-q/dWJ3kcmjBLU4Qc47E4A9kTB4m3wuTY7vkFJDTZKjTs8jhyGQnaUrxa0Ytd0ssMZhbNua9hE+E7Qv1j+DyZwA==" crossorigin="anonymous"></script>
<script type="text/javascript" charset="utf-8">
var socket = io();
socket.on('connect', function() {
socket.emit('my event', {data: 'Im connected!'});
});
</script>
"""
#sio.on('connect')
def connect(sid, environ):
print('Connected')
#sio.on('disconnect')
def disconnect(sid): # without `environ`
print('Disconnected')
#sio.on('my event')
def my_event(sid, environ):
print('my event')
if __name__ == "__main__":
app = socketio.Middleware(sio, app)
eventlet.wsgi.server(eventlet.listen(('', 4567)), app)
I'm using Tornado Web Framework to build a websocket server and I need to connect via WebSocket from javascript. How can I get the server url and port from tornado templates?
I'm thinking in something like this:
<script>
var _url = "{{ (server_url) }}";
var _port = "{{ (server_port) }}";
var ws = new WebSocket("ws://" + _url + ":" + _port + "/socket");
</script>
I think you have to define it as context variables. Tornado doesn't provide this information to templates automatically.
It is a good idea to use tornado.options to set this variables. And then, pass them to your template.
Simplified contents of app.py:
from tornado.options import options, define
define("host", default="localhost", help="app host", type=str)
define("port", default=3000, help="app port", type=int)
class WebsocketHandler(tornado.web.RequestHandler):
def get(self):
self.render(
"index_websocket.html",
server_url=options.host,
server_port=options.port
)
options.parse_command_line() # don't forget to parse command line
app = tornado.web.Application(...)
app.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
In that case, you can run you application and provide defined settings:
python app.py --host=yourserveraddres.com --port=3000
I'm working on my first iOS app that uses push notifications. I have a python script that lets me to send push notifications from my machine but I'm unable to get this working with the Google App Engine Launcher.
When I run this on GAE I get nothing - no errors and no push notifications. What am I doing wrong? I know the code for sending the actual notification is working properly but I'm not able to duplicate this on Google's servers.
Here is the script I'm trying to run with GAE Launcher:
import os
import cgi
import webapp2
from google.appengine.ext.webapp.util import run_wsgi_app
import ssl
import json
import socket
import struct
import binascii
TOKEN = 'my_app_token'
PAYLOAD = {'aps': {'alert':'Push!','sound':'default'}}
class APNStest(webapp2.RequestHandler):
def send_push(token, payload):
# Your certificate file
cert = 'ck.pem'
# APNS development server
apns_address = ('gateway.sandbox.push.apple.com', 2195)
# Use a socket to connect to APNS over SSL
s = socket.socket()
sock = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_SSLv3, certfile=cert)
sock.connect(apns_address)
# Generate a notification packet
token = binascii.unhexlify(token)
fmt = '!cH32sH{0:d}s'.format(len(payload))
cmd = '\x00'
message = struct.pack(fmt, cmd, len(token), token, len(payload), payload)
sock.write(message)
sock.close()
send_push(TOKEN, json.dumps(PAYLOAD))
application = webapp2.WSGIApplication([
('/apns', APNStest)
], debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
So the solution was very simple as I expected. I had enabled billing for the project on cloud.google.com but needed to have billing enabled at appengine.google.com as well. Stupid mistake that set me back 2 days.