I have a web app utilizing Python Flask where I am trying to use Server Sent Events (SSEs) to push messages to web pages without having to poll from the client side or request that data. I'm using Redis to listen for new data which will then be sent to the web page. To start and make sure that I can use SSEs correctly, I've used a template similar to an example like this (How to implement server push in Flask framework?).
The problem I'm running into is that every time the client receives a message, the EventSource onmessage() method is called and delivers the message properly, but then the .onerror() method immediately gets triggered, causing the client to try to reconnect. This results in the '/listen' endpoint being called over and over and over, leading to the creation of many redis pubsub objects that are redundant and subscribe to the same channels.
The python code that runs the flask app is as follows
import flask
from flask_bootstrap import Bootstrap
from redis import Redis
from flask_wtf import FlaskForm
app = flask.Flask(__name__)
bootstrap = Bootstrap(app)
red = Redis(host='localhost', port=6379, db=0)
#app.route('/listen')
def listen():
pubsub = red.pubsub()
pubsub.subscribe('chat')
def stream():
for message in pubsub.listen():
if message['type'] == 'message':
msg = f"data: {message['data'].decode('utf-8')}\n\n"
yield msg
for msg in stream():
return Response(msg, mimetype='text/event-stream')
#app.route('/sse_page', methods=['GET', 'POST'])
def sse_page():
form = FlaskForm()
return render_template('sse_page.html', title='Server Push Testing', form=form)
if __name__ == "__main__":
app.run(port=8000, threaded=True, debug=True)
The corresponding section of sse_page.html where I try to open the EventSource and listen for the events stream is
<body>
<div id="target_div">Watch this space...</div>
</body>
<script>
var source = new EventSource("/listen");
source.onmessage = function (event) {
console.log('data: ', event)
$("#target_div").text(event.data)
};
source.onerror = function (event) {
console.log('error ', event)
};
source.onopen = function (event) {
console.log('open', event)
};
</script>
Using the redis-cli to send messages like those seen
here (and transcribed below)
127.0.0.1:6379> publish chat a
(integer) 1
127.0.0.1:6379> publish chat b
(integer) 2
Result in the console logging messages from Eventsource.onopen(), Eventsource.onmessage(), and Eventsource.onerror() for every single message, as seen
here.
I cannot figure out why the eventsource has an error after every single message that is received or how to prevent that from happening.
The answer to this question is NOT a problem in the code itself.
What this ended up being was an issue with the anti-virus security that was being used on the machine. Using Sophos AV Endpoint was causing each SSE to be treated as a download, and so any of the text data was unable to be streamed until the 'download was complete'.
This is (apparently) a known issue (see link https://community.sophos.com/on-premise-endpoint/f/sophos-endpoint-software/74878/server-sent-events-blocked-by-download-scanner) and there are a couple of ways to deal with it. You can either disable web scanning by Sophos (which does not work if you do not have administrator permissions) or run the flask app securely over HTTPS (https://blog.miguelgrinberg.com/post/running-your-flask-application-over-https has a great tutorial).
Credit too should go to this post (JavaScript EventSource SSE not firing in browser), which is how I was able to find that the AV software was what ended up causing my issues.
Related
I'm trying to wrap my head around this socketio thing and have hit a wall regarding sending a message from my Flask backend to my JavaScript frontend. My app is a consolidated control panel for my church's PTZ camera and OBS Studio. Here are the logical steps I'm trying to implement:
Connect Flask to OBS via the OBS websocket.
Receive notification in Flask when the Graphics source visibility in OBS changes.
Send a message from Flask to my JS frontend to let it know that the Graphics source has changed visibility (making a two-way sync with the button on my control panel and the actual source in OBS).
Steps 1 and 2 are working, but I can't get the 3rd one to work.
Here is my Python where I use the message from OBS to call the function to sync with the JS frontend. (I know the sync_graphics_toggle runs, but the JS never gets the emit message.)
def on_visibility_change(message):
if (message.getItemName() == "Graphics"):
sync_graphics_toggle(message.getItemVisible())
def sync_graphics_toggle(obs_graphics_visible):
socketio.emit('syncGraphicsToggle', obs_graphics_visible, namespace='/main')
In the JavaScript (with just a console log to test):
$(document).ready(function() {
const socketMain = io("/main");
socketMain.on("connect", function() {
console.log("Made connection in graphics-preview", socketMain.id);
});
socketMain.on("syncGraphicsToggle", function(obsGraphicsVisible) {
console.log("Got syncGraphicsToggle command")
});
}
I have verified that I'm connected, but I don't get the message from Flask.
I have other implementations of socketio in my code that work fine, but those are initiated by actions on the webpage. So, I'm thinking there is something I don't understand about sending messages that are initiated by the server.
Any pointers would be greatly appreciated.
John
Windows 10
Python = 3.10.4
Flask = 2.2.2
Python Socket.IO = 5.7.2
JS Socket.IO = 4.5.3
So I'm building a simple Flask social media application that uses Flask's session ids. I want to remove a user's session id whenever they leave the page so that they have to re-login. I did a little research and found out that Flask-SocketIO allows you to do that through the 'connect' and 'disconnect' events. However, when I run the app, and I open up the site/log on to the page, nothing happens. I don't get any output from the connect event nor the disconnect event when I close the tab. Here's the full code.
# Starting/Initialisation of App
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///app.db'
app.config['DEBUG'] = True
app.config['SECRET_KEY'] = SECRET_KEY
socket_io = SocketIO(app=app, logger=True, engineio_logger=True)
db = SQLAlchemy(app)
# Socket IO Events
#socket_io.on('connect')
def on_connect():
print("Connected!")
#socket_io.on('disconnect')
def on_disconnect():
# TODO Fix this socket io disconnect event. It's not working for some reason. Just figure things out with socketio
print("Client Disconnected")
print("YOOOOOOOOOO")
session.pop('id')
#socket_io.on('message')
def on_message():
print("Exit message")
# App Routes
# A page where a user can login to an account
#app.route('/', methods=['GET', 'POST'])
def index():
# If user submits form
if request.method == "POST":
# Get response and user's Account object from db
# if query does not return None aka user is in database
# Check if response's password is the same as the user's password
# If it is, redirect to homepage
# Else if not, return an "invalid password" error
form = request.form # User Response
user = Account.query.filter_by(email=str(request.form['email'])).first() # Query for user
if user is not None: # Validating query
if str(form['password']) == user.password: # Checking password
session['id'] = user.id
return redirect('/home')
else:
return "Invalid password"
else:
return "Invalid email or password"
else:
return render_template('index.html')
# Run App
if __name__ == '__main__':
socket_io.run(app)
I'm don't receive any errors or whatsoever so I'm really confused.
Below is my pip freeze
bidict==0.21.2
click==8.0.1
colorama==0.4.4
Flask==2.0.1
Flask-SocketIO==5.1.0
Flask-SQLAlchemy==2.5.1
greenlet==1.1.0
h11==0.12.0
itsdangerous==2.0.1
Jinja2==3.0.1
MarkupSafe==2.0.1
python-engineio==4.2.0
python-socketio==5.3.0
simple-websocket==0.2.0
SQLAlchemy==1.4.22
Werkzeug==2.0.1
wsproto==1.0.0
Would really appreciate it if anyone has any solutions to this. Thanks in advance.
Few things to consider here:
print statements to stdout won't necessarily show in your output with Flask, I believe stderr does, so you might want to try either that or the logger, e.g.:
print('Connected!', file=sys.stderr)
app.logger.info("Connected!")
Are you sure you have the right version of the client? I don't believe the client is included with Flask-SocketIO, and Flask-SocketIO isn't compatible with the latest client version (4). You should be using client version 3.1.3 (https://socket.io/docs/v3/). This is the version of the SocketIO library you include (this example truncated from the aforementioned SocketIO docs and updated to show an alert box once connected so you can be sure it's worked):
<html>
<body>
...
<script src="https://cdn.socket.io/3.1.3/socket.io.min.js" integrity="sha384-cPwlPLvBTa3sKAg ddT6krw0cJat7egBga3DJepJyrLl4Q9/5WLra3rrnMcyTyOnh" crossorigin="anonymous"></script>
<script type='text/javascript'>
const socket = io("ws://localhost:5000");
socket.on("connect", () => {
// either with send()
alert("You're connected!");
socket.send("Hello!");
// or with emit() and custom event names
socket.emit("salutations", "Hello!", { "mr": "john" }, Uint8Array.from([1, 2, 3, 4]));
});
</script>
</body>
</html>
does your web console in your browser show any errors? Are you sure the client side code is correct and you're connected?
Viewing the Javascript console varies from browser to browser, but will show you any errors encountered in your Javascript. In Chrome for example, you click the 3 dots to the far right of your address bar, then More Tools -> Developer Tools. This is cruicial in debugging Javascript issues.
Are you sure you have included socket.io in client side means in html. you can include this script in your head in html to include it.
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js" integrity="sha512-q/dWJ3kcmjBLU4Qc47E4A9kTB4m3wuTY7vkFJDTZKjTs8jhyGQnaUrxa0Ytd0ssMZhbNua9hE+E7Qv1j+DyZwA==" crossorigin="anonymous" defer></script>
Then you need to define socket in your javascript file or script tag.
<script type="text/javascript">
socket = io.connect(document.href );
socket.on('connect', function() {
// do something when user connects
});
socket.emit("message",data) // data is sent to flask
</script>
for more information, refer to the documentation.
https://flask-socketio.readthedocs.io/en/latest/
So here is the issue...I have had to use two separate SDK's for telnyx due to each offering a functioning service the other did not. One is in Python, one is in Node. Essentially I am sending a message from my phone to a telnyx API endpoint in python, and what I am hoping to do is communicate the number, and the message received in python over to my node.js app, which I will not have a problem formatting. I am just confused about how to communicate data between the two. Open to the smoothest suggestions. It is all server side, no client, just sending messages to a server and responding to them with another server based on the content of the messages.
Python/flask below:
import os
from flask import Flask, request
app = Flask(__name__)
#app.route('/webhooks', methods=['POST'])
def webhooks():
body = request.json
messageSenderNumber = body['data']['payload']['from']['phone_number']
messageSenderMessage = body['data']['payload']['text']
newMessageSenderMessage = messageSenderMessage.lower()
print(f"You have a new message from {messageSenderNumber}: {newMessageSenderMessage}")
return '', 200
if __name__ == "__main__":
app.run(port=5000)
Here is the Node:
var telnyx = require('telnyx').
('KEY0178B7161E6B3873C151BDC3597EE42B_SHXIHH1anGGOgfhcIb3kjQ');
telnyx.messages.create(
{
'from': '+12182101811', // Your Telnyx number
'to': '+18646662611',
'text': 'Hello, World!'
},
function(err, response) {
console.log(response);
}
);
My goal is to take the newMessageSenderMessage and messageSenderNumber variable's data and export it into my node app, that is all I need. Nothing crazy, just a phone number and a string from the python server side code, and same in node.
Thanks in advance.
Replace your return statement in the flask with this.
This should send back a JSON to your Node.js app
return {"newMessageSenderMessage": newMessageSenderMessage, "messageSenderNumber": messageSenderNumber}
I am building a chat app based on Flask SocketIO only, no database. One of the requirements is when a new user connects, the app should show previous messages. I save every message into a array on each send event. Okay, so now the problem is, when User A connects and creates some messages, and then User B connects, all the previous messages from User B is shown, but then User A also gets that messages, so User A ends up duplicated messages.
Client Side JS
function myFunction() {
document.getElementById('demo').style['text-decoration']='underline';
}
const socket = io.connect("http://127.0.0.1:5000");
socket.on('connect', function() {
socket.emit('sync_messages');
socket.emit('sync_channels');
});
Flask App Code
#socketio.on('sync_messages')
def handle_sync():
socketio.emit('show_all_messages', messages)
#socketio.on('sync_channels')
def handle_sync_channels():
socketio.emit('show_all_channels', channels)
Visual representation of what is happening
what the actual bug is
The socketio.emit() function is not context aware, it defaults to broadcasting to all connected users. Try using emit() which is the Flask-friendly wrapper:
#socketio.on('sync_messages')
def handle_sync():
emit('show_all_messages', messages)
#socketio.on('sync_channels')
def handle_sync_channels():
emit('show_all_channels', channels)
I can not use flask flash messages sent with emit and "captured" with #socketio.on, but it works on the html page. How to fix flash from flask_socketio ?
When i look in terminal, it works.
it's at the jonction socket / flash that the problem is .
$('form#send_room').submit(function(event) {
socket.emit('my_room_event', {room: $('#room_name').val(), data: $('#room_data').val()});
return false;
});
events.py
#socketio.on( 'send_room' , namespace='/roomy/roomy')
def broadcast_info_new_box(data):
print("\n\n\n broadcast_info_new_box called with socket on landing events.py")
flash(data)
The flash() function works only in Flask routes, as it relies on an HTTP response delivering an updated session cookie to the client. You are trying to use it in a Socket.IO event handler, which does not have a way to send cookies to the client.
If you want to implement alert popups or similar via Socket.IO you will have to emit these alerts as events to the client, and then use JavaScript in the client to display them.