I am new to web-sockets. I am using tornado/python for my back-end and written the following code.
class BaseWebSocketHandler(websocket.WebSocketHandler):
"""Base Class to establish an websocket connection."""
def open(self):
"""Opening the web socket connection."""
self.write_message('Connection Established.')
def on_message(self, message):
"""On message module send the response."""
pass
def on_close(self):
"""Close the connection."""
self.write_message('bye')
class MeterInfo(BaseWebSocketHandler):
"""Establish an websocket connection and send meter readings."""
def on_message(self, message):
"""On message module send to the response."""
self.write_message({'A': get_meter_reading()})
My JavaScript code is like the following,
var meter = new WebSocket("ws://"+window.location.host+"/socket/meterstatus/");
meter.onopen = function() {
$('#meter-well').text('Establishing connection...');
};
meter.onmessage = function (evt) {
var data = JSON.parse(evt.data)
var text = "<div class='meter'><h2>" + data.A +"</h2></div>";
$('#meter-pre').html(text);
};
meter.onclose = function (evt) {
console.log(JSON.parse(evt.data))
$('#meter-pre').append('\n'+evt.data);
};
window.setInterval(function(){ meter.send('') }, 100);
I am making a blank web-socket request request to the back-end every 100 millisecond. this seems a very bad solution to me. Is there any better way to do it without making multiple send() to the back-end and only notifying the user only on any changes in the meter reading?
Also i have gone through MQTT protocol to do this in a better way, can someone suggest how can i implement that?
You almost found the solution to your issue here:
class MeterInfo(BaseWebSocketHandler):
"""Establish an websocket connection and send meter readings."""
def on_message(self, message):
"""On message module send to the response."""
self.write_message({'A': get_meter_reading()})
As you could notice tornado needs some event to ping client through write_message method. You are using new message from client as such event, try to change to simple timeout call as event, like this:
# BaseWebSocketHandler removed, because we need to track all opened
# sockets in the class. You could change this later.
class MeterInfo(websocket.WebSocketHandler):
"""Establish an websocket connection and send meter readings."""
opened_sockets = []
previous_meter_reading = 0
def open(self):
"""Opening the web socket connection."""
self.write_message('Connection Established.')
MeterInfo.opened_sockets.append(self)
def on_close(self):
"""Close the connection."""
self.write_message('bye')
MeterInfo.opened_sockets.remove(self)
#classmethod
def try_send_new_reading(cls):
"""Send new reading to all connected clients"""
new_reading = get_meter_reading()
if new_reading == cls.previous_meter_reading:
return
cls.previous_meter_reading = new_reading
for socket in cls.opened_sockets:
socket.write_message({'A': new_reading})
if __name__ == '__main__':
# add this after all set up and before starting ioloop
METER_CHECK_INTERVAL = 100 # ms
ioloop.PeriodicCallback(MeterInfo.try_send_new_reading,
METER_CHECK_INTERVAL).start()
# start loop
ioloop.IOLoop.instance().start()
Check out tornado.ioloop documentation for more about PeriodicCallback and other options.
If you want to use tornado to MQTT protocol, it's not possible with tornado. You could try emqtt server for example, but this is actual server, not framework to write apps, so IMHO it would be more comprehensive that ping through web socket with tornado.
Related
I have a websocket server written in python tornado. the server will receive many connections from clients and as you know, we have on_message function that is fired when a websocket message is received. so, here is my question that if a message(say request from client) need 5 sec to be processed then when the server is processing some request, the server goes in blocking mode and can't accept or receive more connection or data. after some research i figure out that Asyncio can resolve my problem but i don't now know to use it. so, how do i call process method to avoid blocking?
following is my code:
class WavesClient(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def open(self):
print("New client connected")
def on_message(self, message):
self.process(message)
def on_close(self):
print("Client disconnected")
def process(self,message):
#it takes 5 sec to complete
I use tornado primarily to host my webapps, therefore I can tell you if any part of your code in tornado is blocking, the whole server will block.
Now to your code:
#tornado.gen.coroutine
def on_message(self, message):
process_results = yield self.process(message)
self.write_message(process_results)
#tornado.gen.coroutine
def process(self, message):
# a long process, this is the equivalent of time.sleep(2) which is blocking
yield tornado.gen.sleep(2)
return 'finished'
With tornado you have to yield from a function to get the return value.
Also, if your function is yielding, you must wrap it using the tornado.gen.coroutine decorator
This question is similar to yours. And the answer is informative as well.
I've a web-socket server which connects with the clients. Following is the code:-
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
class Chat(LineReceiver):
def __init__(self, users):
self.users = users
self.name = None
self.state = "GETNAME"
def connectionMade(self):
self.sendLine("What's your name?")
def connectionLost(self, reason):
if self.users.has_key(self.name):
del self.users[self.name]
def lineReceived(self, line):
if self.state == "GETNAME":
self.handle_GETNAME(line)
else:
self.handle_CHAT(line)
def handle_GETNAME(self, name):
if self.users.has_key(name):
self.sendLine("Name taken, please choose another.")
return
self.sendLine("Welcome, %s!" % (name,))
self.name = name
self.users[name] = self
self.state = "CHAT"
def handle_CHAT(self, message):
# Need to send the message to the connected clients.
class ChatFactory(Factory):
def __init__(self):
self.users = {} # maps user names to Chat instances
def buildProtocol(self, addr):
return Chat(self.users)
reactor.listenTCP(8123, ChatFactory())
reactor.run()
Clients get connected to the above code(server), and sends the data to the server.
Now, I've another python script, basically a scraper which scrapes the web, processes it and finally need to send the data to the connected clients.
script.py
while True:
# call `send_message` function and send data to the connected clients.
How can I achieve it?? Any example would be of great help!!
UPDATE
After using Autobahn
I've a server that fetches data from 3rd party API. I want to send this data to all the connected web-socket clients. Here is my code:-
class MyServerProtocol(WebSocketServerProtocol):
def __init__(self):
self.connected_users = []
self.send_data()
def onConnect(self, request):
print("Client connecting: {0}".format(request.peer))
def onOpen(self):
print("WebSocket connection open.")
self.connected_users.append(self) # adding users to the connected_list
def send_data(self):
# fetch data from the API and forward it to the connected_users.
for u in self.users:
print 1111
u.sendMessage('Hello, Some Data from API!', False)
def onClose(self, wasClean, code, reason):
connected_users.remove(self) # remove user from the connected list of users
print("WebSocket connection closed: {0}".format(reason))
if __name__ == '__main__':
import sys
from twisted.python import log
from twisted.internet import reactor
factory = WebSocketServerFactory(u"ws://127.0.0.1:9000")
factory.protocol = MyServerProtocol
reactor.listenTCP(9000, factory)
reactor.run()
My Server will never receive a message or probably will receive, but as of right now there's no such use-case, hence no need for OnMessage event for this example).
How do I write my send_data function in order to send data to all my connected clients??
You need to avoid this pattern when writing software with Twisted:
while True:
# call `send_message` function and send data to the connected clients.
Twisted is a cooperative multitasking system. "Cooperative" means that you have to give up control of execution periodically so that other tasks get a chance to run.
twisted.internet.task.LoopingCall can be used to replace many while ... loops (particularly while True loops):
from twisted.internet.task import LoopingCall
LoopingCall(one_iteration).start(iteration_interval)
This will call one_iteration every iteration_interval seconds. In between, it will give up control of execution so other tasks can run.
Making one_iteration send a message to a client is just a matter of giving one_iteration a reference to that client (or those clients, if there are many).
This is a variation on the FAQ How do I make Input on One Connection Result in Output on Another.
If you have a ChatFactory with a dict containing all your clients, just pass that factory to one_iteration:
LoopingCall(one_iteration, that_factory)
or
LoopingCall(lambda: one_iteration(that_factory))
I am currently building a client-server-type application with Pythons autobahn (asyncio) framework on both ends. I already have the basic setup of the server, which consists of the server itself and a database managing module. For the client I want to have the networking in a separate module, so I can call <networkingModule>.sendMessage("msg") for example. Is that a good way to do it?
My problem however is, the code used to initialize the client doesn't give me a real object to work with.
import asyncio
factory = WebSocketClientFactory("ws://localhost:9000", debug = False)
factory.protocol = <ClientProtocolName>
loop = asyncio.get_event_loop()
coro = loop.create_connection(factory, '127.0.0.1', 9000)
loop.run_until_complete(coro)
loop.run_forever()
loop.close()
The class itself is:
class <ClientProtocolName>(WebSocketClientProtocol):
def onConnect(self, response):
print("Server connected: {0}".format(response.peer))
def onOpen(self):
print("WebSocket connection open.")
self.sendInput()
def _sendMessage(self, msg):
self.sendMessage(bytes(msg, "utf-8"))
def onMessage(self, payload, isBinary):
if isBinary:
print("Binary message received: {0} bytes".format(len(payload)))
else:
print("Text message received: {0}".format(payload.decode('utf8')))
def onClose(self, wasClean, code, reason):
print("WebSocket connection closed: {0}".format(reason))
So my question is: Is there a way to build a "networking module" so I can access the sendMessage method from outside and would that be a good way to go about this problem, or should I just stuff all my programm logic into the client itself?
I need to send system logs to the browser and so I have a tornado-based websocket server running like so.
class WSHandler(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def get(self, *args, **kwargs):
self.ip = self.get_argument('ip', None)
self.action = self.get_argument('action', None)
super(WSHandler, self).get(self, *args, **kwargs)
def open(self, *args, **kwargs):
clients.append(self)
def on_message(self, message):
ll = eval(message)
for cl in clients:
if cl.ip and cl.ip != ll.user:
continue
if cl.action and cl.action != ll.action:
continue
message = '%s %s' % (ll.action, ll.url)
cl.write_message(message)
def on_close(self):
try:
clients.remove(self)
except ValueError:
pass
The examples I've encountered so far revolve around Tornado-based servers and js-based clients.
What I need, however, is an easy way to connect to this websocket from a Python client, preferably powered by Tornado. The client does not need to receive messages - only send them. I thought I had my answer with this SO post,
How to run functions outside websocket loop in python (tornado)
...but I need to send a message whenever a log event occurs, and preferably from my code that's parsing the events. The examples I've encountered so far revolve around Tornado-based servers and js-based clients. Is there a short & sweet tornado-based client that only sends messages, that can be called from a for-loop?
Also, I developed a complete Tornado WebSocket Client/Server example.
https://github.com/ilkerkesen/tornado-websocket-client-example
If you want WebSocket Authentication/Authorization, look at my other projects trebol and sugar.
There is tornad-websocket-client project. Pay attention on it.
Also there is simple websocket-client to just send messages.
Tornado includes a websocket client: http://www.tornadoweb.org/en/stable/websocket.html#client-side-support
I am building a realtime web application. I want to be able to send broadcast messages from the server-side implementation of my python application.
Here is the setup:
socketio.js on the client-side
TornadIO2 server as Socket.IO server
python on the server-side (Django framework)
I can succesfully send socket.io messages from the client to the server. The server handles these and can send a response. In the following i will describe how i did that.
Current Setup and Code
First, we need to define a Connection which handles socket.io events:
class BaseConnection(tornadio2.SocketConnection):
def on_message(self, message):
pass
# will be run if client uses socket.emit('connect', username)
#event
def connect(self, username):
# send answer to client which will be handled by socket.on('log', function)
self.emit('log', 'hello ' + username)
Starting the server is done by a Django management custom method:
class Command(BaseCommand):
args = ''
help = 'Starts the TornadIO2 server for handling socket.io connections'
def handle(self, *args, **kwargs):
autoreload.main(self.run, args, kwargs)
def run(self, *args, **kwargs):
port = settings.SOCKETIO_PORT
router = tornadio2.TornadioRouter(BaseConnection)
application = tornado.web.Application(
router.urls,
socket_io_port = port
)
print 'Starting socket.io server on port %s' % port
server = SocketServer(application)
Very well, the server runs now. Let's add the client code:
<script type="text/javascript">
var sio = io.connect('localhost:9000');
sio.on('connect', function(data) {
console.log('connected');
sio.emit('connect', '{{ user.username }}');
});
sio.on('log', function(data) {
console.log("log: " + data);
});
</script>
Obviously, {{ user.username }} will be replaced by the username of the currently logged in user, in this example the username is "alp".
Now, every time the page gets refreshed, the console output is:
connected
log: hello alp
Therefore, invoking messages and sending responses works. But now comes the tricky part.
Problems
The response "hello alp" is sent only to the invoker of the socket.io message. I want to broadcast a message to all connected clients, so that they can be informed in realtime if a new user joins the party (for example in a chat application).
So, here are my questions:
How can i send a broadcast message to all connected clients?
How can i send a broadcast message to multiple connected clients that are subscribed on a specific channel?
How can i send a broadcast message anywhere in my python code (outside of the BaseConnection class)? Would this require some sort of Socket.IO client for python or is this builtin with TornadIO2?
All these broadcasts should be done in a reliable way, so i guess websockets are the best choice. But i am open to all good solutions.
I've recently written a very similar application on a similar setup, so I do have several insights.
The proper way of doing what you need is to have a pub-sub backend. There's only so much you can do with simple ConnectionHandlers. Eventually, handling class-level sets of connections starts to get ugly (not to mention buggy).
Ideally, you'd want to use something like Redis, with async bindings to tornado (check out brukva). That way you don't have to mess with registering clients to specific channels - Redis has all that out of the box.
Essentially, you have something like this:
class ConnectionHandler(SockJSConnection):
def __init__(self, *args, **kwargs):
super(ConnectionHandler, self).__init__(*args, **kwargs)
self.client = brukva.Client()
self.client.connect()
self.client.subscribe('some_channel')
def on_open(self, info):
self.client.listen(self.on_chan_message)
def on_message(self, msg):
# this is a message broadcast from the client
# handle it as necessary (this implementation ignores them)
pass
def on_chan_message(self, msg):
# this is a message received from redis
# send it to the client
self.send(msg.body)
def on_close(self):
self.client.unsubscribe('text_stream')
self.client.disconnect()
Note that I used sockjs-tornado which I found to be much more stable than socket.io.
Anyway, once you have this sort of setup, sending messages from any other client (such as Django, in your case) is as easy as opening a Redis connection (redis-py is a safe bet) and publishing a message:
import redis
r = redis.Redis()
r.publish('text_channel', 'oh hai!')
This answer turned out pretty long, so I went the extra mile and made a blog post out of it: http://blog.y3xz.com/blog/2012/06/08/a-modern-python-stack-for-a-real-time-web-application/
I write here, because it's difficult write in comments section. You can view examples for tornadoio2 in examples directory where you can find implementation of chat, and:
class ChatConnection(tornadio2.conn.SocketConnection):
# Class level variable
participants = set()
def on_open(self, info):
self.send("Welcome from the server.")
self.participants.add(self)
def on_message(self, message):
# Pong message back
for p in self.participants:
p.send(message)
As you can see they implemented participants as set ))
If you're already using django, why not have a look at gevent-socketio.