I need to send system logs to the browser and so I have a tornado-based websocket server running like so.
class WSHandler(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def get(self, *args, **kwargs):
self.ip = self.get_argument('ip', None)
self.action = self.get_argument('action', None)
super(WSHandler, self).get(self, *args, **kwargs)
def open(self, *args, **kwargs):
clients.append(self)
def on_message(self, message):
ll = eval(message)
for cl in clients:
if cl.ip and cl.ip != ll.user:
continue
if cl.action and cl.action != ll.action:
continue
message = '%s %s' % (ll.action, ll.url)
cl.write_message(message)
def on_close(self):
try:
clients.remove(self)
except ValueError:
pass
The examples I've encountered so far revolve around Tornado-based servers and js-based clients.
What I need, however, is an easy way to connect to this websocket from a Python client, preferably powered by Tornado. The client does not need to receive messages - only send them. I thought I had my answer with this SO post,
How to run functions outside websocket loop in python (tornado)
...but I need to send a message whenever a log event occurs, and preferably from my code that's parsing the events. The examples I've encountered so far revolve around Tornado-based servers and js-based clients. Is there a short & sweet tornado-based client that only sends messages, that can be called from a for-loop?
Also, I developed a complete Tornado WebSocket Client/Server example.
https://github.com/ilkerkesen/tornado-websocket-client-example
If you want WebSocket Authentication/Authorization, look at my other projects trebol and sugar.
There is tornad-websocket-client project. Pay attention on it.
Also there is simple websocket-client to just send messages.
Tornado includes a websocket client: http://www.tornadoweb.org/en/stable/websocket.html#client-side-support
Related
I have a websocket server written in python tornado. the server will receive many connections from clients and as you know, we have on_message function that is fired when a websocket message is received. so, here is my question that if a message(say request from client) need 5 sec to be processed then when the server is processing some request, the server goes in blocking mode and can't accept or receive more connection or data. after some research i figure out that Asyncio can resolve my problem but i don't now know to use it. so, how do i call process method to avoid blocking?
following is my code:
class WavesClient(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def open(self):
print("New client connected")
def on_message(self, message):
self.process(message)
def on_close(self):
print("Client disconnected")
def process(self,message):
#it takes 5 sec to complete
I use tornado primarily to host my webapps, therefore I can tell you if any part of your code in tornado is blocking, the whole server will block.
Now to your code:
#tornado.gen.coroutine
def on_message(self, message):
process_results = yield self.process(message)
self.write_message(process_results)
#tornado.gen.coroutine
def process(self, message):
# a long process, this is the equivalent of time.sleep(2) which is blocking
yield tornado.gen.sleep(2)
return 'finished'
With tornado you have to yield from a function to get the return value.
Also, if your function is yielding, you must wrap it using the tornado.gen.coroutine decorator
This question is similar to yours. And the answer is informative as well.
I am new to web-sockets. I am using tornado/python for my back-end and written the following code.
class BaseWebSocketHandler(websocket.WebSocketHandler):
"""Base Class to establish an websocket connection."""
def open(self):
"""Opening the web socket connection."""
self.write_message('Connection Established.')
def on_message(self, message):
"""On message module send the response."""
pass
def on_close(self):
"""Close the connection."""
self.write_message('bye')
class MeterInfo(BaseWebSocketHandler):
"""Establish an websocket connection and send meter readings."""
def on_message(self, message):
"""On message module send to the response."""
self.write_message({'A': get_meter_reading()})
My JavaScript code is like the following,
var meter = new WebSocket("ws://"+window.location.host+"/socket/meterstatus/");
meter.onopen = function() {
$('#meter-well').text('Establishing connection...');
};
meter.onmessage = function (evt) {
var data = JSON.parse(evt.data)
var text = "<div class='meter'><h2>" + data.A +"</h2></div>";
$('#meter-pre').html(text);
};
meter.onclose = function (evt) {
console.log(JSON.parse(evt.data))
$('#meter-pre').append('\n'+evt.data);
};
window.setInterval(function(){ meter.send('') }, 100);
I am making a blank web-socket request request to the back-end every 100 millisecond. this seems a very bad solution to me. Is there any better way to do it without making multiple send() to the back-end and only notifying the user only on any changes in the meter reading?
Also i have gone through MQTT protocol to do this in a better way, can someone suggest how can i implement that?
You almost found the solution to your issue here:
class MeterInfo(BaseWebSocketHandler):
"""Establish an websocket connection and send meter readings."""
def on_message(self, message):
"""On message module send to the response."""
self.write_message({'A': get_meter_reading()})
As you could notice tornado needs some event to ping client through write_message method. You are using new message from client as such event, try to change to simple timeout call as event, like this:
# BaseWebSocketHandler removed, because we need to track all opened
# sockets in the class. You could change this later.
class MeterInfo(websocket.WebSocketHandler):
"""Establish an websocket connection and send meter readings."""
opened_sockets = []
previous_meter_reading = 0
def open(self):
"""Opening the web socket connection."""
self.write_message('Connection Established.')
MeterInfo.opened_sockets.append(self)
def on_close(self):
"""Close the connection."""
self.write_message('bye')
MeterInfo.opened_sockets.remove(self)
#classmethod
def try_send_new_reading(cls):
"""Send new reading to all connected clients"""
new_reading = get_meter_reading()
if new_reading == cls.previous_meter_reading:
return
cls.previous_meter_reading = new_reading
for socket in cls.opened_sockets:
socket.write_message({'A': new_reading})
if __name__ == '__main__':
# add this after all set up and before starting ioloop
METER_CHECK_INTERVAL = 100 # ms
ioloop.PeriodicCallback(MeterInfo.try_send_new_reading,
METER_CHECK_INTERVAL).start()
# start loop
ioloop.IOLoop.instance().start()
Check out tornado.ioloop documentation for more about PeriodicCallback and other options.
If you want to use tornado to MQTT protocol, it's not possible with tornado. You could try emqtt server for example, but this is actual server, not framework to write apps, so IMHO it would be more comprehensive that ping through web socket with tornado.
I have a Tornado server with Tornadio2 handling the WebSocket and TornadoRedis by which I publish realtime messages to the clients (code was merged from several examples).
redis_client = tornadoredis.Client(config.get('cache', 'host'), int(config.get('cache', 'port')))
redis_client.connect()
class RealtimeHandler(tornadio2.conn.SocketConnection):
def __init__(self, *args, **kwargs):
super(RealtimeHandler, self).__init__(*args, **kwargs)
self.listen()
#tornado.gen.engine
def listen(self):
self.client = tornadoredis.Client(config.get('cache', 'host'), int(config.get('cache', 'port')))
self.client.connect()
yield tornado.gen.Task(self.client.subscribe, 'invalidation')
self.client.listen(self.on_message)
def on_event(self, name, *args, **kwargs):
if name == 'invalidation':
redis_client.publish('invalidation', kwargs['args'])
def on_message(self, msg):
if msg.kind == 'message':
self.send(msg.body)
def on_close(self):
if self.client.subscribed:
self.client.unsubscribe('invalidation')
self.client.disconnect()
This code works, and now, I want to store the connections in Redis to accomplish 2 things:
1. Support redundancy for my server
2. Be able to use the connections even if I restart Tornado
I saw several examples where a global list is used to save the connections but this of course won't solve my issues above.
So, how do I accomplish this?
On a different note, is this a good setup? How is it compared to node.js with socket.io and redis?
I am building a realtime web application. I want to be able to send broadcast messages from the server-side implementation of my python application.
Here is the setup:
socketio.js on the client-side
TornadIO2 server as Socket.IO server
python on the server-side (Django framework)
I can succesfully send socket.io messages from the client to the server. The server handles these and can send a response. In the following i will describe how i did that.
Current Setup and Code
First, we need to define a Connection which handles socket.io events:
class BaseConnection(tornadio2.SocketConnection):
def on_message(self, message):
pass
# will be run if client uses socket.emit('connect', username)
#event
def connect(self, username):
# send answer to client which will be handled by socket.on('log', function)
self.emit('log', 'hello ' + username)
Starting the server is done by a Django management custom method:
class Command(BaseCommand):
args = ''
help = 'Starts the TornadIO2 server for handling socket.io connections'
def handle(self, *args, **kwargs):
autoreload.main(self.run, args, kwargs)
def run(self, *args, **kwargs):
port = settings.SOCKETIO_PORT
router = tornadio2.TornadioRouter(BaseConnection)
application = tornado.web.Application(
router.urls,
socket_io_port = port
)
print 'Starting socket.io server on port %s' % port
server = SocketServer(application)
Very well, the server runs now. Let's add the client code:
<script type="text/javascript">
var sio = io.connect('localhost:9000');
sio.on('connect', function(data) {
console.log('connected');
sio.emit('connect', '{{ user.username }}');
});
sio.on('log', function(data) {
console.log("log: " + data);
});
</script>
Obviously, {{ user.username }} will be replaced by the username of the currently logged in user, in this example the username is "alp".
Now, every time the page gets refreshed, the console output is:
connected
log: hello alp
Therefore, invoking messages and sending responses works. But now comes the tricky part.
Problems
The response "hello alp" is sent only to the invoker of the socket.io message. I want to broadcast a message to all connected clients, so that they can be informed in realtime if a new user joins the party (for example in a chat application).
So, here are my questions:
How can i send a broadcast message to all connected clients?
How can i send a broadcast message to multiple connected clients that are subscribed on a specific channel?
How can i send a broadcast message anywhere in my python code (outside of the BaseConnection class)? Would this require some sort of Socket.IO client for python or is this builtin with TornadIO2?
All these broadcasts should be done in a reliable way, so i guess websockets are the best choice. But i am open to all good solutions.
I've recently written a very similar application on a similar setup, so I do have several insights.
The proper way of doing what you need is to have a pub-sub backend. There's only so much you can do with simple ConnectionHandlers. Eventually, handling class-level sets of connections starts to get ugly (not to mention buggy).
Ideally, you'd want to use something like Redis, with async bindings to tornado (check out brukva). That way you don't have to mess with registering clients to specific channels - Redis has all that out of the box.
Essentially, you have something like this:
class ConnectionHandler(SockJSConnection):
def __init__(self, *args, **kwargs):
super(ConnectionHandler, self).__init__(*args, **kwargs)
self.client = brukva.Client()
self.client.connect()
self.client.subscribe('some_channel')
def on_open(self, info):
self.client.listen(self.on_chan_message)
def on_message(self, msg):
# this is a message broadcast from the client
# handle it as necessary (this implementation ignores them)
pass
def on_chan_message(self, msg):
# this is a message received from redis
# send it to the client
self.send(msg.body)
def on_close(self):
self.client.unsubscribe('text_stream')
self.client.disconnect()
Note that I used sockjs-tornado which I found to be much more stable than socket.io.
Anyway, once you have this sort of setup, sending messages from any other client (such as Django, in your case) is as easy as opening a Redis connection (redis-py is a safe bet) and publishing a message:
import redis
r = redis.Redis()
r.publish('text_channel', 'oh hai!')
This answer turned out pretty long, so I went the extra mile and made a blog post out of it: http://blog.y3xz.com/blog/2012/06/08/a-modern-python-stack-for-a-real-time-web-application/
I write here, because it's difficult write in comments section. You can view examples for tornadoio2 in examples directory where you can find implementation of chat, and:
class ChatConnection(tornadio2.conn.SocketConnection):
# Class level variable
participants = set()
def on_open(self, info):
self.send("Welcome from the server.")
self.participants.add(self)
def on_message(self, message):
# Pong message back
for p in self.participants:
p.send(message)
As you can see they implemented participants as set ))
If you're already using django, why not have a look at gevent-socketio.
I'm using python-xmpp to send jabber messages. Everything works fine except that every time I want to send messages (every 15 minutes) I need to reconnect to the jabber server, and in the meantime the sending client is offline and cannot receive messages.
So I want to write a really simple, indefinitely running xmpp client, that is online the whole time and can send (and receive) messages when required.
My trivial (non-working) approach:
import time
import xmpp
class Jabber(object):
def __init__(self):
server = 'example.com'
username = 'bot'
passwd = 'password'
self.client = xmpp.Client(server)
self.client.connect(server=(server, 5222))
self.client.auth(username, passwd, 'bot')
self.client.sendInitPresence()
self.sleep()
def sleep(self):
self.awake = False
delay = 1
while not self.awake:
time.sleep(delay)
def wake(self):
self.awake = True
def auth(self, jid):
self.client.getRoster().Authorize(jid)
self.sleep()
def send(self, jid, msg):
message = xmpp.Message(jid, msg)
message.setAttr('type', 'chat')
self.client.send(message)
self.sleep()
if __name__ == '__main__':
j = Jabber()
time.sleep(3)
j.wake()
j.send('receiver#example.org', 'hello world')
time.sleep(30)
The problem here seems to be that I cannot wake it up. My best guess is that I need some kind of concurrency. Is that true, and if so how would I best go about that?
EDIT: After looking into all the options concerning concurrency, I decided to go with twisted and wokkel. If I could, I would delete this post.
There is a good example on the homepage of xmpppy itself (which is another name for python-xmpp), which does almost what you want: xtalk.py
It is basically a console jabber-client, but shouldn't be hard to rewrite into bot you want.
It's always online and can send and receive messages. I don't see a need for multiprocessing (or other concurrency) module here, unless you need to receive and send messages at exact same time.
A loop over the Process(timeout) method is a good way to wait and process any new incoming stanzas while keeping the connection up.