I'm not very experienced with Python and new to Tornado websockets, so I have a problem grasping the idea how it actually works.
Concretely, this is a usual example:
class EchoWebSocket(tornado.websocket.WebSocketHandler):
def open(self):
print("WebSocket opened")
def on_message(self, message):
self.write_message(u"You said: " + message)
def on_close(self):
print("WebSocket closed")
What confuses me is the way the requests are served. I read that Tornado is non-blocking and if understood well single-threaded, where each request is handled by a handler in a fast way and if the operation lasts longer then there are callbacks which are called when certain operation (e.g. database) is finished. This allows a large number of users to be served. But what confuses me with these websockets and examples, is what is self in this case?
I know that it is a similar thing like this in Java and C#, but I don't understand how this self represents different clients each time a client sends a message? Moreover, is there only a single instance of EchoWebSocket per application, or there is an instance per request?
Additionally, how should I keep the references to the clients? I tried two ways:
class EchoWebSocket(tornado.websocket.WebSocketHandler):
clients = []
def open(self):
print("WebSocket opened")
def on_message(self, message):
EchoWebSocket.clients.append(self)
self.write_message(u"You said: " + message)
def on_close(self):
print("WebSocket closed")
And
class EchoWebSocket(tornado.websocket.WebSocketHandler):
def __init__(self, application, request):
super(EchoWebSocket,self).__init__(application, request)
self.clients = []
def open(self):
print("WebSocket opened")
def on_message(self, message):
self.clients.append(self)
self.write_message(u"You said: " + message)
def on_close(self):
print("WebSocket closed")
But the second approach doesn't seem to be working. What is the proper way to do it and why?
In python, by convention self is a reference to the current class' instance object.
AFAIK, the tornado framework creates an instance of the class WebSocketHandler for each client. So you don't really have to keep a reference to clients. Each client is naturally handled by a different instance of the handler. Practically it means that self will be different objects for two distinct clients.
To keep references to connected clients, something like that would work:
class EchoWebSocket(tornado.websocket.WebSocketHandler):
all_clients = set()
def open(self):
self.all_clients.add(self)
def on_message(self, message):
pass
def on_close(self):
self.all_clients.remove(self)
Related
I have a websocket server written in python tornado. the server will receive many connections from clients and as you know, we have on_message function that is fired when a websocket message is received. so, here is my question that if a message(say request from client) need 5 sec to be processed then when the server is processing some request, the server goes in blocking mode and can't accept or receive more connection or data. after some research i figure out that Asyncio can resolve my problem but i don't now know to use it. so, how do i call process method to avoid blocking?
following is my code:
class WavesClient(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def open(self):
print("New client connected")
def on_message(self, message):
self.process(message)
def on_close(self):
print("Client disconnected")
def process(self,message):
#it takes 5 sec to complete
I use tornado primarily to host my webapps, therefore I can tell you if any part of your code in tornado is blocking, the whole server will block.
Now to your code:
#tornado.gen.coroutine
def on_message(self, message):
process_results = yield self.process(message)
self.write_message(process_results)
#tornado.gen.coroutine
def process(self, message):
# a long process, this is the equivalent of time.sleep(2) which is blocking
yield tornado.gen.sleep(2)
return 'finished'
With tornado you have to yield from a function to get the return value.
Also, if your function is yielding, you must wrap it using the tornado.gen.coroutine decorator
This question is similar to yours. And the answer is informative as well.
I've a web-socket server which connects with the clients. Following is the code:-
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
class Chat(LineReceiver):
def __init__(self, users):
self.users = users
self.name = None
self.state = "GETNAME"
def connectionMade(self):
self.sendLine("What's your name?")
def connectionLost(self, reason):
if self.users.has_key(self.name):
del self.users[self.name]
def lineReceived(self, line):
if self.state == "GETNAME":
self.handle_GETNAME(line)
else:
self.handle_CHAT(line)
def handle_GETNAME(self, name):
if self.users.has_key(name):
self.sendLine("Name taken, please choose another.")
return
self.sendLine("Welcome, %s!" % (name,))
self.name = name
self.users[name] = self
self.state = "CHAT"
def handle_CHAT(self, message):
# Need to send the message to the connected clients.
class ChatFactory(Factory):
def __init__(self):
self.users = {} # maps user names to Chat instances
def buildProtocol(self, addr):
return Chat(self.users)
reactor.listenTCP(8123, ChatFactory())
reactor.run()
Clients get connected to the above code(server), and sends the data to the server.
Now, I've another python script, basically a scraper which scrapes the web, processes it and finally need to send the data to the connected clients.
script.py
while True:
# call `send_message` function and send data to the connected clients.
How can I achieve it?? Any example would be of great help!!
UPDATE
After using Autobahn
I've a server that fetches data from 3rd party API. I want to send this data to all the connected web-socket clients. Here is my code:-
class MyServerProtocol(WebSocketServerProtocol):
def __init__(self):
self.connected_users = []
self.send_data()
def onConnect(self, request):
print("Client connecting: {0}".format(request.peer))
def onOpen(self):
print("WebSocket connection open.")
self.connected_users.append(self) # adding users to the connected_list
def send_data(self):
# fetch data from the API and forward it to the connected_users.
for u in self.users:
print 1111
u.sendMessage('Hello, Some Data from API!', False)
def onClose(self, wasClean, code, reason):
connected_users.remove(self) # remove user from the connected list of users
print("WebSocket connection closed: {0}".format(reason))
if __name__ == '__main__':
import sys
from twisted.python import log
from twisted.internet import reactor
factory = WebSocketServerFactory(u"ws://127.0.0.1:9000")
factory.protocol = MyServerProtocol
reactor.listenTCP(9000, factory)
reactor.run()
My Server will never receive a message or probably will receive, but as of right now there's no such use-case, hence no need for OnMessage event for this example).
How do I write my send_data function in order to send data to all my connected clients??
You need to avoid this pattern when writing software with Twisted:
while True:
# call `send_message` function and send data to the connected clients.
Twisted is a cooperative multitasking system. "Cooperative" means that you have to give up control of execution periodically so that other tasks get a chance to run.
twisted.internet.task.LoopingCall can be used to replace many while ... loops (particularly while True loops):
from twisted.internet.task import LoopingCall
LoopingCall(one_iteration).start(iteration_interval)
This will call one_iteration every iteration_interval seconds. In between, it will give up control of execution so other tasks can run.
Making one_iteration send a message to a client is just a matter of giving one_iteration a reference to that client (or those clients, if there are many).
This is a variation on the FAQ How do I make Input on One Connection Result in Output on Another.
If you have a ChatFactory with a dict containing all your clients, just pass that factory to one_iteration:
LoopingCall(one_iteration, that_factory)
or
LoopingCall(lambda: one_iteration(that_factory))
first question here.
So for a club at school we are working on making a IRC client in Python and Twisted.
So I take the example bot that twisted gives you. I've managed to get it connected to a irc channel, and it logs.
I know I've probably gotta use 2 threads to have both reading from the server and input simultaneous, which I can achieve, but only if it command line input. Mind you it is still logging the data from the channel at the same time.
So to do this I used: d = threads.deferToThread(aSillyBlockingMethod)
Which calls my raw_input() loop.
My problem lies in not being able to figure out how to change this from just typing into and printing form the commandline; to being able to actually send messages to the irc server for other people to read.
Any help would be greatly appreciated. I am a novice python programmer and don't know too much web stuff; like protocols, ports, and stuff like that but I am slowly picking them up. If anyone knows of an easier way to do this let me know please I am not committed to using Twisted.
Here is my code, or rather the modified bot I'm tinkering with:
# twisted imports
from twisted.words.protocols import irc
from twisted.internet import reactor, protocol
from twisted.python import log
from twisted.internet import threads
# system imports
import time, sys
class MessageLogger:
"""
An independent logger class (because separation of application
and protocol logic is a good thing).
"""
def __init__(self, file):
self.file = file
def log(self, message):
"""Write a message to the file."""
timestamp = time.strftime("[%H:%M:%S]", time.localtime(time.time()))
self.file.write('%s %s\n' % (timestamp, message))
self.file.flush()
def close(self):
self.file.close()
class LogBot(irc.IRCClient):
"""A logging IRC bot."""
nickname = "twistedbot"
def connectionMade(self):
irc.IRCClient.connectionMade(self)
self.logger = MessageLogger(open(self.factory.filename, "a"))
self.logger.log("[connected at %s]" %
time.asctime(time.localtime(time.time())))
def connectionLost(self, reason):
irc.IRCClient.connectionLost(self, reason)
self.logger.log("[disconnected at %s]" %
time.asctime(time.localtime(time.time())))
self.logger.close()
# callbacks for events
def signedOn(self):
"""Called when bot has succesfully signed on to server."""
self.join(self.factory.channel)
def joined(self, channel):
"""This will get called when the bot joins the channel."""
self.logger.log("[I have joined %s]" % channel)
def privmsg(self, user, channel, msg):
"""This will get called when the bot receives a message."""
user = user.split('!', 1)[0]
self.logger.log("<%s> %s" % (user, msg))
# Check to see if they're sending me a private message
if channel == self.nickname:
msg = "It isn't nice to whisper! Play nice with the group."
self.msg(user, msg)
return
# Otherwise check to see if it is a message directed at me
if msg.startswith(self.nickname + ":"):
msg = "%s: I am a log bot" % user
self.msg(channel, msg)
self.logger.log("<%s> %s" % (self.nickname, msg))
def action(self, user, channel, msg):
"""This will get called when the bot sees someone do an action."""
user = user.split('!', 1)[0]
self.logger.log("* %s %s" % (user, msg))
# irc callbacks
def irc_NICK(self, prefix, params):
"""Called when an IRC user changes their nickname."""
old_nick = prefix.split('!')[0]
new_nick = params[0]
self.logger.log("%s is now known as %s" % (old_nick, new_nick))
# For fun, override the method that determines how a nickname is changed on
# collisions. The default method appends an underscore.
def alterCollidedNick(self, nickname):
"""
Generate an altered version of a nickname that caused a collision in an
effort to create an unused related name for subsequent registration.
"""
return nickname + '^'
def aSillyBlockingMethod(self):
import time
while True:
msg = raw_input()
print msg
class LogBotFactory(protocol.ClientFactory):
"""A factory for LogBots.
A new protocol instance will be created each time we connect to the server.
"""
def __init__(self, channel, filename):
self.channel = channel
self.filename = filename
def buildProtocol(self, addr):
p = LogBot()
p.factory = self
return p
def clientConnectionLost(self, connector, reason):
"""If we get disconnected, reconnect to server."""
connector.connect()
def clientConnectionFailed(self, connector, reason):
print "connection failed:", reason
reactor.stop()
if __name__ == '__main__':
# initialize logging
log.startLogging(sys.stdout)
# create factory protocol and application
f = LogBotFactory("#goon.squad.dev", "test.txt")
# connect factory to this host and port
reactor.connectTCP("irc.freenode.net", 6667, f)
#Use this to keep user input open
d = threads.deferToThread(aSillyBlockingMethod)
# run bot
reactor.run()
--TyrZaraki
I know I've probably gotta use 2 threads to have both reading from the server and input simultaneous, which I can achieve, but only if it command line input. Mind you it is still logging the data from the channel at the same time.
Actually, it's not necessary to use two threads for this. A major strength of Twisted is doing I/O without using threads. The question and answer Michael linked to talk about Twisted's non-threaded support for interacting with standard input and standard output via twisted.internet.stdio.StandardIO.
My problem lies in not being able to figure out how to change this from just typing into and printing form the commandline; to being able to actually send messages to the irc server for other people to read.
IRCClient has a method for sending messages to the IRC server - the sample code you included in your question even uses this method already, IRCClient.msg. All you need to do is call it.
Your threaded code could change like this to do so (again, threads are unnecessary, but in the interest of just showing you how to send a message from your input handling code, I'll base this part of the answer on threads, to avoid other changes to the code which might make the answer harder to understand. You do not need threads to do this.):
def aSillyBlockingMethod(bot):
import time
while True:
msg = raw_input()
bot.threadSafeMsg("#bottest", msg)
class LogBot(irc.IRCClient):
"""A logging IRC bot."""
nickname = "twistedbot"
def connectionMade(self):
irc.IRCClient.connectionMade(self)
self.logger = MessageLogger(open(self.factory.filename, "a"))
self.logger.log("[connected at %s]" %
time.asctime(time.localtime(time.time())))
# The bot is now connected. Start reading input here.
# Pass a reference to this protocol instance, so that
# messages can be sent to this protocol instance.
deferToThread(aSillyBlockingMethod, self)
# Define a helper function for aSillyBlockingMethod to use.
# Since aSillyBlockingMethod runs in a thread, it cannot just call
# IRCClient.msg, since that method - like almost all methods in Twisted -
# is not thread-safe. Instead it must call this thread-safe wrapper.
def threadSafeMsg(self, channel, message):
reactor.callFromThread(self.msg, channel, message)
Notice that all that's happening here is that aSillyBlockingMethod is calling a method on LogBot. The thread-safe wrapper would not be necessary when using StandardIO, since that eliminates the need for threads.
I'm a complete Twisted AND Python noob, so my apologies if any of my terminology is wrong or anything I've done is silly. Nonetheless....
I've implemented my servers in the following way:
def makeServer(application, port):
factory = protocol.ServerFactory()
factory.protocol = MyChat
factory.clients = []
tempServer = internet.TCPServer(port, factory)
tempServer.setServiceParent(application)
return tempServer
application = service.Application("chatserver")
server1 = makeServer(application, port=1025)
server2 = makeServer(application, port=1026)
server3 = makeServer(application, port=1027)
Note that MyChat is an event handling class that has a "receiveMessage" action:
def lineReceived(self, line):
print "received", repr(line)
for c in self.factory.clients:
c.transport.write(message + '\n')
I want server1 to be able to pass messages to server2. Rather, I want server1 to be treated as a client of server2. If server1 receives the message "hi" then I want it to send that same exact message to server2. The only thing server1 needs to be able to do is to send the message it received from its client to server2.
How can I accomplish this?
NOTE: You can totally change the way I'm implementing my server if it helps.
Different parts of your application can interact with each other using method calls.
Send a message to server2 really just means Call a method on one of the objects related to server2.
For example, in MyChat, you might have:
def lineReceived(self, line):
print "received", repr(line)
for c in self.factory.clients:
c.transport.write(message + '\n')
for server in self.factory.otherServers:
server.otherServerMessage(self, line)
This supposes a couple things:
You add a new otherServers attribute to your factory. Its contents are objects related to the other listening servers you have set up. These might be factory objects or protocol objects. It depends on what's most convenient based on what you intend to do with the message.
You give those related objects a new method, otherServerMessage, to handle messages delivered this way. If you were to deliver the messages directly to MyChat.lineReceived (which you easily could, if you wanted) then I would expect you to end up with infinite recursion; having a different method lets you differentiate between messages received from a client and messages received from another server.
You will probably need to implement a separate client. It is possible that an object can be both a client and a server, but I doubt it will be worth it and you are likely to run into trouble.
I suggest that the server instantiates a client object, which you connect to the 'next' server. The client can for example be an instance variable on the server.
Example:
class MyChat(LineReceiver):
def connectionMade(self):
print "Proxy: connected"
factory = protocol.ClientFactory()
class Proxy(protocol.Protocol):
def relayMessage(self, msg):
self.transport.write(msg)
factory.protocol = Proxy
point = TCP4ClientEndpoint(reactor, "localhost", 1025)
conn = point.connect(factory)
conn.addCallback(self.hasConnection)
def hasConnection(self, client):
print "Proxy: Connected to relay", client
self.client = client
def lineReceived(self, line):
print "Proxy: received", repr(line)
self.client.transport.write(line+"\n")
class MyEcho(LineReceiver):
def lineReceived(self, line):
print "Echo: received", repr(line)
factory = protocol.ServerFactory()
factory.protocol = MyChat
reactor.listenTCP(1024, factory)
factory = protocol.ServerFactory()
factory.protocol = MyEcho
reactor.listenTCP(1025, factory)
You need just declare clients inside your server, like this:
factory = SomeClientFactory('ws://127.0.0.1')
connectWS(factory)
and in your Client Class:
class SomeClient(WebSocketClientProtocol):
def __init__(self):
pass
def sendCommand(self):
self.sendMessage('A message to another server')
def onOpen(self):
self.sendCommand()
def onClose(self, wasClean, code, reason):
print(reason)
def onMessage(self, payload, isBinary):
print('A answer from another server')
class SomeClientFactory(WebSocketClientFactory):
def __init__(self, url):
WebSocketClientFactory.__init__(self,url)
self.proto = DeltaClient()
self.proto.factory = self
def buildProtocol(self, addr):
return self.proto
Tip: use a "Controller" class to manage that instances of clients inside your servers.
I'm trying to create a client/server using Twisted.
I'd like to create a daemon, which will be connected to another server as a client and act as a server for other clients.
I've writen something like that which I think describes my problem:
server = sys.argv[1]
control_port = 8001
class ControlClient(protocol.Protocol):
def makeConnection(self, transport):
[some code here -snip-]
self.firstOrder(order, transport)
def firstOrder(self, action, transport):
self.t = transport
self.t.write(action + "\0")
def sendOrder(self, action):
self.t.write(action + "\0")
def dataReceived(self, data):
[some code here -snip-]
[HERE I WANT TO SEND DATA TO CLIENTS CONNECTED TO MY TWISTED SERVER, USING CONTROL SERVER]
class ControlServer(ControlClient):
def dataReceived(self, data):
print "client said " + data
def makeConnection(self, transport):
self.t = transport
self.t.write("make connection")
print "make connection"
def sendData(self, data):
self.t.write("data")
class ClientFactory(protocol.ClientFactory):
protocol = ControlClient
def clientConnectionFailed(self, connector, reason):
print "Connection failed - goodbye!"
reactor.stop()
def clientConnectionLost(self, connector, reason):
print "Connection lost - goodbye!"
reactor.stop()
class ServerFactory(protocol.ServerFactory):
protocol = ControlServer
def main():
c = ClientFactory()
reactor.connectTCP(server, control_port, c)
s = ServerFactory()
reactor.listenTCP(9000, s)
reactor.run()
if __name__ == '__main__':
main()
As you can see, I'd like to send (as a server) some data received (as a client). My problem is of course my ServerControl is not instantiated in my ClientControl so I don't have access to transport which is required to send data to clients.
The only thing you seem to be missing is that you can keep a list of your client connections and make that list available to the code that's trying to send out data to all the clients.
There's an example of this in the Twisted FAQ: http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinputononeconnectionresultinoutputonanother
That example only has one factory, but the idea is the same. To handle your case with two factories, just give one factory a reference to the other.