I'm trying to create my own subclass of socket.socket that will be able to handle custom messages. So far my code looks like this:
self._sockets.append(s)
logging.debug("Waiting for incoming connections on port %d" % (port))
while not self.shutdown:
inputready,outputready,exceptready = select(self._sockets,[],[])
print "Select returned"
for i in inputready:
if s == i:
# handle the server socket
client, address = s.accept()
self._sockets.append(client)
print "%r , %r" % (client, address)
else:
# handle all other sockets
s.handleMessage()
so as you can see I'm either acceptin new connections or if it returned from another socket it'll call handleMessage on that socket. Now the problem is that off course socket.accept() will return a socket.socket and not my subclass which implements the handleMessage function.
What would be the easiest way to get my custom class instead of the default socket.socket?
From your description it appears that you are making a message handler that has-a socket (or sockets). When designing classes has-a indicates composition and delegation while is-a can indicate inheritance.
So it is not appropriate to inherit from socket.socket, and your code is already looking a bit hybrid. Something like this really coarse pseudo-code is probably best suited to the task:
class MyMessageHandler(object):
def __init__(self):
self.sockets = [...]
def wait(self):
debug('waiting...')
i, o, e = select(...)
How difficult is it to setup a dictionary mapping socket descriptors to your socket wrapper objects?
Related
I'm writing a program with multiple sockets using a select statement to manage them using the following code :
while not self.stopped:
input_ready, output_ready, except_ready = select.select(self.input_sockets, [], [], 5)
if (not input_ready) and (not output_ready) and (not except_ready):
print("Timed Out")
else:
for s in input_ready:
s.process_data()
For this to work I have created a class which is abstracted from socket.socket and adds the method process_data. I have a class defined this way, which binds to a socket to listen for inbound connections and when process_data() is called, it accepts the connection using (sock, address) = self.accept() ... I then add sock to the input_sockets array for use in the select, but obviously the accept method returns a socket, not an abstracted class with and therefore has no process_data() method, so this causes errors.
Can anyone think of a way that I could use the accept() method to return my own abstracted socket class rather than a normal socket?
Thanks
EDIT :
I have found a happy work around for now - Instead of creating an abstracted socket class, I have created a standard class as follows:
class DirectConnection():
def __init__(self, sock):
self.socket = sock
def fileno(self):
return self.socket.fileno()
def process_data(self):
print("Got data")
Then from my listening socket
(sock, address) = self.accept()
socketmanager.monitor_socket(DirectConnection(sock))
The select.select makes use of the fileno property of the socket object, so by defining a fileno() method which returns the fileno() of the socket passed into the class, I can now have the select.select call my method on this class, which I can then instruct to send/receive data from the socket passed in.
credit goes to : http://bytes.com/topic/python/answers/437702-subclassing-socket and http://code.activestate.com/recipes/52295/
I'm writing a client implementing a custom protocol, and have a factory for it. My problem is the following: my client has bi-dir communication, and sometimes I want to tell it "send this data". But all I have is the factory object:
class MyFactory(ClientFactory):
protocol = MyProtocol
def __init__(self, recv_callback):
self.recv_callback = recv_callback
def send_message(self, msg):
self.protocol.send_message(msg)
So I create a factory and have a factory object, I don't the protocol object. When send_message above is called I get an error because self.protocol is just a class, not an object.
How can I do this? Should I also expose the protocol for connection in addition to the factory?
Thanks
You have access to all of the objects you want. The factory is responsible for creating protocol instances, so if you want to keep the protocol instance around where the factory can use it, override buildProtocol and save the instance:
class MyFactory(ClientFactory):
protocol = MyProtocol
...
def buildProtocol(self, address):
proto = ClientFactory.buildProtocol(self, address)
self.connectedProtocol = proto
return proto
However, this approach is lacking in one important feature. It does not make it easy to tell when buildProtocol has been called and connectedProtocol has been set. If you try to use this attribute naively:
factory = MyFactory()
reactor.connectTCP(host, port, factory)
factory.connectedProtocol.send_message(...)
The code will fail with an AttributeError because the connection has not yet actually been set up. Since Twisted is event driven, you need to make sure to use this code by responding to an event that says the connection has been set up.
You might do this by firing a callback when the protocol is constructed instead of just setting an attribute. Twisted actually has a helper factory which does something like this already:
from twisted.internet.protocol import ClientCreator
cc = ClientCreator(reactor, MyProtocol)
whenConnected = cc.connectTCP(host, port)
# Or the equivalent with endpoints
# from twisted.internet.endpoints import TCP4ClientEndpoint
# from twisted.internet.protocol import ClientFactory
# endpoint = TCP4ClientEndpoint(reactor, host, port)
# factory = ClientFactory()
# factory.protocol = MyProtocol
# whenConnected = endpoint.connect(factory)
def cbConnected(connectedProtocol):
connectedProtocol.send_message(...)
def ebConnectError(reason):
# Connection attempt failed, perhaps retry
...
whenConnected.addCallbacks(cbConnected, ebConnectError)
You could also save the reference to connectedProtocol in the cbConnected callback so that you can continue to use it later on. You might also start whatever other operations want to use the connected protocol in cbConnected, so that they don't try to use the connection before it is actually available.
I'm trying to make a simple TCP server using Twisted ,which can do some interaction between diffirent client connections.The main code is as below:
#!/usr/bin/env python
from twisted.internet import protocol, reactor
from time import ctime
#global variables
PORT = 22334
connlist = {} #store all the connections
ids = {} #map the from-to relationships
class TSServerProtocol(protocol.Protocol):
def dataReceived(self, data):
from_id,to_id = data.split('|') #get the IDs from standard client input,which looks like "from_id|to_id"
if self.haveConn(from_id): #try to store new connections' informations
pass
else:
self.setConn(from_id)
self.setIds(from_id,to_id)
if to_id in self.csids.keys():
self.connlist[to_id].transport.write(\
"you get a message now!from %s \n" % from_id) #if the to_id target found,push him a message.doesn't work as expected
def setConn(self,sid):
connlist[sid] = self
#some other functions
factory = protocol.Factory()
factory.protocol = TSServerProtocol
print 'waiting from connetction...'
reactor.listenTCP(PORT, factory)
reactor.run()
As the comments mentioned,if a new client connection comes,I'll store its connection handle in a global varaible connlist which is like
connlist = {a_from_id:a_conObj,b_from_id:b_conObj,....}
and also parse the input then map its from-to information in ids.Then I check whether there's a key in the ids matches current "to_id".if does,get the connection handle using connlist[to_id] and push a message to the target connection.But it doesn't work.The message only shows in a same connection.Hope someone can show me some directions about this.
Thanks!
Each time a TCP connection is made, Twisted will create a unique instance of TSServerProtocol to handle that connection. So, you'll only ever see 1 connection in TSServerProtocol. Normally, this is what you want but Factories can be extended to do the connection tracking you're attempting to do here. Specifically, you can subclass Factory and override the buildProtocol() method to track instances of TSServerProtocol. The interrelationship between all the classes in Twisted takes a little time to learn and get used to. In particular, this piece of the standard Twisted documentation should be your best friend for the next while ;-)
Im trying to write a program that would be listening for data (simple text messages) on some port (say tcp 6666) and then pass them to one or more different protocols - irc, xmpp and so on. I've tried many approaches and digged the Internet, but I cant find easy and working solution for such task.
The code I am currently fighting with is here: http://pastebin.com/ri7caXih
I would like to know how to from object like:
ircf = ircFactory('asdfasdf', '#asdf666')
get access to self protocol methods, because this:
self.protocol.dupa1(msg)
returns error about self not being passed to active protocol object. Or maybe there is other, better, easier and more kosher way to create single reactor with multiple protocols and have actions triggeres when a message arrives on any of them, and then pass that message to other protocols for handling/processing/sending?
Any help will be highly appreciated!
Here is sample code to read from multiple connections to port 9001 and write out to a connection on port 9000. You would need multiple "PutLine" implementations, one for XMPP, IRC, MSN, etc.
I used a global to store the output connection PutLine but you would want to create a more complex Factory object that would handle this instead.
#!/usr/bin/env python
from twisted.internet.protocol import Protocol, Factory
from twisted.internet.endpoints import clientFromString, serverFromString
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
queue = []
putter = None
class GetLine(LineReceiver):
delimiter = '\n'
def lineReceived(self, line):
queue.append(line)
putter.have_data()
self.sendLine(line)
class PutLine(LineReceiver):
def __init__(self):
global putter
putter = self
print 'putline init called %s' % str(self)
def have_data(self):
line = queue.pop()
self.sendLine(line)
def main():
f = Factory()
f.protocol = PutLine
endpoint = clientFromString(reactor, "tcp:host=localhost:port=9000")
endpoint.connect(f)
f = Factory()
f.protocol = GetLine
endpoint2 = serverFromString(reactor, "tcp:port=9001")
endpoint2.listen(f)
reactor.run()
if __name__ == '__main__':
main()
Testing:
nc -l 9000
python test.py
nc 9001
Data entered form any number of nc 9001 (or netcat 9001) will appear on nc -l 9000.
This is answered in the FAQ.
http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinputononeconnectionresultinoutputonanother
See doc/core/examples/chatserver.py. There they've added hooks to the Protocol's connectionMade and connectionLost methods to maintain a list of connected clients, and then it iterates through all of them when a message arrives to pass on.
This is problem.
My primary work is : deliver "s" object to "handle" method in TestRequestHandler class.
My first step was : deliver "s" object through "point" method to TestServer class, but here im stuck. How to deliver "s" object to TestRequestHandler? Some suggestions?
import threading
import SocketServer
from socket import *
class TestRequestHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
SocketServer.BaseRequestHandler.__init__(self, request, client_address, server)
return
def setup(self):
return SocketServer.BaseRequestHandler.setup(self)
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
def finish(self):
return SocketServer.BaseRequestHandler.finish(self)
class TestServer(SocketServer.TCPServer):
def __init__(self, server_address, handler_class=TestRequestHandler):
print "__init__"
SocketServer.TCPServer.__init__(self, server_address, handler_class)
return
def point(self,obj):
self.obj = obj
print "point"
def server_activate(self):
SocketServer.TCPServer.server_activate(self)
return
def serve_forever(self):
print "serve_forever"
while True:
self.handle_request()
return
def handle_request(self):
return SocketServer.TCPServer.handle_request(self)
if __name__ == '__main__':
s = socket(AF_INET, SOCK_STREAM)
address = ('localhost', 6666)
server = TestServer(address, TestRequestHandler)
server.point(s)
t = threading.Thread(target=server.serve_forever())
t.setDaemon(True)
t.start()
If I understand correctly, I think you perhaps are misunderstanding how the module works. You are already specifying an address of 'localhost:6666' for the server to bind on.
When you start the server via your call to serve_forever(), this is going to cause the server to start listening to a socket on localhost:6666.
According to the documentation, that socket is passed to your RequestHandler as the 'request' object. When data is received on the socket, your 'handle' method should be able to recv/send from/to that object using the documented socket API.
If you want a further abstraction, it looks like your RequestHandler can extend from StreamRequestHandler and read/write to the socket using file-like objects instead.
The point is, there is no need for you to create an additional socket and then try to force your server to use the new one instead. Part of the value of the SocketServer module is that it manages the lifecycle of the socket for you.
On the flip side, if you want to test your server from a client's perspective, then you would want to create a socket that you can read/write your client requests on. But you would never pass this socket to your server, per se. You would probably do this in a completely separate process and test your server via IPC over the socket.
Edit based on new information
To get server A to open a socket to server B when server A receives data one solution is to simply open a socket from inside your RequestHandler. That said, there are likely some other design concerns that you will need to address based on the requirements of your service.
For example, you may want to use a simple connection pool that say opens a few sockets to server B that server A can use like a resource. There may already be some libraries in Python that help with this.
Given your current design, your RequestHandler has access to the server as a member variable so you could do something like this:
class TestServer(SocketServer.TCPServer):
def point (self, socketB):
self.socketB = socketB # hold serverB socket
class TestRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
self.server.socketB ... # Do whatever with the socketB
But like I said, it may be better for you to have some sort of connection pool or other object that manages your server B socket such that your server A handler can just acquire/release the socket as incoming requests are handled.
This way you can better deal with conditions where server B breaks the socket. Your current design wouldn't be able to handle broken sockets very easily. Just some thoughts...
If the value of s is set once, and not reinitialized - you could make it a class variable as opposed to an instance variable of TestServer, and then have the handler retrieve it via a class method of TestServer in the handler's constructor.
eg: TestServer._mySocket = s
Ok, my main task is this. Construction of the listening server (A-server - localhost, 6666) which during start will open "hard" connection to the different server (B-server - localhost, 7777).
When the customer send data to the A-server this (A-server) sends data (having that hard connection to the B-server) to B-server, the answer receives from the B-server to A-server and answer sends to the customer.
Then again : the customer sends data, A-server receives them, then sends to the B-server, the answer receives data from the B-server and A-server send data to the customer.
And so round and round. The connection to the B-server is closes just when the server A will stop.
All above is the test of making this.