Develop Reference

Proper use of a client and Deferred with Twisted

I implemented a basic SOCKS4 client with socket, but my Twisted translation isn't coming along too well. Here's my current code:
import struct
import socket
from twisted.python.failure import Failure
from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.internet.protocol import Protocol, ClientFactory
class Socks4Client(Protocol):
VERSION = 4
HOST = "0.0.0.0"
PORT = 80
REQUESTS = {
"CONNECT": 1,
"BIND": 2
}
RESPONSES = {
90: "request granted",
91: "request rejected or failed",
92: "request rejected because SOCKS server cannot connect to identd on the client",
93: "request rejected because the client program and identd report different user-ids"
}
def __init__(self):
self.buffer = ""
def connectionMade(self):
self.connect(self.HOST, self.PORT)
def dataReceived(self, data):
self.buffer += data
if len(self.buffer) == 8:
self.validateResponse(self.buffer)
def connect(self, host, port):
data = struct.pack("!BBH", self.VERSION, self.REQUESTS["CONNECT"], port)
data += socket.inet_aton(host)
data += "\x00"
self.transport.write(data)
def validateResponse(self, data):
version, result_code = struct.unpack("!BB", data[1:3])
if version != 4:
self.factory.protocolError(Exception("invalid version"))
elif result_code == 90:
self.factory.deferred.callback(self.responses[result_code])
elif result_code in self.RESPONSES:
self.factory.protocolError(Exception(self.responses[result_code]))
else:
self.factory.protocolError(Exception())
self.transport.abortConnection()
class Socks4Factory(ClientFactory):
protocol = Socks4Client
def __init__(self, deferred):
self.deferred = deferred
def clientConnectionFailed(self, connector, reason):
self.deferred.errback(reason)
def clientConnectionLost(self, connector, reason):
print "Connection lost:", reason
def protocolError(self, reason):
self.deferred.errback(reason)
def result(result):
print "Success:", result
def error(reason):
print "Error:", reason
if __name__ == "__main__":
d = Deferred()
d.addCallbacks(result, error)
factory = Socks4Factory(d)
reactor.connectTCP('127.0.0.1', 1080, factory)
reactor.run()
I have a feeling that I'm abusing Deferred. Is this the right way to send results from my client?
I've read a few tutorials, looked at the documentation, and read through most of the protocols bundled with Twisted, but I still can't figure it out: what exactly is a ClientFactory for? Am I using it the right way?
clientConnectionLosts gets triggered a lot. Sometimes I lose the connection and get a successful response. How is that so? What does this mean, and should I treat it as an error?
How do I make sure that my deferred calls only one callback/errback?
Any tips are appreciated.

I have a feeling that I'm abusing Deferred. Is this the right way to send results from my client?
It's not ideal, but it's not exactly wrong either. Generally, you should try to keep the code that instantiates a Deferred as close as possible to the code that calls Deferred.callback or Deferred.errback on that Deferred. In this case, those pieces of code are quite far apart - the former is in __main__ while the latter is in a class created by a class created by code in __main__. This is sort of like the law of Demeter - the more steps between these two things, the more tightly coupled, inflexible, and fragile the software.
Consider giving Socks4Client a method that creates and returns this Deferred instance. Then, try using an endpoint to setup the connection so you can more easily call this method:
endpoint = TCP4StreamClientEndpoint(reactor, "127.0.0.1", 1080)
d = endpoint.connect(factory)
def connected(protocol):
return protocol.waitForWhatever()
d.addCallback(connected)
d.addCallbacks(result, error)
One thing to note here is that using an endpoint, the clientConnectionFailed and clientConnectionLost methods of your factory won't be called. The endpoint takes over the former responsibility (not the latter though).
I've read a few tutorials, looked at the documentation, and read through most of the protocols bundled with Twisted, but I still can't figure it out: what exactly is a ClientFactory for? Am I using it the right way?
It's for just what you're doing. :) It creates protocol instances to use with connections. A factory is required because you might create connections to many servers (or many connections to one server). However, a lot of people have trouble with ClientFactory so more recently introduced Twisted APIs don't rely on it. For example, you could also do your connection setup as:
endpoint = TCP4StreamClientEndpoint(reactor, "127.0.0.1", 1080)
d = connectProtocol(endpoint, Socks4Client())
...
ClientFactory is now out of the picture.
clientConnectionLosts gets triggered a lot. Sometimes I lose the connection and get a successful response. How is that so? What does this mean, and should I treat it as an error?
Every connection must eventually be lost. You have to decide on your own whether this is an error or not. If you have finished everything you wanted to do and you called loseConnection, it is probably not an error. Consider a connection to an HTTP server. If you have sent your request and received your response, then losing the connection is probably not a big deal. But if you have only received half the response, that's a problem.
How do I make sure that my deferred calls only one callback/errback?
If you structure your code as I described in response to your first question above, it becomes easier to do this. When the code that uses callback/errback on a Deferred is spread across large parts of your program, then it becomes harder to do this correctly.
It is just a matter of proper state tracking, though. Once you give a Deferred a result, you have to arrange to know that you shouldn't give it another one. A common idiom for this is to drop the reference to the Deferred. For example, if you are saving it as the value of an attribute on a protocol instance, then set that attribute to None when you have given the Deferred its result.

Trial unittests using Autobahn WebSocket

I'm trying to write unittests for my application that uses Autobahn.
I want to test my controllers which gets received data from protocol, parses it and reacts to it.
But when my test comes to a point when protocol should be disconnected (self.sendClose) then it raises error
exceptions.AttributeError: 'MyProtocol' object has no attribute 'state'.
I was trying to makeConnection using proto_helpers.StringTransport but then I have errors too
exceptions.AttributeError: StringTransport instance has no attribute 'setTcpNoDelay'`
I'm using trial and I don't want to run dummy server/client for testing purposes only, because it's not recommended.
How should I write my tests so I can test functions that sends data, read data, disconnects etc. using fake connection and trial ?

It is difficult to say exactly what is going on without having a peek at MyProtocol class. The problem sounds a lot like it is caused by the fact that you are directly messing round with low level functions and therefore also the state attribute of WebSocket class, which is, well, a representation of the internal state of the WebSocket connection.
According to the autobahn reference doc, the APIs from the WebSicketProtocol that you could directly use and override are:
onOpen
onMessage
onClose
sendMessage
sendClose
Your approach of using the StringTransport to test your protocol is not ideal. The problem lays in the fact that MyProtocol is a tiny layer on top of the WebSocketProtocol framework provided by autobahn which, for better or worse, hides the details about managing the connection, the transport and the internal protocol state.
If you think about it, you want to test your stuff, not WebSocketProtocol and therefore if you do not want to embed a dummy server or client, your best bet is to test directly the methods that MyProtocol overrides.
An example of what I am saying is the following
class MyPublisher(object):
cbk=None
def publish(self, msg):
if self.cbk:
self.cbk(msg)
class MyProtocol(WebSocketServerProtocol):
def __init__(self, publisher):
WebSocketServerProtocol.__init__(self)
#Defining callback for publisher
publisher.cbk = self.sendMessage
def onMessage(self, msg, binary)
#Stupid echo
self.sendMessage(msg)
class NotificationTest(unittest.TestCase):
class MyProtocolFactory(WebSocketServerFactory):
def __init__(self, publisher):
WebSocketServerFactory.__init__(self, "ws://127.0.0.1:8081")
self.publisher = publisher
self.openHandshakeTimeout = None
def buildProtocol(self, addr):
protocol = MyProtocol(self.listener)
protocol.factory = self
protocol.websocket_version = 13 #Hybi version 13 is supported by pretty much everyone (apart from IE <8 and android browsers)
return protocol
def setUp(self):
publisher = task.LoopingCall(self.send_stuff, "Hi there")
factory = NotificationTest.MyProtocolFactory(listener)
protocol = factory.buildProtocol(None)
transport = proto_helpers.StringTransport()
def play_dumb(*args): pass
setattr(transport, "setTcpNoDelay", play_dumb)
protocol.makeConnection(transport)
self.protocol, self.transport, self.publisher, self.fingerprint_handler = protocol, transport, publisher, fingerprint_handler
def test_onMessage(self):
#Following 2 lines are the problematic part. Here you are manipulating explicitly a hidden state which your implementation should not be concerned with!
self.protocol.state = WebSocketProtocol.STATE_OPEN
self.protocol.websocket_version = 13
self.protocol.onMessage("Whatever")
self.assertEqual(self.transport.value()[2:], 'Whatever')
def test_push(self):
#Following 2 lines are the problematic part. Here you are manipulating explicitly a hidden state which your implementation should not be concerned with!
self.protocol.state = WebSocketProtocol.STATE_OPEN
self.protocol.websocket_version = 13
self.publisher.publish("Hi there")
self.assertEqual(self.transport.value()[2:], 'Hi There')
As you might have noticed, using the StringTransport here is very cumbersome. You must have knowledge of the underline framework and bypass its state management, something you don't really want to do. Unfortunately autobahn does not provide a ready-to-use test object that would permit easy state manipulation and therefore my suggestion of using dummy servers and clients is still valid
Testing your server WITH network
The test provided shows how you can test server push, asserting that what your are getting is what you expect, and using also a hook on how to determine when to finish.
The server protocol
from twisted.trial.unittest import TestCase as TrialTest
from autobahn.websocket import WebSocketServerProtocol, WebSocketServerFactory, WebSocketClientProtocol, WebSocketClientFactory, connectWS, listenWS
from twisted.internet.defer import Deferred
from twisted.internet import task
START="START"
class TestServerProtocol(WebSocketServerProtocol):
def __init__(self):
#The publisher task simulates an event that triggers a message push
self.publisher = task.LoopingCall(self.send_stuff, "Hi there")
def send_stuff(self, msg):
#this method sends a message to the client
self.sendMessage(msg)
def _on_start(self):
#here we trigger the task to execute every second
self.publisher.start(1.0)
def onMessage(self, message, binary):
#According to this stupid protocol, the server starts sending stuff when the client sends a "START" message
#You can plug other commands in here
{
START : self._on_start
#Put other keys here
}[message]()
def onClose(self, wasClean, code, reason):
#After closing the connection, we tell the task to stop sending messages
self.publisher.stop()
The client protocol and factory
Next class is the client protocol. It basically tells the server to start pushing messages. It calls the close_condition on them to see if it is time to close the connection and as a last thing, it calls the assertion function on the messages it received to see if the test was successful or not
class TestClientProtocol(WebSocketClientProtocol):
def __init__(self, assertion, close_condition, timeout, *args, **kwargs):
self.assertion = assertion
self.close_condition = close_condition
self._received_msgs = []
from twisted.internet import reactor
#This is a way to set a timeout for your test
#in case you never meet the conditions dictated by close_condition
self.damocle_sword = reactor.callLater(timeout, self.sendClose)
def onOpen(self):
#After the connection has been established,
#you can tell the server to send its stuff
self.sendMessage(START)
def onMessage(self, msg, binary):
#Here you get the messages pushed from the server
self._received_msgs.append(msg)
#If it is time to close the connection
if self.close_condition(msg):
self.damocle_sword.cancel()
self.sendClose()
def onClose(self, wasClean, code, reason):
#Now it is the right time to check our test assertions
self.assertion.callback(self._received_msgs)
class TestClientProtocolFactory(WebSocketClientFactory):
def __init__(self, assertion, close_condition, timeout, **kwargs):
WebSocketClientFactory.__init__(self, **kwargs)
self.assertion = assertion
self.close_condition = close_condition
self.timeout = timeout
#This parameter needs to be forced to None to not leave the reactor dirty
self.openHandshakeTimeout = None
def buildProtocol(self, addr):
protocol = TestClientProtocol(self.assertion, self.close_condition, self.timeout)
protocol.factory = self
return protocol
The trial based test
class WebSocketTest(TrialTest):
def setUp(self):
port = 8088
factory = WebSocketServerFactory("ws://localhost:{}".format(port))
factory.protocol = TestServerProtocol
self.listening_port = listenWS(factory)
self.factory, self.port = factory, port
def tearDown(self):
#cleaning up stuff otherwise the reactor complains
self.listening_port.stopListening()
def test_message_reception(self):
#This is the test assertion, we are testing that the messages received were 3
def assertion(msgs):
self.assertEquals(len(msgs), 3)
#This class says when the connection with the server should be finalized.
#In this case the condition to close the connectionis for the client to get 3 messages
class CommunicationHandler(object):
msg_count = 0
def close_condition(self, msg):
self.msg_count += 1
return self.msg_count == 3
d = Deferred()
d.addCallback(assertion)
#Here we create the client...
client_factory = TestClientProtocolFactory(d, CommunicationHandler().close_condition, 5, url="ws://localhost:{}".format(self.port))
#...and we connect it to the server
connectWS(client_factory)
#returning the assertion as a deferred purely for demonstration
return d
This is obviously just an example, but as you can see I did not have to mess around with makeConnection or any transport explicitly

Distinguishing outgoing and incoming connections in twisted

I'm working on a P2P application based on the Twisted framework. As such I can have both incoming as well as outgoing connections. Is there a simple way to distinguish them? Currently I just create another Factory that marks the connection as outgoing and delegates all factory calls to the original factory, but there must be a simpler way.
class OutgoingProtocolFactory(MyProtocolFactory):
"""
A rather simple factory that is used to earmark connections as outgoing.
"""
def __init__(self, parentFactory):
self.factory = parentFactory
def buildProtocol(self, addr):
connection = MyProtocolFactory.buildProtocol(self.factory, addr)
connection.factory = self.factory
connection.incoming = False
return connection
def clientConnectionFailed(self, connector, reason):
self.factory.clientConnectionFailed(connector, reason)
def clientConnectionLost(self, connector, reason):
self.factory.clientConnectionLost(connector, reason)
Any thoughts?

Each protocol instance constructed by any of the factories included in Twisted get a free factory attribute that refers back to the factory that created them.
By convention, accepted (server) connections have a ServerFactory (or subclass) instance as their factory and connected (client) connections have a ClientFactory (or subclass) instance.
However, if you really want your code to be completely general, you can't rely on this, and it's better to keep track of it yourself, as it sounds like you're currently doing.
In case the code you currently have for forwarding calls from one factory to another is more cumbersome than necessary, here's an example of how simple it could be:
from twisted.internet.protocol import ClientFactory
from twisted.protocols.policies import WrappingFactory
IN, OUT = 1, 2
class YourFactory(ClientFactory):
# Your application logic
...
class Incoming(WrappingFactory):
direction = IN
def buildProtocol(self, addr):
protocol = self.wrappedFactory.buildProtocol(addr)
protocol.direction = self.direction
return protocol
class Outgoing(Incoming):
direction = OUT
yourFactory = YourFactory(...)
reactor.listenTCP(0, Incoming(yourFactory))
reactor.connectTCP('example.com', 1234, Outgoing(yourFactory))
If that's still too complicated for you, then I don't know what to say. I'm not clever enough to think of a simpler solution.

How to properly trigger a python twisted transport?

I have been asked to write a class that connects to a server, asynchronously sends the server various commands, and then provides the returned data to the client. I've been asked to do this in Python, which is a new language to me. I started digging around and found the Twisted framework which offers some very nice abstractions (Protocol, ProtocolFactory, Reactor) that do a lot of the things that I would have to do if I would roll my own socket-based app. It seems like the right choice given the problem that I have to solve.
I've looked through numerous examples on the web (mostly Krondo), but I still haven't seen a good example of creating a client that will send multiple commands across the wire and I maintain the connection I create. The server (of which I have no control over), in this case, doesn't disconnect after it sends the response. So, what's the proper way to design the client so that I can tickle the server in various ways?
Right now I do this:
class TestProtocol(Protocol)
def connectionMade(self):
self.transport.write(self.factory.message)
class TestProtocolFactory(Factory):
message = ''
def setMessage(self, msg):
self.message = msg
def main():
f = TestProtocolFactory()
f.setMessage("my message")
reactor.connectTCP(...)
reactor.run()
What I really want to do is call self.transport.write(...) via the reactor (really, call TestProtocolFactory::setMessage() on-demand from another thread of execution), not just when the connection is made.

Depends. Here are some possibilities:
I'm assuming
Approach 1. You have a list of commands to send the server, and for some reason can't do them all at once. In that case send a new one as the previous answer returns:
class proto(parentProtocol):
def stringReceived(self, data):
self.handle_server_response(data)
next_command = self.command_queue.pop()
# do stuff
Approach 2. What you send to the server is based on what the server sends you:
class proto(parentProtocol):
def stringReceived(self, data):
if data == "this":
self.sendString("that")
elif data == "foo":
self.sendString("bar")
# and so on
Approach 3. You don't care what the server sends to, you just want to periodically send some commands:
class proto(parentProtocol):
def callback(self):
next_command = self.command_queue.pop()
# do stuff
def connectionMade(self):
from twisted.internet import task
self.task_id = task.LoopingCall(self.callback)
self.task_id.start(1.0)
Approach 4: Your edit now mentions triggering from another thread. Feel free to check the twisted documentation to find out if proto.sendString is threadsafe. You may be able to call it directly, but I don't know. Approach 3 is threadsafe though. Just fill the queue (which is threadsafe) from another thread.
Basically you can store any amount of state in your protocol; it will stay around until you are done. The you either send commands to the server as a response to it's messages to you, or you set up some scheduling to do your stuff. Or both.

You may want to use a Service.
Services are pieces of functionality within a Twisted app which are started and stopped, and are nice abstractions for other parts of your code to interact with. For example, in this case you might have a SayStuffToServerService (I know, terrible name, but without knowing more about its job it was the best I could do here :) ) that exposed something like this:
class SayStuffToServerService:
def __init__(self, host, port):
# this is the host and port to connect to
def sendToServer(self, whatToSend):
# send some line to the remote server
def startService(self):
# call me before using the service. starts outgoing connection efforts.
def stopService(self):
# clean reactor shutdowns should call this method. stops outgoing
# connection efforts.
(That might be all the interface you need, but it should be fairly clear where you can add things to this.)
The startService() and stopService() methods here are just what Twisted's Services expose. And helpfully, there is a premade Twisted Service which acts like a TCP client and takes care of all the reactor stuff for you. It's twisted.application.internet.TCPClient, which takes arguments for a remote host and port, along with a ProtocolFactory to take care of handling the actual connection attempt.
Here is the SayStuffToServerService, implemented as a subclass of TCPClient:
from twisted.application import internet
class SayStuffToServerService(internet.TCPClient):
factoryclass = SayStuffToServerProtocolFactory
def __init__(self, host, port):
self.factory = self.factoryclass()
internet.TCPClient.__init__(self, host, port, self.factory)
def sendToServer(self, whatToSend):
# we'll do stuff here
(See below for the SayStuffToServerProtocolFactory.)
Using this Service architecture is convenient in a lot of ways; you can group Services together in one container, so that they all get stopped and started as one when you have different parts of your app that you want active. It may make good sense to implement other parts of your app as separate Services. You can set Services as child services to application- the magic name that twistd looks for in order to know how to initialize, daemonize, and shut down your app. Actually yes, let's add some code to do that now.
from twisted.application import service
...
application = service.Application('say-stuff')
sttss = SayStuffToServerService('localhost', 65432)
sttss.setServiceParent(service.IServiceCollection(application))
That's all. Now when you run this module under twistd (i.e., for debugging, twistd -noy saystuff.py), that application will be started under the right reactor, and it will in turn start the SayStuffToServerService, which will start a connection effort to localhost:65432, which will use the service's factory attribute to set up the connection and the Protocol. You don't need to call reactor.run() or attach things to the reactor yourself anymore.
So we haven't implemented SayStuffToServerProtocolFactory yet. Since it sounds like you would prefer that your client reconnect if it has lost the connection (so that callers of sendToServer can usually just assume that there's a working connection), I'm going to put this protocol factory on top of ReconnectingClientFactory.
from twisted.internet import protocol
class SayStuffToServerProtocolFactory(protocol.ReconnectingClientFactory):
_my_live_proto = None
protocol = SayStuffToServerProtocol
This is a pretty nice minimal definition, which will keep trying to make outgoing TCP connections to the host and port we specified, and instantiate a SayStuffToServerProtocol each time. When we fail to connect, this class will do nice, well-behaved exponential backoff so that your network doesn't get hammered (you can set a maximum wait time). It will be the responsibility of the Protocol to assign to _my_live_proto and call this factory's resetDelay() method, so that exponential backoff will continue to work as expected. And here is that Protocol now:
class SayStuffToServerProtocol(basic.LineReceiver):
def connectionMade(self):
# if there are things you need to do on connecting to ensure the
# connection is "all right" (maybe authenticate?) then do that
# before calling:
self.factory.resetDelay()
self.factory._my_live_proto = self
def connectionLost(self, reason):
self.factory._my_live_proto = None
del self.factory
def sayStuff(self, stuff):
self.sendLine(stuff)
def lineReceived(self, line):
# do whatever you want to do with incoming lines. often it makes sense
# to have a queue of Deferreds on a protocol instance like this, and
# each incoming response gets sent to the next queued Deferred (which
# may have been pushed on the queue after sending some outgoing
# message in sayStuff(), or whatever).
pass
This is implemented on top of twisted.protocols.basic.LineReceiver, but would work as well with any other sort of Protocol, in case your protocol isn't line-oriented.
The only thing left is hooking up the Service to the right Protocol instance. This is why the Factory keeps a _my_live_proto attribute, which should be set when a connection is successfully made, and cleared (set to None) when that connection is lost. Here's the new implementation of SayStuffToServerService.sendToServer:
class NotConnectedError(Exception):
pass
class SayStuffToServerService(internet.TCPClient):
...
def sendToServer(self, whatToSend):
if self.factory._my_live_proto is None:
# define here whatever behavior is appropriate when there is no
# current connection (in case the client can't connect or
# reconnect)
raise NotConnectedError
self.factory._my_live_proto.sayStuff(whatToSend)
And now to tie it all together in one place:
from twisted.application import internet, service
from twisted.internet import protocol
from twisted.protocols import basic
class SayStuffToServerProtocol(basic.LineReceiver):
def connectionMade(self):
# if there are things you need to do on connecting to ensure the
# connection is "all right" (maybe authenticate?) then do that
# before calling:
self.factory.resetDelay()
self.factory._my_live_proto = self
def connectionLost(self, reason):
self.factory._my_live_proto = None
del self.factory
def sayStuff(self, stuff):
self.sendLine(stuff)
def lineReceived(self, line):
# do whatever you want to do with incoming lines. often it makes sense
# to have a queue of Deferreds on a protocol instance like this, and
# each incoming response gets sent to the next queued Deferred (which
# may have been pushed on the queue after sending some outgoing
# message in sayStuff(), or whatever).
pass
class SayStuffToServerProtocolFactory(protocol.ReconnectingClientFactory):
_my_live_proto = None
protocol = SayStuffToServerProtocol
class NotConnectedError(Exception):
pass
class SayStuffToServerService(internet.TCPClient):
factoryclass = SayStuffToServerProtocolFactory
def __init__(self, host, port):
self.factory = self.factoryclass()
internet.TCPClient.__init__(self, host, port, self.factory)
def sendToServer(self, whatToSend):
if self.factory._my_live_proto is None:
# define here whatever behavior is appropriate when there is no
# current connection (in case the client can't connect or
# reconnect)
raise NotConnectedError
self.factory._my_live_proto.sayStuff(whatToSend)
application = service.Application('say-stuff')
sttss = SayStuffToServerService('localhost', 65432)
sttss.setServiceParent(service.IServiceCollection(application))
Hopefully that gives enough of a framework with which to start. There is sometimes a lot of plumbing to do to handle client disconnections just the way you want, or to handle out-of-order responses from the server, or handle various sorts of timeout, canceling pending requests, allowing multiple pooled connections, etc, etc, but this should help.

The twisted framework is event-based programming; and by nature, its method is all called in async, and result is get by defer object.
The framework's nature is approprivate for protocol developing, just you have to change your minding from traditional sequential programming. The Protocol class is like a finite state machine with events like: connection make, connection lost, receive data.
You can convert your client code into FSM and then will be easily to fit into the Protocol class.
Below is an rough example of what I want to express. A bit of rouge, but this is i can provide now:
class SyncTransport(Protocol):
# protocol
def dataReceived(self, data):
print 'receive data', data
def connectionMade(self):
print 'i made a sync connection, wow'
self.transport.write('x')
self.state = I_AM_LIVING
def connectionLost(self):
print 'i lost my sync connection, sight'
def send(self, data):
if self.state == I_AM_LIVING:
if data == 'x':
self.transport.write('y')
if data == 'Y':
self.transport.write('z')
self.state = WAITING_DEAD
if self.state == WAITING_DEAD:
self.transport.close()

How to deliver instance of object to instance of SocketServer.BaseRequestHandler?

This is problem.
My primary work is : deliver "s" object to "handle" method in TestRequestHandler class.
My first step was : deliver "s" object through "point" method to TestServer class, but here im stuck. How to deliver "s" object to TestRequestHandler? Some suggestions?
import threading
import SocketServer
from socket import *
class TestRequestHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
SocketServer.BaseRequestHandler.__init__(self, request, client_address, server)
return
def setup(self):
return SocketServer.BaseRequestHandler.setup(self)
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
def finish(self):
return SocketServer.BaseRequestHandler.finish(self)
class TestServer(SocketServer.TCPServer):
def __init__(self, server_address, handler_class=TestRequestHandler):
print "__init__"
SocketServer.TCPServer.__init__(self, server_address, handler_class)
return
def point(self,obj):
self.obj = obj
print "point"
def server_activate(self):
SocketServer.TCPServer.server_activate(self)
return
def serve_forever(self):
print "serve_forever"
while True:
self.handle_request()
return
def handle_request(self):
return SocketServer.TCPServer.handle_request(self)
if __name__ == '__main__':
s = socket(AF_INET, SOCK_STREAM)
address = ('localhost', 6666)
server = TestServer(address, TestRequestHandler)
server.point(s)
t = threading.Thread(target=server.serve_forever())
t.setDaemon(True)
t.start()

If I understand correctly, I think you perhaps are misunderstanding how the module works. You are already specifying an address of 'localhost:6666' for the server to bind on.
When you start the server via your call to serve_forever(), this is going to cause the server to start listening to a socket on localhost:6666.
According to the documentation, that socket is passed to your RequestHandler as the 'request' object. When data is received on the socket, your 'handle' method should be able to recv/send from/to that object using the documented socket API.
If you want a further abstraction, it looks like your RequestHandler can extend from StreamRequestHandler and read/write to the socket using file-like objects instead.
The point is, there is no need for you to create an additional socket and then try to force your server to use the new one instead. Part of the value of the SocketServer module is that it manages the lifecycle of the socket for you.
On the flip side, if you want to test your server from a client's perspective, then you would want to create a socket that you can read/write your client requests on. But you would never pass this socket to your server, per se. You would probably do this in a completely separate process and test your server via IPC over the socket.
Edit based on new information
To get server A to open a socket to server B when server A receives data one solution is to simply open a socket from inside your RequestHandler. That said, there are likely some other design concerns that you will need to address based on the requirements of your service.
For example, you may want to use a simple connection pool that say opens a few sockets to server B that server A can use like a resource. There may already be some libraries in Python that help with this.
Given your current design, your RequestHandler has access to the server as a member variable so you could do something like this:
class TestServer(SocketServer.TCPServer):
def point (self, socketB):
self.socketB = socketB # hold serverB socket
class TestRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
if (data):
self.request.send(data)
print data
self.server.socketB ... # Do whatever with the socketB
But like I said, it may be better for you to have some sort of connection pool or other object that manages your server B socket such that your server A handler can just acquire/release the socket as incoming requests are handled.
This way you can better deal with conditions where server B breaks the socket. Your current design wouldn't be able to handle broken sockets very easily. Just some thoughts...

If the value of s is set once, and not reinitialized - you could make it a class variable as opposed to an instance variable of TestServer, and then have the handler retrieve it via a class method of TestServer in the handler's constructor.
eg: TestServer._mySocket = s

Ok, my main task is this. Construction of the listening server (A-server - localhost, 6666) which during start will open "hard" connection to the different server (B-server - localhost, 7777).
When the customer send data to the A-server this (A-server) sends data (having that hard connection to the B-server) to B-server, the answer receives from the B-server to A-server and answer sends to the customer.
Then again : the customer sends data, A-server receives them, then sends to the B-server, the answer receives data from the B-server and A-server send data to the customer.
And so round and round. The connection to the B-server is closes just when the server A will stop.
All above is the test of making this.