Develop Reference

How to avoid ReactorNotRestartable in Autobahn Python

I have a python based page which recieves data by POST, which is then forwarded to the Crossbar server using Autobahn (Wamp). It works well the first 1-2 times but when it's called again after that it throws ReactorNotRestartable.
Now, I need this to work whichever way possible, either by reusing this "Reactor" based on a conditional check or by stopping it properly after every run. (The first one would be preferable because it might reduce the execution time)
Thanks for your help!
Edit:
This is in a webpage (Django View) so it needs to run as many times as the page is loaded/data is sent to it via POST.
from twisted.internet import reactor
from twisted.internet.defer import inlineCallbacks
from twisted.internet.endpoints import TCP4ClientEndpoint
from twisted.application.internet import ClientService
from autobahn.wamp.types import ComponentConfig
from autobahn.twisted.wamp import ApplicationSession, WampWebSocketClientFactory
class MyAppSession(ApplicationSession):
def __init__(self, config):
ApplicationSession.__init__(self, config)
def onConnect(self):
self.join(self.config.realm)
def onChallenge(self, challenge):
pass
#inlineCallbacks
def onJoin(self, details):
yield self.call('receive_data', data=message)
yield self.leave()
def onLeave(self, details):
self.disconnect()
def onDisconnect(self):
reactor.stop()
message = "data from POST[]"
session = MyAppSession(ComponentConfig('realm_1', {}))
transport = WampWebSocketClientFactory(session, url='ws://127.0.0.1:8080')
endpoint = TCP4ClientEndpoint(reactor, '127.0.0.1', 8080)
service = ClientService(endpoint, transport)
service.startService()
reactor.run()
I figured out a probably hacky-and-not-so-good way by using multiprocessing and putting reactor.stop() inside onJoin() right after the function call. This way I don't have to bother with the "twisted running in the main thread" thing because its process gets killed as soon as my work is done.
Is there a better way?

Twisted Python factory methods aren't getting called when using reactor wrappers

I have a simple client / server setup. Here's the client code:
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.endpoints import TCP4ClientEndpoint
class MyProtocol(protocol.Protocol):
def connectionMade(self):
print "Hello!"
def dataReceived(self, data):
print data
class MyProtocolFactory(protocol.ClientFactory):
def startedConnecting(self, connector):
print "Starting to connect!"
def buildProtocol(self, addr):
return MyProtocol()
def clientConnectionLost(self, connector, reason):
print "Lost connection, reason = %s" % reason
def clientConnectionFailed(self, connector, reason):
print "Connection failed, reason = %s" % reason
reactor.stop()
endpoint = TCP4ClientEndpoint(reactor, "127.0.0.1", 54321, timeout=5)
endpoint.connect(MyProtocolFactory())
reactor.run()
For some reason, this client will connect to the server and the protocol works correctly (I can see "Hello!" printed, along with data sent by the server upon a successful connection), but it won't call any of the protocol factory methods. startedConnecting doesn't get called, and if I stop the server, I don't see clientConnectionLost get called. If I try to run the client before the server has started, I would also expect to see clientConnectionFailed get called.
Here's the strange part...If I change the last 3 lines in the code above to the following:
reactor.connectTCP("127.0.0.1", 54321, MyProtocolFactory())
reactor.run()
Then everything works as expected, and all methods get called in all of the cases outlined above.
My understanding of endpoints is that they wrap "connectTCP" (among others) with additional behaviour, but I can't figure out why it works in the second code snippet, but not the first.
Any ideas?

The client endpoint interface does not call the extra connection-state notification methods of ClientFactory.
So, while endpoints do in some sense "wrap" connectTCP et al, it's not true that they have the exact same behavior as using those lower-level methods.
With endpoints, the factory's job is to provide protocol instances. The factory is no longer responsible for other aspects of connection management.

Additional note to supplement my discussion above:
If you’ve used ClientFactory before, keep in mind that the connect method takes a Factory, not a ClientFactory. Even if you pass a ClientFactory to endpoint.connect, its clientConnectionFailed and clientConnectionLost methods will not be called. In particular, clients that extend ReconnectingClientFactory won’t reconnect. The next section describes how to set up reconnecting clients on endpoints.
From the endpoint docs found here: http://twistedmatrix.com/documents/current/core/howto/endpoints.html

Tornado adbapi callback not running

I'm afraid I'm finding it difficult to work with the adbapi interface for sqlite3 ConnectionPools in twisted.
I've initialized my pool like this in a file I've named db.py:
from twisted.enterprise import adbapi
pool = adbapi.ConnectionPool("sqlite3", db=config.db_file)
pool.start()
def last(datatype, n):
cmd = "SELECT * FROM %s ORDER BY Timestamp DESC LIMIT %i" % (datatype, n)
return pool.runQuery(cmd)
Then, I'm importing db.py and using it inside a particular route handler. Unfortunately, it appears the callback is never triggered. datatype is printed, but response is never printed.
class DataHandler(tornado.web.RequestHandler):
#tornado.web.asynchronous
def get(self, datatype):
print datatype
data = db.last(datatype, 500)
data.addCallback(self.on_response)
def on_response(self, response):
print response
self.write(json.dumps(response))
self.finish()
Any ideas?

Mixing Tornado and Twisted requires special attention. Try this, as the first lines executed in your entire program:
import tornado.platform.twisted
tornado.platform.twisted.install()
Then, to start your server:
tornado.ioloop.IOLoop.current().start()
What's happening now is, you start the Tornado IOLoop but you never start the Twisted Reactor.
Your Twisted SQLite connection begins an IO operation when you run your query, but since the Reactor isn't running, the operation never completes. In order for the IOLoop and the Reactor to share your process you must run one of them on top of the other. Tornado provides a compatibility layer that allows you to do that.

txredisapi subscribe and listen async

I'm working in a project with Python, Twisted and Redis. So the team decided to use txredisapi for the communication between the Python modules and Redis. This project does a lot of different things and we need to subscribe to several channels for listen the messages sent by Redis without the other functionalities stops (asynchronously).
Can one execution handle all the work and listen the messages sent by Redis at the same time or must we separate and execute the code in differents flows?
We use the following code for listen the messages:
import txredisapi as redis
class RedisListenerProtocol(redis.SubscriberProtocol):
def connectionMade(self):
self.subscribe("channelName")
def messageReceived(self, pattern, channel, message):
print "pattern=%s, channel=%s message=%s" %(pattern, channel, message)
def connectionLost(self, reason):
print "lost connection:", reason
class RedisListenerFactory(redis.SubscriberFactory):
maxDelay = 120
continueTrying = True
protocol = RedisListenerProtocol
We try to listen the messages with:
self.connRedisChannels = yield redis.ConnectionPool()
I'm interested to know how can I specify that the Connection must use the "RedisListenerFactory", then I guess that the function "messageReceived" will be fired when a message arrives.
Any suggestions, example or correction will be apreciated.
Thanks!
The following code solves the problem:
from twisted.internet.protocol import ClientCreator
from twisted.internet import reactor
defer = ClientCreator(reactor, RedisListenerProtocol).connectTCP(HOST, PORT)
Thanks to Philippe T. for the help.

If you want to use directly the redis.Connection() may be you can do this before:
redis.SubscriberFactory.protocol = RedisListenerProtocol
the package make internal call to is factory for connection.
other way is to rewrite *Connection class and make*Connection factory to use your factory.
to make the connection on other part of your code you can do something like this :
from twisted.internet.protocol import ClientCreator
from twisted.internet import reactor
# some where :
defer = ClientCreator(reactor, RedisListenerProtocol).connectTCP(__HOST__, __PORT__)
# the defer will have your client when the connection is done

handler for python HTTPServer that uses threads

id like to write a handler for the standalone server coming with pywebsocket (http://code.google.com/p/pywebsocket) that uses a thread.
in the example coming with pywebsocket the handler is just a file with a function:
def web_socket_transfer_data(request):
while True:
line = request.ws_stream.receive_message()
if line is None:
return
request.ws_stream.send_message(line)
if line == _GOODBYE_MESSAGE:
return
ive tried to add a thread:
class _Stub(threading.Thread):
def __init__ (self):
threading.Thread.__init__(self)
self._test = 0
def run(self):
while True:
time.sleep(5)
self._test = self._test + 1
but the server crashes without any comment...
so how is this done?
thanks for any pointers.

The standalone server isn't designed to receive messages non-blocking. From the documentation of the class "MessageReceiver" in msgutil.py (at least not when using SSL):
This class receives messages from the client.
This class provides three ways to receive messages: blocking,
non-blocking, and via callback. Callback has the highest precedence.
Note: This class should not be used with the standalone server for wss
because pyOpenSSL used by the server raises a fatal error if the socket
is accessed from multiple threads.