Develop Reference

How to be sure that server received full data fron the client (and if not - resend it) using python - twisted?

I'm new in twisted and in web programming itself.
What I want is to implement server and client (and pass some string data between them). The problem is, that the data is important, so I want client to resend it to server in case of loosing connection or in case it wasn't full on the server side. But I don't want to resend it in case it was fully received, so I don't think that just adding the logic to def connectionLost() will do. How could this be done?
This is my server (just the same as in doc examples) and (after ------------) is the client:
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor
class ConsoleReceiver(Protocol):
def connectionMade(self):
self.transport.write(
"Welcome!\n".encode('utf-8'))
def dataReceived(self, data):
self.transport.write('Data received, thanks'.encode('utf-8'))
data = data.decode('ascii')
print(data)
self.transport.loseConnection()
class ServerFactory(Factory):
def buildProtocol(self, addr):
return ConsoleReceiver()
if __name__ == '__main__':
endpoint = TCP4ServerEndpoint(reactor, 21285)
endpoint.listen(ServerFactory())
reactor.run()```
-----------------------------------------------------------------------------
#some_flask_route.route('/test/')
urgent_information = <getting some urgent information from db with flask_sqlalchemy>
reactor.connectTCP('localhost', 21285, ShiftInfoSenderFactory(urgent_information))
reactor.run()
class ShiftInfoSender(Protocol):
def __init__(self, urgent_information):
self.urgent_information = urgent_information
def connectionMade(self):
self.transport.write('\nInformation to be red:{}\n'.format(self.urgent_information[1]).encode('utf-8'))
for i in self.urgent_information[2]:
self.transport.write('Some unpacked information: {}'.format(i).encode('utf-8')
def dataReceived(self, data):
print(data.decode('ascii'))
class ShiftInfoSenderFactory(ClientFactory):
def __init__(self, urgent_information):
self.urgent_information = urgent_information
def startedConnecting(self, connector):
print('Started to connect')
def buildProtocol(self, addr):
print('Connected')
return ShiftInfoSender(self.urgent_information)
def clientConnectionLost(self, connector, reason):
print('Lost connection. Reason:', reason)
def clientConnectionFailed(self, connector, reason):
print('Connection failed. Reason:', reason) ```

Send in async mode data using twisted in python

I want to send data to server in async mode (whenever I type something in console) not only one time as the below code do. Is there any protocol function within twisted library that can handle this? In the following find the code that only send a message where the connection is established. On the other hand I can receive data in async mode via the function dataReceived. Is any function that will allow me to send messages in async mode as dataReceived is for receiving.
from twisted.internet import reactor, protocol
class QuoteProtocol(protocol.Protocol):
def __init__(self, factory):
self.factory = factory
def connectionMade(self):
self.sendQuote()
def sendQuote(self):
self.message(self.factory.quote)
def dataReceived(self, data):
print "Received quote:", data
#self.transport.loseConnection()
class QuoteClientFactory(protocol.ClientFactory):
def __init__(self, quote):
self.quote = quote
def buildProtocol(self, addr):
return QuoteProtocol(self)
def clientConnectionFailed(self, connector, reason):
print 'connection failed:', reason.getErrorMessage()
reactor.stop()
def clientConnectionLost(self, connector, reason):
print 'connection lost:', reason.getErrorMessage()
reactor.stop()
message = "hello world"
reactor.connectTCP('127.0.0.1', 5000, QuoteClientFactory())
reactor.run()

If you want to asynchronously process keystrokes from a terminal, you might have a look at TerminalProtocol: http://twistedmatrix.com/documents/9.0.0/api/twisted.conch.insults.insults.TerminalProtocol.html

Create client/server with Twisted

I'm trying to create a client/server using Twisted.
I'd like to create a daemon, which will be connected to another server as a client and act as a server for other clients.
I've writen something like that which I think describes my problem:
server = sys.argv[1]
control_port = 8001
class ControlClient(protocol.Protocol):
def makeConnection(self, transport):
[some code here -snip-]
self.firstOrder(order, transport)
def firstOrder(self, action, transport):
self.t = transport
self.t.write(action + "\0")
def sendOrder(self, action):
self.t.write(action + "\0")
def dataReceived(self, data):
[some code here -snip-]
[HERE I WANT TO SEND DATA TO CLIENTS CONNECTED TO MY TWISTED SERVER, USING CONTROL SERVER]
class ControlServer(ControlClient):
def dataReceived(self, data):
print "client said " + data
def makeConnection(self, transport):
self.t = transport
self.t.write("make connection")
print "make connection"
def sendData(self, data):
self.t.write("data")
class ClientFactory(protocol.ClientFactory):
protocol = ControlClient
def clientConnectionFailed(self, connector, reason):
print "Connection failed - goodbye!"
reactor.stop()
def clientConnectionLost(self, connector, reason):
print "Connection lost - goodbye!"
reactor.stop()
class ServerFactory(protocol.ServerFactory):
protocol = ControlServer
def main():
c = ClientFactory()
reactor.connectTCP(server, control_port, c)
s = ServerFactory()
reactor.listenTCP(9000, s)
reactor.run()
if __name__ == '__main__':
main()
As you can see, I'd like to send (as a server) some data received (as a client). My problem is of course my ServerControl is not instantiated in my ClientControl so I don't have access to transport which is required to send data to clients.

The only thing you seem to be missing is that you can keep a list of your client connections and make that list available to the code that's trying to send out data to all the clients.
There's an example of this in the Twisted FAQ: http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinputononeconnectionresultinoutputonanother
That example only has one factory, but the idea is the same. To handle your case with two factories, just give one factory a reference to the other.

Sending data from one Protocol to another Protocol in Twisted?

One of my protocols is connected to a server, and with the output of that I'd like to send it to the other protocol.
I need to access the 'msg' method in ClassA from ClassB but I keep getting: exceptions.AttributeError: 'NoneType' object has no attribute 'write'
Actual code:
from twisted.words.protocols import irc
from twisted.internet import protocol
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.internet import reactor
IRC_USERNAME = 'xxx'
IRC_CHANNEL = '#xxx'
T_USERNAME = 'xxx'
T_PASSWORD = md5.new('xxx').hexdigest()
class ircBot(irc.IRCClient):
def _get_nickname(self):
return self.factory.nickname
nickname = property(_get_nickname)
def signedOn(self):
self.join(self.factory.channel)
print "Signed on as %s." % (self.nickname,)
def joined(self, channel):
print "Joined %s." % (channel,)
def privmsg(self, user, channel, msg):
if not user:
return
who = "%s: " % (user.split('!', 1)[0], )
print "%s %s" % (who, msg)
class ircBotFactory(protocol.ClientFactory):
protocol = ircBot
def __init__(self, channel, nickname=IRC_USERNAME):
self.channel = channel
self.nickname = nickname
def clientConnectionLost(self, connector, reason):
print "Lost connection (%s), reconnecting." % (reason,)
connector.connect()
def clientConnectionFailed(self, connector, reason):
print "Could not connect: %s" % (reason,)
class SomeTClass(Protocol):
def dataReceived(self, data):
if data.startswith('SAY'):
data = data.split(';', 1)
# RAGE
#return self.ircClient.msg(IRC_CHANNEL, 'test')
def connectionMade(self):
self.transport.write("mlogin %s %s\n" % (T_USERNAME, T_PASSWORD))
class tClientFactory(ClientFactory):
def startedConnecting(self, connector):
print 'Started to connect.'
def buildProtocol(self, addr):
print 'Connected.'
return t()
def clientConnectionLost(self, connector, reason):
print 'Lost connection. Reason:', reason
def clientConnectionFailed(self, connector, reason):
print 'Connection failed. Reason:', reason
if __name__ == "__main__":
#chan = sys.argv[1]
reactor.connectTCP('xxx', 6667, ircBotFactory(IRC_CHANNEL) )
reactor.connectTCP('xxx', 20184, tClientFactory() )
reactor.run()
Any ideas please? :-)

Twisted FAQ:
How do I make input on one connection
result in output on another?
This seems like it's a Twisted
question, but actually it's a Python
question. Each Protocol object
represents one connection; you can
call its transport.write to write some
data to it. These are regular Python
objects; you can put them into lists,
dictionaries, or whatever other data
structure is appropriate to your
application.
As a simple example, add a list to
your factory, and in your protocol's
connectionMade and connectionLost, add
it to and remove it from that list.
Here's the Python code:
from twisted.internet.protocol import Protocol, Factory
from twisted.internet import reactor
class MultiEcho(Protocol):
def connectionMade(self):
self.factory.echoers.append(self)
def dataReceived(self, data):
for echoer in self.factory.echoers:
echoer.transport.write(data)
def connectionLost(self, reason):
self.factory.echoers.remove(self)
class MultiEchoFactory(Factory):
protocol = MultiEcho
def __init__(self):
self.echoers = []
reactor.listenTCP(4321, MultiEchoFactory())
reactor.run()

Testing twisted application - Load client

I've written a Twisted based server and I'd like to test it using twisted as well.
But I'd like to write a load test starting a bunch of request at the same time.
But I believe that I didn't get the concepts of Twisted, mainly client side, because I'm stucked with this problem:
from twisted.internet import reactor, protocol
from threading import Thread
from twisted.protocols.basic import LineReceiver
__author__="smota"
__date__ ="$30/10/2009 17:17:50$"
class SquitterClient(LineReceiver):
def connectionMade(self):
self.sendLine("message from " % threading.current_thread().name);
pass
def connectionLost(self, reason):
print "connection lost"
def sendMessage(self, msg):
for m in [ "a", "b", "c", "d", "e"]:
self.sendLine(msg % " - " % m);
class SquitterClientFactory(protocol.ClientFactory):
protocol = SquitterClient
def clientConnectionFailed(self, connector, reason):
print "Connection failed - goodbye!"
reactor.stop()
def clientConnectionLost(self, connector, reason):
print "Connection lost - goodbye!"
reactor.stop()
def createAndRun():
f = SquitterClientFactory()
reactor.connectTCP("localhost", 4010, f)
reactor.run(installSignalHandlers=0)
# this connects the protocol to a server runing on port 8000
def main():
for n in range(0,10):
th=Thread(target=createAndRun)
th.start()
# this only runs if the module was *not* imported
if __name__ == '__main__':
main()
socket_client.py:35:
DeprecationWarning: Reactor already
running! This behavior is deprecated
since Twisted 8.0
reactor.run(installSignalHandlers=0)
What am I missing?
How to test it?
Thank you,
Samuel

The direct cause for your failure is that you attemp to call run() on the reactor multiple times. You are supposed to ever only call run() once. I think you are expecting to have multiple reactors, each in its own thread, but actually you only have one. The bad thing is that having multiple reactors is difficult or impossible - the good thing is that it's also unnecessary. In fact you don't even need multiple threads. You can multiplex multiple client connections in one reactor almost as easily as you can listen for multiple connections.
Modifying your sample code, something like the following should work. The key idea is that you don't need multiple reactors to do things concurrently. The only thing that could ever be concurrent with the regular Python implementation is I/O anyway.
from twisted.internet import reactor, protocol
from twisted.protocols.basic import LineReceiver
__author__="smota"
__date__ ="$30/10/2009 17:17:50$"
class SquitterClient(LineReceiver):
def connectionMade(self):
self.messageCount = 0
# The factory provides a reference to itself, we'll use it to enumerate the clients
self.factory.n += 1
self.name = "Client %d" %self.factory.n
# Send initial message, and more messages a bit later
self.sendLine("Client %s starting!" % self.name);
reactor.callLater(0.5, self.sendMessage, "Message %d" %self.messageCount)
def connectionLost(self, reason):
print "connection lost"
def sendMessage(self, msg):
for m in [ "a", "b", "c", "d", "e"]:
self.sendLine("Copy %s of message %s from client %s!" % (m, msg, self.name))
if self.factory.stop:
self.sendLine("Client %s disconnecting!" % self.name)
self.transport.loseConnection()
else:
self.messageCount += 1
reactor.callLater(0.5, self.sendMessage, "Message %d" %self.messageCount)
class SquitterClientFactory(protocol.ClientFactory):
protocol = SquitterClient
def __init__(self):
self.n = 0
self.stop = False
def stopTest():
self.stop = True
def clientConnectionFailed(self, connector, reason):
print "Connection failed - goodbye!"
def clientConnectionLost(self, connector, reason):
print "Connection lost - goodbye!"
# this connects the protocol to a server running on port 8000
def main():
# Create 10 clients
f = SquitterClientFactory()
for i in range(10):
reactor.connectTCP("localhost", 8000, f)
# Schedule end of test in 10 seconds
reactor.callLater(10, f.stopTest)
# And let loose the dogs of war
reactor.run()
# this only runs if the module was *not* imported
if __name__ == '__main__':
main()