i have a problem with my progam using socket and thread.
I have made a socket server who add client in a thread, but the client thread never start...
here is my code:
socket server
import socket, threading, logging, sys
from client_thread import ClientThread
class SocketServer:
CLIENTS = list()
def __init__(self, server_ip, server_port, max_connections):
try:
self.tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.tcpsock.bind((server_ip, server_port))
self.tcpsock.listen(10)
logging.info('Socket server successfully started !')
except Exception as e:
logging.error(format(e))
def start(self):
from src.realmserver.core.hetwan import EmulatorState, Core
while (Core.STATE == EmulatorState.IN_RUNNING):
try:
(clientsock, (ip, port)) = self.tcpsock.accept()
new_client = threading.Thread(target=ClientThread, args=[len(self.CLIENTS), ip, port, clientsock])
self.CLIENTS.append(new_client)
new_client.start()
except Exception as e:
print format(e)
for client in self.CLIENTS:
client.join()
and client thread
import logging, string, random
class ClientThread:
def __init__(self, client_id, client_ip, client_port, socket):
self.client_id = client_id
self.client_ip = client_ip
self.client_port = client_port
self.socket = socket
logging.debug('(%d) Client join us !', client_id)
def run(self):
key = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase + string.digits) for _ in range(32))
print self.send('HC%s' % key)
while True:
entry = self.socket.recv(4096)
entry.replace("\n", "")
if not entry:
break
else:
logging.debug('(%d) Packet received : %s', self.client_id, str(entry))
self.kill()
def send(self, packet):
return self.socket.send("%s\x00" % packet)
def kill(self):
self.socket.close()
logging.debug('(%d) Client is gone...', self.client_id)
sorry for bad indentation, it's the form, not my file.
Please help me :(
Thank you in advance (sorry for bad english i'm french....)
You have this line of code in your Server instance start function:
new_client = threading.Thread(target=ClientThread,
args=[len(self.CLIENTS), ip, port, clientsock])
The target= argument to threading.Thread needs to be a callable function. Here ClientThread is the name of the constructor function for your class ClientThread, so it is a callable function, returning an instance of that class. Note that it is not actually called yet! The args= argument is more normally a tuple, but a list actually works. These are the arguments that will be passed to the target function once it's eventually called, when you use this particular threading model. (You can also pass keyword arguments using kwargs= and a dictionary.)
What happens now is a bit tricky. Now that the two parameters (target= and args=) have been evaluated, the Python runtime creates a new instance of a threading.Thread class. This new instance is, at the moment, just a data object.
If we add a print statement/function (it's not clear whether this is py2k or py3k code) we can see the object itself:
print('new_client id is', id(new_client))
which will print something like:1
new_client id is 34367605072
Next, you add this to a list and then invoke its start:
self.CLIENTS.append(new_client)
new_client.start()
The list add is straightforward enough, but the start is pretty tricky.
The start call itself actually creates a new OS/runtime thread (whose ID is not related to the data object's ID—the raw thread ID is an internal implementation detail). This new thread starts running at its run method.2 The default run method is in fact:3
try:
if self.__target:
self.__target(*self.__args, **self.__kwargs)
finally:
# Avoid a refcycle if the thread is running a function with
# an argument that has a member that points to the thread.
del self.__target, self.__args, self.__kwargs
Since you are using a regular threading.Thread instance object, you are getting this default behavior, where new_thread.start() creates the new thread itself, which then calls the default run method, which calls its self.__target which is your ClientThread class-instance-creation function.
So now, inside the new thread, Python creates an instance of a ClientThread object, calling its __init__ with the self.__args and self.__kwargs saved in the new_thread instance (which is itself shared between the original Python, and the new thread).
This new ClientThread object executes its __init__ code and returns. This is the equivalent of having the run method read:
def run(self):
ClientThread(**saved_args)
Note that this is not:
def run(self):
tmp = ClientThread(**saved_args)
tmp.run()
That is, the run method of the ClientThread instance is never called. Only the run method of the threading.Thread instance is called. If you modify your ClientThread's __init__ method to print out its ID, you will see that this ID differs from that of the threading.Thread instance:
class ClientThread:
def __init__(self, client_id, client_ip, client_port, socket):
print('creating', id(self), 'instance')
which will print a different ID (and definitely print after the new_client id is line):
new_client id is 34367605072
creating 34367777464 instance
If you add additional prints to your run method you will see that it is never invoked.
What to do about this
You have two main options here.
You can either make your ClientThread a subclass of threading.Thread:
class ClientThread(threading.Thread):
def __init__(self, client_id, client_ip, client_port, socket):
...
threading.Thread.__init__(self)
In this case, you would create the client object yourself, rather than using threading.Thread to create it:
new_thread = ClientThread(...)
...
new_thread.start()
The .start method would be threading.Thread.start since you have not overridden that, and that method would then create the actual OS/runtime thread and then call your run method, which—since you did override it—would be your run.
Or, you can create a standard threading.Thread object, supply it with a target, and have this target invoke your object's run method, e.g.:
new_client = ClientThread(...)
new_thread = threading.Thread(target=new_client.run, ...)
...
new_thread.start()
The choice is yours: to subclass, or to use separate objects.
1The actual ID is highly implementation-dependent.
2The path by which it reaches this run function is somewhat convoluted, passing through bootstrap code that does some internal initialization, then calls self.run for you, passing no arguments. You are only promised that self.run gets entered somehow; you should not rely on the "how".
3At least, this is the code in Python 2.7 and 3.4; other implementations could vary slightly.
Related
I am trying to write some test cases to evaluate the response of a server by creating a client through websockets. I am using autobahn to establish a connection. However, I seem to be unable to send a message to the server because I require the currently active instance of the protocol class in order to run sendMessage. Here is the code:
class SlowSquareClientProtocol(WebSocketClientProtocol):
def onOpen(self):
print "Connection established"
def onMessage(self, payload, isBinary):
if not isBinary:
res = json.loads(payload.decode('utf8'))
print("Result received: {}".format(res))
self.sendClose()
def onClose(self, wasClean, code, reason):
if reason:
print(reason)
reactor.stop()
class NLVRTR(TestFixture,SlowSquareClientProtocol):
#classmethod
def setUpClass(self):
log.startLogging(sys.stdout)
factory = WebSocketClientFactory(u"ws://someURL:8078")
factory.protocol = SlowSquareClientProtocol
reactor.connectTCP("someURL", 8078, factory)
wsThread = threading.Thread(target = reactor.run,
kwargs={'installSignalHandlers':0})
wsThread.start()
def test_00_simple(self):
WSJsonFormatter = WSformat()
x = WSJsonFormatter.formatGetInfo(2)
self.sendMessage(json.dumps(x).encode('utf8'))
print("Request to square {} sent.".format(x))
So just to elaborate, I am started the client in the setUpClass method and I am trying to send some messages in test_00_simple. However, I seem to be getting an error like so
AttributeError: 'NLVRTR' object has no attribute 'state'
State should be an attribute defined inside the WebSocketClientProtoco. Everything works fine if I put the sendmessage inside the onOpen method, but I cannot call it from anywhere else other than inside SlowSquareClientProtocol class. In the doc for autobahn, it was mentionned that
Whenever a new client connects to the server, a new protocol instance will be created
I believe that this is the issue, and that it creates a new protocol instance and the sendmessage method is using that instance. Since I am not calling it inside the slowsquare... class, sendmessage never caught on to this newly created protocol when the client connects thus the error. My question is, is there any way that I can acquire the newly created instance through my code once the client connects?
I found a stupid way around this by using garbage collector to retrieve the instance like so
#Used to get the instance of the protocol
def getIn(self):
for obj in gc.get_objects():
if isinstance(obj, SlowSquareClientProtocol):
protocol = obj
return protocol
def test_00_startSession(self):
WSJsonFormatter = WSformat()
x = WSJsonFormatter.formatCreateSession("eng-USA", sessionId = "837ab900-912e-11e6-b83e-3f30a2b99389")
SlowSquareClientProtocol.sendMessage(self.getIn(),json.dumps(x).encode('utf8'))
print("Request {} sent.".format(x))
So i searched all instances which have the name of the class I am looking for, and then pass it in the sendmessage method. I am still opened to other simpler suggestions :)
I'm writing a program with multiple sockets using a select statement to manage them using the following code :
while not self.stopped:
input_ready, output_ready, except_ready = select.select(self.input_sockets, [], [], 5)
if (not input_ready) and (not output_ready) and (not except_ready):
print("Timed Out")
else:
for s in input_ready:
s.process_data()
For this to work I have created a class which is abstracted from socket.socket and adds the method process_data. I have a class defined this way, which binds to a socket to listen for inbound connections and when process_data() is called, it accepts the connection using (sock, address) = self.accept() ... I then add sock to the input_sockets array for use in the select, but obviously the accept method returns a socket, not an abstracted class with and therefore has no process_data() method, so this causes errors.
Can anyone think of a way that I could use the accept() method to return my own abstracted socket class rather than a normal socket?
Thanks
EDIT :
I have found a happy work around for now - Instead of creating an abstracted socket class, I have created a standard class as follows:
class DirectConnection():
def __init__(self, sock):
self.socket = sock
def fileno(self):
return self.socket.fileno()
def process_data(self):
print("Got data")
Then from my listening socket
(sock, address) = self.accept()
socketmanager.monitor_socket(DirectConnection(sock))
The select.select makes use of the fileno property of the socket object, so by defining a fileno() method which returns the fileno() of the socket passed into the class, I can now have the select.select call my method on this class, which I can then instruct to send/receive data from the socket passed in.
credit goes to : http://bytes.com/topic/python/answers/437702-subclassing-socket and http://code.activestate.com/recipes/52295/
Maybe I'm missing something here in the asynchronous designs of Twisted, but I can't seem to find a way to call the sendMessage() method "externaly". By this I mean, sending messages without being solely at the callback methods of Twisted/AutobahnWebsockets (like at onOpen or when receiving data from server at onMessage())
Of course I could launch a thread and call my_protocol_instance.sendMessage("hello") but that would defeat every purpose of the asynchronous design right?
In a concrete example, I need to have a top wrapper class which opens the connection and manages it, and whenever I need I call my_class.send_my_toplevel_message(msg). How can I implement this?
Hope I've been clear on my explanation.
Thanks
Why do you need a thread to launch protocolInstance.sendMessage() ?
This can be done in a normal reactor loop.
The core of a twisted is reactor and it gives a much easier look at things when you consider twisted itself reactive - meaning it does something as a reaction (response) to something else.
Now I assume that the thread you are talking about, also gets created and made in calling sendMessage because of certain events or activity or status. I can hardly imagine a case where you would just need to send a message out of the blue without any reason to react.
If however there is an event which should trigger sendMessage, there is no need to invoke that in thread: just use twisted mechanisms for catching that event and then calling sendMessage from that particular event's callback.
Now on to your concrete example: can you specify what "whenever I need" means exactly in the context of this question? An input from another connection? An input from the user? Looping activity?
I managed to implement what I needed by running Twisted in another thread, keeping my program free to run and allowing it to trigger send data in Twisted with reactor.callFromThread().
What do you think?
# ----- twisted ----------
class _WebSocketClientProtocol(WebSocketClientProtocol):
def __init__(self, factory):
self.factory = factory
def onOpen(self):
log.debug("Client connected")
self.factory.protocol_instance = self
self.factory.base_client._connected_event.set()
class _WebSocketClientFactory(WebSocketClientFactory):
def __init__(self, *args, **kwargs):
WebSocketClientFactory.__init__(self, *args, **kwargs)
self.protocol_instance = None
self.base_client = None
def buildProtocol(self, addr):
return _WebSocketClientProtocol(self)
# ------ end twisted -------
class BaseWBClient(object):
def __init__(self, websocket_settings):
self.settings = websocket_settings
# instance to be set by the own factory
self.factory = None
# this event will be triggered on onOpen()
self._connected_event = threading.Event()
# queue to hold not yet dispatched messages
self._send_queue = Queue.Queue()
self._reactor_thread = None
def connect(self):
log.debug("Connecting to %(host)s:%(port)d" % self.settings)
self.factory = _WebSocketClientFactory(
"ws://%(host)s:%(port)d" % self.settings,
debug=True)
self.factory.base_client = self
c = connectWS(self.factory)
self._reactor_thread = threading.Thread(target=reactor.run,
args=(False,))
self._reactor_thread.daemon = True
self._reactor_thread.start()
def send_message(self, body):
if not self._check_connection():
return
log.debug("Queing send")
self._send_queue.put(body)
reactor.callFromThread(self._dispatch)
def _check_connection(self):
if not self._connected_event.wait(timeout=10):
log.error("Unable to connect to server")
self.close()
return False
return True
def _dispatch(self):
log.debug("Dispatching")
while True:
try:
body = self._send_queue.get(block=False)
except Queue.Empty:
break
self.factory.protocol_instance.sendMessage(body)
def close(self):
reactor.callFromThread(reactor.stop)
I am trying to activate a function that will, once i'm finished, change the control variables for some motors. the commands are coming in over wifi via socket server. Here is the code;
import SocketServer
import Tkinter as Tk
from Tkinter import *
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.DriveSend = self.request.recv(1024).strip()
self.SteeringSend = self.request.recv(1024).strip()
#print("{} wrote:".format(self.client_address[0]))
#print(self.DriveSend)
#print(self.SteeringSend)
#self.request.sendall(self.DriveSend.upper())
#self.request.sendall(self.SteeringSend.upper())
return (self.DriveSend,self.SteeringSend)
MotorControl()
def MotorControl():
MotorVar = MyTCPHandler()
MotorVar.handle()
MotorVar.DriveSend
MotorVar.SteeringSend
print(MotorVar.DriveSend)
print(MotorVar.SteeringSend)
print('test')
if __name__ == "__main__":
HOST, PORT = "192.168.2.12", 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
As you can see the server runs constantly and looks out for incoming messages, i would like it to run the function MotorControl every time it receives a new message ideally from my client program. I tried this but it doesn't print any values (my way of testing it with something basic before i try and control anything) all the commented out sections are bits of code from the original testing to check the server side of things. all of which works fine.
There are a couple of things wrong here. Firstly, the MyTCPHandler method returns before calling the function. When a return in a function is hit, execution returns immediately and no more code in that function will be executed.
Secondly, you've got the MyTCPHandler method calling the MotorControl function, but then the function instantiates a new instance of the class, which naturally doesn't have any of the information set. Instead, you should pass your instance to the function.
So:
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.DriveSend = self.request.recv(1024).strip()
self.SteeringSend = self.request.recv(1024).strip()
MotorControl(self)
return (self.DriveSend,self.SteeringSend)
def MotorControl(motor_var):
print(motor_var.DriveSend)
print(motor_var.SteeringSend)
print('test')
Although on reflection, it's not clear why you want this in a standalone function anyway. You should perhaps make MotorControl a method on MyTCPHandler, then it will already have access to self.
This is related to the previously answered question here: Logging SMTP connections with Twisted. I have a database resource that I create in each instance of ConsoleMessageDelivery that I need to ensure is cleaned up when the socket is closed. I have a WrappingFactory called DenyFactory and the DenyFactory.unregisterProtocol method is called when the socket is closed, but I have no way (that I can figure out) how to access the resource created in the ConsoleMessageDelivery instance that's being destroyed. I tried a del() method in ConsoleMessageDelivery but that's never called. What's the best way to clean up a resource in this scenario?
class ConsoleMessageDelivery:
implements(smtp.IMessageDelivery)
def receivedHeader(self, helo, origin, recipients):
myHostname, clientIP = helo
headerValue = "by %s from %s with ESMTP ; %s" % (myHostname, clientIP, smtp.rfc822date())
# email.Header.Header used for automatic wrapping of long lines
return "Received: %s" % Header(headerValue)
def validateFrom(self, helo, origin):
# All addresses are accepted
return origin
def validateTo(self, user):
if user.dest.local == "console":
return lambda: ConsoleMessage()
raise smtp.SMTPBadRcpt(user)
class ConsoleMessage:
implements(smtp.IMessage)
def __init__(self):
self.lines = []
def lineReceived(self, line):
self.lines.append(line)
def eomReceived(self):
return defer.succeed(None)
def connectionLost(self):
# There was an error, throw away the stored lines
self.lines = None
class ConsoleSMTPFactory(smtp.SMTPFactory):
protocol = smtp.ESMTP
def __init__(self, *a, **kw):
smtp.SMTPFactory.__init__(self, *a, **kw)
self.delivery = ConsoleMessageDelivery()
def buildProtocol(self, addr):
p = smtp.SMTPFactory.buildProtocol(self, addr)
p.delivery = self.delivery
return p
class DenyFactory(WrappingFactory):
def buildProtocol(self, clientAddress):
if clientAddress.host == '1.3.3.7':
# Reject it
return None
# Accept everything else
return WrappingFactory.buildProtocol(self, clientAddress)
def unregisterProtocol(self, p):
print "Unregister called"
First, don't ever use __del__, particularly if you have some resources you want to clean up. __del__ prevents the garbage collection of objects in reference cycles. (Alternatively, switch to PyPy which can collect such objects by imposing an arbitrary ordering on the collection of objects in the cycle.)
Next, consider opening your database connection (or start a connection pool) in the message delivery factory and sharing it between all of the message delivery objects. This way you don't need to clean the connections of, because you'll re-use them for future messages, and you aren't allocating a new one for each message, so there's no leak.
Finally, if you really need any per-transaction objects, you can clean them up in your eomReceived or connectionLost implementations on the IMessage object. One of these methods will be called once the DATA portion of the SMTP transaction is complete (either because all data was received or because the connection was lost). Note that because SMTP supports delivery of a message to multiple recipients in a single transaction, there may be more than one IMessage object participating, even though there is only a single IMessageDelivery object. So you may want to keep a counter - match up the number of calls to successful validateTo calls on the message delivery object with the number of eomReceived/connectionLost calls on the message objects. When the same number of calls of each have happened, the transaction is done.