Develop Reference

chat-room server not closing threads and not exiting when SIGINT

I have the following server program in Python which simulates a chat-room. The code accepts connections from clients and for each of them it launches a new thread. This thread will wait for messages from this client. The messages can be L so that the server will respond with a list of connected clients, ip:port msg the server will send the message msg to the client ip:port.
On client side there will be 2 threads, one for receiving messages from the server, the other for sending.
import socket
from threading import Thread
#from SocketServer import ThreadingMixIn
import signal
import sys
import errno
EXIT = False
address = []
address2 = []
# handler per il comando Ctrl+C
def sig_handler(signum, frame):
if (signum == 2):
print("Called SIGINT")
EXIT = True
signal.signal(signal.SIGINT, sig_handler) # setto l'handler per i segnali
# Multithreaded Python server : TCP Server Socket Thread Pool
class ClientThread(Thread):
def __init__(self,conn,ip,port):
Thread.__init__(self)
self.conn = conn
self.ip = ip
self.port = port
print ("[+] New server socket thread started for " + ip + ":" + str(port))
def run(self):
while True:
data = self.conn.recv(1024)
print ("Server received data:", data)
if (data=='L'):
#print "QUI",address2
tosend = ""
for i in address2:
tosend = tosend + "ip:"+str(i[0]) + "port:"+str(i[1])+"\n"
self.conn.send(tosend)
#mandare elenco client connessi
else:
#manda ip:port msg
st = data.split(" ")
msg = st[1:]
msg = ' '.join(msg)
print ("MSG 2 SEND: ",msg)
ipport = st[0].split(":")
ip = ipport[0]
port = ipport[1]
flag = False
print ("Address2:",address2)
print ("ip:",ip)
print ("port:",port)
for i in address2:
print (i[0],ip,type(i[0]),type(ip),i[1],type(i[1]),port,type(port))
if str(i[0])==str(ip) and str(i[1])==str(port):
i[2].send(msg)
self.conn.send("msg inviato")
flag = True
break
if flag == False:
self.conn.send("client non esistente")
if __name__ == '__main__':
# Multithreaded Python server : TCP Server Socket Program Stub
TCP_IP = '127.0.0.1'
TCP_PORT = 2004
TCP_PORTB = 2005
BUFFER_SIZE = 1024 # Usually 1024, but we need quick response
tcpServer = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServer.bind((TCP_IP, TCP_PORT))
tcpServerB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpServerB.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServerB.bind((TCP_IP, TCP_PORTB))
threads = []
tcpServer.listen(4)
tcpServerB.listen(4)
while True:
print("Multithreaded Python server : Waiting for connections from TCP clients...")
try:
(conn, (ip,port)) = tcpServer.accept()
except socket.error as e: #(code, msg):
if e.errno != errno.EINTR:
raise
else:
break
address.append((ip,port,conn))
(conn2, (ip2,port2)) = tcpServerB.accept()
address2.append((ip2,port2,conn2))
newthread = ClientThread(conn,ip,port)
newthread.start()
threads.append(newthread)
if EXIT==True:
break
print ("SERVER EXIT")
for t in threads:
t.join()
The code has a signal handler for SIGINT to make the exit cleaner (closing connections, sending a message to the client (still to be implemented) and so on ). The handler writes a global flag EXIT to make the infinite loops terminate.
The code runs both in Python2 and Python3. However there are some problems with SIGINT signal generated by CTRL-C. When there is no client connected the program launched with Python2 exits correctly while the one in Python3 does not. Why this behavioural difference?
Considering only running the program in Python2, when a client connects and I press CTRL-C, the main while exits, like the signal is catched always by the main thread and this interrupts the blocking system call accept. However the other threads do not, I think because of the blocking underlying system call data = self.conn.recv(1024). In C I would block SIGINT signals for one thread and then call pthread_cancel from the other thread. How to exit from all threads when SIGINT is generated in Python?
The client program that for the moment works in Python2 only and suffers from the same problem is:
# Python TCP Client A
import socket
from threading import Thread
class ClientThread(Thread):
def __init__(self,conn):
Thread.__init__(self)
self.conn = conn
def run(self):
while True:
data = self.conn.recv(1024)
print "Ricevuto msg:",data
host = socket.gethostname()
print "host:",host
port = 2004
portB = 2005
BUFFER_SIZE = 2000
tcpClientA = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientB = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpClientA.connect(('127.0.0.1', port))
tcpClientB.connect(('127.0.0.1', portB))
newthread = ClientThread(tcpClientB)
newthread.start()
while(True):
msg = raw_input("Inserisci comando: ")
tcpClientA.send (msg)
data = tcpClientA.recv(BUFFER_SIZE)
print "data received:",data
tcpClientA.close()

As for the difference in behavior with accept() in Python 3, look at the full description in the docs. I think this is the key statement:
Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the method now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).
The other problem, stated in your penultimate sentence:
How to exit from all threads when SIGINT is generated in Python 2?
Take a look at the threading documentation:
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.

Multithreaded TCP socket

I'm trying to create a threaded TCP socket server that can handle multiple socket request at a time.
To test it, I launch several thread in the client side to see if my server can handle it. The first socket is printed successfully but I get a [Errno 32] Broken pipe for the others.
I don't know how to avoid it.
import threading
import socketserver
import graphitesend
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
if data != "":
print(data)
class ThreadedTCPServer(socketserver.ThreadingTCPServer):
allow_reuse_address = True
def __init__(self, host, port):
socketserver.ThreadingTCPServer.__init__(self, (host, port), ThreadedTCPRequestHandler)
def stop(self):
self.server_close()
self.shutdown()
def start(self):
threading.Thread(target=self._on_started).start()
def _on_started(self):
self.serve_forever()
def client(g):
g.send("test", 1)
if __name__ == "__main__":
HOST, PORT = "localhost", 2003
server = ThreadedTCPServer(HOST, PORT)
server.start()
g = graphitesend.init(graphite_server = HOST, graphite_port = PORT)
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
threading.Thread(target = client, args=(g,)).start()
server.stop()

It's a little bit difficult to determine what exactly you're expecting to happen, but I think the proximate cause is that you aren't giving your clients time to run before killing the server.
When you construct a Thread object and call its start method, you're creating a thread, and getting it ready to run. It will then be placed on the "runnable" task queue on your system, but it will be competing with your main thread and all your other threads (and indeed all other tasks on the same machine) for CPU time.
Your multiple threads (main plus others) are also likely being serialized by the python interpreter's GIL (Global Interpreter Lock -- assuming you're using the "standard" CPython) which means they may not have even gotten "out of the gate" yet.
But then you're shutting down the server with server_close() before they've had a chance to send anything. That's consistent with the "Broken Pipe" error: your remaining clients are attempting to write to a socket that has been closed by the "remote" end.
You should collect the thread objects as you create them and put them in a list (so that you can reference them later). When you're finished creating and starting all of them, then go back through the list and call the .join method on each thread object. This will ensure that the thread has had a chance to finish. Only then should you shut down the server. Something like this:
threads = []
for n in range(7):
th = threading.Thread(target=client, args=(g,))
th.start()
threads.append(th)
# All threads created. Wait for them to finish.
for th in threads:
th.join()
server.stop()
One other thing to note is that all of your clients are sharing the same single connection to send to the server, so that your server will never create more than one thread: as far as it's concerned, there is only a single client. You should probably move the graphitesend.init into the client function if you actually want separate connections for each client.
(Disclaimer: I know nothing about graphitesend except what I could glean in a 15 second glance at the first result in google; I'm assuming it's basically just a wrapper around a TCP connection.)

Listening for socket input in multiple threads

I'm using python sockets.
Here's the problem. I've 2 threads:
One thread listens for socket input from remote and reply to it
One thread polls file and if something is present in file then send
to socket and expect a response.
Now the problem is in case of second thread when I send something, the response doesn't come to this thread. Rather it comes to thread mentioned in (1) point.
This is thread (1)
def client_handler(client):
global client_name_to_sock_mapping
client.send(first_response + server_name[:-1] + ", Press ^C to exit")
user_name = None
while True:
request = client.recv(RECV_BUFFER_LIMIT)
if not user_name:
user_name = process_input(client, request.decode('utf-8'))
user_name = user_name.rstrip()
if user_name not in client_name_to_sock_mapping.keys():
client_name_to_sock_mapping[user_name] = client
else:
msg = "Username not available".encode('ascii')
client.send(msg)
else:
process_input(client, request.decode('utf-8'), user_name)
This is run from thread (2)
def send_compute_to_client():
time.sleep(20)
print("Sleep over")
for _, client_sock in client_name_to_sock_mapping.iteritems():
print("client = {}".format(client_sock))
client_sock.sendall("COMPUTE 1,2,3")
print("send completed = {}".format(client_sock))
data = client_sock.recv(1024)
print("Computed results from client {}".format(data))
Can someone please explain this behaviour?

I have faced similar problems in the past. That happens when in one thread you start a blocking action listening for a connection while in the other thread you send through the same socket.
If I understand it well, you always want to receive the response from the previously send data. So in order to solve it I would use locks to force that behaviour, so just create a class:
from threading import Lock
class ConnectionSession:
def __init__(self, address, conn):
self.ip = address[0] # Optional info
self.port = address[1] # Optional info
self.conn = conn
self.lock = Lock()
Here it goes how to create a ConnectionSession object properly when a listening socket is created:
address = ('127.0.0.1', 46140)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(address)
conn, addr = s.accept()
session = ConnectionSession(addr, conn)
And here it goes when a 'sending' connection is created:
address = ('127.0.0.1', 46140)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
session = ConnectionSession(address, s)
Keep in mind that the created session instance is the one that needs to be shared among threads.
Afterwards, to send information through the shared socket you could do in each thread something like:
# Previous code
try:
session.lock.acquire()
session.conn.sendall("Hi there buddy!")
# Do something if needed
message = session.conn.recv(1024)
except Exception as e:
print "Exception e=%s should be handled properly" % e
finally:
if session.lock.locked():
session.lock.release()
# Other code
Note that the finally block is important as it will free the locked connection whether if the action succeeded or not.
You can also wrap the previous code in a class, e.g: SocketManager with the following code in order to avoid having to explicitly acquire and release locks.
I hope it helps

Odd socket behavior in Python 2

I am writing a multithreaded socket application in Python.
Here's some basic skeleton code for what I have:
import socket, threading, time
class listener:
def __init__(self):
# Create a local listener socket
self.socket = socket.socket(AF_INET, SOCK_STREAM)
# Set options
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
def start(self):
# Start listening for incoming connections in a loop. As connections
# come in, start new threads and accept/work with them.
self.socket.bind(('',1001)) # Bind to all addresses
while (True):
self.socket.listen(1) # blocks until a new connection is available
newSocket, addr = self.socket.accept() # Accept connection
thisConnThread = threading.Thread(target=server().runServer, args=( newSocket, addr ) )
thisConnThread.start() # call the connection handler on a new thread
# now return and listen for more connections.
class server:
def RunServer(self, socket, addr):
socket.write("Hello world!\n")
# Create a thread to listen for input from the client
thisListenThread = threading.Thread(target=self.RunServer_Listener, args=(socket.) )
thisListenThread.start()
# To demonstrate async - print a value every so many seconds - this
# needs to happen separately from the listener.
for i in range(0,100):
socket.write("Checkpoint!\n")
time.sleep(60)
socket.write("Your time is UP! Bye!\n")
socket.shutdown(socket.SHUTDOWN_RDWR) # close the connection
def RunServer_Listener(self, socket):
while (True):
inData = socket.read(4096) # blocks until data arrives
if not inData:
break # connection must be closed
socket.write("You wrote: %s\n" % inData)
What I'd expect to happen in this case is that each time a connection comes in, a new instance of server would be created and a new thread would be spawned to run it.
In other words, the line thisConnThread = threading.Thread(target=server().runServer, args=( newSocket, addr ) ) to me should be creating a new instance of server and then executing the function contained therein.
I setup a separate listener thread because on the actual server, the server may need to send data to the client at any time, but it also needs to respond to the client at any time. Since read() blocks until data is available, it made sense to me to create a listening thread that waits for input from the client and then processes it, but the main connection thread can still do what it needs to do and write to the client independent of the listener.
If I have a single connection to this server, this works exactly as expected.
However, if I connect a second client to the server, strange things begin to happen. The most important and most concerning is that if I type into the second instance's client, sometimes the reply will go to the second instance, and sometimes to the FIRST instance, and vice versa. It is almost as if the write() operation is doing some sort of round-robin thing - it goes to each instance alternately.
For this exercise there really isn't a need to keep like a list of active connections. That amy be for a later or more advanced project, but for now, the clients simply live out their time on the server and then go away and the server doesn't have any reason to interact with the other connections.
I'm sure I'm missing something here, maybe my implementation of threading is wrong or maybe my use of sockets is completely wrong. Either way, does anyone have some advice as to how to make the connections to the server completely independent of each other?

After making a bunch of more or less obvious fixes, your code seems to work fine for me in Python 2.7. Here's the patch (note, I changed the port number too, and sped up the timeouts and such).
(Did not add appropriate join or setDaemon calls, did not seem worth doing for this.)
diff --git a/fdmillion.py b/fdmillion.py
index 8109c83..1362d3d 100644
--- a/fdmillion.py
+++ b/fdmillion.py
## -1,40 +1,44 ##
import socket, threading, time
+from socket import AF_INET, SOCK_STREAM
class listener:
def __init__(self):
# Create a local listener socket
- self.socket = socket.socket(AF_INET, SOCK_STREAM)
+ self.sock = socket.socket(AF_INET, SOCK_STREAM)
# Set options
- self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
def start(self):
# Start listening for incoming connections in a loop. As connections
# come in, start new threads and accept/work with them.
- self.socket.bind(('',1001)) # Bind to all addresses
+ self.sock.bind(('',8001)) # Bind to all addresses
while (True):
- self.socket.listen(1) # blocks until a new connection is available
- newSocket, addr = self.socket.accept() # Accept connection
- thisConnThread = threading.Thread(target=server().runServer, args=( newSocket, addr ) )
+ self.sock.listen(1) # blocks until a new connection is available
+ newSocket, addr = self.sock.accept() # Accept connection
+ thisConnThread = threading.Thread(target=server().RunServer, args=( newSocket, addr ) )
thisConnThread.start() # call the connection handler on a new thread
# now return and listen for more connections.
class server:
- def RunServer(self, socket, addr):
- socket.write("Hello world!\n")
+ def RunServer(self, sock, addr):
+ sock.sendall("Hello world!\n")
# Create a thread to listen for input from the client
- thisListenThread = threading.Thread(target=self.RunServer_Listener, args=(socket.) )
+ thisListenThread = threading.Thread(target=self.RunServer_Listener, args=(sock,) )
thisListenThread.start()
# To demonstrate async - print a value every so many seconds - this
# needs to happen separately from the listener.
- for i in range(0,100):
- socket.write("Checkpoint!\n")
- time.sleep(60)
- socket.write("Your time is UP! Bye!\n")
- socket.shutdown(socket.SHUTDOWN_RDWR) # close the connection
+ for i in range(5,0,-1):
+ sock.sendall("Checkpoint! %d...\n" % i)
+ time.sleep(5)
+ sock.sendall("Your time is UP! Bye!\n")
+ sock.shutdown(socket.SHUT_RDWR) # close the connection
- def RunServer_Listener(self, socket):
+ def RunServer_Listener(self, sock):
while (True):
- inData = socket.read(4096) # blocks until data arrives
+ inData = sock.recv(4096) # blocks until data arrives
if not inData:
break # connection must be closed
- socket.write("You wrote: %s\n" % inData)
+ sock.sendall("You wrote: %s\n" % inData)
+
+if __name__ == '__main__':
+ listener().start()
[edited to fix long lines cut off by diff output going through less]

Paramiko SSH Tunnel Shutdown Issue

I'm working on a python script to query a few remote databases over an established ssh tunnel every so often. I'm fairly familiar with the paramiko library, so that was my choice of route. I'd prefer to keep this in complete python so I can use paramiko to deal with key issues, as well as uses python to start, control, and shutdown the ssh tunnels.
There have been a few related questions around here about this topic, but most of them seemed incomplete in answers. My solution below is a hacked together of the solutions I've found so far.
Now for the problem: I'm able to create the first tunnel quite easily (in a separate thread) and do my DB/python stuff, but when attempting to close the tunnel the localhost won't release the local port I binded to. Below, I've included my source and the relevant netstat data through each step of the process.
#!/usr/bin/python
import select
import SocketServer
import sys
import paramiko
from threading import Thread
import time
class ForwardServer(SocketServer.ThreadingTCPServer):
daemon_threads = True
allow_reuse_address = True
class Handler (SocketServer.BaseRequestHandler):
def handle(self):
try:
chan = self.ssh_transport.open_channel('direct-tcpip', (self.chain_host, self.chain_port), self.request.getpeername())
except Exception, e:
print('Incoming request to %s:%d failed: %s' % (self.chain_host, self.chain_port, repr(e)))
return
if chan is None:
print('Incoming request to %s:%d was rejected by the SSH server.' % (self.chain_host, self.chain_port))
return
print('Connected! Tunnel open %r -> %r -> %r' % (self.request.getpeername(), chan.getpeername(), (self.chain_host, self.chain_port)))
while True:
r, w, x = select.select([self.request, chan], [], [])
if self.request in r:
data = self.request.recv(1024)
if len(data) == 0:
break
chan.send(data)
if chan in r:
data = chan.recv(1024)
if len(data) == 0:
break
self.request.send(data)
chan.close()
self.request.close()
print('Tunnel closed from %r' % (self.request.getpeername(),))
class DBTunnel():
def __init__(self,ip):
self.c = paramiko.SSHClient()
self.c.load_system_host_keys()
self.c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.c.connect(ip, username='someuser')
self.trans = self.c.get_transport()
def startTunnel(self):
class SubHandler(Handler):
chain_host = '127.0.0.1'
chain_port = 5432
ssh_transport = self.c.get_transport()
def ThreadTunnel():
global t
t = ForwardServer(('', 3333), SubHandler)
t.serve_forever()
Thread(target=ThreadTunnel).start()
def stopTunnel(self):
t.shutdown()
self.trans.close()
self.c.close()
Although I will end up using a stopTunnel() type method, I've realize that code isn't entirely correct, but more so an experimentation of trying to get the tunnel to shutdown properly and test my results.
When I first call create the DBTunnel object and call startTunnel(), netstat yields the following:
tcp4 0 0 *.3333 *.* LISTEN
tcp4 0 0 MYIP.36316 REMOTE_HOST.22 ESTABLISHED
tcp4 0 0 127.0.0.1.5432 *.* LISTEN
Once I call stopTunnel(), or even delete the DBTunnel object itself..I'm left with this connection until I exit python all together, and what I assume to be the garbage collector takes care of it:
tcp4 0 0 *.3333 *.* LISTEN
It would be nice to figure out why this open socket is hanging around independent of the DBConnect object, and how to close it properly from within my script. If I try and bind a different connection to different IP using the same local port before completely exiting python (time_wait is not the issue), then I get the infamous bind err 48 address in use. Thanks in advance :)

It appears the SocketServer's shutdown method isn't properly shutting down/closing the socket. With the below changes in my code, I retain access to the SocketServer object and access the socket directly to close it. Note that socket.close() works in my case, but others might be interested in socket.shutdown() followed by a socket.close() if other resources are accessing that socket.
[Ref: socket.shutdown vs socket.close
def ThreadTunnel():
self.t = ForwardServer(('127.0.0.1', 3333), SubHandler)
self.t.serve_forever()
Thread(target=ThreadTunnel).start()
def stopTunnel(self):
self.t.shutdown()
self.trans.close()
self.c.close()
self.t.socket.close()

Note that you don't have do the Subhandler hack as shown in the demo code. The comment is wrong. Handlers do have access to their Server's data. Inside a handler you can use self.server.instance_data.
If you use the following code, in your Handler, you would use
self.server.chain_host
self.server.chain_port
self.server.ssh_transport
class ForwardServer(SocketServer.ThreadingTCPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(
self, connection, handler, chain_host, chain_port, ssh_transport):
SocketServer.ThreadingTCPServer.__init__(self, connection, handler)
self.chain_host = chain_host
self.chain_port = chain_port
self.ssh_transport = ssh_transport
...
server = ForwardServer(('', local_port), Handler,
remote_host, remote_port, transport)
server.serve_forever()

You may want to add some synchronization between the spawned thread and the caller so that you don't try to use the tunnel before it is ready. Something like:
from threading import Event
def startTunnel(self):
class SubHandler(Handler):
chain_host = '127.0.0.1'
chain_port = 5432
ssh_transport = self.c.get_transport()
mysignal = Event()
mysignal.clear()
def ThreadTunnel():
global t
t = ForwardServer(('', 3333), SubHandler)
mysignal.set()
t.serve_forever()
Thread(target=ThreadTunnel).start()
mysignal.wait()

You can also try sshtunnel it has two cases to close tunnel .stop() if you want to wait until the end of all active connections or .stop(force=True) to close all active connections.
If you don't want to use it you can check the source code for this logic here: https://github.com/pahaz/sshtunnel/blob/090a1c1/sshtunnel.py#L1423-L1456