Develop Reference

Raise TimeOutError after given time for multiprocessing.connection.Listener.accept()

I'm trying to interrupt multiprocessing.connection.Listener.accept(), but have thus far been unsuccessful. Since it doesn't provide a timeout parameter, I thought perhaps I could use socket.setdefaulttimeout() to interrupt it, as suggested in post I cannot find anymore, here on SO.
This didnt work. I then tried calling close() on the Listener() object. according to this post's answer, this should have worked.
It appears, however, that these objects to not play along with the usual socket-related solutions.
I can confirm that that the Listener is closed by the Timer object as expected, but the accept() call isn't interrupted.
The Code:
import logging
import socket
import os
from multiprocessing.connection import Listener
from queue import Queue, Empty
from threading import Thread, Event, Timer
class Node(Thread):
"""Base Class providing a AF_INET, AF_UNIX or AF_PIPE connection to its
data queue. It offers put() and get() method wrappers, and therefore
behaves like a Queue as well as a Thread.
Data from the internal queue is automatically fed to any connecting client.
"""
def __init__(self, sock_name, max_q_size=None, timeout=None,
*thread_args, **thread_kwargs):
"""Initialize class.
:param sock_name: UDS, TCP socket or pipe name
:param max_q_size: maximum queue size for self.q, default infinite
"""
self._sock_name = sock_name
self.connector = Listener(sock_name)
max_q_size = max_q_size if max_q_size else 0
self.q = Queue(maxsize=max_q_size)
self._running = Event()
self.connection_timer = Timer(timeout, self.connection_timed_out)
super(Node, self).__init__(*thread_args, **thread_kwargs)
def connection_timed_out(self):
"""Closes the Listener and shuts down Node if no Client connected.
:return:
"""
self.connector.close()
self.join()
def _start_connection_timer(self):
self.connection_timer.start()
def start(self):
self._running.set()
super(Node, self).start()
def join(self, timeout=None):
print("clearing..")
self._running.clear()
print("internal join")
super(Node, self).join(timeout=timeout)
print("Done")
def run(self):
while self._running.is_set():
print("Accepting connections..")
self._start_connection_timer()
try:
client = self.connector.accept()
self.connection_timer.cancel()
self.feed_data(client)
except (TimeoutError, socket.timeout):
continue
except Exception as e:
raise
print("Run() Terminated!")
def feed_data(self, client):
try:
while self._running.is_set():
try:
client.send(self.q.get())
except Empty:
continue
except EOFError:
return
if __name__ == '__main__':
import time
n = Node('/home/nils/git/spab2/test.uds', timeout=10)
n.start()
print("Sleeping")
time.sleep(15)
print("Manual join")
n.join()
I realize my question is a duplicate of this question - however, it is almost one year old and has not even received a comment. In addition, I'm using Unix Domain Sockets, as opposed to the linked post's TCP connection.

I managed to set the timeout in the following way in Python 2.7:
self.listener = mpc.Listener((address, port))
self.listener._listener._socket.settimeout(3)
With this, the accept() call is interrupted.
Result:
conn = self.listener.accept()
File "/usr/lib/python2.7/multiprocessing/connection.py", line 145, in accept
c = self._listener.accept()
File "/usr/lib/python2.7/multiprocessing/connection.py", line 275, in accept
s, self._last_accepted = self._socket.accept()
File "/usr/lib/python2.7/socket.py", line 202, in accept
sock, addr = self._sock.accept()
timeout: timed out
Regards,
Henri

Listening for socket input in multiple threads

I'm using python sockets.
Here's the problem. I've 2 threads:
One thread listens for socket input from remote and reply to it
One thread polls file and if something is present in file then send
to socket and expect a response.
Now the problem is in case of second thread when I send something, the response doesn't come to this thread. Rather it comes to thread mentioned in (1) point.
This is thread (1)
def client_handler(client):
global client_name_to_sock_mapping
client.send(first_response + server_name[:-1] + ", Press ^C to exit")
user_name = None
while True:
request = client.recv(RECV_BUFFER_LIMIT)
if not user_name:
user_name = process_input(client, request.decode('utf-8'))
user_name = user_name.rstrip()
if user_name not in client_name_to_sock_mapping.keys():
client_name_to_sock_mapping[user_name] = client
else:
msg = "Username not available".encode('ascii')
client.send(msg)
else:
process_input(client, request.decode('utf-8'), user_name)
This is run from thread (2)
def send_compute_to_client():
time.sleep(20)
print("Sleep over")
for _, client_sock in client_name_to_sock_mapping.iteritems():
print("client = {}".format(client_sock))
client_sock.sendall("COMPUTE 1,2,3")
print("send completed = {}".format(client_sock))
data = client_sock.recv(1024)
print("Computed results from client {}".format(data))
Can someone please explain this behaviour?

I have faced similar problems in the past. That happens when in one thread you start a blocking action listening for a connection while in the other thread you send through the same socket.
If I understand it well, you always want to receive the response from the previously send data. So in order to solve it I would use locks to force that behaviour, so just create a class:
from threading import Lock
class ConnectionSession:
def __init__(self, address, conn):
self.ip = address[0] # Optional info
self.port = address[1] # Optional info
self.conn = conn
self.lock = Lock()
Here it goes how to create a ConnectionSession object properly when a listening socket is created:
address = ('127.0.0.1', 46140)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(address)
conn, addr = s.accept()
session = ConnectionSession(addr, conn)
And here it goes when a 'sending' connection is created:
address = ('127.0.0.1', 46140)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
session = ConnectionSession(address, s)
Keep in mind that the created session instance is the one that needs to be shared among threads.
Afterwards, to send information through the shared socket you could do in each thread something like:
# Previous code
try:
session.lock.acquire()
session.conn.sendall("Hi there buddy!")
# Do something if needed
message = session.conn.recv(1024)
except Exception as e:
print "Exception e=%s should be handled properly" % e
finally:
if session.lock.locked():
session.lock.release()
# Other code
Note that the finally block is important as it will free the locked connection whether if the action succeeded or not.
You can also wrap the previous code in a class, e.g: SocketManager with the following code in order to avoid having to explicitly acquire and release locks.
I hope it helps

Sending data from outside socket object in python

I made a Client socket object, which I instantiate and it keeps alive a connection with the server, which is working fine, but I'm wondering if there is a way to call the socket.send event from outside the instance. I was about to make a stack for the messages and check the stack in the while loop and if it's not empty then send the oldest data to the server, which would be just fine for me, but my problem is that the stack only updates after the while loop(I tried breaking out, then it updated).
So my question would be, is there a way to update the global stack simultaneously with the while loop running? Or is there any other way to call the socket.send event outside the object?
import socket
import sys
import select
import threading
SERVER_IP = '192.168.1.4'
PORT = 8686
TIMEOUT = 5
BUF_SIZE = 1024
MESSAGES = ['testdata1', 'testdata2']
class Client(threading.Thread):
def __init__(self, host=SERVER_IP, port=PORT):
threading.Thread.__init__(self)
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock = socket.create_connection((host, port), 1)
self.sock.setblocking(0)
while 1:
try:
global MESSAGES
ready = select.select([self.sock], [], [], TIMEOUT*1000)
if ready[0]:
buf = self.sock.recv(BUF_SIZE)
print buf
#TODO:do stuff with buf
print 'messages left:'+str(len(MESSAGES))
if len(MESSAGES)>0:
self.sock.send(MESSAGES.pop())
except KeyboardInterrupt:
self.sock.close()
sys.exit(1)
except Exception, e:
print '\n[ERR] %s' % e
self.sock.close()
sys.exit(1)
def run(self):
pass
def sendData(self, data):
global MESSAGES
print 'appending data:%s' % data
MESSAGES.append(data)
def main():
client = Client()
client.start()
client.sendData("test1")
client.sendData("test2")
client.sendData("test3")
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
sys.exit(1)

Client.__init__() does not return because it enters an infinite while loop. Hence control is never returned to the main thread, and the Client thread is not actually started.
Instead you should move the while loop into the run() method. Then the __init__() method will return control to the main thread, which can then start the thread, and request that the client send messages via sendData().
class Client(threading.Thread):
def __init__(self, host=SERVER_IP, port=PORT):
threading.Thread.__init__(self)
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock = socket.create_connection((host, port), 1)
self.sock.setblocking(0)
def run(self):
while 1:
try:
global MESSAGES
ready = select.select([self.sock], [], [], TIMEOUT*1000)
if ready[0]:
buf = self.sock.recv(BUF_SIZE)
print buf
#TODO:do stuff with buf
print 'messages left:'+str(len(MESSAGES))
if len(MESSAGES)>0:
self.sock.send(MESSAGES.pop())
except KeyboardInterrupt:
self.sock.close()
sys.exit(1)
except Exception, e:
print '\n[ERR] %s' % e
self.sock.close()
sys.exit(1)
def sendData(self, data):
global MESSAGES
print 'appending data:%s' % data
MESSAGES.append(data)
Instead of using the global MESSAGES list you should probably create a Queue for communicating between the main thread and the worker thread(s), particularly if more than one worker thread is running. Something like this (untested!):
import Queue
class Client(threading.Thread):
def __init__(self, msg_queue, host=SERVER_IP, port=PORT):
threading.Thread.__init__(self)
self.msg_queue = msg_queue
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock = socket.create_connection((host, port), 1)
self.sock.setblocking(0)
def run(self):
while 1:
try:
ready = select.select([self.sock], [], [], TIMEOUT*1000)
if ready[0]:
buf = self.sock.recv(BUF_SIZE)
print buf
#TODO:do stuff with buf
print 'messages left:'+ str(self.msg_queue.qsize())
try:
msg = self.msg_queue.get_nowait()
self.sock.send(msg)
except Queue.Empty:
pass
except KeyboardInterrupt:
self.sock.close()
sys.exit(1)
except Exception, e:
print '\n[ERR] %s' % e
self.sock.close()
sys.exit(1)
def main():
# create a queue and pass it to the client
msg_queue = Queue.Queue()
client = Client(msg_queue)
client.start()
msg_queue.put("test1")
msg_queue.put("test2")
msg_queue.put("test3")

The thing should work if you move your loop from
__init__() into run()
method instead.
Your thread is not a thread this way, process blocks at client = Client(...).
Why do you mix select and threads? Is this really necessary? If you want asynchronous sending and receiving without threads use asyncore module.
Or remove select from your code. The socket.recv() will block until it receives data in blocking mode, but as this is a thread, I don't see anything wrong about that. If in nonblocking mode, recv() will just return None if there is no data to receive if I remember correctly. So you don't really need select. Just check if recv() returned None. If it does, sleep some time before trying again.
The way you did it troubles your OS twice. Once for reading a socket, and second time to get the status of a socket where timeout is used to simulate sleep() more than anything else. Then the loop checks again making select() system call right after timeout confirmed that there is nothing to do for that socket.

python socketserver timeout

I'm implementing a simple server which should print a message if nothing is received for 3 seconds.
Handler
class SingleTCPHandler(SocketServer.StreamRequestHandler):
def handle(self):
while True:
message = self.rfile.readline().strip()
print message
Server
class SimpleServer(SocketServer.TCPServer):
timeout = 3
def handle_timeout(self):
print "Timeout"
def __init__(self, server_address, RequestHandlerClass):
SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass)
Here I'm extending the TCPServer for testing the timeout method.
I've set the timeout atribute to 3. According to the docs, if that time passes and no messages are sent to the client handle_timeout() is called which, in my case, just prints 'Timeout'.
BaseServer.timeout
Timeout duration, measured in seconds, or None if no timeout is desired.
If handle_request() receives no incoming requests within the
timeout period, the handle_timeout() method is called.
I start the server, and observe it's output. When i connect to it and send some messages, they are normally printed. However, if I don't send anything for 3 seconds or more, nothing happens. As if the timeout and handle_timeout() haven't been implemented.
What could be the source of this behavior?

you must not call server_forever() method for app loop.
try this one instead:
while True:
self.handle_request()
handle_timeout() works for me then.

Can you try declare the timeout at self.timeout (i.e make it a instance field instead of class variable ?)
EDIT (here is the code)
def handle_request(self):
"""Handle one request, possibly blocking.
Respects self.timeout.
"""
# Support people who used socket.settimeout() to escape
# handle_request before self.timeout was available.
timeout = self.socket.gettimeout()
if timeout is None:
timeout = self.timeout
elif self.timeout is not None:
timeout = min(timeout, self.timeout)
fd_sets = select.select([self], [], [], timeout)
if not fd_sets[0]:
self.handle_timeout()
return
self._handle_request_noblock()

Here is the document of serve_forever():
Handle requests until an explicit shutdown() request. Poll for shutdown every poll_interval seconds. Ignores self.timeout. If you need to do periodic tasks, do them in another thread
So the serve_forever() will only check if shutdown() is called every poll_interval whose default value is 0.5 seconds. And only handle_request() care about the timeout.
Here is the code for serve_forever() and handle_request().

In the end, I dropped the socketserver module and went directly with socket module, in which timeout worked.
TIMEOUT = 3
HOST = '192.0.0.202'
PORT = 2000
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(1)
while 1:
conn, addr = s.accept()
conn.settimeout(TIMEOUT)
while 1:
try:
data = conn.recv(1024)
#Do things
except socket.timeout:
#Timeout occurred, do things
if not data or P=='end':
print 'Connection lost. Listening for a new controller.'
break
conn.close()

First of all, what do you mean by "[the server] should print a message if nothing is received for 3 seconds."?
Do you mean that the server should ...
shutdown if it hadn't any new requests in certain period?
close a connection if it isn't finished in a certain period?
In the first case you can use BaseServer.timeout but you would also have to use BaseServer.handle_request() instead of BaseServer.server_forever().
In the second case you should have set the timeout for the SingleTCPHandler:
class SingleTCPHandler(SocketServer.StreamRequestHandler):
timeout = 3
def handle(self):
while True:
message = self.rfile.readline().strip()
print message
For people who want to use their own implementation of BaseRequestHandler:
class MyRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
self.request.settimeout(3)

How do I abort a socket.recvfrom() from another thread in python?

This looks like a duplicate of How do I abort a socket.recv() from another thread in Python, but it's not, since I want to abort recvfrom() in a thread, which is UDP, not TCP.
Can this be solved by poll() or select.select() ?

If you want to unblock a UDP read from another thread, send it a datagram!
Rgds,
Martin

A good way to handle this kind of asynchronous interruption is the old C pipe trick. You can create a pipe and use select/poll on both socket and pipe: Now when you want interrupt receiver you can just send a char to the pipe.
pros:
Can work both for UDP and TCP
Is protocol agnostic
cons:
select/poll on pipes are not available on Windows, in this case you should replace it by another UDP socket that use as notification pipe
Starting point
interruptable_socket.py
import os
import socket
import select
class InterruptableUdpSocketReceiver(object):
def __init__(self, host, port):
self._host = host
self._port = port
self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._r_pipe, self._w_pipe = os.pipe()
self._interrupted = False
def bind(self):
self._socket.bind((self._host, self._port))
def recv(self, buffersize, flags=0):
if self._interrupted:
raise RuntimeError("Cannot be reused")
read, _w, errors = select.select([self._r_pipe, self._socket], [], [self._socket])
if self._socket in read:
return self._socket.recv(buffersize, flags)
return ""
def interrupt(self):
self._interrupted = True
os.write(self._w_pipe, "I".encode())
A test suite:
test_interruptable_socket.py
import socket
from threading import Timer
import time
from interruptable_socket import InterruptableUdpSocketReceiver
import unittest
class Sender(object):
def __init__(self, destination_host, destination_port):
self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
self._dest = (destination_host, destination_port)
def send(self, message):
self._socket.sendto(message, self._dest)
class Test(unittest.TestCase):
def create_receiver(self, host="127.0.0.1", port=3010):
receiver = InterruptableUdpSocketReceiver(host, port)
receiver.bind()
return receiver
def create_sender(self, host="127.0.0.1", port=3010):
return Sender(host, port)
def create_sender_receiver(self, host="127.0.0.1", port=3010):
return self.create_sender(host, port), self.create_receiver(host, port)
def test_create(self):
self.create_receiver()
def test_recv_async(self):
sender, receiver = self.create_sender_receiver()
start = time.time()
send_message = "TEST".encode('UTF-8')
Timer(0.1, sender.send, (send_message, )).start()
message = receiver.recv(128)
elapsed = time.time()-start
self.assertGreaterEqual(elapsed, 0.095)
self.assertLess(elapsed, 0.11)
self.assertEqual(message, send_message)
def test_interrupt_async(self):
receiver = self.create_receiver()
start = time.time()
Timer(0.1, receiver.interrupt).start()
message = receiver.recv(128)
elapsed = time.time()-start
self.assertGreaterEqual(elapsed, 0.095)
self.assertLess(elapsed, 0.11)
self.assertEqual(0, len(message))
def test_exception_after_interrupt(self):
sender, receiver = self.create_sender_receiver()
receiver.interrupt()
with self.assertRaises(RuntimeError):
receiver.recv(128)
if __name__ == '__main__':
unittest.main()
Evolution
Now this code is just a starting point. To make it more generic I see we should fix follow issues:
Interface: return empty message in interrupt case is not a good deal, is better to use an exception to handle it
Generalization: we should have just a function to call before socket.recv(), extend interrupt to others recv methods become very simple
Portability: to make simple port it to windows we should isolate the async notification in a object to choose the right implementation for our operating system
First of all we change test_interrupt_async() to check exception instead empty message:
from interruptable_socket import InterruptException
def test_interrupt_async(self):
receiver = self.create_receiver()
start = time.time()
with self.assertRaises(InterruptException):
Timer(0.1, receiver.interrupt).start()
receiver.recv(128)
elapsed = time.time()-start
self.assertGreaterEqual(elapsed, 0.095)
self.assertLess(elapsed, 0.11)
After this we can replace return '' by raise InterruptException and the tests pass again.
The ready to extend version can be :
interruptable_socket.py
import os
import socket
import select
class InterruptException(Exception):
pass
class InterruptableUdpSocketReceiver(object):
def __init__(self, host, port):
self._host = host
self._port = port
self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._async_interrupt = AsycInterrupt(self._socket)
def bind(self):
self._socket.bind((self._host, self._port))
def recv(self, buffersize, flags=0):
self._async_interrupt.wait_for_receive()
return self._socket.recv(buffersize, flags)
def interrupt(self):
self._async_interrupt.interrupt()
class AsycInterrupt(object):
def __init__(self, descriptor):
self._read, self._write = os.pipe()
self._interrupted = False
self._descriptor = descriptor
def interrupt(self):
self._interrupted = True
self._notify()
def wait_for_receive(self):
if self._interrupted:
raise RuntimeError("Cannot be reused")
read, _w, errors = select.select([self._read, self._descriptor], [], [self._descriptor])
if self._descriptor not in read:
raise InterruptException
def _notify(self):
os.write(self._write, "I".encode())
Now wraps more recv function, implement a windows version or take care of socket timeouts become really simple.

The solution here is to forcibly close the socket. The problem is that the method for doing this is OS-specific and Python does not do a good job of abstracting the way to do it or the consequences. Basically, you need to do a shutdown() followed by a close() on the socket. On POSIX systems such as Linux, the shutdown is the key element in forcing recvfrom to stop (a call to close() alone won't do it). On Windows, shutdown() does not affect the recvfrom and the close() is the key element. This is exactly the behavior that you would see if you were implementing this code in C and using either native POSIX sockets or Winsock sockets, so Python is providing a very thin layer on top of those calls.
On both POSIX and Windows systems, this sequence of calls results in an OSError being raised. However, the location of the exception and the details of it are OS-specific. On POSIX systems, the exception is raised on the call to shutdown() and the errno value of the exception is set to 107 (Transport endpoint is not connected). On Windows systems, the exception is raised on the call to recvfrom() and the winerror value of the exception is set to 10038 (An operation was attempted on something that is not a socket). This means that there's no way to do this in an OS-agnositc way, the code has to account for both Windows and POSIX behavior and errors. Here's a simple example I wrote up:
import socket
import threading
import time
class MyServer(object):
def __init__(self, port:int=0):
if port == 0:
raise AttributeError('Invalid port supplied.')
self.port = port
self.socket = socket.socket(family=socket.AF_INET,
type=socket.SOCK_DGRAM)
self.socket.bind(('0.0.0.0', port))
self.exit_now = False
print('Starting server.')
self.thread = threading.Thread(target=self.run_server,
args=[self.socket])
self.thread.start()
def run_server(self, socket:socket.socket=None):
if socket is None:
raise AttributeError('No socket provided.')
buffer_size = 4096
while self.exit_now == False:
data = b''
try:
data, address = socket.recvfrom(buffer_size)
except OSError as e:
if e.winerror == 10038:
# Error is, "An operation was attempted on something that
# is not a socket". We don't care.
pass
else:
raise e
if len(data) > 0:
print(f'Received {len(data)} bytes from {address}.')
def stop(self):
self.exit_now = True
try:
self.socket.shutdown(socket.SHUT_RDWR)
except OSError as e:
if e.errno == 107:
# Error is, "Transport endpoint is not connected".
# We don't care.
pass
else:
raise e
self.socket.close()
self.thread.join()
print('Server stopped.')
if __name__ == '__main__':
server = MyServer(5555)
time.sleep(2)
server.stop()
exit(0)

Implement a quit command on the server and client sockets. Should work something like this:
Thread1:
status: listening
handler: quit
Thread2: client
exec: socket.send "quit" ---> Thread1.socket # host:port
Thread1:
status: socket closed()

To properly close a tcp socket in python, you have to call socket.shutdown(arg) before calling socket.close(). See the python socket documentation, the part about shutdown.
If the socket is UDP, you can't call socket.shutdown(...), it would raise an exception. And calling socket.close() alone would, like for tcp, keep the blocked operations blocking. close() alone won't interrupt them.
Many suggested solutions (not all), don't work or are seen as cumbersome as they involve 3rd party libraries. I haven't tested poll() or select(). What does definately work, is the following:
firstly, create an official Thread object for whatever thread is running socket.recv(), and save the handle to it. Secondly, import signal. Signal is an official library, which enables sending/recieving linux/posix signals to processes (read its documentation). Thirdly, to interrupt, assuming that handle to your thread is called udpThreadHandle:
signal.pthread_kill(udpthreadHandle.ident, signal.SIGINT)
and ofcourse, in the actual thread/loop doing the recieving:
try:
while True:
myUdpSocket.recv(...)
except KeyboardInterrupt:
pass
Notice, the exception handler for KeyboardInterrupt (generated by SIGINT), is OUTSIDE the recieve loop. This silently terminates the recieve loop and its thread.