In an attempt to solve this, I'm trying to simplify the problem. Suppose I have a receiver listening to both TCP and UDP messages. It will receive several strings, append them to a deque and after receiving "finish" message, it will start processing the deque.
If I receive a UDP message, I need to stop the processing, remove the last item of deque and then continue the processing.
from collections import deque
host = commands.getoutput("hostname -I")
port = 5005
backlog = 5
BUFSIZE = 4096
q = deque()
def read_tcp(s):
conn, addr = s.accept()
print('Connected with', *addr)
while 1:
data = conn.recv(BUFFER_SIZE)
if not data: break
print "received data:", data
conn.send(data) # echo
conn.close()
if (data == 'finish'):
processP(q)
else:
q.append(data)
def read_udp(s):
data,addr = s.recvfrom(1024)
print("received message:", data)
del q[-1]
processP(q):
text = q.popleft()
textReverse = text[::-1]
print(textReverse)
def run():
# create tcp socket
tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcp.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
tcp.bind((host,port))
except socket.error as err:
print('Bind failed', err)
return
tcp.listen(1)
# create udp socket
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
udp.bind((host,port))
print('***Socket now listening at***:', host, port)
input = [tcp,udp]
try:
while True:
inputready,outputready,exceptready = select(input,[],[])
for s in inputready:
if s == tcp:
read_tcp(s)
elif s == udp:
read_udp(s)
else:
print("unknown socket:", s)
# Hit Break / Ctrl-C to exit
except KeyboardInterrupt:
print('\nClosing')
raise
tcp.close()
udp.close()
if __name__ == '__main__':
run()
I have problem in pausing the program upon receiving a UDP message and then returning to the processing phase. Right now, if a UDP message is sent to my program while processing, it won't receive the message until the end of processing (and then the deque is empty). I thought maybe threading or multiprocessing may help, but I can't figure out how to apply them to the code.
Nobody forces you to empty out the dequeue. You can check if an UDP message has arrived before dequeueing the next workload. And that is as far as you can get with threads, as they do not allow you to interrupt arbitrary code. They can always only be terminated cooperatively.
If your single item processing takes too long, then multiprocessing of work-items is an option, as you can kill an external process.
Use select.select to check for incoming data on your sockets with a short timeout, before continuing to process the next workload. Alternatively you could use a thread waiting on input on the thread and manipulate the dequeue.
EDIT This is your code made to work with python3, select.select and a timeout. Triggering read_udp works with netcat with echo foo | nc -4 -u localhost 5005 but then triggers an exception because you assume the existence of elements in the dequeue - which is an application logic problem that is independent of the question how to interleave listening and working.
import socket
import select
from collections import deque
host = "localhost"
port = 5005
backlog = 5
BUFSIZE = 4096
q = deque()
def read_tcp(s):
conn, addr = s.accept()
print('Connected with', *addr)
while 1:
data = conn.recv(BUFFER_SIZE)
if not data: break
print("received data:", data)
conn.send(data) # echo
conn.close()
if (data == 'finish'):
processP(q)
else:
q.append(data)
def read_udp(s):
data,addr = s.recvfrom(1024)
print("received message:", data)
del q[-1]
def processP(q):
text = q.popleft()
textReverse = text[::-1]
print(textReverse)
def run():
# create tcp socket
tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcp.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
tcp.bind((host,port))
except socket.error as err:
print('Bind failed', err)
return
tcp.listen(1)
# create udp socket
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
udp.bind((host,port))
print('***Socket now listening at***:', host, port)
input = [tcp,udp]
try:
while True:
print("select.select")
inputready,outputready,exceptready = select.select(input,[],[], 0.1)
for s in inputready:
if s == tcp:
read_tcp(s)
elif s == udp:
read_udp(s)
else:
print("unknown socket:", s)
# Hit Break / Ctrl-C to exit
except KeyboardInterrupt:
print('\nClosing')
raise
tcp.close()
udp.close()
if __name__ == '__main__':
run()
Related
My server.py
import socket
s = socket.socket()
port = 12345
while True:
s.bind(('', port))
s.listen(5)
c, addr = s.accept()
print ("Socket Up and running with a connection from",addr)
while True:
rcvdData = c.recv(1024).decode()
print("S:",rcvdData)
c.send(rcvdData.encode())
if(sendData == "Bye" or sendData == "bye"):
break
c.close()
My client.py
import socket
s = socket.socket()
s.connect(('127.0.0.1',12345))
while True:
str = input("S: ")
s.send(str.encode());
if(str == "Bye" or str == "bye"):
break
print ("N:",s.recv(1024).decode())
s.close()
I have read to try to add try/except but im not sure where should i add that. Sorry for my english.
I want to accept connections even if no one is waiting to be accepted.
Multiple client connections are handled by calling accept, once per client. Nominally, you could just add a while loop plus exception handling to manage the connections:
import socket
import struct
s = socket.socket()
port = 12345
while True:
s.bind(('', port))
s.listen(5)
while True:
c, addr = s.accept()
print ("Socket Up and running with a connection from",addr)
try:
while True:
rcvdData = c.recv(1024).decode()
print("S:",rcvdData)
c.send(rcvdData.encode())
if(sendData == "Bye" or sendData == "bye"):
break
except (OSError, socket.error) as e:
print("Error:", e)
# reset connection
c.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 0))
c.close()
else:
c.shutdown(socket.SHUT_RDWR)
c.close()
This will only allow a single client connection at at time. If you want to handle multiple connections at once, there are a hundred ways to do it and that's beyond the scope of an SO answer.
I have this code:
import sys
import socket
import asyncio
async def main(dest_addr, max_hops=30, timeout=0.5):
loop = asyncio.get_event_loop()
queue = asyncio.Queue()
port = 33434
rx = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
rx.settimeout(timeout)
rx.bind(("", port))
def reader():
try:
_, addr = rx.recvfrom(512)
addr = addr[0]
except socket.timeout:
addr = None
queue.put_nowait(addr)
loop.add_reader(rx, reader)
tx = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
for ttl in range(1, max_hops + 1):
tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
tx.sendto(b"", (dest_addr, port))
addr = await queue.get()
print(ttl, addr)
loop.remove_reader(rx)
if __name__ == "__main__":
dest_name = sys.argv[1]
dest_addr = socket.gethostbyname(dest_name)
print(f"traceroute to {dest_name} ({dest_addr})")
asyncio.get_event_loop().run_until_complete(main(dest_addr))
I am basically trying to implement traceroute using asyncio.
I'm monitoring the socket file descriptor for read availability and invoke reader when I receive data from a device after using the socket.sendto method and wait for the queue to be filled before going to the next step.
However, I my program hangs right after the first iteration, on the second addr = await queue.get().
It seems like the reader callback is only invoked once and never again so the queue is not filled, which is strange because I have a timeout of 0.5s on the rx socket.
Answering my own question:
I think what happens is that, the device (my front router for example) doesn't respond anything so I am never notified when the file descriptor is ready for reading, so the callback is not invoked.
The workaround is to wrap the queue.get() within asyncio.wait_for with a timeout so it doesn't hang forever:
for ttl in range(1, max_hops + 1):
tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
tx.sendto(b"", (dest_addr, port))
try:
addr = await asyncio.wait_for(queue.get(), timeout)
except asyncio.TimeoutError:
addr = "timeout"
print(ttl, addr)
I'm using threads to have a server listening for both TCP and UDP messages. Here is the code:
from threading import Thread
import time
import socket
Listening_Port = 5005
Listening_IP = "127.0.0.1"
#define UDP listening function
def UDPListen():
BUFFER_SIZE = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # IPv4, UDP
sock.bind((Listening_IP, Listening_Port))
while True:
data, address = sock.recvfrom(BUFFER_SIZE)
print "UDP Messsage from address: ", address
print "Message: ", data
#define a TCP listening function
def TCPListen():
BUFFER_SIZE = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # IPv4, TCP
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((Listening_IP, Listening_Port))
while True:
sock.listen(1)
conn, address = sock.accept()
print "TCP connection from", address
data = conn.recv(BUFFER_SIZE)
print "Mesage: ", data
conn.close()
# main function
def main():
ThreadUDP = Thread(target=UDPListen)
ThreadTCP = Thread(target=TCPListen)
print "Starting Server..."
ThreadUDP.start()
ThreadTCP.start()
print "Server Started!"
if __name__ == "__main__":
main()
The problem is that when I press ctrl + c (even multiple times), the program is not terminated and I should close the console.
I tried something like this for def main(), but it didn't work:
def main():
ThreadUDP = Thread(target=UDPListen)
ThreadTCP = Thread(target=TCPListen)
try:
print "Starting Server..."
ThreadUDP.start()
ThreadTCP.start()
print "Server Started!"
# Hit Break / Ctrl-C to exit
except KeyboardInterrupt:
print('\nClosing')
raise
Updated code according to the solution offered in the answers:
from threading import Thread
import time
import socket
Listening_Port = 5005
Listening_IP = "10.0.0.3"
#define UDP listening function
def UDPListen():
BUFFER_SIZE = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # IPv4, UDP
sock.bind((Listening_IP, Listening_Port))
while not eve.isSet():
data, address = sock.recvfrom(BUFFER_SIZE)
print "UDP Messsage from address: ", address
print "Message: ", data
#define a TCP listening function
def TCPListen():
BUFFER_SIZE = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # IPv4, TCP
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((Listening_IP, Listening_Port))
while not eve.isSet():
sock.listen(1)
conn, address = sock.accept()
print "TCP connection from", address
data = conn.recv(BUFFER_SIZE)
print "Mesage: ", data
conn.close()
# main function
def main():
ThreadUDP = Thread(target=UDPListen)
ThreadTCP = Thread(target=TCPListen)
eve = threading.Event()
print "Starting Server..."
ThreadUDP.start()
ThreadTCP.start()
print "Server Started!"
try:
while True:
eve.wait(2)
except Exception:
eve.set()
if __name__ == "__main__":
main()
but I received an error:
NameError: Global name 'threading' is not defined
the problem you have is that you run your listeners in a thread - meaning that signals caught in the main should somehow signal the threads.
use threading.Event and then in the main function :
eve = threading.Event()
<start your threads>
try:
while True:
eve.wait(2)
except Exception
eve.set()
and in the threads instead of while True use while not eve.isSet():
so right now in order to receive your message you need to receive one
my teachers instructions are (in the main)"Modify the loop so that it only listens for keyboard input and then sends it to the server."
I did the rest but don't understand this, ... help?
import socket
import select
import sys
import threading
'''
Purpose: Driver
parameters: none
returns: none
'''
def main():
host = 'localhost'
port = 5000
size = 1024
#open a socket to the client.
try:
clientSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientSock.connect((host,port))
#exit on error
except socket.error, (value,message):
if clientSock :
clientSock.close()
print "Could not make connection: " + message
sys.exit(1)
thread1 = ClientThread()
thread1.start()
while True:
#wait for keyboard input
line = raw_input()
#send the input to the server unless its only a newline
if line != "\n":
clientSock.send(line)
#wait to get something from the server and print it
data = clientSock.recv(size)
print data
class ClientThread(threading.Thread):
'''
Purpose: the constructor
parameters: the already created and connected client socket
returns: none
'''
def __init__(self, clientSocket):
super(ClientThread, self).__init__()
self.clientSocket = clientSocket
self.stopped = False
def run(self):
while not self.stopped:
self.data = self.clientSocket.recv(1024)
print self.data
main()
I assume your purpose is to create a program that starts two threads, one (client thread) receives keyboard input and sends to the other (server thread), the server thread prints out everything it received.
Based on my assumption, you first need to start a ServerThread listen to a port (it's not like what your 'ClientThread' did). Here's an example:
import socket
import threading
def main():
host = 'localhost'
port = 5000
size = 1024
thread1 = ServerThread(host, port, size)
thread1.start()
#open a socket for client
try:
clientSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientSock.connect((host,port))
except socket.error, (value,message):
if clientSock:
clientSock.close()
print "Could not connect to server: " + message
sys.exit(1)
while True:
#wait for keyboard input
line = raw_input()
#send the input to the server unless its only a newline
if line != "\n":
clientSock.send(line)
# Is server supposed to send back any response?
#data = clientSock.recv(size)
#print data
if line == "Quit":
clientSock.close()
break
class ServerThread(threading.Thread):
def __init__(self, host, port, size):
super(ServerThread, self).__init__()
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.bind((host, port))
self.sock.listen(1)
self.data_size = size
self.stopped = False
def run(self):
conn, addr = self.sock.accept()
print 'Connected by', addr
while not self.stopped:
data = conn.recv(self.data_size)
if data == 'Quit':
print 'Client close the connection'
self.stopped = True
else:
print 'Server received data:', data
# Is server supposed to send back any response?
#conn.sendall('Server received data: ' + data)
conn.close()
if __name__ == '__main__':
main()
And these are the output:
Connected by ('127.0.0.1', 41153)
abc
Server received data: abc
def
Server received data: def
Quit
Client close the connection
You may check here for more details about Python socket: https://docs.python.org/2/library/socket.html?#example
I am writing a simple TCP server in python, and am trying to input a timeout. My current code:
import socket
def connect():
HOST = '' # Symbolic name meaning the local host
PORT = 5007 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
s.settimeout(5)
conn, addr = s.accept()
print 'Connected by', addr
return conn
conn = connect()
while 1:
data = conn.recv(1024)
if not data: break
print data
conn.close()
Issue is when I try to connect I get an error at data = conn.recv(1024)
error: [Errno 10035] A non-blocking socket operation could not be completed immediately
Code works without the timeout.
You can turn on blocking:
# ...
conn.setblocking(1)
return conn
# ...
Try to set the timeout on the socket and the blocking on the connection. Like this:
import socket
def connect():
HOST = '' # Symbolic name meaning the local host
PORT = 5007 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(5)
s.bind((HOST, PORT))
s.listen(1)
return s
s = connect()
while 1:
conn, addr = s.accept()
print 'Connected by', addr
conn.setblocking(1)
data = conn.recv(1024)
conn.close()
if not data: break
print data
s.close()
Ran into the same problem 30 minutes ago. Found a simple non-elegant work around...if you give the socket time to breathe by doing time.sleep(1), catching the 10035 error and doing a retry it works. I'm using 2.7.5...maybe this is a bug that got fixed. Not real sure.
Code sample...please understand this is very simplistic test code I use (only recv 1 byte at a time).So where 's' is the socket with a 10s timeout and 'numbytes' is number of bytes I need...
def getbytes(s,numbytes):
din = ''
socketerror10035count = 0
while True:
try:
r = s.recv(1).encode('hex')
din += r
if len(din)/2 == numbytes:
print 'returning',len(din)/2, 'bytes'
break
except socket.timeout as e:
din = 'socket timeout'
break
except socket.error as e:
if e[0] == 10035 and socketerror10035count < 5:
socketerror10035count = socketerror10035count +1
time.sleep(1)
else:
din = 'socket error'
break
except:
din = 'deaddead'
break
return din
For Python 3 and above, the above code which references e as a scriptable object will need to be changed to "e.errno". And, of course the print statements require parenthesis around the arguments.
Additionally, you may want to change the "except socket.error as e:" line to "except BlockingIOError as e:". However, the code works as is under Python 3.8.5 on Windows.