Develop Reference

Python socket: Errno 9 Bad file descriptor when recv

My problem is like this. I would need to create a client program for quick trial that can send out some commands and listen to data(after which I would need to parse it) from the same socket. So I have created two threads(one to issue command (not shown here), the other to listen to data) to handle this after I created the sockets and connect out to the server. Server is written in other langauges.
As the same socket is to be used, I thought that the socket should be set to be unblocking
the socket after creation is send to the thread as an arguement.
I tried to run the python program. And there is a problem of OSError: [Errno 9] Bad file descriptor. I have narrowed down the problem. It got to do with the recv function.
Most of the solution down in the forum seem to point to socket closure as the main problem but i really could not see how since the socket closure was placed out of while loop.
So need somebody help to point to the problem. And here is my code (as below)
import socket
import errno
import sys
import threading
HOST = "192.168.50.35"
PORT = 2356
def listener(sock, q):
print("status thread created")
while q != True:
try:
data = sock.recv(1024)
except socket.error as socketerr:
if socketerr == errno.EAGAIN or socketerr == errno.EWOULDBLOCK:
sleep(1)
print('Data is unavailable')
continue
else:
print(socketerr)
sys.exit(1)
else:
print(f" new {data!r} recieved")
### processed data
### some processing set but not shown here
continue
sock.close()
def connect():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
try:
s.connect((HOST, PORT))
s.setblocking(0)
s.settimeout(1)
except socket.error as socketerr:
print("Error: ", socketerr)
return s
if __name__ == '__main__':
print_machine_info()
q = False
s = connect()
print("socket created")
t1 = threading.Thread(target=status, args=(s,q))
t1.daemon = False
t1.start()

Handling multiple connections in python with sockets

I have a code which works perfectly for one connection. I have seen two options for multi-client handling but I don't really understand it.
Here is the server socket code:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listening_sock:
listening_sock.bind(('', port))
listening_sock.listen()
client_soc, client_address = listening_sock.accept()
client_soc.sendall('200#Welcome to my server!'.encode())
print(f'Address {client_soc.getsockname()[0]} connected with port {client_soc.getsockname()[1]}')
while True:
# get message
msg = client_soc.recv(1024).decode()
# receive log print:
print(f'"{msg}" sent from {client_soc.getsockname()[0]}')
if 'Quit' in msg:
client_soc.sendall('200#Thanks for using my server!'.encode())
client_soc.close()
elif '0' < msg.split('#')[0] <= '9': # one of the valid actions
answer = call_action(msg.split('#')[0], db, msg.split('#')[1]) # the answer for given parameter
client_soc.sendall("200#".encode() + answer.encode())
If I have only one connection it works good and last thing I need to add is option for multiple-client handling. What is the shortest and easiest way to do it?

The code only calls accept once. Instead, call accept in a while loop and create a thread for each client connection so they are handled in parallel. Use the following pattern as an example:
import socket
import threading
# Thread to handle each "client_soc" connection
def handler(client_soc):
...
client_soc.close()
with socket.socket() as listening_sock:
listening_sock.bind(('', 8000))
listening_sock.listen()
while True:
client_soc, client_address = listening_sock.accept()
# Send each "client_soc" connection as a parameter to a thread.
threading.Thread(target=handler,args=(client_soc,), daemon=True).start()
There is also a built-in socket server that simplifies this process. Here's a tested example echo server that echoes back newline-terminated data:
from socketserver import ThreadingTCPServer,StreamRequestHandler
class echohandler(StreamRequestHandler):
def handle(self):
print(f'Connected: {self.client_address[0]}:{self.client_address[1]}')
while True:
# get message
msg = self.rfile.readline()
if not msg:
print(f'Disconnected: {self.client_address[0]}:{self.client_address[1]}')
break # exits handler, framework closes socket
print(f'Received: {msg}')
self.wfile.write(msg)
self.wfile.flush()
server = ThreadingTCPServer(('',8000),echohandler)
server.serve_forever()

Your code blocks itself.
For instance: client_soc, client_address = listening_sock.accept()
Accepts client, then while True: runs forever, so you can work with 1 connection only, because socket.accept() is called once. You should learn some of these to solve your problem: asyncio, threading, multiprocessing. These libraries will help your code to accept and work with clients concurrently. Sockets can use every, but often they are paired with asyncio: https://asyncio.readthedocs.io/

Socket Server with multiply Clients

I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?

You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.

I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.

Using one socket to both receiving and sending messages in Python

I have a python script, which already sends messages from time to time to peer using socket. I want to add a feature, that socket listens asynchronously for inbound messages and then reacts, for ex. processes content and resends a reply. Is it possible to do it using only one socket? I have to use one socket because my peer creates its socket on localhost:30000.
EDIT:
Here is some code of my tries I have done so far.
Run() method from the listening thread:
def run(self):
print("Started listening to messages from peer...")
while True:
socketLock.acquire()
try:
self.listenToInboundMessages()
finally:
socketFlag.set()
socketLock.notify()
socketLock.release()
def listenToInboundMessages(self):
clientsocket.settimeout(1)
received = clientsocket.recv(BUFF)
print(received)
Run() method for the thread which sends messages:
def run(self):
print("Ready to send messages.")
while True:
print("Attempting to acquire lock...")
socketLock.acquire()
while not socketFlag.isAvailable():
socketLock.wait
print("Attempting to send message...")
clientsocket.send("Message for per")
socketLock.release()
time.sleep(1)
I use socketLock condition variable and socketFlag class to coordinate both threads.

For windows you can use the select class to filter a list of socket connections into 3 lists, sockets that are ready to ready, sockets that are ready to write, and sockets that have errors.
Here is a basic implementation for a p2p socket connection (not class based).
import socket, select, time
s = socket.socket()
s.bind(("10.48.72.71", 8080))
s.listen(10)
connections = [s]
while True:
time.sleep(.1)
recv,write,err = select.select(connections,connections,connections)
for socket in recv:
if socket == s:
client,address = socket.accept()
connections.append(client)
else:
msg = socket.recv(4096).decode("UTF-8")
print("Recieved message from a socket, message was: "+str(msg))
for socket in write:
socket.send(bytes("Hi", "UTF-8"))
for socket in err:
print("Error with a socket")
socket.close()
connections.remove(socket)

Python Sockets, Advanced Chat Box

I want to create a server that handles a lot of clients at the same time (handles: receiving data from clients and sending data to all clients at the same time!!!)
Actually i'm trying to create a chat box. The program will work like this:
1) There's going to be a server which handles the clients.
2) More than one clients can join the server.
3) Clients send messages (Strings) to the server.
4) The server receive's the message from a client and then it send's it to all
the clients except the one he got it from.
And this is how the clients will communicate each other. No private messages available. When someone hits the enter all the clients will see the message on their screen.
The client module is easy to make, because the client communicates only with one socket (The Server).
The server module from the other hand is really complicated, i don't know how to do it (I also know about threads).
This is my atempt:
import socket, threading
class Server:
def __init__(self, ip = "", port = 5050):
'''Server Constructor. If __init__ return None, then you can use
self.error to print the specified error message.'''
#Error message.
self.error = ""
#Creating a socket object.
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Trying to bind it.
try:
self.server.bind( (ip, port) )
pass
#Failed, because socket has been shuted down.
except OSError :
self.error = "The server socket has been shuted down."
return None
#Failed, because socket has been forcibly reseted.
except ConnectionResetError:
self.error = "The server socket has been forcibly reseted."
return None
#Start Listening.
self.server.listen()
#_____Other Variables_____#
#A flag to know when to shut down thread loops.
self.running = True
#Store clients here.
self.clients = []
#_____Other Variables_____#
#Start accepting clients.
thread = threading.thread(target = self.acceptClients)
thread.start()
#Start handling the client.
self.clientHandler()
#Accept Clients.
def acceptClients(self):
while self.running:
self.clients.append( self.server.accept() )
#Close the server.
self.server.close()
#Handle clients.
def clientHandler(self):
while self.running:
for client in self.clients:
sock = client[0]
addr = client[1]
#Receive at most 1 mb of data.
#The problem is that recv will block the loop!!!
data = sock.recv(1024 ** 2)
As you can see, i accept clients using a thread so the server.accept() won't block the program. And then i store the clients into a list.
But the problem is on the clientHandler. How am i going to recv from all
clients at the same time? The first recv will block the loop!!!
I also tried to start new threads (clientHandlers) for every new client
but the problem was the synchronization.
And what about the send? The server must send data to all the clients, so the clientHandler is not yet finished. But if i mix the methods recv and send then the problem become's more complicated.
So what is the proper and best way to do this?
I'd like to give me an example too.

Multithreading is great when the different clients are independant of each other: you write your code as if only one client existed and you start a thread for each client.
But here, what comes from one client must be send to the others. One thread per client will certainly lead to a synchronization nightmare. So let's call select to the rescue! select.select allows to poll a list of sockets and returns as soon as one is ready. Here you can just build a list containing the listening socket and all the accepted ones (that part is initially empty...):
when the listening socket is ready for read, accept a new socket and add it to the list
when another socket is ready for read, read some data from it. If you read 0 bytes, its peer has been shut down or closed: close it and remove it from the list
if you have read something from one accepted socket, loop on the list, skipping the listening socket and the one from which you have read and send data to any other one
Code could be (more or less):
main = socket.socket() # create the listening socket
main.bind((addr, port))
main.listen(5)
socks = [main] # initialize the list and optionaly count the accepted sockets
count = 0
while True:
r, w, x = select.select(socks, [], socks)
if main in r: # a new client
s, addr = main.accept()
if count == mx: # reject (optionaly) if max number of clients reached
s.close()
else:
socks.append(s) # appends the new socket to the list
elif len(r) > 0:
data = r[0].recv(1024) # an accepted socket is ready: read
if len(data) == 0: # nothing to read: close it
r[0].close()
socks.remove(r[0])
else:
for s in socks[1:]: # send the data to any other socket
if s != r[0]:
s.send(data)
elif main in x: # close if exceptional condition met (optional)
break
elif len(x) > 0:
x[0].close()
socks.remove(x[0])
# if the loop ends, close everything
for s in socks[1:]:
s.close()
main.close()
You will certainly need to implement a mechanism to ask the server to stop, and to test all that but it should be a starting place

This my final program and works like a charm.
Server.py
import socket, select
class Server:
def __init__(self, ip = "", port = 5050):
'''Server Constructor. If __init__ return None, then you can use
self.error to print the specified error message.'''
#Error message.
self.error = ""
#Creating a socket object.
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Trying to bind it.
try:
self.server.bind( (ip, port) )
pass
#Failed, because socket has been shuted down.
except OSError :
self.error = "The server socket has been shuted down."
#Failed, because socket has been forcibly reseted.
except ConnectionResetError:
self.error = "The server socket has been forcibly reseted."
#Start Listening.
self.server.listen()
#_____Other Variables_____#
#A flag to know when to shut down thread loops.
self.running = True
#Store clients here.
self.sockets = [self.server]
#_____Other Variables_____#
#Start Handling the sockets.
self.handleSockets()
#Handle Sockets.
def handleSockets(self):
while True:
r, w, x = select.select(self.sockets, [], self.sockets)
#If server is ready to accept.
if self.server in r:
client, address = self.server.accept()
self.sockets.append(client)
#Elif a client send data.
elif len(r) > 0:
#Receive data.
try:
data = r[0].recv( 1024 )
#If the client disconnects suddenly.
except ConnectionResetError:
r[0].close()
self.sockets.remove( r[0] )
print("A user has been disconnected forcible.")
continue
#Connection has been closed or lost.
if len(data) == 0:
r[0].close()
self.sockets.remove( r[0] )
print("A user has been disconnected.")
#Else send the data to all users.
else:
#For all sockets except server.
for client in self.sockets[1:]:
#Do not send to yourself.
if client != r[0]:
client.send(data)
server = Server()
print("Errors:",server.error)
Client.py
import socket, threading
from tkinter import *
class Client:
def __init__(self, ip = "192.168.1.3", port = 5050):
'''Client Constructor. If __init__ return None, then you can use
self.error to print the specified error message.'''
#Error message.
self.error = ""
#Creating a socket object.
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Trying to bind it.
try:
self.server.connect( (ip, port) )
pass
#Failed, because socket has been shuted down.
except OSError :
self.error = "The client socket has been shuted down."
return
#Failed, because socket has been forcibly reseted.
except ConnectionResetError:
self.error = "The client socket has been forcibly reseted."
return
#Failed, because socket has been forcibly reseted.
except ConnectionRefusedError:
self.error = "The server socket refuses the connection."
return
#_____Other Variables_____#
#A flag to know when to shut down thread loops.
self.running = True
#_____Other Variables_____#
#Start the GUI Interface.
def startGUI(self):
#Initialiazing tk.
screen = Tk()
screen.geometry("200x100")
#Tk variable.
self.msg = StringVar()
#Creating widgets.
entry = Entry( textvariable = self.msg )
button = Button( text = "Send", command = self.sendMSG )
#Packing widgets.
entry.pack()
button.pack()
screen.mainloop()
#Send the message.
def sendMSG(self):
self.server.send( str.encode( self.msg.get() ) )
#Receive message.
def recvMSG(self):
while self.running:
data = self.server.recv(1024)
print( bytes.decode(data) )
#New client.
main = Client()
print("Errors:", main.error)
#Start a thread with the recvMSG method.
thread = threading.Thread(target = main.recvMSG)
thread.start()
#Start the gui.
main.startGUI()
#Close the connection when the program terminates and stop threads.
main.running = False
main.server.close()
The program works fine exactly as i wanted.
But i have some more questions.
r, w, x = select.select(self.sockets, [], self.sockets)
r is a list which contains all the ready sockets.
But i did not undesrand what w and x are.
The first parameter is the sockets list, the second the accepted clients
and the third parameter what is it? Why am i giving again the sockets list?