Suppose that I have the following code:
import socket
listener = socket.socket()
listener.bind(('0.0.0.0', 59535))
while True:
conn, addr = listener.accept()
worker_thread = threading.Thread(target=client_handler, args=(conn, addr,)).start()
What will happen if new client will try to connect to our listener socket while we're creating worker thread? Will he wait for the next accept call or will it just rejected? If he'll wait, how many clients can be in that queue simultaneously by default (yeah, I know that I can set it via listen function)?
There is a listen queue in the kernel, so the kernel will deal with new clients while your user space part does something else. If the listen queue in the kernel is full no more clients will be accepted, that is the connect will fail.
Related
The title explain everything. I made Java code that sends messgaes for the python server, but evry time, just the first message is sends because every time, java conneced again to server, and the server keeps waiting to next message from the first client that I send in the first time.
How can the server get message from all clients are connectd? and not just from one?
My python server:
server = socket.socket()
server.bind((socket.gethostname(), 4786))
server.listen(5)
(client, (ipNum, portNum)) = server.accept()
print("Client connected")
while True:
message = str(client.recv(64).decode()) # Check if client send message. I want to change it to check all clients
if(message != ""):
print("Client: " + message)
else:
time.sleep(0.1)
Summary
The server.accept() must be called inside the loop.
TLDR
The socket server returned from the call socket.socket() is a 'listening' socket. It is not used for any data transfer but just for listening incoming connections. When the server is willing to accept incoming connection then calls server.accept(). This call waits till a client connects. When a client connects the accept wakes up and returns a socket that represents a connection to one client. This socket is then used for data send and received and should be closed when the communication with this specific client is done.
When server wants to accept connection from another client it must call server.accept() again to wait for connection from another client and use the unique client socket for each connected client.
If it sufficient to handle client sequentially then you can just move the call accept onto the loop. Furthermore you should close the client socket when the communication with the client is done.
If multiple clients can be connected in parallel then slightly more complicated design is needed. You can start a new thread for each client after accepting the connection. The thread can call recv in a loop and terminates when the client disconnects. See Multi Threaded TCP server in Python for example.
I am writing a UDP server application that serves as a back end to Teltonika FMB630 car mounted devices.
I already took care of the protocol specifics and decoding, the problem I am facing relates to the UDP socket used.
My UDP server has to send an acknowledgement to the client device upon receiving a message (that is the protocol), however, if I send those ACKs, the server socket stops receiving data after a while.
The server's UDP socket object is passed to an concurrent.futures.ThreadPoolExecutor that fires a function (send_ack) that sends the ACK, however this is not the issue because I tried calling send_ack in the main thread, after receiving data and the same issue occurs.
I suspect the problem is the remote device somehow breaks the connection or the ISP or MNO doesn't route the reply packet (this is a GPRS device) and then the socket.send() method that is used to send the acknowledge, somehow freezes other socket operations, specifically recvfrom_into called in the main thread loop.
I wrote two scripts to illustrate the situation:
udp_test_echo.py :
#!/usr/env/bin python
import socket
import concurrent.futures
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.connect(addr)
print("connected to {}".format(addr))
sock.send(ack)
print("ACK sent to {}".format(addr))
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 1337))
data = bytearray([0] * 10)
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
while True:
print("listening")
nbytes, address = s.recvfrom_into(data)
print("Socket Data received {} bytes Address {}".format(nbytes, address))
print("Data received: ", data, " Echoing back to client")
executor.submit(send_ack, s, address, data[:nbytes])
udp_test_client.py:
#!/usr/env/bin python
import socket
import time
import random
def get_random_bytes():
return bytearray([random.randint(0,255) for b in range(10)])
ip = "127.0.0.1"
port = 1337
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect((ip, port))
while True:
stuff_to_send = get_random_bytes()
print("Sending stuff", stuff_to_send)
s.sendall(stuff_to_send)
print("reply: ", s.recvfrom(10))
time.sleep(0.1)
Running udp_test_echo.py in one terminal and udp_test_client.py in another, we see normal operation but if you Ctrl+C the test client and re run it, you will see that the server doesn't respond until it is restarted.
Is there a way to timeout a specific sending operation from a specific call to socket.send() method without affecting other calls ? (I want my socket.recvfrom_into call to block on the main thread)
If I settimeout on the entire socket object, I am going to have to deal with many exceptions while waiting for data in the main thread and I don't like to have to rely on exceptions for proper program operation.
The culprit was the socket.connect() call in send_ack, when being called on the server's socket object it causes the socket to no longer be bound and listen on the port specified in the start of the program.
Instead the send_ack function was changed to be:
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.sendto(ack, addr)
print("ACK sent to {}".format(addr))
socket.sendto(data, address) uses the existing connection instead of starting a new one.
I want to create a multiprocessing echo server. I am currently using telnet as my client to send messages to my echo server.Currently I can handle one telnet request and it echos the response. I initially, thought I should intialize the pid whenever I create a socket. Is that correct?
How do I allow several clients to connect to my server using multiprocessing.
#!/usr/bin/env python
import socket
import os
from multiprocessing import Process
def create_socket():
# Create socket
sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Port for socket and Host
PORT = 8002
HOST = 'localhost'
# bind the socket to host and port
sockfd.bind((HOST, PORT))
# become a server socket
sockfd.listen(5)
start_socket(sockfd)
def start_socket(sockfd):
while True:
# Establish and accept connections woth client
(clientsocket, address) = sockfd.accept()
# Get the process id.
process_id = os.getpid()
print("Process id:", process_id)
print("Got connection from", address)
# Recieve message from the client
message = clientsocket.recv(2024)
print("Server received: " + message.decode('utf-8'))
reply = ("Server output: " + message.decode('utf-8'))
if not message:
print("Client has been disconnected.....")
break
# Display messags.
clientsocket.sendall(str.encode(reply))
# Close the connection with the client
clientsocket.close()
if __name__ == '__main__':
process = Process(target = create_socket)
process.start()
It's probably a good idea to understand which are blocking system calls and which are not. listen for example is not blocking and accept is blocking one. So basically - you created one process through Process(..), that blocks at the accept and when a connection is made - handles that connection.
Your code should have a structure - something like following (pseudo code)
def handle_connection(accepted_socket):
# do whatever you want with the socket
pass
def server():
# Create socket and listen to it.
sock = socket.socket(....)
sock.bind((HOST, PORT))
sock.listen(5)
while True:
new_client = sock.accept() # blocks here.
# unblocked
client_process = Process(target=handle_connection, args=(new_client))
client_process.start()
I must also mention, while this is a good way to just understand how things can be done, it is not a good idea to start a new process for every connection.
The initial part of setting up the server, binding, listening etc (your create_socket) should be in the master process.
Once you accept and get a socket, you should spawn off a separate process to take care of that connection. In other words, your start_socket should be spawned off in a separate process and should loop forever.
I've a python Socket server running and also socket clients.
Now, for example say there are 3 clients connected to same server. Please find below the code of the server.
#!/usr/bin/python # This is server.py file
import socket # Import socket module
import threading
serversocket = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 1234 # Reserve a port for your service.
serversocket.bind((host, port)) # Bind to the port
serversocket.listen(5)
print("Bound the port ",port,"on Machine : ",host,", and ready to accept connections.\n")
def clientThread(connection):
while True:
data=connection.recv(1024)
if not data:
break
connection.send("Thanks")
connection.close()
def sendMessage(connection, message):
connection.send(message)
while 1:
connection, address = serversocket.accept()
start_new_thread(clientthread, (connection,))
serversocket.close();
Now, I need to call sendMessage for a particular client, say out of clients A,B and C, send it to B. In this case, how do I identify the thread and call that function?
You can use Queues and multiple threads per connection to solve this problem.
Basic outline:
Each client connection spawns two threads - one to monitor client input and another which monitors a Queue. Items placed on the queue will be sent to the client. Each client connection will have its own output queue.
You'll also need a global dictionary to map a client name to their output queue.
To send a message to a particular client, find the client's output queue and add the message to it.
You'll also need a way to shutdown the output thread for a client. A common approach is to use a sentinel value (like None) on the queue to inform the output thread to exit its processing loop. When the client's input thread detects EOF it can place the sentinel value on the client's output queue and eventually the output thread will shut itself down.
How can I have a socket server running that accepts incoming connections and deals with that part of the code, while not having code waiting for new connections stuck in that same loop?
I am just starting trying to learn. Would a TCP Handler be useful?
I just need some simple examples on this topic. I'm wanting something like having a commands portion in the server. So i can do certain things while the server is running.
EDIT: What I'm trying to do:
1 - TCP server for multiple clients
2 - Respond to more than one at a time when needed
3 - Text input availability at all time, to be used for getting/setting info
4 - A simple way to get/save client address info. Currently using a list to save them.
You can run your socket server in a thread.
import threading
import SocketServer
server = SocketServer.TCPServer(('localhost', 0), SocketServer.BaseRequestHandler)
th = threading.Thread(target=server.serve_forever)
th.daemon = True
th.start()
Python has builtin support of asynchronous socket handling in asyncore module (http://docs.python.org/library/asyncore.html).
Asynchronous socket handling means that You have to execute at least one iteration of socket processing loop inside Your code (main loop):
asyncore.loop(count=1)
Example taken from documentation:
import asyncore
import socket
class EchoHandler(asyncore.dispatcher_with_send):
def handle_read(self):
data = self.recv(8192)
if data:
self.send(data)
class EchoServer(asyncore.dispatcher):
def __init__(self, host, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.set_reuse_addr()
self.bind((host, port))
self.listen(5)
def handle_accept(self):
pair = self.accept()
if pair is None:
pass
else:
sock, addr = pair
print('Incoming connection from %s' % repr(addr))
handler = EchoHandler(sock)
server = EchoServer('localhost', 8080)
# Note that here loop is infinite (count is not given)
asyncore.loop()
Each time the socket accepts the connection handle_accept is called by the loop. Each time the data is available to read from socket handle_read is called and so on.
You can use both TCP and UDP sockets in this manner.
I'm not exactly sure what you are asking, but normally on the server side, you make socket(), bind() and listen() calls to setup the socket, and then loop around an accept() call. This accept() call blocks until a client connection is made.
For simple servers, you handle whatever request the client makes within the loop. For real-world servers, you need to spawn some other mechanism (e.g. a new thread or process, depending on the language/platform) to handle the request asynchronously, so that the original loop can iterate again on the accept() call and go back to listening for connections.
See the Python socket doc for more info and examples in Python:
http://docs.python.org/howto/sockets.html