Develop Reference

python can't get a socket server to listen in a thread

Having a small problem with a multithreaded socket server in Python 3. I can't seem to start my socket listening in its own thread. I'm probably way off base.
Borrowed the code here: How to make a simple multithreaded socket server in Python that remembers clients
But I need to listen for clients within a thread or at least in the background. Can't figure out what I'm doing wrong.
Here's my code:
import socket
from threading import Thread
from cmd import Cmd
class ThreadedServer(Thread):
def __init__(self):
self.host = "127.0.0.1"
self.port = int(8080)
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind((self.host, self.port))
def listen(self):
self.sock.listen(5)
print("[Info]: Listening for connections on {0}, port {1}".format(self.host,self.port))
while True:
print("Hello?") # Just debug for now
client, address = self.sock.accept()
client.settimeout(60)
threading.Thread(target = self.listenToClient,args = (client,address)).start()
def listenToClient(self, client, address):
size = 1024
while True:
try:
data = client.recv(size)
if data:
# Set the response to echo back the recieved data
response = data
client.send(response)
else:
raise error('Client disconnected')
except:
client.close()
return False
class CommandInput(Cmd):
# Able to accept user input here but is irrelevant right now
pass
print("[Info]: Loading complete.")
clientThread = ThreadedServer().listen()
clientThread.start()
print("[Info]: Server ready!")
prompt = CommandInput()
prompt.prompt = '> '
prompt.cmdloop("[Info]: Type \'help\' for a list of commands and their descriptions/use")
As you can see I have some code after my listening part where I need to be able to accept input on the terminal. However, the code never gets there.
Here's the output quite simply:
[Info]: Loading complete.
[Info]: Listening for connections on 127.0.0.1, port 8080
Hello?
I'm expecting:
[Info]: Loading complete.
[Info]: Listening for connections on 127.0.0.1, port 8080
Hello?
[Info]: Type \'help\' for a list of commands and their descriptions/use
>
with a cursor ready for me to type.
So how can I properly get my program to listen for clients in that loop in a proper thread so that I can enter prompts at the command line and process user input (for example one of the commands I want to implement is that of a client "send" which I would be able to send debug information to connected clients)
Thanks for any assistance you can provide.

This may not be the ideal structure for setting up what you want, but seems to solve the requirement you stated.
After launching this script on a terminal, you can make sample client connections by opening one, or more, browser tabs to localhost:8080
import socket
from threading import Thread
from cmd import Cmd
# basic threading tutorial: https://www.tutorialspoint.com/python3/python_multithreading.htm
class ThreadedServer(Thread):
def __init__(self):
Thread.__init__(self) # change here
self.host = "127.0.0.1"
self.port = int(8080)
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind((self.host, self.port))
def run(self): # change here
self.sock.listen(5)
print("[Info]: Listening for connections on {0}, port {1}".format(self.host,self.port))
while True:
print("Hello?") # Just debug for now
client, address = self.sock.accept()
client.settimeout(60)
Thread(target = self.listenToClient, args = (client,address)).start() # change here
def listenToClient(self, client, address):
size = 1024
while True:
try:
data = client.recv(size)
if data:
# Set the response to echo back the recieved data
response = data
client.send(response)
else:
raise error('Client disconnected')
except:
client.close()
return False
class CommandInput(Cmd):
# Able to accept user input here but is irrelevant right now
pass
if __name__ == "__main__":
print("[Info]: Loading complete.")
server = ThreadedServer() # change here
server.start() # change here
print("[Info]: Server ready!")
prompt = CommandInput()
prompt.prompt = '> '
prompt.cmdloop("[Info]: Type \'help\' for a list of commands and their descriptions/use")

How to fix multiprocessing echo server to handle multiple clients

I want to create a multiprocessing echo server. I am currently using telnet as my client to send messages to my echo server.Currently I can handle one telnet request and it echos the response. I initially, thought I should intialize the pid whenever I create a socket. Is that correct?
How do I allow several clients to connect to my server using multiprocessing.
#!/usr/bin/env python
import socket
import os
from multiprocessing import Process
def create_socket():
# Create socket
sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Port for socket and Host
PORT = 8002
HOST = 'localhost'
# bind the socket to host and port
sockfd.bind((HOST, PORT))
# become a server socket
sockfd.listen(5)
start_socket(sockfd)
def start_socket(sockfd):
while True:
# Establish and accept connections woth client
(clientsocket, address) = sockfd.accept()
# Get the process id.
process_id = os.getpid()
print("Process id:", process_id)
print("Got connection from", address)
# Recieve message from the client
message = clientsocket.recv(2024)
print("Server received: " + message.decode('utf-8'))
reply = ("Server output: " + message.decode('utf-8'))
if not message:
print("Client has been disconnected.....")
break
# Display messags.
clientsocket.sendall(str.encode(reply))
# Close the connection with the client
clientsocket.close()
if __name__ == '__main__':
process = Process(target = create_socket)
process.start()

It's probably a good idea to understand which are blocking system calls and which are not. listen for example is not blocking and accept is blocking one. So basically - you created one process through Process(..), that blocks at the accept and when a connection is made - handles that connection.
Your code should have a structure - something like following (pseudo code)
def handle_connection(accepted_socket):
# do whatever you want with the socket
pass
def server():
# Create socket and listen to it.
sock = socket.socket(....)
sock.bind((HOST, PORT))
sock.listen(5)
while True:
new_client = sock.accept() # blocks here.
# unblocked
client_process = Process(target=handle_connection, args=(new_client))
client_process.start()
I must also mention, while this is a good way to just understand how things can be done, it is not a good idea to start a new process for every connection.

The initial part of setting up the server, binding, listening etc (your create_socket) should be in the master process.
Once you accept and get a socket, you should spawn off a separate process to take care of that connection. In other words, your start_socket should be spawned off in a separate process and should loop forever.

Socket Server with multiply Clients

I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?

You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.

I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.

How to TCP server serves forever?

Hello everyone!
I'm new to python networking programming.
My development environments are as below.
Windows 7
Python 3.4
I am studying with "Python Network Programming Cookbook". In this book, there's an example of ThreadingMixIn socket server application.
This book's code is written in Python 2.7. So I've modified for python 3.4.
The code is...
# coding: utf-8
import socket
import threading
import socketserver
SERVER_HOST = 'localhost'
SERVER_PORT = 0 # tells the kernel to pick up a port dynamically
BUF_SIZE = 1024
def client(ip, port, message):
""" A client to test threading mixin server"""
# Connect to the server
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
message = bytes(message, encoding="utf-8")
sock.sendall(message)
response = sock.recv(BUF_SIZE)
print("Client received: {0}".format(response))
finally:
sock.close()
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
""" An example of threaded TCP request handler """
def handle(self):
data = self.request.recv(1024)
current_thread = threading.current_thread()
response = "{0}: {0}".format(current_thread.name, data)
response = bytes(response, encoding="utf-8")
self.request.sendall(response)
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
"""Nothing to add here, inherited everything necessary from parents"""
pass
if __name__ == "__main__":
# Run server
server = ThreadedTCPServer((SERVER_HOST, SERVER_PORT),
ThreadedTCPRequestHandler)
ip, port = server.server_address # retrieve ip address
# Start a thread with the server -- one thread per request
server_thread = threading.Thread(target=server.serve_forever)
# Exit the server thread when the main thread exits
server_thread.daemon = True
server_thread.start()
print("Server loop running on thread: {0}".format(server_thread))
# Run clients
client(ip, port, "Hello from client 1")
client(ip, port, "Hello from client 2")
client(ip, port, "Hello from client 3")
This code works perfect. Every client's request processed by new thread. And when the client's request is over, program ends.
I want to make server serves forever. So when the additional client's request has come, server send its response to that client.
What should I do?
Thank you for reading my question.
P.S: Oh, one more. I always write say hello in top of my post of stack overflow. In preview it shows normally. But when the post has saved, first line always gone. Please anyone help me XD

Your program exits because your server thread is a daemon:
# Exit the server thread when the main thread exits
server_thread.daemon = True
You can either remove that line or add server_thread.join() at the bottom of the code to prevent the main thread from exiting early.

You will have to run on an infinite loop and on each loop wait for some data to come from client. This way the connection will be kept alive.
Same infinite loop for the server to accept more clients.
However, you will have to somehow detect when a client closes the connection with the server because in most times the server won't be notified.

Why does my python script not keep listening

Hello stackoverflow users, so I have this problem where i am trying to code a web server but the script ends before I can even test if it works. So my question is how can I make the script so that it will keep running forever?
#import threading
#import socket
#import signal # Signal support (server shutdown on signal receive)
import multiprocessing
#import queue
def MasterProcessA():
import socket
import multiprocessing
import threading
HOST = '97.107.139.231' # Symbolic name, meaning all available interfaces
PORT = 8080 # Arbitrary non-privileged port
#print(PORT)
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#print 'Socket created'
#Bind socket to local host and port
#try:
socket.bind((HOST, PORT))
#except socket.error as msg:
##print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
#print 'Socket bind complete'
#Start listening on socket
socket.listen(100)
print "starting server"
def ConnProcessA():
print "thread step one"
Conn, Address = socket.accept()
t = threading.Thread(target=ConnectionProcessorA)
print "thread step two"
#t.daemon = True
t.start()
#print("A Got connection from:", Address)
DataRecived = Conn.recv(1024) #receive data from client
DataRecived = bytes.decode(DataRecived) #decode it to string
print DataRecived
Conn.send("HELLO World")
Conn.clouse()
ConnProcessA = threading.Thread(target=ConnProcessA)
#t.daemon = True
ConnProcessA.start()
MasterProcessA = multiprocessing.Process(target=MasterProcessA)
MasterProcessA.start()

There are several issues with your codes.
The thread doesn't run. You need to modify:
ConnProcessA.start()
ConnProcessA.join()
ConnectionProcessorA is not included in the codes you pasted. So I have to comment out these 2 lines:
t = threading.Thread(target=ConnectionProcessorA)
t.start()
Name shadowing. Refer to these lines:
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ConnProcessA = threading.Thread(target=ConnProcessA)
MasterProcessA = multiprocessing.Process(target=MasterProcessA)
You named instances with existing module/function names. Name shadowing is very dangerous. Just try to execute any of the 3 lines twice, you will see the error.
On the other side, it's ok to reuse a variable to hold different things, if straightforward enough. E.g., following two lines are close enough to avoid confusion:
DataRecived = Conn.recv(1024) #receive data from client
DataRecived = bytes.decode(DataRecived) #decode it to string
Seems your socket server is to continuously listen to a same port, then you probably need to add s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1). As said at the most bottom of the doc.
A less severe point, in Python coding convention, CapWords is usually used for class names ref. You used it for both function names and variable names. Better to keep your codes consistent.