Develop Reference

Socket Server with multiply Clients

I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?

You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.

I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.

python multithreading server

I am new to networking programming and python.
I am trying to figure out how to run different jobs at the server side.
For example, I want one function to create connections for incoming clients but in the same time I can still do some administration work from the terminal.
My code is as below but it doesn't work:
Edited: it doesn't work means it will get stuck in the init_conn() function
Like:
starting up on localhost port 8887
Thread: 0 Connected with 127.0.0.1:48080
# waiting
I am looking into SocketServer framework but don't know how that works.
from thread import *
import socket
def init_conn():
thread_count =0
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8887)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(10)
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = sock.accept()
print 'Thread: '+ str(thread_count) + ' Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
thread_count +=1
sock.close()
def clientthread(conn):
# receive data from client and send back
def console():
print 'this is console'
option = raw_input('-v view clients')
if option == 'v':
print 'you press v'
def main():
start_new_thread( init_conn(),() )
start_new_thread( console(),() )
if __name__ == "__main__":
main()

Your problem is probably that you start the program, sometimes it prints "this is console" and then it ends.
The first bug is that you call the methods instead of passing the handle to start_new_thread. It must be:
start_new_thread( init_conn, () )
i.e. no () after the function name.
The program doesn't do much because start_new_thread() apparent starts a thread and then waits for it to stop. The documentation is pretty unclear. It's better to use the new threading module; See http://pymotw.com/2/threading/
def main():
t = threading.Thread( target=init_conn )
t.daemon = True
t.start()
console()
so the code will run until console() ends.
I suggest to split the server and the command line tool. Create a client which accepts commands from the command line and sends them to the server. That way, you can start the console from anywhere and you can keep the code for the two separate.

Seeing that you're new to python, have you tried taking a look at the threading module that comes with the standard library?
import threading
... #rest of your code
while conditions==True:
i = threading.Thread(target=init_conn)
c = threading.Thread(target=console)
i.start()
c.start()
Can't say I've done too much with networking programming with python, so I don't really have much to say in that manner, but at least this should get you started with adding multithreading to your project.

Using SocketServer you may implement a client/server system. The documentation gives small examples which may be useful for you. Here is an extended example from there:
server.py :
import SocketServer
import os
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
class MyServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
# By setting this we allow the server to re-bind to the address by
# setting SO_REUSEADDR, meaning you don't have to wait for
# timeouts when you kill the server and the sockets don't get
# closed down correctly.
allow_reuse_address = True
request_queue_size = 10
def __init__(self, port):
self.host = os.uname()[1]
self.port = port
SocketServer.TCPServer.__init__(self, (self.host,self.port), MyTCPHandler)
logging.info( "Server has been started on {h}:{p}".format(h=self.host,p=self.port) )
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
# max length is here 1024 chars
self.data = self.request.recv(1024).strip()
logging.info( "received: {d}".format(d=self.data) )
# here you may execute different functions according to the
# request string
# here: just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
PORT = 8887
if __name__ == "__main__":
# Create the server, binding to localhost on port 8887
#server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server = MyServer( PORT )
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
client.py
import socket
import sys
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
HOST, PORT = "workstation04", 8887
logging.info( "connect to server {h}:{p}".format(h=HOST,p=PORT ) )
# read command line
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
logging.info( "Sent: {}".format(data) )
logging.info( "Received: {}".format(received) )
The output looks something like:
server side:
> python server.py
[2015-05-28 11:17:49,263] Server has been started on disasterarea:8887
[2015-05-28 11:17:50,972] received: my message
client side:
[2015-05-28 11:17:50,971] connect to server disasterarea:8887
[2015-05-28 11:17:50,972] Sent: my message
[2015-05-28 11:17:50,972] Received: MY MESSAGE
You can run several clients (from different consoles) in parallel. You may implement a request processor on the server side which processes the incoming requests and executes certain functions.
Alternatively, you may use the python module ParallelPython which executes python code locally on a multicore system or on a cluster and clusters. Check the http examples.
I had to force pip to install this module:
pip install --allow-external pp --allow-unverified pp pp

Python Server Client program error: "OSError: [WinError 10048]"

So I'm studying Python from the book Fundamentals of Python by Kenneth Lambert and I'm having trouble with an error from one of the programs in the book.
Here in chapter 10 talks about Clients and Servers. My professor asked us to key those programs in Python to see how they work. The first programs worked perfectly, but in a program I'm getting an error which seems to be a Windows error instead of a Python one.
This is the program in page 339:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 5000
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
When I run this program, it displays the "Waiting for connection" message in the Shell. However, when I try to connect to the program using the Command Prompt, it displays the following error:
C:\Python33>python multi-client-server.py
Traceback (most recent call last):
File "multi-client-server.py", line 30, in <module>
server.bind(ADDRESS)
OSError: [WinError 10048] Only one usage of each socket address (protocol/networ
k address/port) is normally permitted
We haven't studied this in class a lot. So I'm just wondering why this happens and how to fix it.
Thanks!

So, as per your questions:
We haven't studied this in class a lot. So I'm just wondering why this
happens and how to fix it.
Why:
You are trying to run the same code snippet from two different CMD on a Windows OS. So, when you initially execute the code snippet, the server starts listening on the port number 5000, then when you execute the same code snippet from the second CMD wndow it conflicts with the socket that is already being used by the first one.
I tested this on Windows 8.
How to fix:
To fix this issue, you have to simply use a different port number when you execute the code snippet for the second time, so that the socket(IP+port) doesn't conflicts with the previous one. Simply edit your code and put PORT = 15200 and save this file with a different name.(I have provided the code below too.) Now try executing the first code snippet file from a CMD windows and then execute the second code snippet file that you created right now from the second CMD window. The issue will be solved!
Code:
from socket import *
from time import ctime
from threading import Thread
class ClientHandler(Thread):
"""Handles a client request."""
def __init__(self, client):
Thread.__init__(self)
self._client = client
def run(self):
self._client.send(bytes(ctime() + '\nHave a nice day!' , 'ascii'))
self._client.close()
HOST = "localhost"
PORT = 15200 # Port number was changed here
BUFSIZE = 1024
ADDRESS = (HOST, PORT)
server = socket(AF_INET, SOCK_STREAM)
server.bind(ADDRESS)
server.listen(5)
# The server now just waits for connections from clients
# and hands sockets off to client handlers
while True:
print('Waiting for connection')
client, address = server.accept()
print('...connected from:', address)
handler = ClientHandler(client)
handler.start()
If you prefer then have a look here for the basic client-server issues.

Socket Server in Python

I am trying to set up a local server so that other PCs on the same local network can connect to it. When trying to do so, on the client side, I get the following error:
[Errno 10061] No connection could be made because the target machine actively refused it
I have been searching around for hours and still couldn't resolve this issue. I tried turning off my Firewall too, but nothing.
These are my server and client codes:
Server Code:
import socket
import threading
import SocketServer
import datetime
ver_codes = []
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
print threading.current_thread().isDaemon()
data = self.request.recv(1024)
command = data.split()[0]
if(command=="login"):
if(logged_in(data.split()[1])==False):
self.request.sendall(login(data.split()[1], data.split()[2]))
else:
self.request.sendall("already in")
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print "Received: {}".format(response)
finally:
sock.close()
def logged_in(id_num):
for i in ver_codes:
if(i[0]==id_num):
return True
return False
def login(username, password):
login_file = open("Login.txt", "r")
match = login_file.readline()
while(match!="*"):
if(match.split()[0]==username):
if(match.split()[1]==password):
ver_codes.append([match.split()[0], encryption_code(match.split()[2])])
login_file.close()
return "{} {}".format(match.split()[2], encryption_code(match.split()[2]))
print "And Here"
match = login_file.readline()
return "Denied"
login_file.close()
def encryption_code(to_encrypt):
now = datetime.datetime.now()
return int(str(now.microsecond)) * int(to_encrypt)
if __name__ == "__main__":
HOST, PORT = "localhost", 7274
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
print server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = False
server_thread.start()
print "Server loop running in thread:", server_thread.name
Client Code:
import socket
import sys
HOST, PORT = "localhost", 7274
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.sendall("login mahdiNolan m1373")
received = sock.recv(1024)
finally:
sock.close()
I really appreciate any help you could give me!
Thanks A LOT beforehand!

Your issue is because you're listening on localhost - this will only accept connections from the local machine.
If you want to accept connections from anywhere, instead of "localhost" just pass the empty string "". This is equivalent to specifying INADDR_ANY to the C sockets API - see the ip man page for more information, or this page also looks like it has some useful explanation. In short, this means "accept connections on any local interface".
Instead of the empty string you can instead specify an IP address of a local interface to only accept connections on that interface - it's unlikely you need to do this unless you machine has multiple network cards inside it (e.g. acting as a gateway) and you only want to serve requests on one of the networks.
Also, on the client side you should use the actual address of the machine - replace "localhost" with the IP address or hostname of the server machine. For example, something like "192.168.0.99". If you want to find the IP address of the server under Windows, open a DOS window and run the ipconfig command, look for the line with IPv4 Address (assuming you've got an IPv4 network which is very likely).
The Windows firewall will also block the server from accepting connections as you've already found, but you shouldn't need to disable it - as soon as you run your server you should see a popup window where you can instruct it to accept connections (that was on Windows 7, it might be different on other versions). In any case, turning the software firewall off should allow everything to work, although whether that's a security risk is a matter outside of the scope of this question.

Why is host aborting connection?

I'm teaching myself Python networking, and I recalled that back when I was teaching myself threading, I came across this page, so I copied the scripts, updated them for Python 3.1.1 and ran them. They worked perfectly.
Then I made a few modifications. My goal is to do something simple:
The client pickles an integer and sends it to the server.
The server receives the pickled integer, unpickles it, doubles it, then pickles it and sends it back to the client.
The client receives the pickled (and doubled) integer, unpickles it, and outputs it.
Here's the server:
import pickle
import socket
import threading
class ClientThread(threading.Thread):
def __init__(self, channel, details):
self.channel = channel
self.details = details
threading.Thread.__init__ ( self )
def run(self):
print('Received connection:', self.details[0])
request = self.channel.recv(1024)
response = pickle.dumps(pickle.loads(request) * 2)
self.channel.send(response)
self.channel.close()
print('Closed connection:', self.details [ 0 ])
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('', 2727))
server.listen(5)
while True:
channel, details = server.accept()
ClientThread(channel, details).start()
And here is the client:
import pickle
import socket
import threading
class ConnectionThread(threading.Thread):
def run(self):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 2727))
for x in range(10):
client.send(pickle.dumps(x))
print('Sent:',str(x))
print('Received:',repr(pickle.loads(client.recv(1024))))
client.close()
for x in range(5):
ConnectionThread().start()
The server runs fine, and when I run the client it successfully connects and starts sending integers and receiving them back doubled as expected. However, very quickly it exceptions out:
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Python30\lib\threading.py", line 507, in _bootstrap_inner
self.run()
File "C:\Users\Imagist\Desktop\server\client.py", line 13, in run
print('Received:',repr(pickle.loads(client.recv(1024))))
socket.error: [Errno 10053] An established connection was aborted by the softwar
e in your host machine
The server continues to run and receives connections just fine; only the client crashes. What's causing this?
EDIT: I got the client working with the following code:
import pickle
import socket
import threading
class ConnectionThread(threading.Thread):
def run(self):
for x in range(10):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 2727))
client.send(pickle.dumps(x))
print('Sent:',str(x))
print('Received:',repr(pickle.loads(client.recv(1024))))
client.close()
for x in range(5):
ConnectionThread().start()
However, I still don't understand what's going on. Isn't this just opening and closing the socket a bunch of times? Shouldn't there be time limitations to that (you shouldn't be able to open a socket so soon after closing it)?

Your client is now correct - you want to open the socket send the data, receive the reply and then close the socket.
The error original error was caused by the server closing the socket after it sent the first response which caused the client to receive a connection closed message when it tried to send the second message on the same connection.
However, I still don't understand
what's going on. Isn't this just
opening and closing the socket a bunch
of times?
Yes. This is acceptable, if not the highest performance way of doing things.
Shouldn't there be time
limitations to that (you shouldn't be
able to open a socket so soon after
closing it)?
You can open a client socket as quickly as you like as every time you open a socket you will get a new local port number, meaning that the connections won't interfere. In the server code above, it will start a new thread for each incoming connection.
There are 4 parts to every IP connection (source_address, source_port, destination_address, destination_port) and this quad (as it is known) must change for ever connection. Everything except source_port is fixed for a client socket so that is what the OS changes for you.
Opening server sockets is more troublesome - if you want to open a new server socket quickly, your
server.bind(('', 2727))
Above then you need to read up on SO_REUSEADDR.