Develop Reference

Socket Server with multiply Clients

I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?

You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.

I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.

Asynchronous receiving TCP packets in python

I have an instant messaging app, with client written in python.
Code that connects to a server:
def connect(self, host, port, name):
host = str(host)
port = int(port)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.send('CONNECT:' + name)
print s.recv(1024)
return s
Then s will be stored in a self.socket.
Here is function that receives tcp packets from server and prints it to command line.
def chat(self):
while True:
data = self.socket.recv(4096)
if not data:
pass
else:
print data
So in my mind it should receive everything server sends and prints out, but it isn't happens. Anyone knows how to bring it to life?

There is a way with select function to monitor multiple streams, make a list of all the streams you need to handle and use the select function for it, for the user input use sys.stdin and all the sockets that you expect to chat with.
check this: https://docs.python.org/2/library/select.htmlBut still the best way to do asynchronous chat will be with udp, it will work really fine

Getting iOS app to talk to my home server

Currently, I have a home server in my closet basically doing nothing. It has Ubuntu Server 8.0.4 installed, apache for web development ill be using later, ssh, and python/twisted installed.
Here's the issue:
I created an app to talk to "localhost" port-40, using socket implementation, here is a link of what I did but what I want to develop off of : http://www.raywenderlich.com/3932/how-to-create-a-socket-based-iphone-app-and-server
Now connecting to the localhost is no problem but I want to expand this to work with my server.
I implemented the python protocol onto my server and changed the ip address im accessing in the iOS app. Here's the implementation I have, it's exactly the same as the tutorial except for the port i'm accessing.
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
class IphoneChat(Protocol):
def connectionMade(self):
self.factory.clients.append(self)
print "clients are ", self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
a = data.split(':')
print a
#b = password.split(':')
#print b
if len(a) > 1:
command = a[0]
content = a[1]
#username = a[2]
#password = a[2]
msg = ""
#msg2 = ""
if command == "username":
self.name = data
msg = self.name + " has joined"
#self.name = password
#msg2 = self.name + " this is his password"
#print msg
elif command == "password":
self.name = data
msg = self.name + " this is their password"
elif command == "msg":
msg = self.name + ": " + data
print msg
#msg2 = self.name + ": " + password
#print msg2
for c in self.factory.clients:
c.message(msg)#1, msg2)
def message(self, message):
self.transport.write(message + '\n')
factory = Factory()
factory.protocol = IphoneChat #here1#
factory.clients = []
reactor.listenTCP(40, factory)
print "Iphone Chat server started"
reactor.run()
So the REAL issue is, I cannot connect clients to my server or something..... I'm sorry but I am very new to networking.
Any opinion will help.

If you've not established networking communication to your server from the internet before you'll want to establish that you can do that with simple test cases first, there's a lot that can stop traffic from the internet reaching your program on your server.
Is your server connected to the internet through a router? If so, can you communicate with the server from inside the local network (i.e. use the 192.168.xxx.xxx ip address), using something like netcat or telnet?
If that works you should try from outside the network (i.e. using the other ip address, from whatismyip.net or similar). If you've really no prior experience with networking you may have neglected to set up port forwarding, this is a setting on your router.
There are a lot of tutorials around teaching you how to set up a Ubuntu home server, I suggest learning how to host a (very) simple webpage as a means of learning how to network, this will help a lot with debugging a networked program like the one you're making.

Streaming continuous data over a network with python

I have a device that continually outputs data and I would like to send that data to a client on the same network as it is produced and I'm not finding a good solution. Here is what I'm trying.
Server:
import SocketServer
from subprocess import Popen,PIPE
class Handler(SocketServer.BaseRequestHandler):
def handle(self):
if not hasattr(self, 'Proc'):
self.Proc = Popen('r.sh', stdout=PIPE)
socket = self.request[1]
socket.sendto(self.Proc.stdout.readline(),self.client_address)
if __name__ == "__main__":
HOST, PORT = "192.168.1.1", 6001
server = SocketServer.UDPServer((HOST, PORT), Handler)
server.serve_forever()
Client:
import socket
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(data + "\n", ("192.168.1.1", 6001))
try:
received = sock.recv(1024)
while True:
print "Sent: {}".format(data)
print "Received: {}".format(received)
sock.sendto('more' + "\n", ("192.168.1.1", 6001))
received = sock.recv(1024)
except:
print "No more messages"
arg[1] for the client is a program that outputs lines of data for several minutes that I need to process as it is created. The problem seems to be that every time the client sends another request, a new Handler object is created, so I loose Proc. How can I stream Proc.stdout?
Edit: The device is a Korebot2, so I have limited access to other python libraries due to space.

Using UDP you get a new "connection" each time you send a datagram, which is the reason you notice that a new object instance is created each time you send something. You're probably using the wrong kind of protocol here... UDP is used mostly for sending distinct "datagrams", or when a longer connection is not needed. TCP is also called a "streaming" protocol, and is often used for data that has no fixed end.
Also remember that UDP is not a reliable protocol, if used over a network it is almost guaranteed that you will loose packets.

Python chat client-server modification goes horribly wrong

As an exercise, I looked for a simple Python chat client-server system to modify and toy with. The first major failing I found in the system was that it used a single tcp connection for the server and client to communicate. The second was that you could only have two people (one using the client, and the other using the server) communicate. Thirdly, consecutive posts were impossible. One person sent a message, then had to wait for the other person to send a single message before talking again. Very, very limiting.
So I began threading it and experimenting with the sockets. Clients connect to the server once, give their IP addresses, create a listening thread, and then reconnect to the server's message receiver.
All posts are sent to that receiver, which iterates through a list of connected clients and connects to each of them and sends the message (with the sender's name in the beginning; misc feature).
(I know that opening a new connection so often like that is inefficient, but I wanted to keep with tcp connections until I had it working, and THEN go to UDP)
However, weird crap began happening. Suffice it to say that I have nightmares of Error 91.
Could anyone identify how to render this code operable within this structure and feature-set?
(Python version 2.6 yey; ignore the infinite loop that is just a placeholder)
SERVER CODE:
from socket import *
from time import time, ctime
import Queue, threading
IP = ''
PORT = 5000
PORTPlus = 2
PORTRec = 1000
ADS = (IP, PORT)
namelist = []
clientlist = []
class clientRec(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
print "I'm this far:", (IP, (PORT + PORTRec))
self.receiver = socket(AF_INET, SOCK_STREAM)
self.receiver.bind((IP, PORT + PORTRec))
self.sender = socket(AF_INET, SOCK_STREAM)
def run(self):
global clientlist, namelist
self.receiver.listen(10)
connected = True
while connected:
tcpcli, addr = receiver.accept()
message = tcpcli.recv(1024) # Accept clien't IP for home-dialing
for i in range(clientlist.__len__()): # For each connected client
try:
sender.connect(clientlist(i)) # connect
sender.send(namelist[i] + message) # and deliver message with sender's name
sender.close()
except:
del clientlist[i], namelist[i]
print "ADS:", (IP, 5000)
handle = clientRec()
tcpsoc = socket(AF_INET, SOCK_STREAM) # Paperwork
tcpsoc.bind(ADS) # Bind self to port
tcpsoc.listen(5) # Listen on that port0
handle.start() # Start thread
# Main
while 1:
print "Waiting for connection"
tcpcli, addr = tcpsoc.accept() # Accept unknown client
print "Connection received; handling..."
namelist.append(tcpcli.recv(1024)) # Accept client's name
client_IP = tcpcli.recv(1024) # Accept clien't IP for home-dialing
client_port = int(tcpcli.recv(1024))# Accept clien't listening port
port_assign = PORT + PORTRec
tcpcli.send(str(port_assign)) # Tell the client that port
tcpcli.close() # Close client connection
clientlist.append((client_IP, client_port))# Add client to send-list
print "Handled."
tcpsoc.close()
CLIENT CODE:
#!/usr/bin/env python
from socket import *
import threading, cgi, os
IP = ''
PORT = 5000
PORTmy = 100
ADS = (IP, PORT)
class iListen(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.receiver = socket(AF_INET, SOCK_STREAM)# Paperwork
self.receiver.bind(('', PORT + PORTmy)) # Listen on that port
self.receiver.listen(5) # Listen for posts
def run(self):
while listening:
tcpcli, addr = receiver.accept() # Accept unknown client
message = tcpcli.recv(1024)
if message == "/q":
listening = False
tcpcli.close()
# Initial CONNECT
myname = raw_input("Tell me yer name partnah: ")
tcpsoc = socket(AF_INET, SOCK_STREAM)
tcpsoc.connect(ADS) # First Connect
tcpsoc.send(myname) # Declare name
tcpsoc.send(gethostbyname(gethostname()))# Give IP address
tcpsoc.send(str(PORT + PORTmy)) # Give listening port
ADS = (IP, int(tcpsoc.recv(1024))) # Get new connect details
tcpsoc.close() # Close old connection
listen = iListen() # Create listener thread
listen.start() # Start listening
# RECONNECT
print ADS
tcpsoc = socket(AF_INET, SOCK_STREAM)
tcpsoc.connect(ADS) # reconnect to new port
connected = True
# Main Chat-loop
while connected:
mes = raw_input(">>>")
tcpsoc.send(mes)
if mes == "/q":
tcpsoc.close()
connected = False
time.sleep(4)
sys.exit()

I am working on something alot like this, but I am instead going to implement text encryption. I see you are suing lists for the client list... but I would say there is a better way of doing that. I am using a dictionary.
if you are familiar with dictionaries, skip the next paragraph.
Dicionaries can handle basicly 2 variables, and are defined using the {}.
>>> stuff = {'a':'hello','b':'world'}
>>> print stuff['a']
hello
>>> print stuff['a'],stuff['b']
hello world
so using this, you can can make a dictionary like {'username':'ipaddr'} this way you can make it so that both usernames and ips are all in one variable. If you want the end product like me, you will be making it so all the server does is repeat the message, and send it to everyone who is connected. then the server can just cycle through the usernames.
as another note, I think the tcpsoc.listen(5) is how many people can beconnected at once... I think thats what i read somewhere.
I have no idea why you would be having that error, but if you want to look at my halfway constucted code, you are more than welcome too. (ignore the import random, this is not yet used, but will be part of the encryption system)
http://piratepad.net/PwQzdU0bkk