I have this code
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
_data = self.request.recv(1024)
Utils.log("Received from %s: %s" % (self.client_address, _data))
calling it with
kamcon_server = ThreadedTCPServer((HOST, 3011), ThreadedTCPRequestHandler)
server_thread = threading.Thread(target = kamcon_server.serve_forever)
server_thread.setDaemon(True)
server_thread.start()
I can connect to the host, and the server can send the data, but when the client sends something to the server, the connection is automatically closed. Why? Thank you.
Your handle() method is only calling recv() once per connection. If you want to handle multiple messages from the client, you need to loop. You should think about your protocol as well, so that you can handle request/response messages larger than 1024 bytes (e.g. parse _data and figure out if you have a complete message, buffer partial requests, etc).
For example:
def handle(self):
close = 0
while not close:
_data = self.request.recv(1024)
if not _data:
# EOF, client closed, just return
return
Utils.log("Received from %s: %s" % (self.client_address, _data))
self.request.send('got %d bytes\r\n' % len(_data))
if 'quit' in _data:
close = 1
Client session:
% telnet localhost 3011
hi
got 4 bytes
bye
got 5 bytes
telnet> quit
Try this code, it allows multiple connections to same port and not close socket until client do it:
import SocketServer
import socket, threading
class MyTCPHandler(SocketServer.BaseRequestHandler):
BUFFER_SIZE = 4096
def handle(self):
while 1:
#get input with wait if no data
data = self.request.recv(self.BUFFER_SIZE)
#suspect many more data (try to get all - without stop if no data)
if (len(data)==self.BUFFER_SIZE):
while 1:
try: #error means no more data
data += self.request.recv(self.BUFFER_SIZE, socket.MSG_DONTWAIT)
except:
break
#no data found exit loop (posible closed socket)
if (data == ""): break
#processing input
print "%s (%s) wrote: %s" % (self.client_address[0], threading.currentThread().getName(), data.strip())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
server = SocketServer.ThreadingTCPServer((HOST, PORT), MyTCPHandler)
server.serve_forever()
You can also use ForkingTCPServer instead ThreadingTCPServer.
the handle() method is called once a new TCP connection is set up, not once every time there is data available. You're supposed to handle all comminication of a TCP session in this method.
Since all you do is read one chunk of data, the connection getst closed when the handle() method returns.
Look into how TCP connections work, at the end of every 'Stream' of TCP data the connection is closed, to tell the listener that no more data is incoming in that batch. Roughly a TCP connection is as follows:
Open Connection
Send Data
Close Connection
If you're just looking at sending packets of data, try using UDP instead that way each packet will be read on arrival, provided you're listening for it.
It might be worth telling us what your planning on using this server for...
Anyway hope that helps.
Related
I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?
You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.
I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.
I'm trying to create a simple server that receiving an option and returning a method.
So I looked for a few examples and I found this one:
def start_server(ip="0.0.0.0", port=8820):
sock = socket.socket()
sock.bind((ip, port))
sock.listen(1)
print "Server started listening on %s:%d" % (ip, port)
return sock
def new_client(sock):
c_sock, c_addr = sock.accept()
print "got new client %s" % c_addr
return c_sock, c_addr
How do I create the start server and client calls?
I want the client to send an option and the server return the method
with a loop that ends in call the quit method.
like an example method if the client pressed "name":
def get_name():
return "My name is Server!"
thanks!
Here's a quick sample that creates an "echo" server. (It just sends all the data it gets from a client back to the client.)
Add this to the end of your existing code:
import select
def main():
lsock = start_server()
rfds=[lsock]
while True:
readyfds, _, _ = select.select(rfds, [], [])
for fd in readyfds:
if fd == lsock:
csock, caddr = new_client(lsock)
rfds.append(csock)
else:
rdata = fd.recv(1024)
if rdata:
print("Got some data from client socket %s" % fd)
# Send data back to client
fd.send(rdata)
else:
print("Client socket closed %s" % fd)
rfds.remove(fd)
fd.close()
main()
Annotation: We are using select here to allow a single thread to wait on multiple file descriptors at once. Whenever a connection request comes in from a client, the listening socket becomes "ready"; we then accept the connection and add the new socket to the set of descriptors in our selection list.
When we receive data from any other socket, we call recv. If that succeeds, we send the same data back to the client (obviously a real server would do something more useful with the data). If the recv returns zero, that means the client closed the socket, so then we remove the descriptor from the selection list.
One more thing: In the line print "got new client %s" % c_addr, c_addr is a tuple (address and port), hence the formatting fails due to TypeError: not all arguments converted during string formatting. Easy fix is to make it: print "got new client %s" % str(c_addr)
I have a client-server model where the client will constantly checking a log file and as soon as a new line comes in the log file it sends that line to the server.
Somehow I managed to work this thing using the following code.
server.py
import SocketServer
class MyTCPSocketHandler(SocketServer.BaseRequestHandler):
def handle(self):
# self.request is the TCP socket connected to the client
data = self.request.recv(1024).strip()
print data
# process the data..
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPSocketHandler)
server.serve_forever()
client.py
import time
import socket
def follow(thefile):
thefile.seek(0, 2)
while True:
line = thefile.readline()
if not line:
time.sleep(0.1)
continue
yield line
def connect_socket():
HOST, PORT = "localhost", 9999
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
return sock
if __name__ == '__main__':
logfile = open("my_log.log")
loglines = follow(logfile)
for line in loglines:
sock = connect_socket()
# send data
sock.sendall(bytes(line))
the problem is every time I need to call the connect_socket() method to send a new line.
I'm quite new to this topic so somebody please let me know is there any workaround for this to work in a such a way that once the connection is established between client and server I need to send data continuously to the server without making a new connection again and again.
If I'm connecting only one time and using the same socket object to send data it was throwing
socket.error: [Errno 32] Broken pipe
Some StackOverflow links which I have followed are given below,
1, 2, 3
One thing I found is
Broken Pipe occurs when one end of the connection tries sending data while the other end has already closed the connection.
How can I keep the connection open on both ends?
For this use case should I go for an asynchronous method and if so which framework will be the best match tornado or twisted?
After a line is transmitted, you close the connection on the client, but don't close it on the server.
From the docs:
RequestHandler.finish()
Called after the handle() method to perform any clean-up actions required. The default implementation does nothing.
So you should implement finish as well and close the socket (self.request) there.
Another option is not to close the connection on the client:
sock = connect_socket()
for line in loglines:
# send data
sock.sendall(bytes(line))
sock.sendall(b'bye') # EOF
sock.close()
However in this case you should modify the server code and make sure it can serve multiple clients and it understands when to close the socket. With all these difficulties it is nevertheless the preferable way of doing things. Ideally, a client has a single TCP connection per session for they're costly to establish.
I wrote a python client to communicate with server side. Each time when I finished sanding out data, I have to call sock.shutdown(socket.SHUT_WR), otherwise the server would not do any response. But after calling sock.shutdown(socket.SHUT_WR), I have to reconnect the connection as sock.connect((HOST, PORT)), other wise I can not send data to server. So how can I keep the connection alive without close it.
My sample code as following:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.sendall(data)
sock.shutdown(socket.SHUT_WR)
received = sock.recv(1024)
while len(received)>0:
received = sock.recv(1024)
sock.sendall(newdata) # this would throw exception
The Server Side code as following:
def handle(self):
cur_thread = threading.current_thread()
while True:
self.data = self.rfile.read(bufsiz=100)
if not self.data:
print 'receive none!'
break
try:
data = self.data
print 'Received data, length: %d' % len(data)
self.wfile.write('get received data\n')
except Exception:
print 'exception!!'
You didn't show any server side code but I suspect it simply reads bytes until it gets none anymore.
You can't do this as you found out, because then the only way to tell the server the message is complete is by killing the connection.
Instead you'll have to add some form of framing in your protocol. Possible approaches include a designated stop character that the server recognises (such as a single newline character, or perhaps a 0-byte), sending frames in fixed sizes that your client and server agree upon, or send the frame size first as a network encoded integer followed by exactly the specified number of bytes. The server then first reads the integer and then exactly that same number of bytes from the socket.
That way you can leave the connection open and send multiple messages.
I have a device that continually outputs data and I would like to send that data to a client on the same network as it is produced and I'm not finding a good solution. Here is what I'm trying.
Server:
import SocketServer
from subprocess import Popen,PIPE
class Handler(SocketServer.BaseRequestHandler):
def handle(self):
if not hasattr(self, 'Proc'):
self.Proc = Popen('r.sh', stdout=PIPE)
socket = self.request[1]
socket.sendto(self.Proc.stdout.readline(),self.client_address)
if __name__ == "__main__":
HOST, PORT = "192.168.1.1", 6001
server = SocketServer.UDPServer((HOST, PORT), Handler)
server.serve_forever()
Client:
import socket
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(data + "\n", ("192.168.1.1", 6001))
try:
received = sock.recv(1024)
while True:
print "Sent: {}".format(data)
print "Received: {}".format(received)
sock.sendto('more' + "\n", ("192.168.1.1", 6001))
received = sock.recv(1024)
except:
print "No more messages"
arg[1] for the client is a program that outputs lines of data for several minutes that I need to process as it is created. The problem seems to be that every time the client sends another request, a new Handler object is created, so I loose Proc. How can I stream Proc.stdout?
Edit: The device is a Korebot2, so I have limited access to other python libraries due to space.
Using UDP you get a new "connection" each time you send a datagram, which is the reason you notice that a new object instance is created each time you send something. You're probably using the wrong kind of protocol here... UDP is used mostly for sending distinct "datagrams", or when a longer connection is not needed. TCP is also called a "streaming" protocol, and is often used for data that has no fixed end.
Also remember that UDP is not a reliable protocol, if used over a network it is almost guaranteed that you will loose packets.