I wrote a python client to communicate with server side. Each time when I finished sanding out data, I have to call sock.shutdown(socket.SHUT_WR), otherwise the server would not do any response. But after calling sock.shutdown(socket.SHUT_WR), I have to reconnect the connection as sock.connect((HOST, PORT)), other wise I can not send data to server. So how can I keep the connection alive without close it.
My sample code as following:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.sendall(data)
sock.shutdown(socket.SHUT_WR)
received = sock.recv(1024)
while len(received)>0:
received = sock.recv(1024)
sock.sendall(newdata) # this would throw exception
The Server Side code as following:
def handle(self):
cur_thread = threading.current_thread()
while True:
self.data = self.rfile.read(bufsiz=100)
if not self.data:
print 'receive none!'
break
try:
data = self.data
print 'Received data, length: %d' % len(data)
self.wfile.write('get received data\n')
except Exception:
print 'exception!!'
You didn't show any server side code but I suspect it simply reads bytes until it gets none anymore.
You can't do this as you found out, because then the only way to tell the server the message is complete is by killing the connection.
Instead you'll have to add some form of framing in your protocol. Possible approaches include a designated stop character that the server recognises (such as a single newline character, or perhaps a 0-byte), sending frames in fixed sizes that your client and server agree upon, or send the frame size first as a network encoded integer followed by exactly the specified number of bytes. The server then first reads the integer and then exactly that same number of bytes from the socket.
That way you can leave the connection open and send multiple messages.
Related
I've got a server side socket that sends a message very frequently (it updates the same message with new data). The client side is processing that information as soon as the message is received.
My problem is that while the client is processing the message, the server side might have already send a few messages.
How could the client receive only the last message and drop all the pending packages?
This is the code I use for my client side. I made it non-blocking hoping it would solve my problem, but it didn't, so I don't know if it is even needed now.
import select
import socket
import time
class MyClient:
def __init__(self):
self.PORT = 5055
self.SERVER = socket.gethostbyname(socket.gethostname())
self.client = self.connect()
self.client.setblocking(False)
def connect(self):
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((self.SERVER, self.PORT))
return client
client = MyClient()
while True:
inputs = [client.client]
while inputs:
readable, writable, exceptional = select.select(inputs, [], inputs, 0.5)
for s in readable:
data = client.client.recv(2048)
print(data)
time.sleep(1)
There is no concept of a message in TCP in the first place, which also means that TCP cannot have any semantic for "last message". Instead TCP is a byte stream. One cannot rely on a 1:1 relation between a single send on one side and a single recv on the other side.
It is only possible to read data from the stream, not to skip data. Skipping data must be done by reading the data and throwing these away. To get the last message an application level protocol must first define what a message means and then the application logic must read data, detect message boundaries and then throw away all messages except the last.
I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)
recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...
I am trying to send commands to my remote server using Python sockets. My remote server is listening on port 50000 (server socket is not configured with python). When I log onto the server and echo the command I want I get the correct results back as followed:
echo mycommand | nc 127.0.0.1 50000
I get back three lines to stdout:
>ServerOn=
>Command_received
>Cmd=mycommand
I want to achieve the same on the client side using python3. I know Iam able to connect to the server, however, I am not getting the full response like above. I tried the following:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('myhost', 50000))
s.sendall('mycommand')
data = s.recv(1024)
s.close()
print(repr(data))
but I only get back the first line
>ServerOn=
Any ideas why I am missing part of the response?
The socket recv() returns when a chunk of data has arrived. There's no guarantee on what size that chunk is, other than that it's less than the passed size, 1024 in this case. Here it looks like the client has sent part of the data when it saw an end-of-line.
If you want to check/parse what you have received, it will be necessary to read it into a buffer until you have the amount of data you expect.
In the short term, keep reading until the socket closes:
connected = True
while connected:
data = s.recv(2048)
if not data:
connected = False
print("### Client Disconnected")
else:
print("Received data: %u octets" % (len(data)))
try:
as_string = str(data,'utf-8')
print(as_string)
except:
pass
I have just started learning python network programming. I was reading Foundations of Python Network Programming and could not understand the use of s.shutdown(socket.SHUT_WR) where s is a socket object.
Here is the code(where sys.argv[2] is the number of bytes user wants to send, which is rounded off to a multiple of 16) in which it is used:
import socket, sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
HOST = '127.0.0.1'
PORT = 1060
if sys.argv[1:] == ['server']:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(1)
while True:
print 'Listening at', s.getsockname()
sc, sockname = s.accept()
print 'Processing up to 1024 bytes at a time from', sockname
n = 0
while True:
message = sc.recv(1024)
if not message:
break
sc.sendall(message.upper()) # send it back uppercase
n += len(message)
print '\r%d bytes processed so far' % (n,),
sys.stdout.flush()
print
sc.close()
print 'Completed processing'
elif len(sys.argv) == 3 and sys.argv[1] == 'client' and sys.argv[2].isdigit():
bytes = (int(sys.argv[2]) + 15) // 16 * 16 # round up to // 16
message = 'capitalize this!' # 16-byte message to repeat over and over
print 'Sending', bytes, 'bytes of data, in chunks of 16 bytes'
s.connect((HOST, PORT))
sent = 0
while sent < bytes:
s.sendall(message)
sent += len(message)
print '\r%d bytes sent' % (sent,),
sys.stdout.flush()
print
s.shutdown(socket.SHUT_WR)
print 'Receiving all the data the server sends back'
received = 0
while True:
data = s.recv(42)
if not received:
print 'The first data received says', repr(data)
received += len(data)
if not data:
break
print '\r%d bytes received' % (received,),
s.close()
else:
print >>sys.stderr, 'usage: tcp_deadlock.py server | client <bytes>'
And this is the explanation that the author provides which I am finding hard to understand:
Second, you will see that the client makes a shutdown() call on the socket after it finishes sending its transmission. This solves an important problem: if the server is going to read forever until it sees end-of-file, then how will the client avoid having to do a full close() on the socket and thus forbid itself from doing the many recv() calls that it still needs to make to receive the server’s response? The solution is to “half-close” the socket—that is, to permanently shut down communication in one direction but without destroying the socket itself—so that the server can no longer read any data, but can still send any remaining reply back in the other direction, which will still be open.
My understanding of what it will do is that it will prevent the client application from further sending the data and thus will also prevent the server side from further attempting to read any data.
What I cant understand is that why is it used in this program and in what situations should I consider using it in my programs?
My understanding of what it will do is that it will prevent the client
application from further sending the data and thus will also prevent
the server side from further attempting to read any data.
Your understanding is correct.
What I cant understand is that why is it used in this program …
As your own statement suggests, without the client's s.shutdown(socket.SHUT_WR) the server would not quit waiting for data, but instead stick in its sc.recv(1024) forever, because there would be no connection termination request sent to the server.
Since the server then would never get to its sc.close(), the client on his part also would not quit waiting for data, but instead stick in its s.recv(42) forever, because there would be no connection termination request sent from the server.
Reading this answer to "close vs shutdown socket?" might also be enlightening.
The explanation is half-baked, it applies only to this specific code and overall I would vote with all-fours that this is bad practice.
Now to understand why is it so, you need to look at a server code. This server works by blocking execution until it receives 1024 bytes. Upon reception it processes the data (makes it upper-case) and sends it back. Now the problem is with hardcoded value of 1024. What if your string is shorter than 1024 bytes?
To resolve this you need to tell the server that - hey there is no more data coming your way, so return from message = sc.recv(1024) and you do this by shutting down the socket in one direction.
You do not want to fully close the socket, because then the server would not be able to send you the reply.
I have a SocketServer.StreamRequestHandler server that calls self.rfile.readline() to read a request and then calls self.wfile.write(data) to send back some data:
class FileServerHandler(SocketServer.StreamRequestHandler):
def handle(self):
# self.rfile is a file-like oject created by the handler
data = self.rfile.readline()
if data == "msg":
self.wfile.write(someOtherData)
I want my client to be able to send the request and receive the "someOtherData" from the server:
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.send("msg")
print sock.recv(1024)
sock.close()
But the client hangs when I try this. Where am I going wrong? Also is it necessary to know how much data the socket recv's or is there a way to just receive all the data the server writes?
As your server is doing a self.rfile.readline() it is constantly reading until it receives a newline ("\n") character. Thus your client needs to send sock.send("msg\n") for it to terminate the read command.
beside Jan answers I like to mention if you want to receive your exact message, you need to use strip to drop '\n' that you have put at the end of your string .