I am using socket to connect to a simple server to receive some data:
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host = "X.X.X.X"
port = Y
s.connect((host,port))
data = s.recv(512)
print data
Not all of the expected received data is received; some of it is cut off. So any operations that I need to perform on the received data is thrown off.
I've tried using s.settimeout(1) and putting the socket in non-blocking mode. Strangely, it works just fine in ipython.
Even if 512 bytes are sent from the server to the client, if they're sent through multiple packets, and that one packet has TCP Push flag set, the kernel will stop buffering and hand over the data to userland, ie, your application.
You should loop on data = s.recv(512) and print data. I'd even remove the '512', which is useless.
while True:
if not data: break
data = s.recv()
print data
Related
I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)
recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...
I am writing a simple client server program using Sockets in python. In my example, 1st I want to send a simple "text" in a packet for example, 2nd I will send an image for example.
In the client I want that if I receive a text, I print it on my console. But, if I receive a file (like an image), I save it on my hard drive.
How can I differentiate between the packets I receive on the client side?
#Server Code
import socket
host = socket.gethostname()
s_client=socket.socket()
port_server = 8081
s_client.bind((host, port_server))
s_client.listen(1)
print("Waiting for any incoming connections from clients ...")
conn_client, addr_client = s_client.accept()
#Send a text
conn_client.send("text".encode())
#Send a file (image for example)
f = open("image", "rb")
l = f.read(1024)
while (l):
conn_client.send(l)
l = f.read(1024)
#Client code
import socket
s = socket.socket()
host = socket.gethostname()
port_server = 8081
s.connect((host, port_server))
print("Connected...")
while True:
data = s.recv(1024)
#I WANT TO DIFFERENTIATE HERE
if data:
print(data)
Python's socket object is designed for low-level network communication. In other words, you can use it to send some raw data from one communication endpoint to another one. The socket object itself does not have any means to differentiate between data types or content. This typically takes place on a higher protocol layer.
For example, when a browser receives some data from a server via http, usually the content type is transmitted so the browser knows how to handle the data.
For your purpose, it might be perfectly sufficient if the server would in a similar way send some kind of prefix before the actual data - for example the string 'TXT' if the subsequent data is a text, and 'IMG' if the subsequent data is an image. The client could then evaluate these first three characters and interpret the data appropriately.
I am trying to send commands to my remote server using Python sockets. My remote server is listening on port 50000 (server socket is not configured with python). When I log onto the server and echo the command I want I get the correct results back as followed:
echo mycommand | nc 127.0.0.1 50000
I get back three lines to stdout:
>ServerOn=
>Command_received
>Cmd=mycommand
I want to achieve the same on the client side using python3. I know Iam able to connect to the server, however, I am not getting the full response like above. I tried the following:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('myhost', 50000))
s.sendall('mycommand')
data = s.recv(1024)
s.close()
print(repr(data))
but I only get back the first line
>ServerOn=
Any ideas why I am missing part of the response?
The socket recv() returns when a chunk of data has arrived. There's no guarantee on what size that chunk is, other than that it's less than the passed size, 1024 in this case. Here it looks like the client has sent part of the data when it saw an end-of-line.
If you want to check/parse what you have received, it will be necessary to read it into a buffer until you have the amount of data you expect.
In the short term, keep reading until the socket closes:
connected = True
while connected:
data = s.recv(2048)
if not data:
connected = False
print("### Client Disconnected")
else:
print("Received data: %u octets" % (len(data)))
try:
as_string = str(data,'utf-8')
print(as_string)
except:
pass
I wrote a python client to communicate with server side. Each time when I finished sanding out data, I have to call sock.shutdown(socket.SHUT_WR), otherwise the server would not do any response. But after calling sock.shutdown(socket.SHUT_WR), I have to reconnect the connection as sock.connect((HOST, PORT)), other wise I can not send data to server. So how can I keep the connection alive without close it.
My sample code as following:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.sendall(data)
sock.shutdown(socket.SHUT_WR)
received = sock.recv(1024)
while len(received)>0:
received = sock.recv(1024)
sock.sendall(newdata) # this would throw exception
The Server Side code as following:
def handle(self):
cur_thread = threading.current_thread()
while True:
self.data = self.rfile.read(bufsiz=100)
if not self.data:
print 'receive none!'
break
try:
data = self.data
print 'Received data, length: %d' % len(data)
self.wfile.write('get received data\n')
except Exception:
print 'exception!!'
You didn't show any server side code but I suspect it simply reads bytes until it gets none anymore.
You can't do this as you found out, because then the only way to tell the server the message is complete is by killing the connection.
Instead you'll have to add some form of framing in your protocol. Possible approaches include a designated stop character that the server recognises (such as a single newline character, or perhaps a 0-byte), sending frames in fixed sizes that your client and server agree upon, or send the frame size first as a network encoded integer followed by exactly the specified number of bytes. The server then first reads the integer and then exactly that same number of bytes from the socket.
That way you can leave the connection open and send multiple messages.
i have a bunch of devices that send NMEA sentences to a URL/ip.
that look like this
"$GPGGA,200130.0,3447.854659,N,11014.636735,W,1,11,0.8,41.4,M,-24.0,M,,*53"
i want to read this data in, parse it and upload the key parts to a database. i know how to parse it and upload it to the DB but i am at a complete loss on how to "read"/accept/get the data into a python program so that i can parse and upload.
my first thought was to point it at a Django page and then have Djanog parse it and upload to the database (data will be accessed from Django site) but its a NMEA sentence not a HTTP request so Django rejects it as "message Bad request syntax"
what is the best (python) way to read NMEA sentences sent to a url/IP?
thanks
I assume you have some hardware that has an ethernet connection, and it pipes out the NMEA string over its ethernet connection. this probably defaults to having some random 192.168.0.x ip address and spitting out data over port 12002 or something
you would typically create a socket to listen for this incomming data
server.py
import socket
host = "" #Localhost
port = 12002
PACKET_SIZE=1024 # how many characters to read at a time
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.bind((host,port))
sock.listen(5) # we should never have more than one client
def work_thread(client):
while True: #continuously read and handle data
data = client.recv(PACKET_SIZE)
if not data:
break # done with this client
processData(data)
while True:
client,addr = sock.accept() # your script will block here waiting for a connection
t = threading.Thread(target=work_thread,args=(client,))
t.start()
sometimes however you need to ping the device to get the data
client.py
import socket
host = "192.168.0.204" #Device IP
port = 12002
PACKET_SIZE=1024 # how many characters to read at a time
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect((host,port)) #connect to the device
while True: #continuously read and handle data
data = sock.recv(PACKET_SIZE)
processData(data)