Develop Reference

Recvall with while loop doesn't work between two devices in python

I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)

recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...

How to use a Python 3 socket client/server setup over the internet (not locally)?

I am trying to implement a simple chat program that uses sockets to transmit data via a UDP connection. However, I can't figure out how to correctly set it up so that people from outside my local network can access it if I am hosting it on my laptop. I am utilizing port 5000, and have port-forwarded that port on my router for my laptop. The port-forwarding doesn't seem to be the issue; at least the "Port Forward Network Utilities" from portforward.com seems to detect it as properly forwarded. Maybe I am mixing up the IP addresses I need to host from and connect with? The code in question is below:
import socket
import threading
import sys
class Server:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
connections = []
def __init__(self):
self.sock.bind(('192.168.1.5', 5000))
self.sock.listen(1)
def handler(self, c, a):
while True:
data = c.recv(1024)
for connection in self.connections:
print(data.decode())
connection.send(data)
if not data:
break
def run(self):
while True:
c, a = self.sock.accept()
cThread = threading.Thread(target=self.handler, args=(c, a))
cThread.daemon = True
cThread.start()
self.connections.append(c)
print(self.connections)
class Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
usr_name = ""
def sendMsg(self):
while True:
self.sock.send(bytes(self.usr_name + ": " + input("-> "), 'utf-8'))
def __init__(self, address):
self.sock.connect((address, 5000))
self.usr_name = input("Enter a username: ")
iThread = threading.Thread(target=self.sendMsg)
iThread.daemon = True
iThread.start()
while True:
data = self.sock.recv(1024)
if not data:
break
print(data.decode())
if len(sys.argv) > 1:
client = Client(sys.argv[1])
else:
server = Server()
server.run()
As you can see, I have my current local IP address inputted for hosting the server, while the client asks for an IP to connect to. I'm not sure what to do now for hosting this over the internet, but I have tried every IP combination I can think of and it returns a number of errors.
Thanks in advance.
Edit:
The two main errors I was getting are:
Timeout Error [WinError 10060]
My friend received this when trying to connect from another network
[WinError 10061]
I would receive this when trying to connect using my public IP from the same computer
I'm sorry that I can't be more detailed in my errors and provide a full printout, and I will try to update this if I'm able to replicate them.
Edit:
I was able to rewrite it and get it to work, I don't need anymore help with this.
Thanks.

You're port-forwarding UDP port 5000 to 5000.
But you're opening TCP streams, not UDP. That's what SOCK_STREAM means. If you want UDP, you need to use SOCK_DGRAM.
So, you need to make these two consistent. The only problem is, I'm not sure which one you actually want here.
On the one hand, your code is doing connection-oriented recv, and seems to be assuming reliable transmission, which means you probably want TCP.
On the other hand, your code seems to be assuming that each recv(1024) is going to get exactly one send from the other side, which is only true for UDP; TCP sockets are byte streams, not message streams. When you do a recv(1024), you could easily get just the first 12 bytes of an 80-byte line, which means it could end in the middle of a UTF-8 character, which means decode will throw an exception.
I think you want TCP, but with a framing protocol on top of it. The simplest protocol that would probably make sense here is lines of text. Which is pretty easy to do on your own, but even easier to do with socket.makefile, given that you're dedicating a thread to each connection.

Failing to receive broadcast udp packets in python

I'm trying to receive UDP Broadcast packets sent from FPGA connected via a LAN cable. the FPGA sends continuous packets to port 5001.
My python receiver code is simple:
from socket import *
s=socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
s.bind(('', 5001))
print "trying to receive"
msg = s.recvfrom(1024)[0]
print msg
print "I'm outta here! Bye!"
I checked using Wireshark, and I found that the PC receives the packets. However, my Python code doesn't. I also checked sending packets from another local python code (to the same address and port) and my receiver got those packets.
Wireshark captures:

The issue was the firewall permissions for python

How to capture UDP packets with Python

Facts & context elements:
I need to capture data (latitude,longitude) coming out of a GPS device rework them and make them suitable for another application (QGIS). To this end I've tried to perform (What I thought at first would be a simple one) a python based module.
According to wire shark analysis.
Source Destination Protocol length info
192.168.0.1 225.2.5.1 UPD 136 source port : 1045 destination port:6495
I've tried this code found on various sources, like this one.
import socket
import os
UDP_IP = "225.2.5.1"
UDP_PORT = 6495
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))
while True:
data, addr = sock.recvfrom(4096) # buffer size is 1024 bytes
print "received message:", data
os.system("pause")
The problem
This code doesn't work for me.The console windows whether collapse (despite the os.system("pause") or run indefinitely. As I'm not very skilled in python programming nor networking I've tested the provided code with the other IP address and port. As no result came from it I've also started to mix both of them. And finally, gave up and decided to share my issue with the community.
The aim :
I need to be able to access the data contains in this UDP frame with python 2.7 save them in a variable (data) for the next step of my programming project.
Thanks for reading and for your help

You should start your python program from the windows cmd-console or powershell, not from the explorer, then the window stays open and you see error messages. Remove the indentation error and the last line. Be sure, that your computer has the given IP-address. Bind your socket to any address:
import socket
UDP_IP = "0.0.0.0"
UDP_PORT = 6495
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))
while True:
data, addr = sock.recvfrom(4096)
print "received message:", data

What is this address that I'm getting from recvfrom?

I am trying to learn about network communications and sockets. Here is some code that I wrote:
Client code:
from socket import *
sock = socket(AF_INET, SOCK_DGRAM)
sock.bind('127.0.0.1', 3000)
data, addr = sock.recvfrom(1024)
print "Received data '" + data + "' from address:", addr
Server code:
from socket import *
sock = socket(AF_INET, SOCK_DGRAM)
sock.sendto("HELLO WORLD", ('127.0.0.1', 3000))
sock.close()
The client prints out: Received data 'HELLO WORLD!' from address: ('127.0.0.1', 60788)
To my understanding, the second member of the tuple is supposed to be the port. Furthermore, if I send several messages the number increase by 1 every time. Why is it not 3000?
As a side-note, are the htons and htonl functions necessary with the python API?

First of all: The scripts that you've written here would typically be regarded as a server and client, respectively, not client and server. This is because the server (the first script) is binding to a known port and waiting for a connection, while the client (the second script) is connecting to it from a random port and sending data.
To answer your questions directly:
Since you aren't binding the client to any specific port, it's choosing a new epheremal port for each socket, and those are being used sequentially. If the server were to send a packet back to that same host and port, it would be received by the client.
The htons and htonl functions are not generally necessary in Python. The socket module takes care of endian-swapping addresses and port numbers for you.

The port you are seeing is the port the data was sent from, not the port the data was sent to. When you don't specify the source port, and your "server" doesn't, the system assigns the socket a source port.