I have a UDP broadcast of some data. I'm able to open the following client in python 2.6.1, under OSX 10.6.8, and it works. I can catch the data, all is well.
But: this code "consumes" the port, in that I can't open another one, the 2nd attempt to bind fails... and I must allow for more than one listener. Here's the code that opens the port:
import select, socket
port = 58083 # port msg is broadcast upon
# Create listening port
# ---------------------
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
s.bind(('', port))
except:
print 'failure to bind'
s.close()
raise
s.setblocking(0)
...Since UDP is intended for broadcast to multiple clients (among other things), I assume I'm doing something wrong. I just can't figure out what.
I found an example on activestate that suggested:
s.bind(('<broadcast>',port))
...but that simply fails every time. Binding to 0.0.0.0 works, but also suffers from the "one client" problem. Binding to the local IP (e.g. 192.168.1.100) doesn't work at all. Removing the bind doesn't work at all.
Anyone?
If you need multiple processes to listen on 58083, you need to set SO_REUSEADDR on the socket before socket.bind()
import select, socket
port = 58083 # port msg is broadcast upon
# Create listening port
# ---------------------
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # Allow other processes to
# bind to port
try:
s.bind(('0.0.0.0', port))
except:
print 'failure to bind'
s.close()
raise
s.setblocking(0)
After adding an infinite loop at the bottom, and running this twice on my linux server (once as root, and the other as unpriv user), I see:
root#tsunami# lsof | grep 58083
python 25908 root 3u IPv4 284835 0t0 UDP *:58083
python 25945 mpenning 3u IPv4 284850 0t0 UDP *:58083
root#tsunami#
Related
I am new to python programming. I have the task to read the broadcast feed on UDP port 4012.I have code of visual basic and it is working fine. The code is as follows.
#Dim receivingUdpClient As New UdpClient(4012)
#Dim RemoteIpEndPoint As New IPEndPoint(IPAddress.Any, 0)
#receiveBytes = receivingUdpClient.Receive(RemoteIpEndPoint)
#returnData = Encoding.ASCII.GetString(receiveBytes)
#Dim TestArray() As String = Split(returnData, ";")
I made the following program in python to read the broadcast feed on UPD port 4012, but was unable to achieve it with the following python program. The program is working and shows the cmd window message "waiting for 4012 localhost from 4012".
Can anybody help me out with this? If the code is correct then, how can i checked resolve this issue? i also want to read good material about socket programming in python specially about the UDP socket Broad Cast reading, if anybody can recommend any video or material for read.
import socket
s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_host = 'localhost'
udp_port = 4012
s.connect((udp_host,udp_port))
print("waiting for 4012",udp_host, "from" ,udp_port)
data , addr= s.recvfrom(1024)
print("Received Messages: ", data ,"from", addr)
You should use broadcast IP to listen.
Currently you are listening 'localhost', but broadcast IP is usually your subnet maximum IP (for 255.255.255.0 mask it is IP with number 255 in last octet)
You need to get right IP from somewhere. Manually you can do it with ifconfig on *nix, or ipconfig on Win:
inet 192.168.100.7 netmask 0xffffff00 broadcast 192.168.100.255
so you need 192.168.100.255
Also, easy way is to listen all IP's. To listen all IP's you could bind socket to '0.0.0.0' or just ''. But in this case you'll catch both broadcast and direct packets.
import socket
s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_host = ''
udp_port = 4012
s.connect((udp_host,udp_port))
print("waiting for 4012",udp_host, "from" ,udp_port)
data , addr= s.recvfrom(1024)
print("Received Messages: ", data ,"from", addr)
this snippet is something i use quite often do create basic socket server stuff...
socket_config = {
'udp_ip_address': 'your.ip.here.bla',
'udp_port_no': '6789',
'max_send_size': '1024'
}
#
# socket creation
#
serverSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
serverSock.bind((socket_config['udp_ip_address'],
int(socket_config['udp_port_no'])))
def receive_loop():
# eternal loop starts here
while True:
data, addr = serverSock.recvfrom(int(socket_config['max_send_size']))
data = data.decode('utf-8')
logger.debug("Message:" + data)
I'm using the socket module from Python 3.7 (shouldn't matter, as I tried activating a different Python version from different venv's).
The problem is that I've created a TCP connection listening at port 65432, an arbitrary number that I selected for this simple demo.
server.py looks like the following:
import socket
HOST = '127.0.0.1' # Standard loopback interface address (localhost)
PORT = 65432 # Non-privileged ports are > 1024
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
print('Connected by', addr)
while True:
data = conn.recv(1024)
if not data:
break
conn.sendall(data)
client.py is relatively straightforward as it makes a connection with 127.0.0.1:65432.
import socket
HOST = '127.0.0.1' # The server's hostname or IP address
PORT = 65432 # Port used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
# Send its message and then read the server's reply and prints it
s.sendall(b'Hello, world')
data = s.recv(1024)
print('Received', repr(data))
Executing server.py to open the port 65432 for listening (in first console) and then executing client.py to send a simple 'hello world' message (in a second console). This is what got printed to the first console:
Connected by ('127.0.0.1', 56051)
So far so good. Port 56051 connecting to port 65432, right? No.
I execute netstat -am (command tool utility to see state of sockets on the host machine) and found this:
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 127.0.0.1.51495 *.* LISTEN
Instead of 127.0.0.1.65432 as local address, it is using port 51495 instead.
Doing another verification check, this time firing off lsof -i -n:
COMMAND PID FD TYPE DEVICE SIZE/OFF NODE NAME
Code\x20H 51214 37u IPv4 0x1af15eb424ba89f3 0t0 TCP 127.0.0.1:51495 (LISTEN)
Both verifications confirmed that port 51495 is being used instead of 65432 as specified in my server.py and client.py scripts. Any leads or tips? Many thanks in advance!
65432 is the port number of your server socket, not your client socket. As the client end is not attached with any specific port number, it will be dynamically allocated with port number, every time you run the client code.
As far as I understood, you mentioned -
Connected by ('127.0.0.1', 56051)
is shown on the first console which is your server console. so this port number is port number of client socket. not the server socket.
In the server code, you are using, s.accept(), this function returns the connection temporary id and the address of the client which made the request. same thing you are trying to print in the code.
As #ottomeister pointed out, the process name was the first giveaway. The process name should have been Python but it showed VS Code instead, which is indicative that the port 51495 is opened by the VS Code process and has nothing to do with our socket module code.
The way the context manager was setup means that the connection will be closed the moment the last line (in this case, socket.sendall()) is executed. So the server socket is not active anymore.
I run netstat after the client socket has connected, by this point the server port is closed.
When I monitor the ports status while the server port is open (before the client socket connects with it) then sure enough 65432 is what appeared. This is confirmed in netstat, lsof and also nmap. A simple print statement after the socket connection is successful will also confirmed that the server port is in fact using the specified port number, which is 65432.
Sorry for the inconvenience, and again much appreciation to Ottomeister for first pointing this out.
I want to create a small TCP server that takes incoming TCP connections from a device that is hooked up via Ethernet to my computer.
The physical port for that has the IP 192.168.1.100 statically assigned to it.
The scripts I use as a client and server are listed at the bottom.
The setup works if I want to send messages between the python scripts. However, I am unable to receive anything from the external device (screenshot from Wireshark capture below). From what I have read I can define an interface to listen to by defining its IP. So I defined the IP of the interface as the host variable. However, I do not receive anything in my script but the messages sent by the other script. I had a similar situation already here on stackoverflow. I thought that defining the correct IP as the host would resolve this issue but it did not.
I am also having a hard time capturing the traffic between the two scripts with Wireshark at all. They did not show up anywhere.
I need to pick up these connections on the eth0 interface with the static IP 192.168.1.100:
tcp_server.py
import socket
# create a socket object
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# get local machine name
# host = socket.gethostname()
host = "192.168.1.100"
port = 9002
# bind to the port
serverSocket.bind((host, port))
# queue up to 5 requests
serverSocket.listen(5)
while True:
# establish a connection
clientSocket, addr = serverSocket.accept()
print("Got a connection from %s" % str(addr))
msg = 'Thank you for connecting' + "\r\n"
clientSocket.send(msg.encode('ascii'))
clientSocket.close()
and this as a client:
tcp_client.py
import socket
# create a socket object
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# get local machine name
# host = socket.gethostname()
host = "192.168.1.100"
port = 9002
# connection to hostname on the port.
s.connect((host, port))
# Receive no more than 1024 bytes
msg = s.recv(1024)
s.close()
print(msg.decode('ascii'))
Linux kernels >= 3.9 allow sharing of sockets between processes with in-kernel load-balancing by setting SO_REUSEPORT: http://lwn.net/Articles/542629/
How can this be used for sockets of type AF_UNIX?
It seems, it only works with TCP, not Unix domain sockets.
Here is a Python test program:
import os
import socket
if not hasattr(socket, 'SO_REUSEPORT'):
socket.SO_REUSEPORT = 15
if True:
# using TCP sockets
# works. test with: "echo data | nc localhost 8888"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s.bind(('', 8888))
else:
# using Unix domain sockets
# does NOT work. test with: "echo data | nc -U /tmp/socket1"
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
try:
os.unlink("/tmp/socket1")
except:
pass
s.bind("/tmp/socket1")
s.listen(1)
while True:
conn, addr = s.accept()
print('Connected to {}'.format(os.getpid()))
data = conn.recv(1024)
conn.send(data)
conn.close()
Start 2 instances, and test by running the following multiple times:
echo data | nc localhost 8888 for TCP
echo data | nc -U /tmp/socket1 for Unix domain sockets
When using TCP, the incoming clients will get balanced to both servers. With Unix domain sockets, the incoming clients all get connected to the last started server.
This specific kernel patch is documented here:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c617f398edd4db2b8567a28e899a88f8f574798d
As you can see from the list of patched files, the patch only affected the net/ipv4 and net/ipv6 sockets. Unix domain sockets are implemented in net/unix. So, the answer is: no, SO_REUSEPORT will not work with sockets of type AF_UNIX.
A small patch, that adds support for SO_REUSEPORT on UNIX sockets was posted, but has been rejected. This patch however did not implement load-balancing over multiple sockets, it only caused bind() not to fail, if the socket file already exists.
This use case was deemed
a really weird corner case from a user's perspective
So there is still the possibility, that a different patch, that would implement load-balancing for UNIX sockets via SO_REUSEPORT, would get accepted.
This following is a straightforward IPv4 UDP broadcast, followed by listening on all interfaces.
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, True)
sock.bind(("", 1337))
sock.sendto("hello world", ("255.255.255.255", 1337))
while True:
data, addr = sock.recvfrom(0x100)
print "received from {0}: {1!r}".format(addr, data)
I want to adjust this to send and receive both IPv4 and IPv6.
I've poked around and read as much as I can and believe the following is roughly the route I need to take:
Create an IPv6 socket.
Add the socket to a link or site local multicast group.
Send the UDP packets to the multicast address for the group in use.
Further info I have is that I may need to bind to several interfaces, and tell the socket using setsockopt() that it should also receive multicast packets. Lastly getaddrinfo() can be used across the board to gracefully "drop back" to IPv4 where IPv6 isn't available.
I have much of this implemented, but stumble primarily on the multicasting parts. A complete code example in Python, or vivid description of the constants and addresses needed are preferred.
Here's a link to python mcast demo, does both IPv4 and IPv6.
I'm currently asking a question over here that involves getting the multicast address of a received message, but the source code answers your question!
To listen:
# Initialise socket for IPv6 datagrams
sock = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
# Allows address to be reused
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Binds to all interfaces on the given port
sock.bind(('', 8080))
# Allow messages from this socket to loop back for development
sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_LOOP, True)
# Construct message for joining multicast group
mreq = struct.pack("16s15s".encode('utf-8'), socket.inet_pton(socket.AF_INET6, "ff02::abcd:1"), (chr(0) * 16).encode('utf-8'))
sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mreq)
data, addr = sock.recvfrom(1024)
and to send:
# Create ipv6 datagram socket
sock = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
# Allow own messages to be sent back (for local testing)
sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_LOOP, True)
sock.sendto("hello world".encode('utf-8'), ("ff02::abcd:1", 8080))
This is for python3.6, with python 2.7 I don't think the encodes are necessary. Also in the struct.pack line, I've seen variations of "16s15s" such as "4s", but I don't know what it is and what I have worked for me!