I have started to learn UDP sockets but for some reason this code is throwing an error. why does this code:
s=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
x=s.recvfrom(1024)
throw an error of invalid argument supplied?
UDP datagram communications take place between source and destination ports. You can assign a port with a bind call or let the network stack choose one for you by simply calling one of the send/recv methods. If you call sendto (and you have no bound a port), the stack will assign a number in the dynamic port range. If you call recvfrom, the stack will typically assign port 0. But there is no port 0. What happens next is platform dependent. Windows will attempt to bind you as a promiscuous listener. But that is a privileged call and you will likely get an invalid argument error. But it may work if you are administrator - i'm not sure.
To start a conversation, the first entity to send something needs to know what destination port to use. That means that the entity that receives the first communication needs to bind to a port number that the other side knows about in advance. This could be a well-known port number, a port hardwired into your code, something in a config file or even something advertised with a name service protocol such as LDAP.
Once that first datagram is received the receiving entity now has the senders address and port number so can talk back.
In your example, your entities could agree on a well-known port and begin a conversation. here is a datagram echoer listening on port 9999.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(('', 9999))
while 1:
data, addr = s.recvfrom(8096)
s.sendto(data, addr)
and a client that talks to it
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for i in range(10):
s.sendto(str(i), ('', 9999))
print(s.recvfrom(1024))
Related
I'm having a strange problem with python udp sockets. When I send data over them, they open an udp port on 0.0.0.0 and I cannot find out why they do, what they do listen to and how to deactivate that behaviour. Our system administrators don't like ports to be listened on 0.0.0.0 (reasonably).
Minimum example:
import socket
fam = socket.AF_INET
family, _, _, _, addr = socket.getaddrinfo('localhost', 9999, fam, socket.SOCK_DGRAM)[0]
sock = socket.socket(family, socket.SOCK_DGRAM)
sock.sendto('foobar'.encode('ascii'), addr)
Right after the last method call, the python program listens to:
udp 0 0 0.0.0.0:41972 0.0.0.0:* 1000 308716 17777/python3
And this seems to stay until the python executable stops. So my question is, does anyone here have the same problem and how can I avoid it?
Thank you very much!
Which port do you expect your packets to come from?
Sockets have ports at both ends. Packets come from a port and go to a port. You didn't pick a port (with bind), so the operating system chose one for you.
It's not a problem, it's how sockets work.
0.0.0.0 means "any IP address" by the way.
just to elaborate on my comment, here's how I'd put the socket calls together:
import socket
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
sock.bind(('127.0.0.1', 0)) # optional
sock.connect(('localhost', 9999))
sock.send(b'foobar')
notes:
connecting a UDP socket should cause it to be bound to something more appropriate than 0.0.0.0, and hence why I put an optional comment
getaddrinfo doesn't help you much, hence I'm just passing names and letting Python resolve them internally
using a name for the connect call and dotted-quad notation for bind looks a little strange. I'd suggest using just one format for consistency, or just using connect and not using bind
getaddrinfo is really useful when you want to be able to transparently handle IPv6 along with IPv4 addresses, especially for hosts that resolve to multiple addresses. see the Happy Eyeballs algorithm for an example
So, I've found a workaround, even if I think it's not the cleanest way to do that, I couldn't find another one.
I used the method bind(addr: tuple) on the udp socket to make it listen to only a specific IP. In my case this looks like:
import socket
fam = socket.AF_INET
family, _, _, _, addr = socket.getaddrinfo('localhost', 9999, fam, socket.SOCK_DGRAM)[0]
sock = socket.socket(family, socket.SOCK_DGRAM)
sock.bind(('127.0.0.1', 0)) # This is the new line where I bind only to 127.0.0.1
sock.sendto('foobar'.encode('ascii'), addr)
Thanks to #user253751 for leading me in the right direction.
I've learned how to write a python server, and figured out that I have a hole in my knowledge.
Therefore, I would glad to know more about the differences between the commands bind(), listen() of the module called socket.
In addition, when I use bind() with a specific port as a parameter, Is the particular port being in use already, before using the listen() method?!
I found a tutorial which explains in detail:
... bind() is used to associate the socket with the server address.
Calling listen() puts the socket into server mode, and accept() waits for an incoming connection.
listen() is what differentiates a server socket from a client.
Once bind() is called, the port is now reserved and cannot be used again until either the program ends or the close() method is called on the socket.
A test program that demonstrates this is as follows:
import socket
import time
HOST = '127.0.0.1'
PORT = 65432
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
while 1:
time.sleep(1)
when running two instances of this program at once, you can see that the one started last has the error:
Which proves that the port is reserved before listen() is ever called.
I currently have a raspberry pi that is set up as a wifi to ethernet bridge. The raspberry pi acts as an access point to the entire ethernet subnet that I have. The subnet and the network bridge work perfectly but when I try to get my python program on the raspberry pi to listen to requests on the ethernet interface/subnet it doesn't seem to do that. It is set to bind the socket to ('',10000) but it never receives any messages. However, it is able to send messages via sockets to the subnet just fine, just not receive. I think it is listening to the wifi interface rather than the ethernet one but I'm not sure how to specify which interface the socket is suppose to listen to.
here is my receiving code
receive_group = ('',10000)
receive = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
receive.bind(receive_group)
while(True):
data, address = receive.recv(65536)
print(data)
The bind part should be correct. The receive part is wrong because recv only return the (bytes) data, you should use recvfrom to also get the sender address. But this should work:
import socket
receive = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
receive.bind(('', 10000))
while True:
data, address = receive.recvfrom(64)
print(data, address)
I used this code for the send part:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.sendto(b'foo', (addr, 10000))
where addr is one of the (reachable) addresses of the receiver, and the receiver could successfully receive data from any of its interfaces.
To get something from socket.recv something must connect to this socket and send something. Are you sure that some program on the network is doing this?
For listening/sniffing to packets and network traffic, better use pyshark.
Turns out it wasn't anything with python. When I created the access point on the pi it created a firewall rule that blocked that port even though I never configured the firewall that way. Adding an exception to that port fixed my problem
I've written server and client programs with Python.
Server.py
import socket
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 5555
sock.bind((host, port))
sock.listen(1)
conn, addr = sock.accept()
data = "Hello!"
data = bytes(data, 'utf-8')
conn.send(data)
sock.close()
Client.py on Linux
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 5555
sock.connect((host, port))
data = sock.recv(2048)
data = str(data, "utf-8")
print(data)
sock.close()
When I run the server and then the client on my local machine (a Linux Mint), it works correctly. I got "Hello!" in bash, and everything is fine. BUT when I ran my client program on another machine (a Windows 8) and ran it (previously I ran server on Linux, of course, and change IP address in client to my static Linux mint's IP) it says:
ConnectionRefusedError: [WinError 10061] No connection could be made
because the target machine actively refused it
client.py on Windows
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "here is my static ip"
port = 5555
sock.connect((host, port))
data = sock.recv(2048)
data = str(data, "utf-8")
print(data)
sock.close()
I must say that I had done port forwarding in my router settings on port 5555. Earlier, I had done same thing to port 80 and my own site worked correctly, but now it doesn't work to 5555 with Python sockets! Why? I can't get it! And one more thing: I tried to change the port to 80 in my server and client files, but it didn't work too. PLease, help.
You have to change the socket.gethostname() in the server script to the empty string (or just directly call socket.bind(('', port))).
Your problem is not in Python but in the usage of sockets generally. When you create a socket, you just prepare your process to receive/send some data from/to another process.
Server
The first step for creating a socket is to specify what kind of protocol will be used for communication between those processes. In your case it is the socket.AF_INET which is constant for use of IP protocol and the socket.SOCK_STREAM is specify reliable stream-oriented service. The reliable stream-oriented service means that you want to be sure that every single sent byte will be delivered to the other side and nothing can be lost during the communication (the underlying OS will use the TCP protocol for that). From this point we are using the IPv4 protocol (because we set the socket.AF_INET).
The second step is bind it to an address. The bind process assigns an address where you expect a client will join (with your socket's settings it's a IP address and the TCP port). Your PC has multiple IP address (well, at least two). It always has 127.0.0.1, which is called "callback" and works only when your applications communicate on the same PC (that is you Linux - Linux scenario in the question), and then you have your external IP address, for communication with other computers (let's pretend it is 10.0.0.1).
When you call socket.bind(('127.0.0.1', 5555)), you're setting the socket to listen only for communication from the same PC. If you call socket.bind(('10.0.0.1', 5555)), then the socket setting is ready to receive data targeted to the 10.0.0.1 address.
But what if you have 10 IPs or more and you want to receive everything (with the right TCP port)? For those scenarios you can leave the IP address in bind() empty, and it does exactly what you want.
With Python's version of bind(), you can also enter a "computer name" instead of the concrete IP. The socket.gethostname() call returns your computer's name. The problem is in the translation of "computer name" to the IP which Python performs behind your back. The translation has some rules but generally your "computer name" can be translated into any IP address which you have set on your computer. In your case, the your computer's name is converted into 127.0.0.1, and that's why communication works only between processes on the same computer.
After socket.bind(), you have the socket ready to use but it is still "inactive". The call to socket.listen() activates the socket and causes it to wait until it receives an attempted connection. When a socket receives a new connection request, it will put it into a queue and wait for processing.
That's what socket.accept() does. It pulls the connection request from the queue, accepts it, and establishes the stream (remember the socket.SOCK_STREAM when you set up the socket) between the server and the client. The new stream is actually a new socket, but is ready to communicate with other side.
What happened with the old socket? Well, it's still alive, and you can call socket.listen() again to get another stream (connection).
How is it possible to have multiple sockets on the same port?
Every connection within computer's network is defined by flow which is 5-item tuple of:
L4 protocol (usually TCP or UDP)
Source IP address
Source L4 port
Destination IP address
Destination L4 port
When you create a new connection with a client, the flow can look like this: (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555). Just for clarification, the server's response flow is (TCP, 10.0.0.1, 55555, 192.168.0.1, 12345), but it isn't important for us. If you create another connection with a client, that it will differ at source TCP port (if you do it from another computer that it will differ also at the source IP). Only from this information you can distinguish every connection created to your computer.
When you create a server socket in your code and call socket.listen(), it listens for any flow with this pattern (TCP, *, *, *, 55555) (the * means "match everything"). So when you get a connection with (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555), then socket.accept() creates another socket which works only with this one concrete flow while the old socket carries on accepting new connections which haven't yet been established.
When the operating system receives a packet, it looks in the packet and checks the flow. At this point, several scenarios can take place:
The packet's flow matches all 5 items exactly (without usage of *). Then the packet's content is delivered to the queue associated with that socket (you're reading the queue when you call socket.recv()).
The packet's flow matched socket with associated flow contains * then it is considered as new connection and you can call scoket.accept().
The operating system doesn't contain open socket which would match the flow. In that case the OS refuse connection (or just ignore the packet it depends on firewall settings).
Probably an example can clarify these scenarios. The operating system has something like a table where it maps flows to sockets. When you call socket.bind(), it will assign a flow to the socket. After the call, the table can look like this:
+=====================================+========+
| Flow | Socket |
+=====================================+========+
| (TCP, *, *, *, 55555) | 1 |
+-------------------------------------+--------+
When it receive a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 10) then it won't match any flow (last port won't match). So, the connection is refused. If it receives a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 55555), the packet is delivered to the socket 1 (because there is a match). The socket.accept() call creates a new socket and record in the table.
+=====================================+========+
| Flow | Socket |
+=====================================+========+
| (TCP, 1.1.1.1, 10, 10.0.0.1, 55555) | 2 |
+-------------------------------------+--------+
| (TCP, *, *, *, 55555) | 1 |
+-------------------------------------+--------+
Now you have 2 sockets for 1 port. Every received packet which matches the flow associated with the socket 2 also matches the flow associated with socket 1 (on the contrary, it does not apply). It's not a problem because the socket 2 has a preciser match (is doesn't use the *), so any data with that flow will be delivered to socket 2.
How to serve multiple connections
If you want to implement a "real" server, your application should be able to process multiple connections without restarting. There are 2 basic approaches:
Sequential processing
try:
l = prepare_socket()
while True:
l.listen()
s, a = socket.accept()
process_connection(s) # before return you should call s.close()
except KeyboardInterrupt:
l.close()
In this case, you can process only one client while others clients have to wait for accept. If the process_connection() takes too long, then others clients will timeout.
Parallel processing
import threading
threads = []
try:
l = prepare_socket()
while True:
l.listen()
s, a = socket.accept()
t = threading.Thread(target=process_connection, s)
threads.append(t)
t.start()
except KeyboardInterrupt:
for t in threads:
t.join()
l.close()
Now when you receive a new connection, it will create a new thread so that every connection is processed in parallel. The main disadvantage of this solution is that you have to solve common troubles with threading (like access to shared memory, deadlocks etc.).
Beware that the above snippets are only examples, and are not complete! For example, they don't contain code for graceful exit on unexpected exceptions.
Servers in Python
Python also contains a module called socketserver, which contains shortcuts for creating servers in Python. You can find examples of how to use it here.
Client
With the client, it's much more simpler than with the server. You just have to create a socket with some settings (same as server side), and then tell it where the server is (what its IP and TCP port are). This is accomplished through the socket.connect() call. As a bonus, it also establishes the stream between your client and server, so from this point you can communicate.
You can find more information about sockets at the Beej's Guide to Network Programming. It's written for usage with C, but the concepts are the same.
I was stuck with the same problem months ago and also wasn't able to do port forwarding. I found a way out of port forwarding Ngrock
For your information what Ngrock does is it is a useful utility to create secure tunnels to locally hosted applications using a reverse proxy. It is a utility to expose any locally hosted application over the web
For How To use it, please see the steps shown below :
If you are in Mac write this command in your terminal to download ngrock
brew install ngrok
For windows
choco install ngrok
After installing You need need to sign up on the Ngrok website
You will get your Ngrock authentication token then paste this command in the terminal
For Mac and Windows
ngrok config add-authtoken <token>
Now that Ngrock is all setup you can start a tunnel using
ngrok tcp <Your Port Number Used In Server.py>
ngrok tcp 5321
Note : Please Give the command inside the directory in which the Python Socket Server File in Located
That's it Your Socket can connect you any computer over the internet anywhere in the world
If You are still struggling to see the detailed explanation in this video
You can also refer ngrock documentation here
An eye-tracking application I use utilizes UDP to send packets of data. I made a python socket on the same computer to listen and dump the data into a .txt file. I already have this much working.
A separate application also written in python (what the eye-tracked subject is seeing) is running on a separate computer. Because the eye-tracking application is continuous and sends unnecessary data, so far I've had to manually parse out the instances when the subject is looking at desired stimuli. I did this based on a manually synchronized start of both the stimuli and eye-tracking applications and then digging through the log file.
What I want to do is have the second computer act as a second UDP client, sending a packet of data to the socket on the eye-tracking computer everytime the subject is looking at stimuli (where a marker is inserted into the .txt file previously mentioned). Is it possible to have a socket listening to two IP addresses at one time?
Here's my socket script:
#GT Pocket client program
import datetime
import socket
now = datetime.datetime.now()
filename = 'C:\gazelog_' + now.strftime("%Y_%m_%d_%H_%M") + '.txt'
UDP_IP = '127.0.0.1' # The remote host (in this case our local computer)
UDP_PORT = 6666 # The same port as used by the GT server by default
sock = socket.socket(socket.AF_INET, #internet
socket.SOCK_DGRAM) #UDP
sock.bind( (UDP_IP, UDP_PORT) )
while True:
data, addr = sock.recvfrom( 1024) #I assume buffer size is 1024 bytes.
print "Received Message:", data
with open(filename, "a") as myfile:
myfile.write(str(data + "\n"))
sock.close()
myfile.close()
EDIT:
#abarnert I was able to bind to the host address on the Ethernet interface and send a message from computer B to computer A, but computer A was no long able to receive packets from itself. When I specified UDP_IP = '0.0.0.0' computer B was no longer able to send data across the Ethernet. When I specified UDP_IP = '' I received the `error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
This have to do with the script I used on the Computer B to send the data:
import socket
UDP_IP = "169.254.35.231" # this was the host address I was able to send through.
UDP_PORT = 6666
MESSAGE = "Start"
print ("UDP target IP:"), UDP_IP
print ("UDP target port:"), UDP_PORT
print ("message:"), MESSAGE
sock = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM)
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT) )
I didn't know where (or if at all) I needed to specify INADDR_ANY, so I didn't. But I did try once where import socket.INADDR_ANY but got ImportError: No module named INADDR_ANY
Seems like a simple issue based on your response, so I'm not sure where I'm messing up.
EDIT2: I just reread your answer again and understand why socket.INADDR_ANY doesn't work. Please disregard that part of my previous edit
EDIT3: Okay so the reason that I wasn't picking up data when specifying the host IP was that the application I was collecting data from on Computer A was still specified to send to 127.0.0.1. So I figured it out. I am still curious why 0.0.0.0 didn't work though!
No. A socket can only be bound to a single address at a time.*
If there happens to be a single address that handles both things you want, you can use a single socket to listen to it. In this case, the INADDR_ANY host (0.0.0.0) may be exactly what you're looking for—that will handle any (IPv4) connections on all interfaces, both loopback and otherwise. And even if there is no pre-existing address that does what you want, you may be able to set one up via, e.g., an ipfilter-type interface.
But otherwise, you have to create two sockets. Which means you need to either multiplex with something like select, or create two threads.
In your case, you want to specify a host that can listen to both the local machine, and another machine on the same Ethernet network. You could get your host address on the Ethernet interface and bind that. (Your machine can talk to itself on any of its interfaces.) Usually, getting your address on "whatever interface is the default" works for this too—you'll see code that binds to socket.gethostname() in some places, like the Python Socket Programming HOWTO. But binding to INADDR_ANY is a lot simpler. Unless you want to make sure that machines on certain interfaces can't reach you (which is usually only a problem if you're, e.g., building a server intended to live on a firewall's DMZ), you'll usually want to use INADDR_ANY.
Finally, how do you bind to INADDR_ANY? The short answer is: just use UDP_IP = '', or UDP_IP = '0.0.0.0' if you want to be more explicit. Anyone who understands sockets, even if they don't know any Python, will understand what '0.0.0.0' means in server code.(You may wonder why Python doesn't have a constant for this in the socket module, especially when even lower-level languages like C do. The answer is that it does, but it's not really usable.**)
* Note that being bound to a single address doesn't mean you can only receive packets from a single address; it means you can receive packets from all networks where that single address is reachable. For example, if your machine has a LAN connection, where your address is 10.0.0.100, and a WAN connection, where your address is 8.9.10.11, if you bind 10.0.0.100, you can receive packets from other LAN clients like 10.0.0.201 and 10.0.0.202. But you can't receive packets from WAN clients like 9.10.11.12 as 10.0.0.100.
** In the low-level sockets API, dotted-string addresses like '0.0.0.0' are converted to 32-bit integers like 0. Python sometimes represents those integers as ints, and sometimes as 4-byte buffers like b'\0\0\0\0'. Depending on your platform and version, the socket.INADDR_ANY constant can be either 0 or b'\0\0\0\0'. The bind method will not take 0, and may not take b'\0\0\0\0'. And you can't convert to '0.0.0.0' without first checking which form you have, then calling the right functions on it. This is ugly. That's why it's easier to just use '0.0.0.0'.
I believe you can bind a raw socket to an entire interface, but you appear to be using two different interfaces.
It's probably best to use two sockets with select().