Develop Reference

Python Binding Socket: “Address already in use”, after closing socket

I know there is a similar question already, but none of the solutions solve my problem. Over ssh I am starting a script on a remote client with
nohup python script.py &
This script contains the following:
TCP_PORT = 5005
host = ""
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(40)
s.bind((host, TCP_PORT))
s.listen(0)
c, addr = s.accept()
...some code...
try:
while True:
c.send(str(1).ljust(16).encode())
except Exception as e:
print("exiting main")
print(e)
c.close()
s.close()
When I run the code two times in e row, the second time I always get the above mentioned error. The log of the python output:
exiting main
[Errno 32] Broken pipe
Traceback (most recent call last):
File "LogImages.py", line 204, in <module>
main(interv)
File "LogImages.py", line 114, in main
s.bind((host, TCP_PORT))
OSError: [Errno 98] Address already in use
So obviously the process calls c.close() and s.close(). So how can the address still be in use?

Closing a socket just releases the handle to any underlying connection. It can still take the implementation some amount of time to complete the orderly shutdown of the connection and, during that time, the address is still in use.
For example, if you have an active connection and the other side isn't reading from it, the implementation will give it time to read the data that was sent. During that time, the address is still in use.

Python server-client

I need some help. I have a simple server:
host="localhost"
port=4447
from socket import *
import thread
def func():
while 1:
data = conn.recv(1024)
if not data:
continue
else:
print("%s said: %s")%(player, data)
conn.close()
s=socket(AF_INET, SOCK_STREAM)
s.bind((host,port))
s.listen(2)
print("Waiting for clients on localhost, port %s")%port
while 1:
conn, addr = s.accept()
player = addr[1]
print(conn)
thread.start_new_thread(func,())
And a simple client:
import socket
TCP_IP = '127.0.0.1'
TCP_PORT = 4447
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
while 1:
data = raw_input("Input: ")
s.send(data)
So when I connect to the server I can type anything and it is printed in the server's terminal. When I open another terminal and start second client I can also type anything and it is sent to the server, but when I go back to the first client's terminal and type several messages, it returns:
Traceback (most recent call last):
File "Client.py", line 18, in <module>
s.send(data)
socket.error: [Errno 32] Broken pipe
So I fixed that with adding conn as a parameter in func(), but I don't understand why this error happened? Could anyone please explain it to me?
Thanks!

Your func, apart from needing a better name, uses global state in your program to communicate with a client. No matter how many threads you start to handle client connections, there's still only one global conn variable. Each time a new client connects, your main thread loop rebinds conn to the new connection. The old socket is thrown away and automatically closed by the Python runtime.
You can fix this by removing the use of global variables to track per-connection state. A better route to explore, though, is Twisted.

why can't bind to 0.0.0.0:80 and 192.168.1.1:80 simultaneously?

My python test code:
import socket
s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s1.bind(('192.168.1.1', 80))
s1.listen(5)
s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s2.bind(('0.0.0.0', 80))
s2.listen(5)
I got this error:
fpemud-workstation test # ./test.py
Traceback (most recent call last):
File "./test.py", line 11, in <module>
s2.bind(('0.0.0.0', 80))
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
192.168.1.1 is the ip address of my eth0 interface.
I think 0.0.0.0:80 and 192.168.1.1:80 should be able to co-exist.
Packets with dst-addr 192.168.1.1 goes to socket s1, packets with other dst-addr goes to socket s2.

You cannot bind to both 0.0.0.0:80 and any other IP on port 80, because 0.0.0.0 covers every IP that exists on the machine, including your 192.168.1.1 address. It doesn't mean 'any other destination address', it means 'all interfaces on this box'.

Because it's a contradiction in terms. 0.0.0.0 means 'accept connections from any local IP address'. 192.168.1.1 means 'accept connections only that are addressed to 192.168.1.1'. What exactly do you expect to happen if someone connects to 192.168.1.1?

Despite what other answers have said, this should be possible - it's just that the way bind works is implementation dependent.
On Windows, for example, your code will probably work fine as is. On some *nix operating systems I believe you can get it to work by setting the SO_REUSEADDR socket option. On Linux, I've been able to get it to work using the SO_REUSEPORT socket option, but only on kernel version 3.9 or later.
Unfortunately the SO_REUSEPORT property isn't directly supported in current versions of python, so we have to define it manually.
Basically your code should look like this:
# This adds support for the SO_REUSEPORT constant if not already defined.
if not hasattr(socket, 'SO_REUSEPORT'):
socket.SO_REUSEPORT = 15
s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s1.bind(('192.168.1.1', 80))
s1.listen(5)
s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s2.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s2.bind(('0.0.0.0', 80))
s2.listen(5)

'SO_REUSEPORT' is not defined on Windows 7

I found multiple answers that suggest using the SO_REUSEPORT socket option when multiple UDP clients need to listen for broadcasts on the same port. However I'm getting an error that this option is not available. Using python 2.7
from socket import *
s = socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1)
try:
s.bind(('', MYPORT))
except:
print "Error connecting to the UDP stream."
Traceback (most recent call last):
File "qsorder.py", line 119, in <module>
s.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1)
NameError: name 'SO_REUSEPORT' is not defined
I've tried SO_REUSEADDR and it does not give an error but only one client receives broadcasts. Any idea how to work around this?

You need to set SO_BROADCAST option on each socket:
s.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
SO_REUSEPORT is not standard and usually means same thing as SO_REUSEADDR where supported.

Python client / server question

I'm working on a bit of a project in python. I have a client and a server. The server listens for connections and once a connection is received it waits for input from the client. The idea is that the client can connect to the server and execute system commands such as ls and cat. This is my server code:
import sys, os, socket
host = ''
port = 50105
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
print("Server started on port: ", port)
s.listen(5)
print("Server listening\n")
conn, addr = s.accept()
print 'New connection from ', addr
while (1):
rc = conn.recv(5)
pipe = os.popen(rc)
rl = pipe.readlines()
file = conn.makefile('w', 0)
file.writelines(rl[:-1])
file.close()
conn.close()
And this is my client code:
import sys, socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = 'localhost'
port = input('Port: ')
s.connect((host, port))
cmd = raw_input('$ ')
s.send(cmd)
file = s.makefile('r', 0)
sys.stdout.writelines(file.readlines())
When I start the server I get the right output, saying the server is listening. But when I connect with my client and type a command the server exits with this error:
Traceback (most recent call last):
File "server.py", line 21, in <module>
rc = conn.recv(2)
File "/usr/lib/python2.6/socket.py", line 165, in _dummy
raise error(EBADF, 'Bad file descriptor')
socket.error: [Errno 9] Bad file descriptor
On the client side, I get the output of ls but the server gets screwed up.

Your code calls conn.close() and then loops back around to conn.recv(), but conn is already closed.

If you want your client to repeat what it's doing, just add a loop in there ;)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = 'localhost'
port = input('Port: ')
s.connect((host, port))
while True:
cmd = raw_input('$ ')
s.send(cmd)
file = s.makefile('r', 0)
sys.stdout.writelines(file.readlines())
Should probably be closer to what you want.
Other comments:
s.listen(1)
This statement should probably be moved outside of the while loop. You only need to call listen once.
pipe = os.popen(rc)
os.popen has been deprecated, use the subprocess module instead.
file = s.makefile('r', 0)
You're opening a file, yet you never close the file. You should probably add a file.close() after your sys.stdout.writelines() call.
EDIT: to answer below comment; done here due to length and formatting
As it stands, you read from the socket once, and then immediately close it. Thus, when the client goes to send the next command, it sees that the server closed the socket and indicates an error.
The solution is to change your server code so that it can handle receiving multiple commands. Note that this is solved by introducing another loop.
You need to wrap
rc = conn.recv(2)
pipe = os.popen(rc)
rl = pipe.readlines()
fl = conn.makefile('w', 0)
fl.writelines(rl[:-1])
in another while True: loop so that it repeats until the client disconnects, and then wrap that in a try-except block that catches the IOError that is thrown by conn.recv() when the client disconnects.
the try-except block should look like
try:
# the described above loop goes here
except IOError:
conn.close()
# execution continues on...