Develop Reference

Python Binding Socket: “Address already in use”, after closing socket

I know there is a similar question already, but none of the solutions solve my problem. Over ssh I am starting a script on a remote client with
nohup python script.py &
This script contains the following:
TCP_PORT = 5005
host = ""
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(40)
s.bind((host, TCP_PORT))
s.listen(0)
c, addr = s.accept()
...some code...
try:
while True:
c.send(str(1).ljust(16).encode())
except Exception as e:
print("exiting main")
print(e)
c.close()
s.close()
When I run the code two times in e row, the second time I always get the above mentioned error. The log of the python output:
exiting main
[Errno 32] Broken pipe
Traceback (most recent call last):
File "LogImages.py", line 204, in <module>
main(interv)
File "LogImages.py", line 114, in main
s.bind((host, TCP_PORT))
OSError: [Errno 98] Address already in use
So obviously the process calls c.close() and s.close(). So how can the address still be in use?

Closing a socket just releases the handle to any underlying connection. It can still take the implementation some amount of time to complete the orderly shutdown of the connection and, during that time, the address is still in use.
For example, if you have an active connection and the other side isn't reading from it, the implementation will give it time to read the data that was sent. During that time, the address is still in use.

Python 2.7 Socket programming ports

I had an exercise of port carousel which means that I need to build a server-client which the server asks the client for a port and then they starting to listen to the port that given, and this is the loop I got a error and I don't know how to fix it.
server:
import socket
import random
def main():
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('0.0.0.0', 1729))
server_socket.listen(1)
(client_socket, server_socket) = server_socket.accept()
done = False
while not done:
port = client_socket.recv(4096)
client_socket.send('i got the port' + port)
port = int(port)
if port != 1:
server_socket.bind(('0.0.0.0', port))
continue
else:
done = True
if __name__ == '__main__':
main()
client:
import socket
import random
def main():
print 'hi at anytime enter 1 to break the loop'
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('127.0.0.1', 1729))
done = False
while not done:
port = client_socket.send(raw_input("enter port:"))
data = client_socket.recv(4096)
print data
port = int(port)
if port != 1:
client_socket.connect(('127.0.0.1', port))
continue
else:
done = True
client_socket.close()
if __name__ == '__main__':
main()
the error output for the server:
File "C:/Cyber/ServerFolder/ports_carrousel.py", line 18, in main
server_socket.bind(('0.0.0.0', port))
AttributeError: 'tuple' object has no attribute 'bind'

In your main function, you do the following:
(client_socket, server_socket) = server_socket.accept()
but, server_socket.accept() actually returns two objects. The first, is a socket object, and the second one is a tuple that contains (sourceIPString, sourcePort).
Thus, by using this line of code, outlined above, you are essentially overriding the server_socket by a tuple object.
Notice that later, in line 18, you are trying to access the "bind" function of a socket, but, using a reference to a tuple object, that does not implement such a function.
What you should be doing is something along the lines of
(client_socket, client_connection_info) = server_socket.accept()
and adjust your code accordingly.

Just a couple of things wrong here. First, accept returns a 2-tuple containing the newly-connected socket, and the client's address (which is itself a 2-tuple of IP address and port number). It does not return two sockets. But you're overwriting your server_socket variable with the second returned value. That doesn't make sense and it's why the interpreter is telling you that the 2-tuple has no bind attribute: it's not a socket object. The accept call should look something like this:
client_socket, client_addr = server_socket.accept()
Next, after receiving the new port number from the client, you must create a new socket (you cannot re-use the same listening socket), then bind that new socket to the new port, then listen; finally you can accept a new client connection from the new listening socket.
You should also close sockets you're finished with so that you don't continually leak file descriptors. That means each time you receive a new port number from the client, you should close the client socket, and the listening socket, then create a new listening socket (and bind and listen), then accept the new client socket.
Altogether that will mean restructuring your code in the server significantly. You need to pull the creation of a listening socket down into your main while not done loop.
Another thing to keep in mind. On the client side, immediately after sending the port number to the server, you're attempting a connect to that new port number. However, it's almost certain that your connect request will reach the server before the server has had a chance to create a new listening socket, and bind it. So your client will either need to delay a moment before attempting to connect, or it will need to have logic to retry the connect for some period of time.
EDIT:
Also, you must create a new socket on the client side too when reconnecting. Once a stream socket has been bound to a port (which also happens automatically when you connect), you can never use it to bind or connect to a different address/port.

How to connect() with a non-blocking socket?

In Python, I would like to use socket.connect() on a socket that I have set to non-blocking. When I try to do this, the method always throws a BlockingIOError. When I ignore the error (as below) the program executes as expected. When I set the socket to non-blocking after it is connected, there are no errors. When I use select.select() to ensure the socket is readable or writable, I still get the error.
testserver.py
import socket
import select
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setblocking(0)
host = socket.gethostname()
port = 1234
sock.bind((host, port))
sock.listen(5)
while True:
select.select([sock], [], [])
con, addr = sock.accept()
message = con.recv(1024).decode('UTF-8')
print(message)
testclient.py
import socket
import select
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setblocking(0)
host = socket.gethostname()
port = 1234
try:
sock.connect((host, port))
except BlockingIOError as e:
print("BlockingIOError")
msg = "--> From the client\n"
select.select([], [sock], [])
if sock.send(bytes(msg, 'UTF-8')) == len(msg):
print("sent ", repr(msg), " successfully.")
sock.close()
terminal 1
$ python testserver.py
--> From the client
terminal 2
$ python testclient.py
BlockingIOError
sent '--> From the client\n' successfully.
This code works correctly except for the BlockingIOError on the first connect(). The documentation for the error reads like this: Raised when an operation would block on an object (e.g. socket) set for non-blocking operation.
How do I properly connect() with a socket set to non-blocking? Can I make connect() non-blocking? Or is it appropriate to just ignore the error?

When using socket.connect with a non-blocking socket, it is somewhat expected to get a BlockingIOError at first. See TCP Connect error 115 Operation in Progress What is the Cause? for an explanation of the cause. Basically, the socket isn't ready yet and raises BlockingIOError: [Errno 115] Operation now in progress, also known as EINPROGRESS.
The solution is to either catch and ignore the exception or to use socket.connect_ex instead of socket.connect because that method doesn't raise an exception. Especially note the last sentence from its description in the Python docs:
socket.connect_ex(address)
Like connect(address), but return an error indicator instead of raising an exception for errors returned by the C-level connect() call (other problems, such as “host not found,” can still raise exceptions). The error indicator is 0 if the operation succeeded, otherwise the value of the errno variable. This is useful to support, for example, asynchronous connects.
Source: https://docs.python.org/3/library/socket.html#socket.socket.connect_ex
If you want to keep using socket.connect, you can catch and ignore the responsible EINPROGRESS error:
>>> import socket
>>>
>>> # bad
>>> s = socket.socket()
>>> s.setblocking(False)
>>> s.connect(("127.0.0.1", 8080))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
BlockingIOError: [Errno 115] Operation now in progress
>>>
>>> # good
>>> s = socket.socket()
>>> s.setblocking(False)
>>> try:
... s.connect(("127.0.0.1", 8080))
... except OSError as exc:
... if exc.errno != 115: # EINPROGRESS
... raise
...
>>>

The trick here is that when the select completes the first time, then you need to call sock.connect again. The socket is not connected until you have received a successful return status from connect.
Just add these two lines after the first call to select completes:
print("first select completed")
sock.connect((host, port))
EDIT:
Followup. I was wrong to have stated that an additional call to sock.connect is required. It is however a good way to discover whether the original non-blocking call to connect succeeded if you wish to handle the connection failure in its own code path.
The traditional way of achieving this in C code is explained here: Async connect and disconnect with epoll (Linux)
This involves calling getsockopt. You can do this in python too but the result you get back from sock.getsockopt is a bytes object. And if it represents a failure, you then need to convert it into an integer errno value and map that to a string (or exception or whatever you require to communicate the issue to the outside world). Calling sock.connect again maps the errno value to an appropriate exception already.
Solution 2:
You can also simply defer calling sock.setblocking(0) until after the connect has completed.

Python server-client

I need some help. I have a simple server:
host="localhost"
port=4447
from socket import *
import thread
def func():
while 1:
data = conn.recv(1024)
if not data:
continue
else:
print("%s said: %s")%(player, data)
conn.close()
s=socket(AF_INET, SOCK_STREAM)
s.bind((host,port))
s.listen(2)
print("Waiting for clients on localhost, port %s")%port
while 1:
conn, addr = s.accept()
player = addr[1]
print(conn)
thread.start_new_thread(func,())
And a simple client:
import socket
TCP_IP = '127.0.0.1'
TCP_PORT = 4447
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
while 1:
data = raw_input("Input: ")
s.send(data)
So when I connect to the server I can type anything and it is printed in the server's terminal. When I open another terminal and start second client I can also type anything and it is sent to the server, but when I go back to the first client's terminal and type several messages, it returns:
Traceback (most recent call last):
File "Client.py", line 18, in <module>
s.send(data)
socket.error: [Errno 32] Broken pipe
So I fixed that with adding conn as a parameter in func(), but I don't understand why this error happened? Could anyone please explain it to me?
Thanks!

Your func, apart from needing a better name, uses global state in your program to communicate with a client. No matter how many threads you start to handle client connections, there's still only one global conn variable. Each time a new client connects, your main thread loop rebinds conn to the new connection. The old socket is thrown away and automatically closed by the Python runtime.
You can fix this by removing the use of global variables to track per-connection state. A better route to explore, though, is Twisted.

Python ForkingTCPServer error

Help to build python ForkingTCPServer. This server returns an error when client connects to the server. When I use ThreadingTCPServer instead of ForkingTCPServer it works fine. socketFileIO module contains functions for pickle to/from socket:
http://code.activestate.com/recipes/577667-pickle-tofrom-socket/
server.py:
import SocketServer, time
from socketFileIO import write, read
class MyClientHandler(SocketServer.BaseRequestHandler):
def handle(self):
print self.client_address
data=read(self.request)
print 'Client:', data
time.sleep(10) # for testing
write(self.request,data)
self.request.close()
HOST, PORT = '', 50007
server = SocketServer.ForkingTCPServer((HOST, PORT), MyClientHandler)
server.serve_forever()
client.py:
import socket
from socketFileIO import write, read
HOST = '127.0.0.1'
PORT = 50007
data=["A","B","C","End"]
for x in data:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
write(s,x)
y=read(s)
print 'Server:', y
s.close()
Error:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 1469)
Traceback (most recent call last):
File "c:\Python26\lib\SocketServer.py", line 283, in _handle_request_noblock
self.process_request(request, client_address)
File "c:\Python26\lib\SocketServer.py", line 525, in process_request
pid = os.fork()
AttributeError: 'module' object has no attribute 'fork'
----------------------------------------
I have a several other questions:
How to shutdown the server by sending data "End" from client ?
How to restrict the number of connections (for example no more then 5) ?

Your fork question was already answered in comments.
For "how to shut down the server": in a similar application (but much older Python version and using xmlrpc), I used a loop invoking one request at a time until a "stop" variable was set. This appears to be supported directly in Python 2.7 now: in your threaded server, call shutdown() on the server instance, from one of the request-handler threads. (Not sure if this is available in 2.6, I de-installed my old 2.6 version a couple of months ago.) I.e.:
class MyClientHandler(SocketServer.BaseRequestHandler):
def handle(self):
...
# if we're supposed to shut down the server, do that
if somecond:
self.server.shutdown()
For limiting the number of connections: there's nothing built-in that I can see, but if you redefined the process_request handler you could count the number of active threads and wait for one to finish if there are "too many" outstanding (or work it however else you like). See the ThreadingMixIn class in SocketServer.py and compare it to the max_children code in ForkingMixIn.