I tried with opening the socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sending a message, and then closing it
sock.close()
but I don't know how to open it again. I use UDP socket, and the sock.connect() is a TCP command I think... leastways I tried it but it caused error.
You can't reopen a socket. As close explains:
The underlying system resource (e.g. a file descriptor) is also closed when all file objects from makefile() are closed. Once that happens, all future operations on the socket object will fail.
This is inherent in the BSD sockets model: closing a socket means the kernel throws away all resources related to that socket and releases the file descriptor. As POSIX defines it:
If fildes refers to a socket, close() shall cause the socket to be destroyed.
If you try to use that file descriptor again, you get an EBADFD error if you're lucky—and if you're unlucky, you get some completely different socket or file that reused the same number. In fact, this is true for all file descriptors, not just sockets. Quoting POSIX again:
Once a file is closed, the file descriptor no longer exists, since the integer corresponding to it no longer refers to a file.
What you can do, of course, is create a brand new socket, the same way you created the first one:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
You may end up with a different file descriptor, but for most sockets code, that won't make a difference.
Or, of course, you can just not close the socket. Whatever reason you had for wanting to close the socket, the fact that you want to use it again implies that you were probably wrong.
Related
When you setup a simple TCP listening socket using the Python 'socket' module, what are the different steps involved doing?
The code I'm talking about looks like this:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 50000))
s.listen(1)
conn, addr = s.accept()
The s = ... seems pretty straightforward - you are expressing your intent to create an ipv4 TCP socket, without having done anything yet.
What I'm curious about is this:
What does it mean to bind to a socket, without listening?
How does limiting the number of unaccepted connections using listen(n) work?
If you have listen(1), you're in the middle of dealing with the first connection you accepted, and a second client tries to connect, is the second client waiting for the SYN-ACK? Or does the 3 way handshake happen, and he's waiting for actual data?
What happens if a third client tries to connect - does he immediately get a TCP RST?
Does setting the number of unaccepted connections here set some option in the kernel to indicate how many connections it should accept? Or is this all handled in Python?
How can you be listening without accepting? What does it mean to accept a connection?
Every article I've come across seems to just assume these steps make sense to everyone, without explaining what exactly it is that each one does. They just use generic terms like
listen() starts listening for connections
bind() binds to a socket
accept() just accepts the connection
Defining a word by using that word in the definition is kind of a dumb way to explain something.
it's basically a 1-to-1 from the POSIX c calls and as such I'm including links to the man pages, so that you can read their explanation and corresponding c code:
socket creates a communication endpoint by means of a file-descriptor in the namespace of the address-family you specified but assigns neither address nor port.
bind assigns an address and port to said socket, a port which may be chosen randomly if you request a port for which you do not have the privilige. (like < 1024 for non-root user)
listen makes the specific socket and hence address and port a passive one, meaning that it will accept incoming connections with the accept call. To handle multiple connections one after the other, you get to specify a backlog containing them, connections that arrive while you're handling one get appended. Once the backlog is full, the system will respond as such to those systems with an approach that makes them reconnect by withholding SYN, withholding ACK response etc..
As usual you can find someone explaining the previous to you a lot better.
accept then creates a new non-listening socket associated with a new file descriptor that you then use for communication with said connecting party.
accept also works as a director for your flow of execution, effectively blocking further progress until a connection is actually available in the queue for it to take, like a spinlock. The only way around that is to declare the socket non-blocking in which case it would return immediately with an error.
I created a program which listens to particular socket in python, however I ctrl+c'd script which resulted in .close() nor called, however how can I free the socket now.
The socket is closed when the process exits. The port it was using may hang around for a couple of minutes, that's normal, then it will disappear. If you need to re-use the port immediately, set SO_REUSEADDR before binding or connecting.
Set the SO_REUSEADDR socket option before calling the bind method, like this:
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
That will instruct the socket to freely reuse the ports left in a waiting state by recent runs of the program.
Or, use the SocketServer.TCPServer class from the standard library, which will automatically do this if you set the allow_reuse_address property on the server instance to a true value.
I have a Python application which opens a simple TCP socket to communicate with another Python application on a separate host. Sometimes the program will either error or I will directly kill it, and in either case the socket may be left open for some unknown time.
The next time I go to run the program I get this error:
socket.error: [Errno 98] Address already in use
Now the program always tries to use the same port, so it appears as though it is still open. I checked and am quite sure the program isn't running in the background and yet my address is still in use.
SO, how can I manually (or otherwise) close a socket/address so that my program can immediately re-use it?
Update
Based on Mike's answer I checked out the socket(7) page and looked at SO_REUSEADDR:
SO_REUSEADDR
Indicates that the rules used in validating addresses supplied in a bind(2) call should
allow reuse of local addresses. For AF_INET sockets this means that a socket may bind,
except when there is an active listening socket bound to the address. When the listen‐
ing socket is bound to INADDR_ANY with a specific port then it is not possible to bind
to this port for any local address. Argument is an integer boolean flag.
Assume your socket is named s... you need to set socket.SO_REUSEADDR on the server's socket before binding to an interface... this will allow you to immediately restart a TCP server...
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((ADDR, PORT))
You might want to try using Twisted for your networking. Mike gave the correct low-level answer, SO_REUSEADDR, but he didn't mention that this isn't a very good option to set on Windows. This is the sort of thing that Twisted takes care of for you automatically. There are many, many other examples of this kind of boring low-level detail that you have to pay attention to when using the socket module directly but which you can forget about if you use a higher level library like Twisted.
You are confusing sockets, connections, and ports. Sockets are endpoints of connections, which in turn are 5-tuples {protocol, local-ip, local-port, remote-ip, remote-port}. The killed program's socket has been closed by the OS, and ditto the connection. The only relic of the connection is the peer's socket and the corresponding port at the peer host. So what you should really be asking about is how to reuse the local port. To which the answer is SO_REUSEADDR as per the other answers.
When I close the socket on one end of a connection, the other end gets an error the second time it sends data, but not the first time:
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("localhost", 12345))
server.listen(1)
client = socket.create_connection(("localhost",12345))
sock, addr = server.accept()
sock.close()
client.sendall("Hello World!") # no error
client.sendall("Goodbye World!") # error happens here
I've tried setting TCP_NODELAY, using send instead of sendall, checking the fileno(), I can't find any way to get the first send to throw an error or even to detect afterwards that it failed. EDIT: calling sock.shutdown before sock.close doesn't help. EDIT #2: even adding a time.sleep after closing and before writing doesn't matter. EDIT #3: checking the byte count returned by send doesn't help, since it always returns the number of bytes in the message.
So the only solution I can come up with if I want to detect errors is to follow each sendall with a client.sendall("") which will raise an error. But this seems hackish. I'm on a Linux 2.6.x so even if a solution only worked for that OS I'd be happy.
This is expected, and how the TCP/IP APIs are implemented (so it's similar in pretty much all languages and on all operating systems)
The short story is, you cannot do anything to guarantee that a send() call returns an error directly if that send() call somehow cannot deliver data to the other end. send/write calls just delivers the data to the TCP stack, and it's up to the TCP stack to deliver it when it can.
TCP is also just a transport protocol, if you need to know if your application "messages" have reached the other end, you need to implement that yourself(some form of ACK), as part of your application protocol - there's no other free lunch.
However - if you read() from a socket, you can get notified immediatly when an error occurs, or when the other end closed the socket - you usually need to do this in some form of multiplexing event loop (that is, using select/poll or some other IO multiplexing facility).
Just note that you cannot read() from a socket to learn whether the most recent send/write succeded, Here's a few cases as of why (but it's the cases one doesn't think about that always get you)
several write() calls got buffered up due to network congestion, or because the tcp window was closed (perhaps a slow reader) and then the other end closes the socket or a hard network error occurs, thus you can't tell if if was the last write that didn't get through, or a write you did 30 seconds ago.
Network error, or firewall silently drops your packets (no ICMP replys are generated), You will have to wait until TCP times out the connection to get an error which can be many seconds, usually several minutes.
TCP is busy doing retransmission as you call send - maybe those retransmissions generate an error.(really the same as the first case)
As per the docs, try calling sock.shutdown() before the call to sock.close().
If my program crashes before a socket is closed, the next time I run in, I get an error that looks like this;
socket.error: [Errno 48] Address already in use
Changing the port fixes the problem.
Is there any way to avoid this, and why does this happen (when the program exits, shouldn't the socket be garbage collected, and closed)?
Use .setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) on your listening socket.
A search for those terms will net you many explanations for why this is necessary. Basically, after your first program closes down, the OS keeps the previous listening socket around in a shutdown state for TIME_WAIT time. SO_REUSEADDR says that you want to use the same listening port regardless.
Most OSes take up to 2 minutes to close the socket when the program doesn't properly close it first. I've hit this many times with C programs that SEGFAULT (and I don't have it handled) or similar.
Edit:
Thanks to ephemient for pointing out RFC 793 (TCP) which defines this timeout.
Other people who are getting this error may be getting it because the port is in use by another process. So check if the port is being used by any other processes and either run your program in another port or kill the blocking processes.