Is it possible to create a socket connection (from either Python or a nc based listener), and then 'join' it from another Python process, sending data from the same socket to the same remote, and vice versa?
It is possible for a Python process to create a listening socket, bind() it to a port, and listen()s for incoming TCP connections on that port, and then at some point after that, another Python process can connect() to that port, at which point the first process can accept() the incoming TCP connection and the two processes can send data to each other over it. So if that is what you are asking about, the Python socket module has the APIs you are looking for.
If OTOH you are asking about splicing a third party in as a middleman between two existing processes that are already communicating with each other over TCP, that is not possible without some serious low-level hackery, since TCP was designed as a 1-to-1 communications mechanism only.
Related
I have a python socket server that listens on a port, and accepts all incoming connections using:
(conn, address) = socket.accept()
However, I wish to accept connections only from certain ip address.
Currently, I close the connection if the address isn't registered, to accomplish this.
But is there a better way to do this, by directly rejecting connections from unregistered addresses, instead of accepting connections and then closing them?
It's not possible to indicate Connection refused to clients from some IP addresses, and to establish the connection to clients from other IP addresses. This is not a Python limitation, but a lower-level, BSD socket layer limitation. You can't do it even from C.
The closest behavior in general you can do in Python is closing the connection quickly after it has been accepted:
sock, addr = server_socket.accept()
if addr[0] != '12.34.56.78':
sock.close()
return
...
Then the client would see the connection being accepted, and very shortly after that the client would see EOF when reading from it, and it wouldn't be able to write to it.
However it's possible to limit by interface (i.e. network card) at bind time, by using one of:
server_socket.bind(('', 65432)) # Bind on any interface.
server_socket.bind(('127.0.0.1', 65432)) # Bind on loopback (localhost clients only).
server_socket.bind(('34.56.78.91', 65432))
So in the 127.0.0.1 version, telnet 127.0.0.1 65432 (as a client) would work, but telnet myhostname 65432 would yield Connection refused (and the server_socket.accept() call won't get this connection).
If you read the docs you can find the BaseServer.verify_request(request, client_address) which tells you this:
Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.
Microsoft appears to support this functionality via the SO_CONDITIONAL_ACCEPT socket option
This appears to require usage of WSAAccept to accept connections
This constant does not appear in pythons socket module on my windows 8 machine. I don't think there is an option to use WSAAccept via python's builtin socket module.
If I understand correctly, this will allow your server to respond to SYN packets immediately with RST packets when configured to do so instead of finishing the handshake and exchanging FIN packets. Note that usage of this flag removes responsibility to handle connections from the operating system and places it on the application, so there is plenty of room for errors and performance hits to occur. If a performance boost was the goal, it might not be not worth pursuing
It is possible to do at the C level on windows. Pythons ctypes module allows interfacing with C code, so it is technically possible to do via a python interface. But it likely requires a non trivial amount of effort. If you are certain you require this feature, it may be less effort to find a C socket library that supports this out of the box, then you could make a ctypes wrapper for that.
When you setup a simple TCP listening socket using the Python 'socket' module, what are the different steps involved doing?
The code I'm talking about looks like this:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 50000))
s.listen(1)
conn, addr = s.accept()
The s = ... seems pretty straightforward - you are expressing your intent to create an ipv4 TCP socket, without having done anything yet.
What I'm curious about is this:
What does it mean to bind to a socket, without listening?
How does limiting the number of unaccepted connections using listen(n) work?
If you have listen(1), you're in the middle of dealing with the first connection you accepted, and a second client tries to connect, is the second client waiting for the SYN-ACK? Or does the 3 way handshake happen, and he's waiting for actual data?
What happens if a third client tries to connect - does he immediately get a TCP RST?
Does setting the number of unaccepted connections here set some option in the kernel to indicate how many connections it should accept? Or is this all handled in Python?
How can you be listening without accepting? What does it mean to accept a connection?
Every article I've come across seems to just assume these steps make sense to everyone, without explaining what exactly it is that each one does. They just use generic terms like
listen() starts listening for connections
bind() binds to a socket
accept() just accepts the connection
Defining a word by using that word in the definition is kind of a dumb way to explain something.
it's basically a 1-to-1 from the POSIX c calls and as such I'm including links to the man pages, so that you can read their explanation and corresponding c code:
socket creates a communication endpoint by means of a file-descriptor in the namespace of the address-family you specified but assigns neither address nor port.
bind assigns an address and port to said socket, a port which may be chosen randomly if you request a port for which you do not have the privilige. (like < 1024 for non-root user)
listen makes the specific socket and hence address and port a passive one, meaning that it will accept incoming connections with the accept call. To handle multiple connections one after the other, you get to specify a backlog containing them, connections that arrive while you're handling one get appended. Once the backlog is full, the system will respond as such to those systems with an approach that makes them reconnect by withholding SYN, withholding ACK response etc..
As usual you can find someone explaining the previous to you a lot better.
accept then creates a new non-listening socket associated with a new file descriptor that you then use for communication with said connecting party.
accept also works as a director for your flow of execution, effectively blocking further progress until a connection is actually available in the queue for it to take, like a spinlock. The only way around that is to declare the socket non-blocking in which case it would return immediately with an error.
I'm trying to write a server for a chat program. I want the server to have a tcp connection with every chat user. Is there way for the server to have multiple tcp connections at the same time without creating a socket for every connection? And if yes, how?
No. Unlike UDP sockets, TCP sockets work are connection-oriented. Whatever data is written into a socket, "magically" appears to come out of the socket at the other end as a stream of data. For that, both sockets maintain a virtual connection, a state. Among other things, the state defines both endpoints of the conntection - IP and port numbers of both sockets. So a single TCP socket can only talk to a single TCP socket at the other end.
UDP sockets on the other hand operate on a per-packet basis (connectionless), allowing you to send a receive packets to/from any destination using the same socket. However, UDP does not guarantee reliability and in-order delivery.
Btw, your question has nothing to do with python. All sockets (except raw sockets) are operating system sockets that work in the same way in all languages.
I want to create a simple video streaming (actually, image streaming) server that can manage different protocols (TCP Push/Pull, UDP Push/Pull/Multicast).
I managed to get TCP Push/Pull working with the SocketServer.TCPServer class and ThreadinMixIn for processing each connected client in a different thread.
But now that I'm working on the UDP protocol, I just realized that ThreadinMixIn creates a thread per call of handle() per client query (as there's nothing such as a "connection" in UDP).
The problem is I need to process a sequence of queries by the same client, for all the clients. How could I manage that ?
The only way I see I could handle that is to have a list of (client adresses, processing thread) and send each query to the matching thread (or create a new one if the client haven't sent any thread yet). Is there an easier way to do that ?
Thanks !
P.S : I can't use any external or too "high-level" library for this as it's a school subject meant to understand how sockets work.
Take a look at Twisted. This will remove the need to do any thread dispatch from your application. You still have to match up packets to a particular session in order to handle them, but this isn't difficult (use a port per client and dispatch based on the port, or require packets in a session to always come from the same address and use the peer address, or use one of the existing protocols that solves this problem such as SIP).
I have a Python application which opens a simple TCP socket to communicate with another Python application on a separate host. Sometimes the program will either error or I will directly kill it, and in either case the socket may be left open for some unknown time.
The next time I go to run the program I get this error:
socket.error: [Errno 98] Address already in use
Now the program always tries to use the same port, so it appears as though it is still open. I checked and am quite sure the program isn't running in the background and yet my address is still in use.
SO, how can I manually (or otherwise) close a socket/address so that my program can immediately re-use it?
Update
Based on Mike's answer I checked out the socket(7) page and looked at SO_REUSEADDR:
SO_REUSEADDR
Indicates that the rules used in validating addresses supplied in a bind(2) call should
allow reuse of local addresses. For AF_INET sockets this means that a socket may bind,
except when there is an active listening socket bound to the address. When the listen‐
ing socket is bound to INADDR_ANY with a specific port then it is not possible to bind
to this port for any local address. Argument is an integer boolean flag.
Assume your socket is named s... you need to set socket.SO_REUSEADDR on the server's socket before binding to an interface... this will allow you to immediately restart a TCP server...
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((ADDR, PORT))
You might want to try using Twisted for your networking. Mike gave the correct low-level answer, SO_REUSEADDR, but he didn't mention that this isn't a very good option to set on Windows. This is the sort of thing that Twisted takes care of for you automatically. There are many, many other examples of this kind of boring low-level detail that you have to pay attention to when using the socket module directly but which you can forget about if you use a higher level library like Twisted.
You are confusing sockets, connections, and ports. Sockets are endpoints of connections, which in turn are 5-tuples {protocol, local-ip, local-port, remote-ip, remote-port}. The killed program's socket has been closed by the OS, and ditto the connection. The only relic of the connection is the peer's socket and the corresponding port at the peer host. So what you should really be asking about is how to reuse the local port. To which the answer is SO_REUSEADDR as per the other answers.