I have a question regarding client socket on TCP/IP network. Let's say I use
try:
comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(1)
try:
comSocket.bind(('', 5555))
comSocket.connect()
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(2)
The socket created will be bound to port 5555. The problem is that after ending the connection
comSocket.shutdown(1)
comSocket.close()
Using wireshark, I see the socket closed with FIN,ACK and ACK from both sides, I can't use the port again. I get the following error:
[ERROR] Address already in use
I wonder how can I clear the port right away so that next time I still can use that same port.
comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
setsockopt doesn't seem to be able to resolve the problem
Thank you!
Try using the SO_REUSEADDR socket option before binding the socket.
comSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Edit:
I see you're still having trouble with this. There is a case where SO_REUSEADDR won't work. If you try to bind a socket and reconnect to the same destination (with SO_REUSEADDR enabled), then TIME_WAIT will still be in effect. It will however allow you to connect to a different host:port.
A couple of solutions come to mind. You can either continue retrying until you can gain a connection again. Or if the client initiates the closing of the socket (not the server), then it should magically work.
Here is the complete code that I've tested and absolutely does NOT give me a "address already in use" error. You can save this in a file and run the file from within the base directory of the HTML files you want to serve. Additionally, you could programmatically change directories prior to starting the server
import socket
import SimpleHTTPServer
import SocketServer
# import os # uncomment if you want to change directories within the program
PORT = 8000
# Absolutely essential! This ensures that socket resuse is setup BEFORE
# it is bound. Will avoid the TIME_WAIT issue
class MyTCPServer(SocketServer.TCPServer):
def server_bind(self):
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.socket.bind(self.server_address)
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = MyTCPServer(("", PORT), Handler)
# os.chdir("/My/Webpages/Live/here.html")
httpd.serve_forever()
# httpd.shutdown() # If you want to programmatically shut off the server
According to this link
Actually, SO_REUSEADDR flag can lead to much greater consequences:
SO_REUSADDR permits you to use a port that is stuck in TIME_WAIT, but
you still can not use that port to establish a connection to the last
place it connected to. What? Suppose I pick local port 1010, and
connect to foobar.com port 300, and then close locally, leaving that
port in TIME_WAIT. I can reuse local port 1010 right away to connect
to anywhere except for foobar.com port 300.
However you can completely avoid TIME_WAIT state by ensuring that the remote end initiates the closure (close event). So the server can avoid problems by letting the client close first. The application protocol must be designed so that the client knows when to close. The server can safely close in response to an EOF from the client, however it will also need to set a timeout when it is expecting an EOF in case the client has left the network ungracefully. In many cases simply waiting a few seconds before the server closes will be adequate.
I also advice you to learn more about networking and network programming. You should now at least how tcp protocol works. The protocol is quite trivial and small and hence, may save you a lot of time in future.
With netstat command you can easily see which programs ( (program_name,pid) tuple) are binded to which ports and what is the socket current state: TIME_WAIT, CLOSING, FIN_WAIT and so on.
A really good explanation of linux network configurations can be found https://serverfault.com/questions/212093/how-to-reduce-number-of-sockets-in-time-wait.
In case you face the problem using TCPServer or SimpleHTTPServer,
override SocketServer.TCPServer.allow_reuse_address (python 2.7.x)
or socketserver.TCPServer.allow_reuse_address (python 3.x) attribute
class MyServer(SocketServer.TCPServer):
allow_reuse_address = True
server = MyServer((HOST, PORT), MyHandler)
server.serve_forever()
You need to set the allow_reuse_address before binding. Instead of the SimpleHTTPServer run this snippet:
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer(("", PORT), Handler, bind_and_activate=False)
httpd.allow_reuse_address = True
httpd.server_bind()
httpd.server_activate()
httpd.serve_forever()
This prevents the server from binding before we got a chance to set the flags.
As Felipe Cruze mentioned, you must set the SO_REUSEADDR before binding. I found a solution on another site - solution on other site, reproduced below
The problem is that the SO_REUSEADDR socket option must be set before
the address is bound to the socket. This can be done by subclassing
ThreadingTCPServer and overriding the server_bind method as follows:
import SocketServer, socket
class MyThreadingTCPServer(SocketServer.ThreadingTCPServer):
def server_bind(self):
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.socket.bind(self.server_address)
I found another reason for this exception.
When running the application from Spyder IDE (in my case it was Spyder3 on Raspbian) and the program terminated by ^C or an exception, the socket was still active:
sudo netstat -ap | grep 31416
tcp 0 0 0.0.0.0:31416 0.0.0.0:* LISTEN 13210/python3
Running the program again found the "Address already in use"; the IDE seems to start the new 'run' as a separate process which finds the socket used by the previous 'run'.
socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
did NOT help.
Killing process 13210 helped.
Starting the python script from command-line like
python3 <app-name>.py
always worked well when SO_REUSEADDR was set to true. The new Thonny IDE or Idle3 IDE did not have this problem.
socket.socket() should run before socket.bind() and use REUSEADDR as said
I know you've already accepted an answer but I believe the problem has to do with calling bind() on a client socket. This might be OK but bind() and shutdown() don't seem to play well together. Also, SO_REUSEADDR is generally used with listen sockets. i.e. on the server side.
You should be passing and ip/port to connect(). Like this:
comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
comSocket.connect(('', 5555))
Don't call bind(), don't set SO_REUSEADDR.
For me the better solution was the following. Since the initiative of closing the connection was done by the server, the setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) had no effect and the TIME_WAIT was avoiding a new connection on the same port with error:
[Errno 10048]: Address already in use. Only one usage of each socket address (protocol/IP address/port) is normally permitted
I finally used the solution to let the OS choose the port itself, then another port is used if the precedent is still in TIME_WAIT.
I replaced:
self._socket.bind((guest, port))
with:
self._socket.bind((guest, 0))
As it was indicated in the python socket documentation of a tcp address:
If supplied, source_address must be a 2-tuple (host, port) for the socket to bind to as its source address before connecting. If host or port are ‘’ or 0 respectively the OS default behavior will be used.
another solution, in development environment of course, is killing process using it, for example
def serve():
server = HTTPServer(('', PORT_NUMBER), BaseHTTPRequestHandler)
print 'Started httpserver on port ' , PORT_NUMBER
server.serve_forever()
try:
serve()
except Exception, e:
print "probably port is used. killing processes using given port %d, %s"%(PORT_NUMBER,e)
os.system("xterm -e 'sudo fuser -kuv %d/tcp'" % PORT_NUMBER)
serve()
raise e
I think the best way is just to kill the process on that port, by typing in the terminal fuser -k [PORT NUMBER]/tcp, e.g. fuser -k 5001/tcp.
I had the same problem and I couldn't find any other solution (reuse options didn't work) except restarting Raspberry Pi each time. Then I found a workaround;
comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
comSocket.close()
comSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
comSocket.connect(('', 5555))
This means, define socket first, close it, then define again, so you can use the same port if it is stuck.
Related
Good day everyone! I'm still doing research on this so please pardon if I make any mistake. I'm currently working on a small project that need socket connection between 2 device, problem is, when ever I used the client and the server on the same device, it worked out okay. But when I moved the client into a different device, then started the process again(same LAN connection), it just gave me the time out error [WinError 10060]. Here is my code:
Server side:
HOST = '10.0.0.32'
PORT = 44132
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST, PORT))
server.listen()
client, address = server.accept()
Client side:
HOST = '10.0.0.32'
PORT = 44132
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((HOST, PORT))
I have tried disabling the Firewall and restart computer, changing port or trying to check in cmd if server is really Listening or not, is there anything that I'm missing here? Thank you.
The full error report is: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
edit:
Here is also my port listening on 44132 using netstat:
Proto Local Address Foreign Address State
TCP 0.0.0.0:44132 0.0.0.0:* LISTENING
edit2: Another update on my end, I've tried turning off the firewall on target machine and ping it, the ping now went through successfully but the client and server still refused to reconnect and continue on timing out. Could it be that there is another firewall between my 2 devices and are implemented by the router to prevent the connection taking place?
Probably you are using the wrong ip address, my advice is to use the command arp -a to check if the server's ip is correct (if you have access to the router you could check there otherwise).
Moreover, be aware when you use socket.gethostbyname(socket.gethostname()), take a look here Python socket.gethostname
So i'm trying to write a TCP server and client so that when the client connects, a file is sent back from the server. Here's my code for the server:
import socket
import threading
bind_ip = '0.0.0.0'
bind_port = 9999
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(10)
file_to_send = ('file_to_send.txt')
print '[*] Listening on %s:%d' % (bind_ip,bind_port)
def handle_client(client_socket):
request = client_socket.recv(1024)
print '[*] Received %s' % request
client_socket.send('')
client_socket.close(file_to_send)
while True:
client,addr = server.accept()
print '[*] Accepted connection from: %s:%d' % (addr[0],addr[1])
client_handler = threading.Thread(target=handle_client,args=(client,))
client_handler.start()
And here is my code for the client:
import socket
target_host = '0.0.0.0'
target_port = 9999
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host,target_port))
client.send('ACK!')
response = client.recv(4096)
print response
When the server and client are run, the server returns the error 'Only one usage of each socket address (protocol/network address/port) is normally permitted' and when the client is run I get the error ' The requested address is not valid in its context'
Does anyone know why these errors are occurring or how I might be able to fix it.
I think this is a case of inaccurate example code. The server code you posted does not cause the issue you're describing (the client code does, but we'll get to that).
Server
The issue with the server code is that you're binding to the same (address, port) twice. Whether this is from wrong indentation or wrong logic it's tough to say, but that error message comes from binding the same protocol to the same address with the same port number more than once at the same time.
The rest of the code seems fine, though you can't do client_socket.close("some string") as you're doing here. socket.close does not accept any arguments.
Client
There's a simple solution here -- just change the target_host to something reasonable. You cannot connect to 0.0.0.0. This should be the addressable IP of the server in the smallest scope possible. Presumably if this is a toy program this is something like localhost.
(N.B. you bind the server to '0.0.0.0' to tell it to accept connections going to any destination IP. You could bind the server instead to '127.0.0.1' to inform the server that it will ONLY be known as localhost and never anything else.)
I've had a similar issue before and it was that I was running old versions of the program on the same port, restarting the PC or closing the processes in task manager should fix it.
I'm aware that this was asked over a year ago, so OP has probably restarted their PC since then, but hopefully this will help someone looking for a solution for a similar problem.
On Windows 7:
Given this server code:
# in server.py
if __name__ == '__main__':
serversock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# uncommenting this won't help
#serversock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serversock.bind(('',8888))
serversock.listen(5)
# accept and receive dummy data from client
clientsock,address = serversock.accept()
data = clientsock.recv(1024)
# as long as calc.exe is running, I can't do this again
subprocess.Popen(r"c:\windows\system32\calc.exe")
# letting client close first still won't help
time.sleep(3)
# closing won't help either
clientsock.close()
serversock.close()
And the client code
# in client.py
if __name__ == '__main__':
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 8888))
s.sendall('Hello, world')
# close early to help prevent TIME_WAIT on server, but doesn't help
s.close()
Running server first then client will launch the calculator app.
While the calculator app is still running, I can't run server again. It will complain about
python server.py (ok)
python client.py (ok)
python server.py (boom!)
socket.error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
If I close the Calculator app, running server is ok again...
This does not happen on Mac.
Enabling SO_REUSEADDR will only make the error go away, but the server is unreachable from the client.
In the example above, I specifically let the client close first so that the server socket don't go into TIME_WAIT.
So the questions:
Am I running into the TIME_WAIT problem on the server?
Are any sockets/filedescriptors left unclosed in the server?
Why SO_REUSEADDR won't help in this case? could the client be coming from the same port?
Could the child process be hanging on to some descriptors?
What can I do about this?
The SOLUTION:
The problem IS with the parent process of Calculator holding on to some file descriptor.
So adding close_fds=True to the Popen will ensure everything is released properly.
subprocess.Popen(r"c:\windows\system32\calc.exe", close_fds=True)
Closing socket after subprocess.Popen leaves socket in TIME_WAIT as long as child process is still running
No it doesn't. It leaves it in TIME_WAIT for a fixed amount of time, 2 or 4 minutes. After the close it has nothing to do with the child process at all.
I'm new to Sockets, please excuse my complete lack of understanding.
I have a server script(server.py):
#!/usr/bin/python
import socket #import the socket module
s = socket.socket() #Create a socket object
host = socket.gethostname() #Get the local machine name
port = 12397 # Reserve a port for your service
s.bind((host,port)) #Bind to the port
s.listen(5) #Wait for the client connection
while True:
c,addr = s.accept() #Establish a connection with the client
print "Got connection from", addr
c.send("Thank you for connecting!")
c.close()
and client script (client.py):
#!/usr/bin/python
import socket #import socket module
s = socket.socket() #create a socket object
host = '192.168.1.94' #Host i.p
port = 12397 #Reserve a port for your service
s.connect((host,port))
print s.recv(1024)
s.close
I go to my desktop terminal and start the script by typing:
python server.py
after which, I go to my laptop terminal and start the client script:
python client.py
but I get the following error:
File "client.py", line 9, in
s.connect((host,port))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 111] Connection refused
I've tried using different port numbers to no avail. However, I was able to get the host name using the same ip and the gethostname() method in the client script and I can ping the desktop (server).
Instead of
host = socket.gethostname() #Get the local machine name
port = 12397 # Reserve a port for your service
s.bind((host,port)) #Bind to the port
you should try
port = 12397 # Reserve a port for your service
s.bind(('', port)) #Bind to the port
so that the listening socket isn't too restricted. Maybe otherwise the listening only occurs on one interface which, in turn, isn't related with the local network.
One example could be that it only listens to 127.0.0.1, which makes connecting from a different host impossible.
This error means that for whatever reason the client cannot connect to the port on the computer running server script. This can be caused by few things, like lack of routing to the destination, but since you can ping the server, it should not be the case. The other reason might be that you have a firewall somewhere between your client and the server - it could be on server itself or on the client. Given your network addressing, I assume both server and client are on the same LAN, so there shouldn't be any router/firewall involved that could block the traffic. In this case, I'd try the following:
check if you really have that port listening on the server (this should tell you if your code does what you think it should): based on your OS, but on linux you could do something like netstat -ntulp
check from the server, if you're accepting the connections to the server: again based on your OS, but telnet LISTENING_IP LISTENING_PORT should do the job
check if you can access the port of the server from the client, but not using the code: just us the telnet (or appropriate command for your OS) from the client
and then let us know the findings.
Assume s = socket.socket()
The server can be bound by following methods:
Method 1:
host = socket.gethostname()
s.bind((host, port))
Method 2:
host = socket.gethostbyname("localhost") #Note the extra letters "by"
s.bind((host, port))
Method 3:
host = socket.gethostbyname("192.168.1.48")
s.bind((host, port))
If you do not exactly use same method on the client side, you will get the error: socket.error errno 111 connection refused.
So, you have to use on the client side exactly same method to get the host, as you do on the server. For example, in case of client, you will correspondingly use following methods:
Method 1:
host = socket.gethostname()
s.connect((host, port))
Method 2:
host = socket.gethostbyname("localhost") # Get local machine name
s.connect((host, port))
Method 3:
host = socket.gethostbyname("192.168.1.48") # Get local machine name
s.connect((host, port))
Hope that resolves the problem.
host = socket.gethostname() # Get the local machine name
port = 12397 # Reserve a port for your service
s.bind((host,port)) # Bind to the port
I think this error may related to the DNS resolution.
This sentence host = socket.gethostname() get the host name, but if the operating system can not resolve the host name to local address, you would get the error.
Linux operating system can modify the /etc/hosts file, add one line in it. It looks like below( 'hostname' is which socket.gethostname() got).
127.0.0.1 hostname
in your server.py file make : host ='192.168.1.94' instead of host = socket.gethostname()
Pay attention to change the port number. Sometimes, you need just to change the port number. I experienced that when i made changes over changes over syntax and functions.
I was being able to ping my connection but was STILL getting the 'connection refused' error. Turns out I was pinging myself! That's what the problem was.
I was getting the same problem in my code, and after thow days of search i finally found the solution, and the problem is the function socket.gethostbyname(socket.gethostname) doesnt work in linux so instead of that you have to use socket.gethostbyname('put the hostname manually') not socket.gethostbyname('localhost'), use socket.gethostbyname('host') looking with ifconfig.
try this command in terminal:
sudo ufw enable
ufw allow 12397
I am currently using this lib to stress test a kafka server that I have set up: https://github.com/dsully/pykafka
import kafka
import time
def test_kafka_server(n=1):
for i in range(0,n):
producer = kafka.producer.Producer('test',host='10.137.8.192')
message = kafka.message.Message(str(time.time()))
producer.send(message)
producer.disconnect()
def main():
test_kafka_server(100000)
if __name__ == '__main__':
main()
What just ends up happening is that I end up overloading my own local machine.
I get error 10055, which according to google means that "Windows has run out of TCP/IP socket buffers because too many connections are open at once." According to netstat, producer.disconnect() is not closing the socket, but rather putting it in a TIME_WAIT state.
The ipython debugger points to this line:
C:\Python27\lib\socket.pyc in meth(name, self, *args)
222 proto = property(lambda self: self._sock.proto, doc="the socket protocol")
223
--> 224 def meth(name,self,*args):
225 return getattr(self._sock,name)(*args)
226
as the culprit, but this then seems to get into messing with things at a lower level than I am comfortable with.
I had searched and found this Python socket doesn't close connection properly which recommended doing:
setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
so, I rebuilt the pykafka lib using that option in the io.py file:
def connect(self):
""" Connect to the Kafka server. """
global socket
self.socket = socket.socket()
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.socket.connect((self.host, self.port))
and I still get the same error.
Am I not putting the setsockopt line in the right spot? Is there anything else I could be trying?
What you are describing is normal TCP behavior at the socket level. When a user level program closes a socket the kernel does not free the socket right away. It enters TIME_WAIT state:
TIME-WAIT (either server or client) represents waiting for enough
time to pass to be sure the remote TCP received the acknowledgment of
its connection termination request. [According to RFC 793 a connection
can stay in TIME-WAIT for a maximum of four minutes known as a MSL
(maximum segment lifetime).
So the socket is closed. The socket.SO_REUSEADDR is for listeners (servers), doesn't effect client connections. Well, really used when binding the socket.