I have a simple socket server set-up to send some numbers to a client in the intranet. While testing, I stop the server.py script from terminal(CTRL+C) which later causes busy server error in Safari when I try to reach the same page.
I saw this serverfault question But the issues are:
I could not find the /etc/init.d/networking restart file. I am using Mac and this is for Linux. Also it is an overkill for every-time I test my server. At least 10 times an hour.
Inserting the option SO_REUSEADDR in the code did help Address already in use but I reckon that my server is up but is in a busy state. Using the answer here, I had edited the code to this:
host = <my machine address>
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
port = 8080
s.bind((host, port))
What changes do I make on my server to remove this error in Safari?
Safari can’t open the page "my-ip" because the server unexpectedly dropped the connection. This sometimes occurs when the server is busy. Wait for a few minutes and then try again.
Output of lsof -i:8080, after stopping the script was
python3.7 11881 <user> 3u IPv4 <Device> 0t0 TCP <my machine address>?:http-alt (LISTEN)
I did kill 11881 and then re-run the code but got the same message in Safari.
Output of netstat -na | grep "8080" was
tcp4 0 0 <my machine address>.8080 *.* LISTEN
while the script was running and nothing when I KeyboardInterrupt it.
Firefox shows the page momentarily and then shows the error page
The connection was reset
Chrome shows an empty page only.
Update: It works in python 2.7 but not in 3.5. The difference in the code comes at:
c.send('\n')
in 2.7 and
c.send(bytes('\n'.encode('utf-8')))
in 3.5 where
c is
c, (client_host, client_port) = s.accept()
Rest is all the same.
I had accidentally commented out a line
c.recv(1000)
Where c is mentioned in the question.
I found it when I compared this to an older version of the same file.
I've written server and client programs with Python.
Server.py
import socket
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 5555
sock.bind((host, port))
sock.listen(1)
conn, addr = sock.accept()
data = "Hello!"
data = bytes(data, 'utf-8')
conn.send(data)
sock.close()
Client.py on Linux
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 5555
sock.connect((host, port))
data = sock.recv(2048)
data = str(data, "utf-8")
print(data)
sock.close()
When I run the server and then the client on my local machine (a Linux Mint), it works correctly. I got "Hello!" in bash, and everything is fine. BUT when I ran my client program on another machine (a Windows 8) and ran it (previously I ran server on Linux, of course, and change IP address in client to my static Linux mint's IP) it says:
ConnectionRefusedError: [WinError 10061] No connection could be made
because the target machine actively refused it
client.py on Windows
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "here is my static ip"
port = 5555
sock.connect((host, port))
data = sock.recv(2048)
data = str(data, "utf-8")
print(data)
sock.close()
I must say that I had done port forwarding in my router settings on port 5555. Earlier, I had done same thing to port 80 and my own site worked correctly, but now it doesn't work to 5555 with Python sockets! Why? I can't get it! And one more thing: I tried to change the port to 80 in my server and client files, but it didn't work too. PLease, help.
You have to change the socket.gethostname() in the server script to the empty string (or just directly call socket.bind(('', port))).
Your problem is not in Python but in the usage of sockets generally. When you create a socket, you just prepare your process to receive/send some data from/to another process.
Server
The first step for creating a socket is to specify what kind of protocol will be used for communication between those processes. In your case it is the socket.AF_INET which is constant for use of IP protocol and the socket.SOCK_STREAM is specify reliable stream-oriented service. The reliable stream-oriented service means that you want to be sure that every single sent byte will be delivered to the other side and nothing can be lost during the communication (the underlying OS will use the TCP protocol for that). From this point we are using the IPv4 protocol (because we set the socket.AF_INET).
The second step is bind it to an address. The bind process assigns an address where you expect a client will join (with your socket's settings it's a IP address and the TCP port). Your PC has multiple IP address (well, at least two). It always has 127.0.0.1, which is called "callback" and works only when your applications communicate on the same PC (that is you Linux - Linux scenario in the question), and then you have your external IP address, for communication with other computers (let's pretend it is 10.0.0.1).
When you call socket.bind(('127.0.0.1', 5555)), you're setting the socket to listen only for communication from the same PC. If you call socket.bind(('10.0.0.1', 5555)), then the socket setting is ready to receive data targeted to the 10.0.0.1 address.
But what if you have 10 IPs or more and you want to receive everything (with the right TCP port)? For those scenarios you can leave the IP address in bind() empty, and it does exactly what you want.
With Python's version of bind(), you can also enter a "computer name" instead of the concrete IP. The socket.gethostname() call returns your computer's name. The problem is in the translation of "computer name" to the IP which Python performs behind your back. The translation has some rules but generally your "computer name" can be translated into any IP address which you have set on your computer. In your case, the your computer's name is converted into 127.0.0.1, and that's why communication works only between processes on the same computer.
After socket.bind(), you have the socket ready to use but it is still "inactive". The call to socket.listen() activates the socket and causes it to wait until it receives an attempted connection. When a socket receives a new connection request, it will put it into a queue and wait for processing.
That's what socket.accept() does. It pulls the connection request from the queue, accepts it, and establishes the stream (remember the socket.SOCK_STREAM when you set up the socket) between the server and the client. The new stream is actually a new socket, but is ready to communicate with other side.
What happened with the old socket? Well, it's still alive, and you can call socket.listen() again to get another stream (connection).
How is it possible to have multiple sockets on the same port?
Every connection within computer's network is defined by flow which is 5-item tuple of:
L4 protocol (usually TCP or UDP)
Source IP address
Source L4 port
Destination IP address
Destination L4 port
When you create a new connection with a client, the flow can look like this: (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555). Just for clarification, the server's response flow is (TCP, 10.0.0.1, 55555, 192.168.0.1, 12345), but it isn't important for us. If you create another connection with a client, that it will differ at source TCP port (if you do it from another computer that it will differ also at the source IP). Only from this information you can distinguish every connection created to your computer.
When you create a server socket in your code and call socket.listen(), it listens for any flow with this pattern (TCP, *, *, *, 55555) (the * means "match everything"). So when you get a connection with (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555), then socket.accept() creates another socket which works only with this one concrete flow while the old socket carries on accepting new connections which haven't yet been established.
When the operating system receives a packet, it looks in the packet and checks the flow. At this point, several scenarios can take place:
The packet's flow matches all 5 items exactly (without usage of *). Then the packet's content is delivered to the queue associated with that socket (you're reading the queue when you call socket.recv()).
The packet's flow matched socket with associated flow contains * then it is considered as new connection and you can call scoket.accept().
The operating system doesn't contain open socket which would match the flow. In that case the OS refuse connection (or just ignore the packet it depends on firewall settings).
Probably an example can clarify these scenarios. The operating system has something like a table where it maps flows to sockets. When you call socket.bind(), it will assign a flow to the socket. After the call, the table can look like this:
+=====================================+========+
| Flow | Socket |
+=====================================+========+
| (TCP, *, *, *, 55555) | 1 |
+-------------------------------------+--------+
When it receive a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 10) then it won't match any flow (last port won't match). So, the connection is refused. If it receives a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 55555), the packet is delivered to the socket 1 (because there is a match). The socket.accept() call creates a new socket and record in the table.
+=====================================+========+
| Flow | Socket |
+=====================================+========+
| (TCP, 1.1.1.1, 10, 10.0.0.1, 55555) | 2 |
+-------------------------------------+--------+
| (TCP, *, *, *, 55555) | 1 |
+-------------------------------------+--------+
Now you have 2 sockets for 1 port. Every received packet which matches the flow associated with the socket 2 also matches the flow associated with socket 1 (on the contrary, it does not apply). It's not a problem because the socket 2 has a preciser match (is doesn't use the *), so any data with that flow will be delivered to socket 2.
How to serve multiple connections
If you want to implement a "real" server, your application should be able to process multiple connections without restarting. There are 2 basic approaches:
Sequential processing
try:
l = prepare_socket()
while True:
l.listen()
s, a = socket.accept()
process_connection(s) # before return you should call s.close()
except KeyboardInterrupt:
l.close()
In this case, you can process only one client while others clients have to wait for accept. If the process_connection() takes too long, then others clients will timeout.
Parallel processing
import threading
threads = []
try:
l = prepare_socket()
while True:
l.listen()
s, a = socket.accept()
t = threading.Thread(target=process_connection, s)
threads.append(t)
t.start()
except KeyboardInterrupt:
for t in threads:
t.join()
l.close()
Now when you receive a new connection, it will create a new thread so that every connection is processed in parallel. The main disadvantage of this solution is that you have to solve common troubles with threading (like access to shared memory, deadlocks etc.).
Beware that the above snippets are only examples, and are not complete! For example, they don't contain code for graceful exit on unexpected exceptions.
Servers in Python
Python also contains a module called socketserver, which contains shortcuts for creating servers in Python. You can find examples of how to use it here.
Client
With the client, it's much more simpler than with the server. You just have to create a socket with some settings (same as server side), and then tell it where the server is (what its IP and TCP port are). This is accomplished through the socket.connect() call. As a bonus, it also establishes the stream between your client and server, so from this point you can communicate.
You can find more information about sockets at the Beej's Guide to Network Programming. It's written for usage with C, but the concepts are the same.
I was stuck with the same problem months ago and also wasn't able to do port forwarding. I found a way out of port forwarding Ngrock
For your information what Ngrock does is it is a useful utility to create secure tunnels to locally hosted applications using a reverse proxy. It is a utility to expose any locally hosted application over the web
For How To use it, please see the steps shown below :
If you are in Mac write this command in your terminal to download ngrock
brew install ngrok
For windows
choco install ngrok
After installing You need need to sign up on the Ngrok website
You will get your Ngrock authentication token then paste this command in the terminal
For Mac and Windows
ngrok config add-authtoken <token>
Now that Ngrock is all setup you can start a tunnel using
ngrok tcp <Your Port Number Used In Server.py>
ngrok tcp 5321
Note : Please Give the command inside the directory in which the Python Socket Server File in Located
That's it Your Socket can connect you any computer over the internet anywhere in the world
If You are still struggling to see the detailed explanation in this video
You can also refer ngrock documentation here
Why is my python script behaving this way?
I give it the instruction to connect via port 7777 but instead it is going over 45604.
I am NOT using socket.bind((socket.gethostname(),port))
Instead I work either with socket.bind(("0.0.0.0",port))
or with socket.bind(("127.0.0.1",port))
so I'm working local here. Why does my computer reroute the ports?
There should be no need for that, shouldn't it? Can I somehow disable it locally?
I am answering in the absence of any of your actual code.. So I have to make assumptions here:
1) You have server (right side in picture) listening on port 7777.
2) You are running a client on the same machine (left side of picture) that is connecting to the server.
So, the client (on the left shell) is connecting to the server (right shell window). The server is listening on 7777 and the client is connecting to the server from 45604 (client and server cannot occupy the same port on the same machine!)
Put another way, the client is "sending" to port 7777 from port 45604. Maybe that makes better sense?
A TCP connection is defined by 4 numbers: source IP address, source port, destination IP address, destination port.
The connection goes from 127.0.0.1 port 45604 to 127.0.0.1 port 7777.
The source port (45604) is a value chosen by the system from a wide range of unused ports (it is called an ephemeral port), because your program did not set a specific source port.
I want to write a totally arbitrary SMB server on my computer.
I'm using Impact (http://code.google.com/p/impacket/source/browse/trunk/impacket/smbserver.py), but the problem (probably) is I can't listen to port 445, as it is already in use by windows.
I've killed the LanmanServer service, but it didn't help.
Running netstat -a I can clearly see port 445 is still in used.
Using CurrPorts I can see that the process that is listening to this port is PID 4 (a.k.a. System process) but obviously I can't (and don't want to) kill it.
On the other hand, just for the sake of it, I've tried to listen to some random port (5002), and check out the SMB server by connecting to \\[my internal ip = 10.0.0.4]:5002\, weirdly enough, this also didn't work out.
I could see some request in Wireshark.. and I know my python process was listening to this port, but it just didn't answer.
On the server I just run this:
SMBSERVER(('10.0.0.4', 5002))
I am learning network programing in python and I'm trying to write a Toy vpn forked from android sdk https://github.com/android/platform_development/tree/master/samples/ToyVpn.
My Toy vpn is https://github.com/325862401/ToyVPN.
It's only for Linux.
My home network is behind NAT.
I can use this vpn to surf the internet after connect to remote sever.
But about half an hour or some time later the client udp socket stops receiving any data but the server can receive and send normally.
At this point I must terminate my client and run ToyVpnClient again.
It works normal for some time until it stop receiving again.
Please help me check the client logs.
>2013-08-24 11:42:38 INFO receive data from the tunnel timeout`
you can see that when problem happens, the socket always sends, not receive.
> means send, < means receive
I want to know why the udp socket stops receiving data.
Is there any debug method to find the cause?
For now I've just used logging to debug my program.
Since you're trying your client on the Internet, there is the whole universe of possible causes represented by all the Internet newtwork.
There's not a simple way of debugging here. Possible causes could be of course a software error but also some intermediate network configurations between you and the remote server.
You should capture the udp traffic using the good wireshark or the commandline tcpdump between you and the server and check if you're stopping sending packets or if the server is stopping receiving them.
If you send packets but your server doesn't receive them ( tcpdump on the server ) then there is something on the network which decides to filter your packets. And if it's not on the server (firewall rules to rate limit packets for example or something like that) then there's nothing you can do to that without modifying the logic of your program. Like changing UDP port every X seconds or using a persistent tcp connection.
A udp socket is not stable and may become null once a scanning or other event occupy your network interface for a while (especially true on Android). Using tcp avoids this problem. If you wants to maintain a stable udp, keep monitoring the status of your udp socket; if it becomes null or any unusual things happens, delete this socket and create a new one. Put this reactivating staff in a loop so that your udp socket is always alive.