Develop Reference

How can I create a TCP connection in Python between 2 PCs

So far I have made a VERY basic client/server application that creates a TCP connection. I have a lot of programming experience, just never did this low-level stuff and especially nothing with networks. Note that all the prints are just to help me figuring out what is going on. One of the known issues is that jsonip sometimes gives me an IPv4 and sometimes v6, I don't know why but that doesn't matter for now, just to warn anyone who wants to recreate my code.
Server:
import socket
import requests
port = int(input("Enter port you want to open:\n"))
#todo: add errorhandling
print("Adding socket...")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
hostname = socket.gethostname()
print(f"Hostname: {hostname}")
ip_address = socket.gethostbyname(hostname)
print(f"Host address: {ip_address}")
r = requests.get(r'http://jsonip.com')
public_ip_address = r.json()['ip']
s.bind((ip_address, port))
print("Is open for connections on IP: "+public_ip_address+" and Port: "+str(port))
s.listen(5)
print("Done initialisation, listening for incoming connections...")
while True:
clientsocket, address = s.accept()
print(f"Connection from {address} has been established")
clientsocket.send(bytes(f"You have connected to server: {hostname}", "utf-8"))
Client:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip = input("Enter IP to connect to:\n")
port = int(input("Enter Port to connect to:\n"))
print(f"Connecting to server {ip} ...")
s.connect((ip, port))
msg = s.recv(1024)
print(msg.decode("utf-8"))
On my local machine: Open 20000 in my server.py, it tells me the host is 127.0.1.1, I then enter 127.0.1.1 into my client script and 20000, and they connect. So the Socket has been bound with the 127.0.1.1. (Side question: What is this IP address, is it like the internal IP address of processes in my PC or something? If running ip a on my other machine it is the first one shown of 2)
Using Virtmanager on my machine and running one Linux Server (command line only) and one normal Ubuntu, the server tells me the host is, again, 127.0.1.1 which I don't need to enter into the other VM to know it won't work, what does work however, is getting the IP-address of the Server via ip a, which in this case is 192.168.122.37, and when I enter this IP address into the client, it connects. But in the socket here I bind, again, the 127.0.1.1, so is it arbitrary what I put here? What SHOULD I bind here, the public, the weird or the 192. address?
The first thing I could not get to work was using 2 physical devices. When opening a server on my Linux machine, I cannot connect with my windows machine at all, no matter if I use my public, my 127. or my 192. IP-address. Now my end goal is doing this over the internet so I am walking myself up, describing here the steps I took to try and get where I want to be but here I hit a brick wall where I don't know what is wrong. Am I binding the wrong address on the server, is my router being a problem, is there something else wrong?
I also tried leaving my network using my friends pc a few countries over, but this also just results in a timeout (my theory is that the Router port he is trying to open is closed and I have now idea how I can make the router send data to his PC, which should be not impossible as firefox and every application using internet does it without me having to manually forward every port, I just don't know how). This is my end goal, creating a connection between my friends PC and mine, and this is how far I got (I wouldn't mind skipping the local network if it is not relevant for fixing the global connection problem), so, tl;dr: what did i do wrong, what do i need to bind and what do i need to do for the final result to work?

There are many questions to answer.
Addresses 127.X.X.X are reserved for the loopback interface, most common one is 127.0.0.1. The loopback is a virtual, but important interface and as you have probably guessed, it is usable on the local machine only. You cannot use 127.X.X.X address to make two hosts to communicate with each other.
Addresses 192.168.X.X (and also 10.X.X.X and 172.16-31.X.X.) are reserved for local LANs. They are not valid on the Internet.
You cannot use these addresses to make two hosts to communicate with each other over the public Internet (unless you create a tunnel, an advanced networking topic)
Almost everybody uses them, because we ran out of IPv4 addresses long time ago, they were difficult to get, expensive, etc. Also such hosts are isolated from the Internet, they can be reached only via a router that translates addresses. Such router feature is called NAT. A typical router has one valid Internet address and all connections to the Internet appear as coming from the router. If you contant a service like jsonip.com from a PC, you get your router's address, not your PC's address.
See also: Finding local IP addresses using Python's stdlib
To make your program working, make it to accept connections on all interfaces. See the first example in the socket docs. On Linux, use port numbers >= 1024. Ports < 1024 are reserved, not available to regular users.
Final point is that a firewall may prevent connections to your server. It depends on your system and setup.

Python - socket.gethostbyaddr() fails in one PC but works on another

I have a Python script to get hostname for a list of IP addresses. The script works when I run from one computer but same script returns error ([Errno 11004] host not found) when run from another computer.
Both computers are W10 (v1809) with same Python version (3.7.4), sitting next to each other and connected to same corporate network. I don't even know where to look at for potential difference. Would appreciate any hints where/what I should be looking or if there's a different way to get hostname from IP.
Here's my code
for ip in ipList:
try:
retVal = socket.gethostbyaddr(ip)
except socket.error as serr:
logger.debug('IP2Host for {} failed with Error {}'.format(ip, serr))
retVal = 'FAILED'
return retVal

This is probably related to network settings.The hostname resolution is usually done by DNS ethernet protocole or hosts file and compare your netwotrk settings (ipconfig /ALL)
check the result of system command nslookup ip in a terminal.

Python. Need to be sure the connection is made from the local machine?

Imagine you have a HTTP server on your local machine, this is a typical Python/Twisted application. This server is used to access your local data, server is used just as a GUI interface. So user can use his web browser or special application ( acts like a web browser ) to access his local data.
Now you want to be sure that only local user who physically sit near this machine get access to the HTTP server.
Also I will have FTP server and it must be protected the same way too.
At the moment I am running such code for my HTTP server:
class LocalSite(server.Site):
def buildProtocol(self, addr):
if addr.host != '127.0.0.1':
print 'WARNING connection from ' + str(addr)
return None
try:
res = server.Site.buildProtocol(self, addr)
except:
res = None
return res
So I am just check the IP address at the moment and I am not sure this is enough.
Is there any ways to emulate local IP from remote machine.?
Well, If a bad guy get access over my OS I have no way to protect - but this is not my deal. My firewall and antivirus should care about this, right?
Anyway, I would like to listen any extra ideas about increase security of such HTTP server.
May be we can use MAC address to verify connection.?
Check the processes on local machine and detect which is actually executes connection?
We can use HTTPS, but in my understanding this acts in opposite direction: this is for user to trust to the server, not server to trust to the user.
Using CAPTCHA is a kind of solution. But I do not like this at all (it strains users) and this will not work for FTP server.
I am also use random port number every time application starts.
The type of internet connection is not defined - this is a p2p application. Any user in the WEB can use my software and it must be protected against remote access.

I believe the way you handled it is good enough. About it being cross-platform, I believe it is as Windows(starting from windows 7) too maps localhost to 127.0.0.1 but for previous versions, you have to define localhost in the main hosts file.

socket.error:[errno 99] cannot assign requested address and namespace in python

My server software says errno99: cannot assign requested address while using an ip address other than 127.0.0.1 for binding.
But if the IP address is 127.0.0.1 it works.
Is it related to namespaces?
I am executing my server and client codes in another python program by calling execfile().
I am actually editing the mininet source code.I edited net.py and inside that I used execfile('server.py') execfile('client1.py') and execfile('client2.py').So as soon as "sudo mn --topo single,3" is called along with the creation of 3 hosts my server and client codes will get executed.I have given my server and client codes below.
#server code
import select
import socket
import sys
backlog = 5
size = 1024
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("10.0.0.1",9999))
server.listen(backlog)
input = [server]
running = 1
while running:
inputready,outputready,exceptready = select.select(input,[],[])
for s in inputready:
if s == server:
client, address = server.accept()
input.append(client)
else:
l = s.recv(1024)
sys.stdout.write(l)
server.close()
#client code
import socket
import select
import sys
import time
while(1) :
s,addr=server1.accept()
data=int(s.recv(4))
s = socket.socket()
s.connect(("10.0.0.1",9999))
while (1):
f=open ("hello1.txt", "rb")
l = f.read(1024)
s.send(l)
l = f.read(1024)
time.sleep(5)
s.close()

Stripping things down to basics this is what you would want to test with:
import socket
server = socket.socket()
server.bind(("10.0.0.1", 6677))
server.listen(4)
client_socket, client_address = server.accept()
print(client_address, "has connected")
while True:
recvieved_data = client_socket.recv(1024)
print(recvieved_data)
This works assuming a few things:
Your local IP address (on the server) is 10.0.0.1 (This video shows you how)
No other software is listening on port 6677
Also note the basic concept of IP addresses:
Try the following, open the start menu, in the "search" field type cmd and press enter.
Once the black console opens up type ping www.google.com and this should give you and IP address for google. This address is googles local IP and they bind to that and obviously you can not bind to an IP address owned by google.
With that in mind, you own your own set of IP addresses.
First you have the local IP of the server, but then you have the local IP of your house.
In the below picture 192.168.1.50 is the local IP of the server which you can bind to.
You still own 83.55.102.40 but the problem is that it's owned by the Router and not your server. So even if you visit http://whatsmyip.com and that tells you that your IP is 83.55.102.40 that is not the case because it can only see where you're coming from.. and you're accessing your internet from a router.
In order for your friends to access your server (which is bound to 192.168.1.50) you need to forward port 6677 to 192.168.1.50 and this is done in your router.
Assuming you are behind one.
If you're in school there's other dilemmas and routers in the way most likely.

This error will also appear if you try to connect to an exposed port from within a Docker container, when nothing is actively serving the port.
On a host where nothing is listening/bound to that port you'd get a No connection could be made because the target machine actively refused it error instead when making a request to a local URL that is not served, eg: localhost:5000. However, if you start a container that binds to the port, but there is no server running inside of it actually serving the port, any requests to that port on localhost will result in:
[Errno 99] Cannot assign requested address (if called from within the container), or
[Errno 0] Error (if called from outside of the container).
You can reproduce this error and the behaviour described above as follows:
Start a dummy container (note: this will pull the python image if not found locally):
docker run --name serv1 -p 5000:5000 -dit python
Then for [Errno 0] Error enter a Python console on host, while for [Errno 99] Cannot assign requested address access a Python console on the container by calling:
docker exec -it -u 0 serv1 python
And then in either case call:
import urllib.request
urllib.request.urlopen('https://localhost:5000')
I concluded with treating either of these errors as equivalent to No connection could be made because the target machine actively refused it rather than trying to fix their cause - although please advise if that's a bad idea.
I've spent over a day figuring this one out, given that all resources and answers I could find on the [Errno 99] Cannot assign requested address point in the direction of binding to an occupied port, connecting to an invalid IP, sysctl conflicts, docker network issues, TIME_WAIT being incorrect, and many more things. Therefore I wanted to leave this answer here, despite not being a direct answer to the question at hand, given that it can be a common cause for the error described in this question.

Try like this:
server.bind(("0.0.0.0", 6677))

When you bind localhost or 127.0.0.1, it means you can only connect to your service from local.
You cannot bind 10.0.0.1 because it not belong to you, you can only bind ip owned by your computer
You can bind 0.0.0.0 because it means all ip on your computer, so any ip can connect to your service if they can connect to any of your ip

This is not directly answering the question, but is a debugging direction in case above solutions failed.
When you are not on a native environment, let's say you are on a VM or WSL, the inside network might not be transparent to external computer due to NATing. So make sure you can ping the IP from wherever you are trying to bind. If not, then consider switching to the correct environment or consider network bridging.
If you are looking for a WSL2 specific solution, you may try this link:
Bridging WSL2 network adapter with Windows
In Virtual Box you may change Network Adapter -> Attached To: Bridged Adapter.
The other consideration is if you are trying to bind to a port <1023 you need admin privilege.

This was what I need on a remote VM:
jupyter notebook --ip=0.0.0.0 --port=8888
Copied from here

python Socket server with real ip address

I am playing with my python server, but I'm through with using localhost and I want to go over the internet. My code thus-far is:
import socket
import threading
import socketserver
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
cur_thread = threading.current_thread()
response = "{}: {}".format(cur_thread.name, data)
self.request.sendall(b'worked')
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print("Received: {}".format(response))
finally:
sock.close()
if __name__ == "__main__":
# Port 0 means to select an arbitrary unused port
HOST, PORT = "0.0.0.0", 9001
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
# Start a thread with the server -- that thread will then start one
# more thread for each request
server_thread = threading.Thread(target=server.serve_forever)
# Exit the server thread when the main thread terminates
server_thread.daemon = True
server_thread.start()
print("Server loop running in thread:", server_thread.name)
ip = '12.34.56.789' #Not my real ip address This is just to hide my ip
print(ip, PORT)
client(ip, PORT, b'Hello World 1')
#client(ip, port, b'Hello World 2')
#client(ip, port, b'Hello World 3')
server.shutdown()
When I run this i get the error:
Server loop running in thread: Thread-1
12.34.56.789 9001
Traceback (most recent call last):
File "C:/Python32/serverTesty.py", line 43, in <module>
client(ip, PORT, b'Hello World 1')
File "C:/Python32/serverTesty.py", line 18, in client
sock.connect((ip, port))
socket.error: [Errno 10061] No connection could be made because the target machine actively refused it
I know the port works because when I use canyouseeme.org on port 9001 when my program is running it says its active and working. So I think I just have my connection wrong somewhere.

ip = '12.34.56.789' #Not my real ip address, its the one i got from whatismyip.org
The first problem is that '12.34.56.789' isn't a valid IP address at all. Each component has to fit in 8 bits (0-255); 789 is impossible. But I assume that isn't the actual code you're running, because the output shows 12.45.29.122.
The second problem is that you're using an address that isn't your real address.
Your machine presumably has an internal IP address, that can only be accessed from your LAN. Then, your router has an external IP address. The router uses a technique called Network Address Translation to let each machine on your LAN pretend that external address belongs to them, when they're acting as clients (which is why whatismyip.org shows you that address). But that doesn't work when they're acting as servers.
If you think about it, there's really no way it could work. If you make an outbound connection, and someone replies, the router knows that the reply should go to your machine. But if someone just comes along and talks to the router out of the blue, how could it know which machine to send the connection to?
If you're trying to connect from inside the same LAN, there's a very easy solution: use the server's real internal address, not the router's external address.
If you need to connect from outside, you can't, without some extra work. There are four ways around this:
Give your machine a real publicly-addressable IP address (e.g., by putting it on the router's DMZ). This is generally not even an option for home users, and it's a bad option for people who don't know what they're doing (unless you want your machine to be part of someone's botnet by lunchtime).
Set up static port forwarding in your router's configuration. This is different for each router, but the idea is that you tell it "if someone comes looking for port 9001, always send them to machine 192.168.1.64".
Use UPnP to set up port forwarding dynamically.
Set up a NAT hole punching.
Options 3 and 4 are more complex, and I think option 2 is the one you want, so I won't explain them.
On top of all that:
HOST, PORT = "192.168.1.64", 9001
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
You've told the server explicitly "listen on 192.168.1.64". Even if you put your server machine on the DMZ, so it had addresses 192.168.1.64 and 12.45.29.122, your program is only listening for connections on the first one, so nobody would be able to reach it using the second. If you want to listen on all addresses, use 0.0.0.0.
In the edited version, you're now listening on 0.0.0.0, and connecting to the router's public IP, and you claim to have set up port forwarding on the router, and you're still getting the exact same error.
If that's all correct, there are three obvious things that could be going wrong:
You're not actually port forwarding; something is wrong with the setup.
You're not actually listening on 0.0.0.0:9001.
You've got a firewall blocking the connection.
There are a few tests you can do to narrow things down.
Open two terminals. In one, type nc -kl 9001. In the other, type nc 12.34.56.78 9001. They should connect up, so anything you type into one window appears in the other (maybe only after you hit Return). If that works, the port forwarding is working, and there's no firewall problem, so it's a problem in your code.
If that didn't work, please post exactly what you saw in each window. Then Ctrl-C the second nc, and type nc 192.168.1.64 9001. If that now works, either the port forwarding isn't set up right, unless you have a clever firewall that allows same-host (or same-interface) connections but not remote connections.
If neither one worked, it's probably a firewall problem. (Unless you're wrong about your IP addresses or something.) You can probably find logs somewhere, but without knowing what platform you're on and what firewall you're using it's hard to offer much help. (Also, that's probably a problem for a different site than SO.)
If you're on Windows, or some linux distros, you need to get a copy of nc (netcat) from somewhere; on most linux distros, and Mac, it should be built in. Also, GNU, BSD, and Hobbit nc are slightly different, so if nc -kl 6000 gives you an error, you might have to read the man page or --help. (If I remember right, Hobbit nc requires -l -p6000, BSD requires -l 6000, GNU allows either.)
Or you may want ncat, a re-implementation of netcat that I know can handle the syntax I used above, and has a single-file static executable for Windows.
If you can't get started with nc, at least try changing your code to connect to 192.168.1.64 instead of 12.34.56.78. If that fixes the problem, at least you'll know it's either port forwarding or a firewall that allows same-host/interface connections but not remote.