I see examples where people create the socket object from a host and port but then also send a GET request, and include the Host header inside the http request.
import socket
HOST = 'daring.cwi.nl'
PORT = 80
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
s.sendall("GET / HTTP/1.1\r\n\r\nHost: daring.cwi.nl")
data = s.recv(1024)
print('Received', repr(data))
Why do we need to provide the host once to create socket object and again to send the request?
First, if you are using just a normal tcp connection, you can send anything you want, you can s.sendall("HAHAHA").
Second, In your code, it is simulating a Get http request. To send parameters, you just need to concatenate them after url. /?parameter1=value1¶meter2=value2.
Finally, why we need to include Host again? Assume you are sending this request to a load-balance server, the application server cannot get real Host without you explicitly sending it.
The Host header is required by the HTTP standard (at least version 1.1), i.e. the standard which defines the protocol spoken with a web server. The reason for this is that you can have multiple domains share the same IP address and thus the web server somehow needs to find out which of the domains you want to access. Since the lower layer (i.e. TCP) does not contain this information but only the IP address you need to provide this information at the application layer (i.e. HTTP).
As for using the hostname inside the connect: This is actually not needed and you could also provide the IP address there. All what it does when providing the hostname and not the IP is to lookup the IP address for the host and connect to this.
Related
I'm relatively new to Python, so apologies for what me a simple question, I just cannot find the solution. First off, I am not looking for the client's hostname. My situation is that I have a simple socket server (basically this https://docs.python.org/3/library/socketserver.html#socketserver-tcpserver-example) which clients connect to. The exact server code is;
import socketserver
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The request handler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "0.0.0.0", 8080
# Create the server, binding to localhost on port 9999
with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server:
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
The clients connect successfully and are sending data which the server is receiving. My issue is that need to know the hostname that the client used to connect. The architecture will be like;
client1 will connect to client1.mydomain.net:8080
client2 will connect to client2.mydomain.net:8080
client3 will connect to client3.mydomain.net:8080
The DNS entry for client1.mydomain.net, client2.mydomain.net and client3.mydomain.net will all map to 123.123.123.123 so behind the scenes there is only one server.
The 3 clients will connect to their respective server and send their data. I have no control over the payload and I cannot augment it with a string or parm like "client=1".
So my question is, is there a way in python sockets (on the server) to know the hostname that a client connected to, so for example I know when I'm process data from client1.
Thanks!
Nothing at the TCP level reveals which hostname the client is connected to. This means there is no way for a generic TCP server to get this information.
Various protocols on top of TCP contains such information though. For example HTTP has a Host header so that different domains with different contents can be server on the same IP and port. TLS has the server_name extension in the TLS handshake so that the expected certificate can be given matching the hostname used by the client.
Thus, if you need this information you need to define your application protocol so that the client will include this information.
I have a Tornado server that listens on both an address/port, and on a socket. I create the server roughly like so (stripped down heavily):
from tornado import httpserver
from tornado.netutil import bind_unix_socket
server = httpserver.HTTPserver(
request_callback=some_callback,
io_loop=some_loop,
)
unix_socket = bind_unix_socket("mysock.sock")
server.add_socket(unix_socket)
server.listen(address="some_host", port=1234)
I'd like to be able to differentiate when a request is received via the socket, ie something like:
curl -XGET --unix-socket mysock.sock http:/ping
As opposed to:
curl http://some_address:1234/ping
I looked at the HTTPServerRequest that Tornado uses when a request is received, but I'm not sure what the best way is to tell the difference between the two. I can look at remote_ip to see if it comes from localhost, but I don't think that's ideal.
localhost is an internet-domain interface with a well-known IP address. It's not the same as a unix-domain socket, which has no IP address.
A brief look at the source suggests that the remote_ip attribute will contain '0.0.0.0' for a connection received on the unix socket.
(The remote_ip would presumably be '127.0.0.1' for a connection received via a localhost connection.)
I'm trying to code a small web server in python to catch an HTTP post.
But I'm having an issue with the socket.gethostname part of it
here is my sample code
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serversocket.bind((socket.gethostname(), 8089))
serversocket.listen(1)
while True:
connection, address = serversocket.accept()
buf = connection.recv(164)
print buf
If i change
serversocket.bind((socket.gethostname(), 8089))
to
serversocket.bind(("localhost", 8089))
Everything is fine I can telnet into it, but I need to be able to connect from another web server on the internet so I need to use socket.gethostname but this block my telnet.
You are using a clever trick to get your servers "real" address when potentially several network interfaces are open. serversocket.bind((socket.gethostname(), 8089)) can be broken down to
hostname = socket.gethostname()
dns_resolved_addr = socket.gethostbyname(hostname)
serversocket.bind((dns_resolved_addr, 8089))
You get your local hostname and then ask DNS what it thinks your IP address is, and bind to that. That's the IP address external connections will use so you should use it too.
But it doesn't always work. DNS may not know what your server name is or your server may have a different name in DNS. One example is my home network where I don't have a DNS server and the DHCP addresses handed out by my modem don't appear in a name server anywhere. A similar problem exists if your corporate DHCP doesn't register your hostname with its local DNS.
On my machine, when I go through the steps I get
>>> socket.gethostbyname(socket.gethostname())
'127.0.1.1'
Notice it was 127.0.1.1 ... I think that's some weird Ubuntu thing to keep its routing tables from getting confused. Anyway, one solution is to try to resolve the address and if you don't like it, fall back to a default.
>>> my_ip = socket.gethostbyname(socket.gethostname())
>>> if my_ip.startswith('127.0.'):
... my_ip = '0.0.0.0'
...
>>> my_ip
'0.0.0.0'
i am relatively new to python, so please be considerate...
i'm implementing a server and a client via raw_sockets.
i have the necessary privileges.
now, the server i defined so:
host = socket.gethostbyname(socket.gethostname())
address = (host, 22224)
sockSer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
sockSer.bind(address)
sockSer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
packet, addr = sockSer .recvfrom(4096) # wait for packet from client
Q1) why can't i simply type: hosts = 'localhost'.
if i do so, it doesn't allow me to write the line: sockSer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON). and then the server doesn't receive my client's messages.
only when doing gethostbyname(socket.gethostname()) i get 192.168.1.101
and then it works.
in a different class:
the client socket:
host = socket.gethostbyname(socket.gethostname())
address = (host, 22224)
sockCli = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
Q2) do i also need to type: sockCli.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
or maybe sockCli.connect(address)? seems that it works without the connect command.
for the client socket?
now, the problems arise when i do the following:
1) send a packet from client to server:
header=...
payload='a'
sockCli.sendto(header + payload, address)
2) receive packet in server and send something back to client:
while(true):
data, addr = sockSer.recvfrom(4096)
header2=...
payload2='b'
sockSer.sendto(header2 + payload2, addr)
now, my important question is:
Q3) the server sent only 1 packet to client, with payload 'b'.
what happens is, my client actually receives 2 packets in the while loop:
first packet is what the client itself sent to server, and the other packet is what the client got from the server.
hence my output is 'ab' instead of simply 'b'
why is this happening???
NOTE: i didn't type the entire code, but i think my syntax,parsing,header composition etc.. are correct.
is there an obvious problem in my code?
if necessary i'll upload the entire code.
thanks
I got this too.
my solution is add a judge in the receive code,such as if I send Ping package so I only want ECHO　Reply( type 0 code 0), I write
if type != 0:
continue
and you also can write as
if addr == my_ip:
continue
It seems not has any smooth solution
Q1: I was able to bind to localhost and call IOCTL with both parameters just fine. Assuming your client is also running on the same system, ensure the client is sending to "localhost", otherwise your server will never receive the packets. If your client is on another system, obviously your server will never receive the packets.
Q2: You do not need IOCTL for sending the packet. Just send it via sendto().
Q3: The reason you're seeing two replies is, the kernel is also processing the echo request, in addition to your own user-space code.
Although you can use ICMP for arbitrary message passing, as someone else pointed out this isn't its intended design. You may find that your data portion is truncated out in message replies. For example, when sending echo requests, your reply likely will contain everything you sent; however, a reply that is type 3 code 3 may not include your data, but only the first 8 bytes of the ICMP header.
i already have a post which is quite similiar, but i am getting more and more frustrated because it seems nothing is wrong with my network setup. Other software can be seen from the outside (netcat listen servers etc.) but not my scripts.. How can this be??
Note: It works on LAN but not over the internet.
Server:
import socket
host = ''
port = 80001
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(1)
print 'Listening..'
conn, addr = s.accept()
print 'is up and running.'
print addr, 'connected.'
s.close()
print 'shut down.'
Client:
import socket
host = '80.xxx.xxx.xxx'
port = 80001
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.close()
Somebody please help me.
Any help is greatly appreciated.
Jake
Edited again to add:
I think you may be missing some basics on socket communication. In order for sockets to work, you need to ensure that the sockets on both your client and server will meet. With your latest revision, your server is now bound to port 63001, but on the local loopback adapter: 127.0.0.1
Computers have multiple network adapters, at least 2: one is the local loopback, which allows you to make network connections to the same machine in a fast, performant manner (for testing, ipc etc), and a network adapter that lets you connect to an actual network. Many computers may have many more adapters (virtual adapters for vlans, wireless vs wired adapters etc), but they will have at least 2.
So in your server application, you need to instruct it to bind the socket to the proper network adapter.
host = ''
port = 63001
bind(host,port)
What this does in python is binds the socket to the loopback adapter (or 127.0.0.1/localhost).
In your client application you have:
host = '80.xxx.xxx.xxx'
port = 63001
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
Now what your client attempts to do is to connect to a socket to port 63001 on 80.xxx.xxx.xxx (which is your wireless internet adapter).
Since your server is listening on your loopback adapter, and your client is trying to connect on your wireless adapter, it's failing, because the two ends don't meet.
So you have two solutions here:
Change the client to connect to localhost by host = 127.0.0.1
Change the server to bind to your internet adapter by changing host = 80.xxx.xxx.xxx
Now the first solution, using localhost, will only work when your server and client are on the same machine. Localhost always points back to itself (hence loopback), no matter what machine you try. So if/when you decide to take your client/server to the internet, you will have to bind to a network adapter that is on the internet.
Edited to add:**
Okay with your latest revision it still won't work because 65535 is the largest post available.
Answer below was to the original revision of the question.
In your code posted, you're listening (bound) on port 63001, but your client application is trying to connect to port 80. Thats why your client can't talk to your server. Your client needs to connect using port 63001 not port 80.
Also, unless you're running an HTTP server (or your python server will handle HTTP requests), you really shouldn't bind to port 80.
In your client code change:
import socket
host = '80.xxx.xxx.xxx'
port = 63001
And in your Server Code:
import socket
host = ''
port = 63001
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((socket.gethostbyname(socket.gethostname()), port ))
In your server script you have port = 80, but you don't ever use it. It looks like the server is listening on 63001. And the client is connecting to 80.
If you're going to use 80, make sure you don't have an http server trying to use the port at the same time as well.