Develop Reference

How to send a request using socket and check status code?

So I am very new to Python and am now trying to understand how to send a request using TCP in Python. The sample code and document is not very helpful (to me, as I don't understand Java).
The document:
https://www.sharekhan.com/Upload/General/TradeTigerAPIForClient.pdf
I have the following till now
import socket
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server = '192.168.43.211'
port = 800
request = "DataLength = 196|Transcode = 1|LoginId = ***|MemberPassword = sh*|TradingPassword = S77*| IP = 192.618.31.211|Reserved = |"
s.connect((server,port))
s.send(request.encode())
result = s.recv(4096)
If I use this the program shows I am connected and the result is b''
I also tried
request = "|DataLength =108|Transcode = 21|Exchange Code=NC|Reserved=|"
result is b'Hurray you are connected'
How do I use the commands from the document to get data?

According to this, you can send a TCP packet like this:
import socket
server = '192.168.31.211'
port = 80
buffer_size = 4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((server, port))
sock.send(message)
data = sock.recv(buffer_size)
sock.close()
Note I didn't include your message here, nor did I encode your data. However, as was pointed out in the comments, your IP address is incorrectly written and refers to a private IP address, according to the IANA so if you're trying to send out over a public network, this won't work. Additionally, I have noticed a couple of other problems with your code:
Your message looks like you've manually included TCP header information. Be aware that by using the socket library the way you are and the way I have suggested, you are making a TCP request. The header information will therefore be included with your request so you don't need to include it yourself.
What you have here is the client code and you didn't include any server code. Have you written code for your server? If not, you'll need some.
Otherwise, I can't see any problems with your code.

Why include host in python socket request as well?

I see examples where people create the socket object from a host and port but then also send a GET request, and include the Host header inside the http request.
import socket
HOST = 'daring.cwi.nl'
PORT = 80
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
s.sendall("GET / HTTP/1.1\r\n\r\nHost: daring.cwi.nl")
data = s.recv(1024)
print('Received', repr(data))
Why do we need to provide the host once to create socket object and again to send the request?

First, if you are using just a normal tcp connection, you can send anything you want, you can s.sendall("HAHAHA").
Second, In your code, it is simulating a Get http request. To send parameters, you just need to concatenate them after url. /?parameter1=value1&parameter2=value2.
Finally, why we need to include Host again? Assume you are sending this request to a load-balance server, the application server cannot get real Host without you explicitly sending it.

The Host header is required by the HTTP standard (at least version 1.1), i.e. the standard which defines the protocol spoken with a web server. The reason for this is that you can have multiple domains share the same IP address and thus the web server somehow needs to find out which of the domains you want to access. Since the lower layer (i.e. TCP) does not contain this information but only the IP address you need to provide this information at the application layer (i.e. HTTP).
As for using the hostname inside the connect: This is actually not needed and you could also provide the IP address there. All what it does when providing the hostname and not the IP is to lookup the IP address for the host and connect to this.

socket.connect() gives socket.gaierror: [Errno 11004] getaddrinfo failed

I have already checked answers regarding my problem, but I couldn't find what's wrong. I am new to Python and that might be a problem. I have written this simple code to connect to a site, but I get this error:
socket.gaierror: [Errno 11004] getaddrinfo failed
This is my code:
import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('http://www.py4e.com', 80))
mysock.send('GET http://www.py4e.com/code3/mbox-short.txt HTTP/1.0\n\n')
while True:
data = mysock.recv(512)
if(len(data) < 1):
break
print (data)
mysock.close()

import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('www.py4e.com', 80))
mysock.send('GET http://www.py4e.com/code3/mbox-short.txt HTTP/1.0\n\n')
while True:
data = mysock.recv(512)
if(len(data) < 1):
break
print (data)
mysock.close()
Quite simple, don't use http:// in your host declaration on .connect().
http:// is a protocol and www.py4e.com is a host (or A record in a DNS server). The standard socket library doesn't know anything regarding protocols and there for requires only a host and a port number.
If you want automated processes check out urllib.request or #Mego's answer using Requests which handles the connection and HTTP parsing for you.
Also if you're using Python3 which you probably should, you need to send bytes data when doing .send().
There's two ways of converting your string to bytes data:
mysock.send(b'GET http://www.py4e.com/code3/mbox-short.txt HTTP/1.0\n\n')
mysock.send(bytes('GET http://www.py4e.com/code3/mbox-short.txt HTTP/1.0\n\n', 'UTF-8'))
Both does the same thing basically.
Finally, in a GET request you don't request http:// either.
Instead you just send the path to the file you want to retrieve:
mysock.send(b'GET /code3/mbox-short.txt HTTP/1.0\n\n')
The reason is (again) that http:// is a protocol descriptor and not part of the actual protocol data being sent. You also don't need the host declaration in your GET request because the server that you connected to already knows which host you're on - since you're... connected to it.
Instead the server expects you to supply a Host: <hostname>\r\n header if the host is serving multiple virtual hosts.
You might need a few other headers tho to be able to request actual content from certain web-servers.
But this is the basic jist of things.
Continue reading
Here's a good start:
https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Client_request
It shows you what a raw GET request looks like.
An in the future I recommend using your browsers built-in Network Debugger which can show raw headers, raw responses and a whole bunch of other things.

Simple Python Web Server trouble

I'm trying to write a python web server using the socket library. I've been through several sources and can't figure out why the code I've written doesn't work. Others have run very similar code and claim it works. I'm new to python so I might be missing something simple.
The only way it will work now is I send the data variable back to the client. The browser prints the original GET request. When I try to send an HTTP response, the connection times out.
import socket
##Creates several variables, including the host name, the port to use
##the size of a transmission, and how many requests can be handled at once
host = ''
port = 8080
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
while 1:
client, address = s.accept()
data = client.recv(16)
if data:
client.send('HTTP/1.0 200 OK\r\n')
client.send("Content-Type: text/html\r\n\r\n")
client.send('<html><body><h1>Hello World</body></html>')
client.close()
s.close()

You need to consume the input before responding, and you shouldn't close the socket in your while loop:
Replace client.recv(16) with client.recv(size), to consume the request.
Move your last line, s.close() back one indent, so that it is not in your while loop. At the moment you are closing the connection, then trying to accept from it again, so your server will crash after the first request.
Unless you are doing this as an exercise, you should extend SimpleHTTPServer instead of using sockets directly.
Also, adding this line after your create the socket (before bind) fixes any "Address already in use" errors you might be getting.
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Good luck!

python icmp raw socket implementation

i am relatively new to python, so please be considerate...
i'm implementing a server and a client via raw_sockets.
i have the necessary privileges.
now, the server i defined so:
host = socket.gethostbyname(socket.gethostname())
address = (host, 22224)
sockSer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
sockSer.bind(address)
sockSer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
packet, addr = sockSer .recvfrom(4096) # wait for packet from client
Q1) why can't i simply type: hosts = 'localhost'.
if i do so, it doesn't allow me to write the line: sockSer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON). and then the server doesn't receive my client's messages.
only when doing gethostbyname(socket.gethostname()) i get 192.168.1.101
and then it works.
in a different class:
the client socket:
host = socket.gethostbyname(socket.gethostname())
address = (host, 22224)
sockCli = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
Q2) do i also need to type: sockCli.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
or maybe sockCli.connect(address)? seems that it works without the connect command.
for the client socket?
now, the problems arise when i do the following:
1) send a packet from client to server:
header=...
payload='a'
sockCli.sendto(header + payload, address)
2) receive packet in server and send something back to client:
while(true):
data, addr = sockSer.recvfrom(4096)
header2=...
payload2='b'
sockSer.sendto(header2 + payload2, addr)
now, my important question is:
Q3) the server sent only 1 packet to client, with payload 'b'.
what happens is, my client actually receives 2 packets in the while loop:
first packet is what the client itself sent to server, and the other packet is what the client got from the server.
hence my output is 'ab' instead of simply 'b'
why is this happening???
NOTE: i didn't type the entire code, but i think my syntax,parsing,header composition etc.. are correct.
is there an obvious problem in my code?
if necessary i'll upload the entire code.
thanks

I got this too.
my solution is add a judge in the receive code,such as if I send Ping package so I only want ECHO　Reply( type 0 code 0), I write
if type != 0:
continue
and you also can write as
if addr == my_ip:
continue
It seems not has any smooth solution

Q1: I was able to bind to localhost and call IOCTL with both parameters just fine. Assuming your client is also running on the same system, ensure the client is sending to "localhost", otherwise your server will never receive the packets. If your client is on another system, obviously your server will never receive the packets.
Q2: You do not need IOCTL for sending the packet. Just send it via sendto().
Q3: The reason you're seeing two replies is, the kernel is also processing the echo request, in addition to your own user-space code.
Although you can use ICMP for arbitrary message passing, as someone else pointed out this isn't its intended design. You may find that your data portion is truncated out in message replies. For example, when sending echo requests, your reply likely will contain everything you sent; however, a reply that is type 3 code 3 may not include your data, but only the first 8 bytes of the ICMP header.