I am trying to issue a simple HTTP GET request to a website through Python socket library. The sample website I used here is https://azlyrics.com/lyrics/charlieputh/attention.html. My code is:
from socket import *
serverName = 'azlyrics.com';
serverPort = 80;
clientSocket = socket(AF_INET, SOCK_STREAM);
print(clientSocket.connect((serverName, serverPort)));
message = '''GET /lyrics/charlieputh/attention.html HTTP/1.1
Host: www.azlyrics.com
Connection: keep-alive
''';
print(message);
clientSocket.send(message.encode());
modifiedMessage = clientSocket.recv(2048).decode();
print(modifiedMessage);
clientSocket.close();
But I get no response message in return. Also the object I get in return of connect() is None. I have tried the same URL with Python Request Library and it works fine. What am I doing wrong here?
HTTP requests body should end with \r\n (CR-LF),
please try this:
from socket import *
serverName = 'azlyrics.com';
serverPort = 80;
clientSocket = socket(AF_INET, SOCK_STREAM);
print(clientSocket.connect((serverName, serverPort)));
message = '''GET /lyrics/charlieputh/attention.html HTTP/1.1
Host: www.azlyrics.com
Connection: keep-alive
''';
print(message);
clientSocket.send(message.encode());
modifiedMessage = clientSocket.recv(2048).decode();
print(modifiedMessage);
clientSocket.close();
Related
Socket can't receive any data from the server, when there is a successful response, but with bad requests it can. Also server responds, just the socket can't receive data (checked in WireShark)
import socket
import ssl
HOST, PORT = 'example.com', 443
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssock = ssl.wrap_socket(sock)
ssock.connect((HOST, PORT))
raw_req = [f'GET / HTTP/1.1', 'Host: {HOST}', 'Connection: keep-alive']
req = '\n'.join(raw_req)
ssock.send(req.encode())
msg = ssock.recv(4096).decode()
print(msg)
ssock.close()
First, the HTTP GET expects a sequence of CR LF characters after each header line not just a single '\n' character and an extra CR LF after the last header line. Also, the join() adds the separator between each pair but not at the end so must append data with CR LF + CR LF to be a valid HTTP request.
Second, the 'Host: {HOST}' must be a f-string otherwise the "{HOST}" is not replaced.
import socket
import ssl
HOST, PORT = 'stackoverflow.com', 443
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
ssock = ssl.wrap_socket(sock)
ssock.connect((HOST, PORT))
raw_req = [f'GET / HTTP/1.1', f'Host: {HOST}', 'Connection: keep-alive']
req = ('\r\n'.join(raw_req) + "\r\n\r\n").encode()
print("Request:", req)
ssock.send(req)
msg = ssock.recv(4096).decode()
print("\nResponse:")
print(msg)
Output:
Request: b'GET / HTTP/1.1\r\nHost: stackoverflow.com\r\nConnection: keep-alive\r\n\r\n'
Response:
HTTP/1.1 200 OK
Connection: keep-alive
content-type: text/html; charset=utf-8
...
If the HTTP response is larger than 4096 bytes then you would need to call ssock.recv() in a loop until it returns a byte array of length 0.
A server listens packet and we send http GET request packet to this listener.If we use auth header with username/pass server does not accept connection and it fails.Is there any way to parse this auth header info (username/pass) on listener ? Because we want to perform authentication based on user/pass comparison
NOTE : Without auth header in GET packet http listener accept connection and it works fine
HTTP PACKET LISTENER
import socket
serverSocket = socket(AF_INET, SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
while True:
print('Ready to serve...')
try :
connectionSocket, addr = serverSocket.accept()
except :
print (f"Socket error occured for 127.0.0.1 {serverPort} ")
HTTP CLIENT
import requests
from requests.auth import HTTPBasicAuth
r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
Thank you for your helps !
Here is a working example of what you need.
tl;dr: as pointed out in comments, with sockets you are working at the transport level. The HTTP Basic Auth lies at a higher level in the TCP/IP (or OSI) stack. If you do not want to embrace the HTTP protocol (do you?), you need to process requests and headers manually, mimicking the HTTP protocol. Indeed, python requests manages full-fledged HTTP requests.
I slightly modified your code to parse http headers and to manage a HTTP-like auth. There you go (comments and explanation in the code):
import socket, base64
from http.server import BaseHTTPRequestHandler
from io import BytesIO
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
# Auth section
user = 'user'
password = 'pass'
# The token you want the client to provide (string)
server_token = base64.b64encode(bytes(f'{user}:{password}','utf-8')).decode('utf-8')
# Use this simple class to parse you HTTP headers
# Read more here: https://stackoverflow.com/a/5955949/4820341
class HTTPRequest(BaseHTTPRequestHandler):
def __init__(self, request_text):
self.rfile = BytesIO(request_text)
self.raw_requestline = self.rfile.readline()
self.error_code = self.error_message = None
self.parse_request()
def send_error(self, code, message):
self.error_code = code
self.error_message = message
while True:
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
data = connectionSocket.recv(1024)
# Those are your data coming from the client
print(data.decode('utf-8'))
# parse your headers
http_headers = HTTPRequest(data)
try:
# get the incoming auth token
client_token = http_headers.headers['Authorization'].strip('Basic ')
if server_token != client_token:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'Wrong credetials', 'utf-8'))
else:
# process the request and do your stuff here
connectionSocket.sendall(bytes("HTTP/1.1 200 OK\n\n" + 'Ok, all is fine here', 'utf-8'))
except AttributeError:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'No credentials provided', 'utf-8'))
finally:
connectionSocket.close()
Here is how a requests.get with auth looks like server side:
Ready to serve...
GET / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: python-requests/2.26.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Authorization: Basic dXNlcjpwYXNz
And now, let's see it in action:
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
>>> r.status_code
200
>>> r.text
'Ok, all is fine here'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'wrongpass'))
>>> r.status_code
401
>>> r.text
'wrong credentials'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080')
>>> r.status_code
401
>>> r.text
'No credentials provided'
I'm a beginner compsci student and I'm trying to code a simple server in python that takes a .HTML page stored in the same directory and sends it to a client on the same network using a TCP connection.
This is my code:
from socket import *
serverPort = 8000
serverSocket = socket(AF_INET, SOCK_STREAM)
# Prepare a sever socket
serverSocket.bind(('', serverPort)) # binds socket to port 8000
serverSocket.listen(1) # waiting for client to initiate connection
while True:
# Establish the connection
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
try:
message = connectionSocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:].decode())
outputdata = f.read()
# Send one HTTP header line into socket
http_response = 'HTTP/1.1 200 OK\n'
connectionSocket.send(http_response.encode())
# Send the content of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
connectionSocket.close()
except IOError:
connectionSocket.send("\r\n".encode())
# DO LATER
serverSocket.close()
sys.exit()
And this is my simple html page:
<!DOCTYPE html>
<html>
<body>
<h1>My First Web Page</h1>
<p>You have successfully accessed the Web Server</p>
</body>
</html>
So far whenever I run my server and direct my browser to it, I only get the following served to me:
<p>You have successfully accessed the Web Server</p>
Along with the body and html tags after this. Checking the page source there's no header.
I ran Wireshark while trying to access my server and indeed it seems like I'm only sending through "You have successfully accessed the Web server" and onwards. This is despite the fact a print function shows I am definitely sending all the data in the file through the TCP connection.
Does anyone know what the issue is?
After sending the protocol answer and headers, the actual response comes after two \r\n sequences.
Use this fixed code:
from socket import *
serverPort = 8000
serverSocket = socket(AF_INET, SOCK_STREAM)
# Prepare a sever socket
serverSocket.bind(('', serverPort)) # binds socket to port 8000
serverSocket.listen(1) # waiting for client to initiate connection
while True:
# Establish the connection
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
try:
message = connectionSocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:].decode())
outputdata = f.read()
# Send one HTTP header line into socket
http_response = 'HTTP/1.1 200 OK\n'
connectionSocket.send(http_response.encode())
connectionSocket.send("\r\n".encode())
connectionSocket.send("\r\n".encode())
# Send the content of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.close()
except IOError:
# DO LATER
serverSocket.close()
sys.exit()
I would use the http.server library
import http.server
import socketserver
PORT = 8080
Handler = http.server.SimpleHTTPRequestHandler
with socketserver.TCPServer(("", PORT), Handler) as httpd:
httpd.serve_forever()
source: https://www.afternerd.com/blog/python-http-server/
I am trying to write a simple HTTP client program using raw sockets in Python 3. However, the server does not return a response despite having been sent a simple HTTP request. My question is why the server doesn't return a response.
Here is my code:
from socket import *
BUF_LEN = 8192 * 100000
info = getaddrinfo('google.com', 80, AF_INET)
addr = info[-1][-1]
print(addr)
client = socket(AF_INET, SOCK_STREAM)
client.connect(addr)
client.send(b"GET /index.html HTTP1.1\r\nHost: www.google.com\r\n")
print(client.recv(BUF_LEN).decode("utf-8")) # print nothing
You've missed a blank line at the end and mis-specified the HTTP version without a slash:
>>> client.send(b"GET /index.html HTTP1.1\r\nHost: www.google.com\r\n")
Should be:
>>> client.send(b"GET /index.html HTTP/1.1\r\nHost: www.google.com\r\n\r\n")
50
>>> client.recv(BUF_LEN).decode("utf-8")
u'HTTP/1.1 302 Found\r\nCache-Control: private\r\nContent-Type: text/html; charset=UTF-8\r\nLocation: http://www.google.co.uk/index.html?gfe_rd=cr&ei=fIR7WJ7QGejv8AeZzbWgCw\r\nContent-Length: 271\r\nDate: Sun, 15 Jan 2017 14:17:32 GMT\r\n\r\n<HTML><HEAD><meta http-equiv....
The blank line tells the server its the end of the headers, and since this is a GET request there's no payload and so it can then return the content.
Without the / in the HTTP/1.1 spec Google's servers will return an Error: 400 Bad Request response.
I am using socket to visit a localhost website. Get the following error:
HTTP/1.1 408 Request Timeout
Content-Length: 0
Content-Type: text/plain
The socket code as below:
import socket
mysock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
mysock.connect(('0.0.0.0',8080))
mysock.send('GET http://localhost:8080/hello HTTP/1.0\r\n')
while True:
data = mysock.recv(512)
if (len(data)<1):
break
print data
mysock.close()
HTTP request header should end with two newlines. If you send only one blank line, the server will wait until timeout.
mysock.send('GET http://localhost:8000/hello HTTP/1.0\r\n\r\n')