Socket can't receive any data from the server, when there is a successful response, but with bad requests it can. Also server responds, just the socket can't receive data (checked in WireShark)
import socket
import ssl
HOST, PORT = 'example.com', 443
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssock = ssl.wrap_socket(sock)
ssock.connect((HOST, PORT))
raw_req = [f'GET / HTTP/1.1', 'Host: {HOST}', 'Connection: keep-alive']
req = '\n'.join(raw_req)
ssock.send(req.encode())
msg = ssock.recv(4096).decode()
print(msg)
ssock.close()
First, the HTTP GET expects a sequence of CR LF characters after each header line not just a single '\n' character and an extra CR LF after the last header line. Also, the join() adds the separator between each pair but not at the end so must append data with CR LF + CR LF to be a valid HTTP request.
Second, the 'Host: {HOST}' must be a f-string otherwise the "{HOST}" is not replaced.
import socket
import ssl
HOST, PORT = 'stackoverflow.com', 443
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
ssock = ssl.wrap_socket(sock)
ssock.connect((HOST, PORT))
raw_req = [f'GET / HTTP/1.1', f'Host: {HOST}', 'Connection: keep-alive']
req = ('\r\n'.join(raw_req) + "\r\n\r\n").encode()
print("Request:", req)
ssock.send(req)
msg = ssock.recv(4096).decode()
print("\nResponse:")
print(msg)
Output:
Request: b'GET / HTTP/1.1\r\nHost: stackoverflow.com\r\nConnection: keep-alive\r\n\r\n'
Response:
HTTP/1.1 200 OK
Connection: keep-alive
content-type: text/html; charset=utf-8
...
If the HTTP response is larger than 4096 bytes then you would need to call ssock.recv() in a loop until it returns a byte array of length 0.
Related
Right now in my code, the index.html file is read on the server and then sent from the server to the client connection. I need to make it so that I have a separate client application that formulates the HTTP request and sends it to the server to send HTTP response. Just not quite sure how to do that. I have an echo-client.py file but it doesn't work at the moment. The server application works and displays the index.html at localhost:14000 with the server.py code below:
server.py
"""
Implements a simple HTTP/1.0 Server
"""
import socket
# Define socket host and port
SERVER_HOST = '127.0.0.1'
SERVER_PORT = 14000
# Create socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((SERVER_HOST, SERVER_PORT))
server_socket.listen(1)
print('Listening on port %s ...' % SERVER_PORT)
while True:
# Wait for client connections
client_connection, client_address = server_socket.accept()
# Get the content of index.html
# HTTP Request
fin = open('index.html')
content = fin.read()
fin.close()
# Send HTTP response
response = 'HTTP/1.0 200 OK\n\n' + content
client_connection.sendall(response.encode())
# client_connection.close()
# Close socket
server_socket.close()
I've tried configuring the client.py application like this but it says:
File "/Users/jamesmeegan/Desktop/COSC 350 Data Comm/Problem3_HW3/echo-client.py", line 33, in <module>
s.sendall((content))
TypeError: a bytes-like object is required, not 'str'
and I'm not sure how I would even receive it on the server end.
client.py
# echo-client.py
import socket
HOST = "127.0.0.1" # The server's hostname or IP address
PORT = 14000 # The port used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
fin = open('index.html')
content = fin.read()
fin.close()
s.sendall((content))
data = s.recv(1024)
print(f"Received {data!r}")
A server listens packet and we send http GET request packet to this listener.If we use auth header with username/pass server does not accept connection and it fails.Is there any way to parse this auth header info (username/pass) on listener ? Because we want to perform authentication based on user/pass comparison
NOTE : Without auth header in GET packet http listener accept connection and it works fine
HTTP PACKET LISTENER
import socket
serverSocket = socket(AF_INET, SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
while True:
print('Ready to serve...')
try :
connectionSocket, addr = serverSocket.accept()
except :
print (f"Socket error occured for 127.0.0.1 {serverPort} ")
HTTP CLIENT
import requests
from requests.auth import HTTPBasicAuth
r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
Thank you for your helps !
Here is a working example of what you need.
tl;dr: as pointed out in comments, with sockets you are working at the transport level. The HTTP Basic Auth lies at a higher level in the TCP/IP (or OSI) stack. If you do not want to embrace the HTTP protocol (do you?), you need to process requests and headers manually, mimicking the HTTP protocol. Indeed, python requests manages full-fledged HTTP requests.
I slightly modified your code to parse http headers and to manage a HTTP-like auth. There you go (comments and explanation in the code):
import socket, base64
from http.server import BaseHTTPRequestHandler
from io import BytesIO
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
# Auth section
user = 'user'
password = 'pass'
# The token you want the client to provide (string)
server_token = base64.b64encode(bytes(f'{user}:{password}','utf-8')).decode('utf-8')
# Use this simple class to parse you HTTP headers
# Read more here: https://stackoverflow.com/a/5955949/4820341
class HTTPRequest(BaseHTTPRequestHandler):
def __init__(self, request_text):
self.rfile = BytesIO(request_text)
self.raw_requestline = self.rfile.readline()
self.error_code = self.error_message = None
self.parse_request()
def send_error(self, code, message):
self.error_code = code
self.error_message = message
while True:
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
data = connectionSocket.recv(1024)
# Those are your data coming from the client
print(data.decode('utf-8'))
# parse your headers
http_headers = HTTPRequest(data)
try:
# get the incoming auth token
client_token = http_headers.headers['Authorization'].strip('Basic ')
if server_token != client_token:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'Wrong credetials', 'utf-8'))
else:
# process the request and do your stuff here
connectionSocket.sendall(bytes("HTTP/1.1 200 OK\n\n" + 'Ok, all is fine here', 'utf-8'))
except AttributeError:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'No credentials provided', 'utf-8'))
finally:
connectionSocket.close()
Here is how a requests.get with auth looks like server side:
Ready to serve...
GET / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: python-requests/2.26.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Authorization: Basic dXNlcjpwYXNz
And now, let's see it in action:
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
>>> r.status_code
200
>>> r.text
'Ok, all is fine here'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'wrongpass'))
>>> r.status_code
401
>>> r.text
'wrong credentials'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080')
>>> r.status_code
401
>>> r.text
'No credentials provided'
As I wrote on title, I have already successfully connected the server and client.
But the client can't display the HTML file.
I checked file path and send function. But can't find any fault.
When running the code, the code runs normally until connectionSocket.close().
But browser can't display the HTML file, just blank.
So, I checked the details and I found that connectionSocket.send(outputdata[i].encode()) send values, 1 or 3.
I don't know the reason but I'm sure that that is the cause.
Please give me your insight.
from socket import *
serverSocket = socket(AF_INET, SOCK_STREAM)
# Prepare a sever socket
TCPPort = 8000
BufferSize = 1024
serverSocket.bind((host, TCPPort))
serverSocket.listen(1)
while True:
# Establish the connection
print('Ready to serve...')
(connectionSocket,addr) = serverSocket.accept()
print('connectionSocket is:',connectionSocket)
try:
message = connectionSocket.recv(BufferSize)
print('message is:',message)
#filename = message.split()[1]
#print('filename is:', filename)
f = open('\HTML.html','r',encoding='UTF-8')
outputdata = f.read()
# Send one HTTP header line into socket
connectionSocket.send('HTTP/1.1 200 OK\r\n'.encode('UTF-8'))
# Send the content of the requested file to the client
for i in range(0,len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.close()
except IOError:
connectionSocket.send('HTTP/1.1 404 Not Found'.encode('UTF-8'))
connectionSocket.send("<html><head></head><body><h1>404 Not Found</h1></body></html> ".encode('UTF-8'))
# Close client socket
connectionSocket.close()
serverSocket.close()
You need to make your server to respond by the HTTP protocol. In HTTP there are 2 newlines between headers and body and you need to send both together:
from socket import *
serverSocket = socket(AF_INET, SOCK_STREAM)
# Prepare a sever socket
TCPPort = 8000
BufferSize = 1024
serverSocket.bind(('127.0.0.1', TCPPort))
serverSocket.listen(1)
while True:
# Establish the connection
print('Ready to serve...')
(connectionSocket, addr) = serverSocket.accept()
print('connectionSocket is:', connectionSocket)
try:
message = connectionSocket.recv(BufferSize)
print('message is:', message)
#filename = message.split()[1]
#print('filename is:', filename)
#f = open('\HTML.html','r',encoding='UTF-8')
outputdata = "<html><body>foo</body></html>"
# Send one HTTP header line into socket
response = 'HTTP/1.1 200 OK\nConnection: close\n\n' + outputdata
connectionSocket.send(response.decode())
# Send the content of the requested file to the client
connectionSocket.close()
except IOError:
connectionSocket.send('HTTP/1.1 404 Not Found'.encode('UTF-8'))
connectionSocket.send(
"<html><head></head><body><h1>404 Not Found</h1></body></html> ".
encode('UTF-8')
)
# Close client socket
connectionSocket.close()
serverSocket.close()
Test, using: curl -X GET http://localhost:8000
Out:
<html><body>foo</body></html>
I am trying to write a simple HTTP client program using raw sockets in Python 3. However, the server does not return a response despite having been sent a simple HTTP request. My question is why the server doesn't return a response.
Here is my code:
from socket import *
BUF_LEN = 8192 * 100000
info = getaddrinfo('google.com', 80, AF_INET)
addr = info[-1][-1]
print(addr)
client = socket(AF_INET, SOCK_STREAM)
client.connect(addr)
client.send(b"GET /index.html HTTP1.1\r\nHost: www.google.com\r\n")
print(client.recv(BUF_LEN).decode("utf-8")) # print nothing
You've missed a blank line at the end and mis-specified the HTTP version without a slash:
>>> client.send(b"GET /index.html HTTP1.1\r\nHost: www.google.com\r\n")
Should be:
>>> client.send(b"GET /index.html HTTP/1.1\r\nHost: www.google.com\r\n\r\n")
50
>>> client.recv(BUF_LEN).decode("utf-8")
u'HTTP/1.1 302 Found\r\nCache-Control: private\r\nContent-Type: text/html; charset=UTF-8\r\nLocation: http://www.google.co.uk/index.html?gfe_rd=cr&ei=fIR7WJ7QGejv8AeZzbWgCw\r\nContent-Length: 271\r\nDate: Sun, 15 Jan 2017 14:17:32 GMT\r\n\r\n<HTML><HEAD><meta http-equiv....
The blank line tells the server its the end of the headers, and since this is a GET request there's no payload and so it can then return the content.
Without the / in the HTTP/1.1 spec Google's servers will return an Error: 400 Bad Request response.
Are there any straightforward ways to make a HTTP request and get at the raw, unparsed response (specifically the headers)?
Using the socket module directly:
import socket
CRLF = "\r\n"
request = [
"GET / HTTP/1.1",
"Host: www.example.com",
"Connection: Close",
"",
"",
]
# Connect to the server
s = socket.socket()
s.connect(('www.example.com', 80))
# Send an HTTP request
s.send(CRLF.join(request))
# Get the response (in several parts, if necessary)
response = ''
buffer = s.recv(4096)
while buffer:
response += buffer
buffer = s.recv(4096)
# HTTP headers will be separated from the body by an empty line
header_data, _, body = response.partition(CRLF + CRLF)
print header_data
HTTP/1.0 302 Found
Location: http://www.iana.org/domains/example/
Server: BigIP
Connection: Keep-Alive
Content-Length: 0