URL = "MY HTTP REQUEST URL"
XML = "<port>0</port>"
parameter = urllib.urlencode({'XML': XML})
response = urllib.urlopen(URL, parameter)
print response.read()
IOError: ('http protocol error', 0, 'got a bad status line', None)
I am trying to send XML to a server and get back XML. Is there any way to fix / ignore this exception?
I know that the status line is empty which is raising this error.
Try to have a look what your server actually returns! It probably isn't a valid HTTP response. You could use something like this to send a raw http request to the server:
from socket import socket
host = 'localhost'
port = 80
path = "/your/url"
xmlmessage = "<port>0</port>"
s = socket()
s.connect((host, port))
s.send("POST %s HTTP/1.1\r\n" % path)
s.send("Host: %s\r\n" % host)
s.send("Content-Type: text/xml\r\n")
s.send("Content-Length: %d\r\n\r\n" % len(xmlmessage))
s.send(xmlmessage)
for line in s.makefile():
print line,
s.close()
The response should look something like:
HTTP/1.1 200 OK
<response headers>
<response body>
Related
A server listens packet and we send http GET request packet to this listener.If we use auth header with username/pass server does not accept connection and it fails.Is there any way to parse this auth header info (username/pass) on listener ? Because we want to perform authentication based on user/pass comparison
NOTE : Without auth header in GET packet http listener accept connection and it works fine
HTTP PACKET LISTENER
import socket
serverSocket = socket(AF_INET, SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
while True:
print('Ready to serve...')
try :
connectionSocket, addr = serverSocket.accept()
except :
print (f"Socket error occured for 127.0.0.1 {serverPort} ")
HTTP CLIENT
import requests
from requests.auth import HTTPBasicAuth
r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
Thank you for your helps !
Here is a working example of what you need.
tl;dr: as pointed out in comments, with sockets you are working at the transport level. The HTTP Basic Auth lies at a higher level in the TCP/IP (or OSI) stack. If you do not want to embrace the HTTP protocol (do you?), you need to process requests and headers manually, mimicking the HTTP protocol. Indeed, python requests manages full-fledged HTTP requests.
I slightly modified your code to parse http headers and to manage a HTTP-like auth. There you go (comments and explanation in the code):
import socket, base64
from http.server import BaseHTTPRequestHandler
from io import BytesIO
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverPort = 8080
serverSocket.bind(("127.0.0.1", serverPort))
serverSocket.listen(1)
# Auth section
user = 'user'
password = 'pass'
# The token you want the client to provide (string)
server_token = base64.b64encode(bytes(f'{user}:{password}','utf-8')).decode('utf-8')
# Use this simple class to parse you HTTP headers
# Read more here: https://stackoverflow.com/a/5955949/4820341
class HTTPRequest(BaseHTTPRequestHandler):
def __init__(self, request_text):
self.rfile = BytesIO(request_text)
self.raw_requestline = self.rfile.readline()
self.error_code = self.error_message = None
self.parse_request()
def send_error(self, code, message):
self.error_code = code
self.error_message = message
while True:
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
data = connectionSocket.recv(1024)
# Those are your data coming from the client
print(data.decode('utf-8'))
# parse your headers
http_headers = HTTPRequest(data)
try:
# get the incoming auth token
client_token = http_headers.headers['Authorization'].strip('Basic ')
if server_token != client_token:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'Wrong credetials', 'utf-8'))
else:
# process the request and do your stuff here
connectionSocket.sendall(bytes("HTTP/1.1 200 OK\n\n" + 'Ok, all is fine here', 'utf-8'))
except AttributeError:
connectionSocket.sendall(bytes("HTTP/1.1 401 Unauthorized\n\n" + 'No credentials provided', 'utf-8'))
finally:
connectionSocket.close()
Here is how a requests.get with auth looks like server side:
Ready to serve...
GET / HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: python-requests/2.26.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Authorization: Basic dXNlcjpwYXNz
And now, let's see it in action:
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'pass'))
>>> r.status_code
200
>>> r.text
'Ok, all is fine here'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080',auth = HTTPBasicAuth('user', 'wrongpass'))
>>> r.status_code
401
>>> r.text
'wrong credentials'
>>>
>>>
>>> r = requests.get('http://127.0.0.1:8080')
>>> r.status_code
401
>>> r.text
'No credentials provided'
I would like to make various http requests and display the actual response status code and reason regardless of any http exceptions, for e.g. if it returns 503 or 404 then just want to display that status code and handle it rather than throwing exception stack.
However, what happens currently in the following is reason variable is never populated if the request is unsuccessful so the request summary result is never displayed.
Any suggestions?
import http.server
import socketserver
import socket
import requests
PORT = 5000
URL1 = "https://foo/"
# URL2 =
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write(("<br>Running on: %s" % socket.gethostname()).encode('utf-8'))
if self.path == '/status':
self.wfile.write(("<h1>status</h1>").encode('utf-8'))
try:
response = requests.get(URL1,verify=False)
self.wfile.write(("<br>Request client connection : {}, Response Status: {}, Response Reason: {}".format(response.url, response.status_code, response.reason)).encode('utf-8'))
except:
self.wfile.write(("exception").encode('utf-8'))
#self.wfile.write(("<br>Request client connection : {}, Response Status: {}, Response Reason: {}".format(response.url, response.status_code, response.reason)).encode('utf-8'))
return
return
httpd = socketserver.TCPServer(("", PORT), Handler)
print("serving at port: %s" % PORT)
httpd.serve_forever()
In this case the code is not checking for unsuccessful requests, the try will catch some exceptions but not all. What you want is the following function raise_for_status() that will raise an exception in case of a failed status code. See also Response raise for status
I can't connect to anything on my network using the IP address of the host. I can open a browser and connect and I can ping the host just fine. Here is my code:
from httplib import HTTPConnection
addr = 192.168.14.203
conn = HTTPConnection(addr)
conn.request('HEAD', '/')
res = conn.getresponse()
if res.status == 200:
print "ok"
else:
print "problem : the query returned %s because %s" % (res.status, res.reason)
The following error gets returned:
socket.error: [Errno 51] Network is unreachable
If I change the addr var to google.com I get a 200 response. What am I doing wrong?
You should check the address and your proxy settings.
For making HTTP requests I recommend the requests library. It's much more high-level and user friendly compared to httplib and it makes it easy to set proxies:
import requests
addr = "http://192.168.14.203"
response = requests.get(addr)
# if you need to set a proxy:
response = requests.get(addr, proxies={"http": "...proxy address..."})
# to avoid using any proxy if your system sets one by default
response = requests.get(addr, proxies={"http": None})
I need to build a http server without using an HTTP library.
I have the server running and an html page beeing loaded but my <img src="..."/> tags are not beeing loaded, I recive the call but cannot preset the png/JPEG in the page.
httpServer.py
# Define socket host and port
SERVER_HOST = '0.0.0.0'
SERVER_PORT = 8000
# Create socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((SERVER_HOST, SERVER_PORT))
server_socket.listen(1)
print('Listening on port %s ...' % SERVER_PORT)
while True:
# Wait for client connections
client_connection, client_address = server_socket.accept()
# Handle client request
request = client_connection.recv(1024).decode()
content = handle_request(request)
# Send HTTP response
if content:
response = 'HTTP/1.1 200 OK\n\n'
response += content
else:
response = 'HTTP/1.1 404 NOT FOUND\n\nFile Not Found'
client_connection.sendall(response.encode())
client_connection.close()
# Close socket
server_socket.close()
Function where handles the call
def handle_request(request):
http = HttpHandler.HTTPHandler
# Parse headers
print(request)
headers = request.split('\n')
get_content = headers[0].split()
accept = headers[6].split()
type_content = accept[1].split('/')
try:
# Filename
filename = get_content[1]
if get_content[0] == "GET":
content = http.get(None, get_content[1], type_content[0])
return content
except FileNotFoundError:
return None
class to handle the http verbs
class HTTPHandler:
def get(self, args, type):
if args == '/':
args = '/index.html'
fin = open('htdocs' + args)
if type != "image":
fin = open('htdocs/' + args)
if type == "image":
fin = open('htdocs/' + args, 'rb')
# Read file contents
content = fin.read()
fin.close()
return content
Realize that I´m trying to make an HTTP 1.1, if you see anything out of pattern fell free to say thanks in advance.
I don't know where you've learnt how HTTP works but I'm pretty sure that you did not study the actual standard which you should do when implementing a protocol. Some notes about your implementation:
Line ends should be \r\n not \n. This is true for both responses from the server as requests from the client.
You are assuming that the clients requests is never larger than 1024 bytes and that it can be read within a single recv. But, requests can have arbitrary length and there is no guarantee that you get all within a single recv (TCP is a streaming protocol and not a message protocol).
While it is kind of ok to simply close the TCP connection after the body it would be better to include the length of the body in the Content-length header or use chunked transfer encoding.
The type of the content should be given by using the Content-Type header, i.e. Content-type: text/html for HTML and Content-type: image/jpeg for JPEG images. Without this browser might guess correctly or wrongly what the type might be or depending on the context might also insist on a proper content-type header.
Apart from that, if you debug such problems it is helpful to find out what gets actually exchanged between client and server. It might be that you've checked this for yourself but you did not include such information into your question. Your only error description is "...I recive the call but cannot preset the png/JPEG in the page" and then a dump of your code.
httpServer.py
Ended up like:
while True:
# Wait for client connections
client_connection, client_address = server_socket.accept()
# Handle client request
request = client_connection.recv(10240).decode()
content = handle_request(request)
# Send HTTP response
if content:
if str(content).find("html") > 0:
client_connection.send('HTTP/1.1 200 OK\n\n'.encode())
client_connection.send(content.encode())
else:
client_connection.send('HTTP/1.1 200 OK\r\n'.encode())
client_connection.send("Content-Type: image/jpeg\r\n".encode())
client_connection.send("Accept-Ranges: bytes\r\n\r\n".encode())
client_connection.send(content)
else:
response = 'HTTP/1.1 404 NOT FOUND\r\nFile Not Found'
client_connection.close()
And the Get method like:
class HTTPHandler:
def get(self, args, type):
if args == '/':
args = '/index.html'
fin = open('htdocs' + args)
if type != "image":
fin = open('htdocs/' + args)
if type.find("html") == -1:
image_data = open('htdocs/' + args, 'rb')
bytes = image_data.read()
# Content-Type: image/jpeg, image/png \n\n
content = bytes
fin.close()
return content
# Read file contents
content = fin.read()
fin.close()
return content
import socket
host = 'www.google.com'
port = 80
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try :
client.connect((host, port))
except socket.error:
print ("Err")
package = '00101'
package.encode('utf-8')
client.sendall(package.encode(encoding = 'utf-8'))
response = client.recv(4096)
print (response.decode('UTF-8')
I kept getting b'' as my return, so I'm trying to decode it. The error I receive is unexpected EOF while parsing. Should I not include the decoding() function in my printing? I've tried printing only response, the .decode() function did not decode. What should I try?
You need to send a valid HTTP request. For example:
package = b'''GET /HTTP/1.1
Host: www.google.com
'''
client.sendall(package)
Which correctly returns a redirect on my machine. Note the empty line at the end of package, which ends the request.
When you send b'00101' and start reading, the google server has not yet processed your request and returns nothing. By sending a trailing newline (package = b'00101\n') it will start processing your request, and you will get:
...
<p>Your client has issued a malformed or illegal request. <ins>That’s all we know.</ins>