I'd like to raise a Python-standard exception when an HTTP response code from querying an API is not 200, but what specific exception should I use? For now I raise an OSError:
if response.status_code != 200:
raise OSError("Response " + str(response.status_code)
+ ": " + response.content)
I'm aware of the documentation for built-in exceptions.
You can simply call Response.raise_for_status() on your response:
>>> import requests
>>> url = 'http://stackoverflow.com/doesnt-exist'
>>> r = requests.get(url)
>>>
>>> print r.status_code
404
>>> r.raise_for_status()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "requests/models.py", line 831, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found
This will raise a requests.HTTPError for any 4xx or 5xx response.
See the docs on Response Status Code for a more complete example.
Note that this does not exactly do what you asked (status != 200): It will not raise an exception for 201 Created or 204 No Content, or any of the 3xx redirects - but this is most likely the behavior you want: requests will just follow the redirects, and the other 2xx are usually just fine if you're dealing with an API.
The built-in Python exceptions are probably not a good fit for what you are doing. You will want to subclass the base class Exception, and throw your own custom exceptions based on each scenario you want to communicate.
A good example is how the Python Requests HTTP library defines its own exceptions:
In the event of a network problem (e.g. DNS failure, refused
connection, etc), Requests will raise a ConnectionError exception.
In the rare event of an invalid HTTP response, Requests will raise an
HTTPError exception.
If a request times out, a Timeout exception is raised.
If a request exceeds the configured number of maximum redirections, a
TooManyRedirects exception is raised.
All exceptions that Requests explicitly raises inherit from
requests.exceptions.RequestException.
Related
i was opening a url by using urllib.request.urlopen.
The following exception was raised
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
HTTPError
I went through the documentation of the requests library
*exception urllib.error.HTTPError
Though being an exception (a subclass of URLError), an HTTPError can also function as a non-exceptional file-like return value (the same thing that urlopen() returns). This is useful when handling exotic HTTP errors, such as requests for authentication.
code
An HTTP status code as defined in RFC 2616. This numeric value corresponds to a value found in the dictionary of codes as found in http.server.BaseHTTPRequestHandler.responses.
reason
This is usually a string explaining the reason for this error.
headers
The HTTP response headers for the HTTP request that caused the HTTPError.*
But in my case there was no error code or string which gave reason for the exception.
enter image description here
If you catch the exception you can see the reason, like this:
try:
urllib.request.urlopen(req)
except urllib.error.HTTPError as e:
print(e.reason)
Is there a way to receive the status code of a request if the request throws an exception? I am trying to send a request, but there is a ConnectionTimeout. I've tried printing the error, but there's not useful status info in the error message. The actual object that is returned by requests.request() doesn't seem to get created when the exception is thrown, at least in my case. Could this be due to internal server configurations or something?
What I have attempted:
try:
response = requests.request(
method=some_http_method,
url=some_url,
auth=some_auth,
data=some_data,
timeout=30,
verify=some_certificate)
except requests.exceptions.ConnectTimeout as e:
response = e.response
print(e)
print(response.status_code) # should print a status code, instead response is still None
The exception ConnectTimeout means that the request timed out while trying to connect to the remote server. Thus, there's no response to get the status code from. it is normal that response variable is None
It can be some network issue, make sure the URL you're trying to query is reachable (use curl to verify)
Using the HTTPretty library for Python, I can create mock HTTP responses of choice and then pick them up i.e. with the requests library like so:
import httpretty
import requests
# set up a mock
httpretty.enable()
httpretty.register_uri(
method=httpretty.GET,
uri='http://www.fakeurl.com',
status=200,
body='My Response Body'
)
response = requests.get('http://www.fakeurl.com')
# clean up
httpretty.disable()
httpretty.reset()
print(response)
Out: <Response [200]>
Is there also the possibility to register an uri which cannot be reached (e.g. connection timed out, connection refused, ...) such that no response is received at all (which is not the same as an established connection which gives an HTTP error code like 404)?
I want to use this behaviour in unit testing to ensure that my error handling works as expected (which does different things in case of 'no connection established' and 'connection established, bad bad HTTP status code'). As a workaround, I could try to connect to an invalid server like http://192.0.2.0 which would time out in any case. However, I would prefer to do all my unit testing without using any real network connections.
Meanwhile I got it, using a HTTPretty callback body seems to produce the desired behaviour. See inline comments below.
This is actually not exactly the same as I was looking for (it is not a server that cannot be reached and hence the request times out but a server that throws a timeout exception once it is reached, however, the effect is the same for my usecase.
Still, if anybody knows a different solution, I'm looking forward to it.
import httpretty
import requests
# enable HTTPretty
httpretty.enable()
# create a callback body that raises an exception when opened
def exceptionCallback(request, uri, headers):
# raise your favourite exception here, e.g. requests.ConnectionError or requests.Timeout
raise requests.Timeout('Connection timed out.')
# set up a mock and use the callback function as response's body
httpretty.register_uri(
method=httpretty.GET,
uri='http://www.fakeurl.com',
status=200,
body=exceptionCallback
)
# try to get a response from the mock server and catch the exception
try:
response = requests.get('http://www.fakeurl.com')
except requests.Timeout as e:
print('requests.Timeout exception got caught...')
print(e)
# do whatever...
# clean up
httpretty.disable()
httpretty.reset()
Looking at the documentation here: http://docs.python-requests.org/en/latest/user/quickstart/
This should print 200 and it does.
import requests
r = requests.get('http://souke.xdf.cn/Category/1-40-0-0.html?v=5&page=1&pagesize=50')
print r.status_code
This should print 404 but it prints 200
import requests
r = requests.get('http://souke.xdf.cn/CategoryXXX/1-40-0-0.html?v=5&page=1&pagesize=50')
print r.status_code
Why is that?
Is there another way to recognize a 404 error has occurred?
The problem isn't with requests but with the site you're accessing. It's returning 200.
You can confirm this by looking at the headers using something like the Chrome developer tools:
Request URL:http://souke.xdf.cn/CategoryXXX/1-40-0-0.html?v=5&page=1&pagesize=50
Request Method:GET
Status Code:200 OK
The page you are looking for is found on the server , therefore the server responded with a 200 OK.
Nevertheless you can use Requests's raise_for_status() , to raise an exception whenever a server error is found, like 404 , 401 and so on.
import requests
>>>>r = requests.get('http://something.com/404/')
>>>>print r.status_code
404
>>>>r.raise_for_status()
Traceback (most recent call last):
File "requests/models.py", line 832, in raise_for_status
raise http_error
requests.exceptions.HTTPError: 404 Client Error
.raise_for_status()
this will raise error if not 200
this is better than using
.status_code
I want to detect wheter the http or https service is ok, in python.
Now I have known is to use httplib module.
Use the httplib.HTTPConnection to get the status, and check whether it is 'OK'(code is 200), and the same to https by using HTTPSConnection
but I don't know whether this way is right?or there is another more good way?
I have a script that does this kind of check, I use urllib2 for that, whatever the protocol (http or https):
result = False
error = None
try:
# Open URL
urllib2.urlopen(url, timeout=TIMEOUT)
result = True
except urllib2.URLError as exc:
error = 'URL Error: {0}'.format(str(exc))
except urllib2.HTTPError as exc:
error = 'HTTP Error: {0}'.format(str(exc))
except Exception as exc:
error = 'Unknow error: {0}'.format(str(exc))