I have a function call an api to get an access token :
def get_token():
try:
url = 'https://call-api/accesstoken'
token = 'authentication-token'
headers = {"Authorization": "Bearer %s" % token}
session = requests.Session()
response = session.get(url, headers=headers)
return response.json()
except HTTPError as http_err:
print(f'HTTP error occurred: {http_err}')
except Exception as err:
print(f'Other error occurred: {err}')
I want to test this function with pytest and verify the try/except block.
Here is my test function :
import responses
import functions as m_functions
#responses.activate
def test_get_token():
url = 'https://call-api/accesstoken'
responses.add(responses.GET, url,
json = {
"accessToken": "tmp-token1"
})
response = m_functions.get_token()
assert response['accessToken'] == 'tmp-token1'
The part inside the block try is tested, but how to test the exceptions part ?
For the moment, only part inside try block is covered by tests. I would like my code to be fully covered.
Any idea? Thanks
As shown in the documentation, you can specify an exception as the response body for the request to trigger an exception:
responses.add(responses.GET, url, body=SomeExceptionObject(...))
Therefore, you can make two tests: one where you specify HTTPError as the exception and another tests with a different exception. You can then use the capsys fixture to capture the standard output.
Related
I have some python test code that is requesting web pages from a embedded controller. For general .htm pages it works but when I request a .txt file stored on it I get a
"401 Client Error: Unauthorized for url: http://192.168.61.30/fs/work/sys/crashdump/log0.txt"
HTTP error
The code is as follows:-
for url in ['http://192.168.61.30/get_version.htm']:
try:
cookies = {'Auth': access_token}
response = requests.get(url, cookies=cookies)
# If the response was successful, no Exception will be raised
response.raise_for_status()
except HTTPError as http_err:
print(f'HTTP error occurred: {http_err}') # Python 3.6
except Exception as err:
print(f'Other error occurred: {err}') # Python 3.6
else:
print("*****Access obtained******")
print(response.content);
print('Success!')
for url in ['http://192.168.61.30/fs/work/sys/crashdump/log0.txt']:
try:
cookies = {'Auth': access_token}
response = requests.get(url, cookies=cookies)
# If the response was successful, no Exception will be raised
response.raise_for_status()
except HTTPError as http_err:
print(f'HTTP error occurred: {http_err}') # Python 3.6
except Exception as err:
print(f'Other error occurred: {err}') # Python 3.6
else:
print("*****Access obtained******")
print(response.content);
print('Success!')
The 1st one works fine, however the 2nd one always fails with 401 client error and as far as I can work out never gets as far as the web server. Looking at the returned headers I don't see any authorisation requests. I have also disabled the proxies.
If I send the same request via a web browser I get no problems
The headers response is
{'Server': 'HPCi Controller Web server', 'Connection': 'close', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Type': 'text/html'}
I have tried adding WWW-Authentication: Basic to the headers, but no success
I have such task in locustfile:
class Tasks(FastHttpUser):
headers = {'content-type': 'application/json', 'Connection': 'close'}
#task
def task(self):
payload = "some data"
try:
with self.client.post("/endpoint", data=payload, headers=self.headers,
name="request", catch_response=True) as response:
result = json.loads(response.content)
if len(result["result"]) == 0:
response.failure(result)
log.error(result)
except (TypeError, JSONDecodeError) as err:
response.failure(response.text)
log.error(f'{type(err).__name__} because of : {response.status_code} - {response.text}')
I expected that if response will not pass statement or there will be some unexpected answer, this response will be marked as failed and be added to statistic as failure, but it isn't happen, I see only error in logs.
Is it the right way to use failure attribute or it's better to use something different if I want to check response's content?
You need to do response.failure(...) inside the with-block. The way you are doing, the response has already been lost because of the unexpected termination of the with-block.
Use something like this:
with self.client.post("/endpoint", data=payload, headers=self.headers,
name="request", catch_response=True) as response:
try:
result = json.loads(response.content)
if len(result["result"]) == 0:
response.failure(result)
log.error(result)
except (TypeError, JSONDecodeError) as err:
response.failure(response.text)
log.error(f'{type(err).__name__} because of : {response.status_code} - {response.text}')
I am trying to pass a list of dictionaries(strings) to a for a put request. I am getting this error:
TypeError: POST data should be bytes, an iterable of bytes.
Is this the right way to make a put request with list of dictionaries(strings) in python.
list looks like the following:
list1 = ['{"id" : "","email" : "John#fullcontact.com","fullName": "John Lorang"}', '{"id" : "","email" : "Lola#fullcontact.com","fullName": "Lola Dsilva"}']
myData = json.dumps(list1)
myRestRequestObj = urllib.request.Request(url,myData)
myRestRequestObj.add_header('Content-Type','application/json')
myRestRequestObj.add_header('Authorization','Basic %s')
myRestRequestObj.get_method = lambda : 'PUT'
try:
myRestRequestResponse = urllib.request.urlopen(myRestRequestObj)
except urllib.error.URLError as e:
print(e.reason)
As you said in a comment, you cannot use requests (that's pretty sad to hear!), so I did another snippet using urllib (the short answer: you must .encode('utf-8') json.dumps and decode('utf-8') response.read()):
import urllib.request
import urllib.error
import json
url = 'http://httpbin.org/put'
token = 'jwtToken'
list1 = ['{"id" : "","email" : "John#fullcontact.com","fullName": "John Lorang"}', '{"id" : "","email" : "Lola#fullcontact.com","fullName": "Lola Dsilva"}']
# Request needs bytes, so we have to encode it
params = json.dumps(list1).encode('utf-8')
headers = {
'Content-Type': 'application/json',
'Authorization': 'Basic {token}'.format(token=token)
}
# Let's try to create our request with data, headers and method
try:
request = urllib.request.Request(url, data=params, headers=headers, method='PUT')
except urllib.error.URLError as e:
# Unable to create our request, here the reason
print("Unable to create youro request: {error}".format(error=str(e)))
else:
# We did create our request, let's try to use it
try:
response = urllib.request.urlopen(request)
except urllib.error.HTTPError as e:
# An HTTP error occured, here the reason
print("HTTP Error: {error}".format(error=str(e)))
except Exception as e:
# We got another reason, here the reason
print("An error occured while trying to put {url}: {error}".format(
url=url,
error=str(e)
))
else:
# We are printing the result
# We must decode it because response.read() returns a bytes string
print(response.read().decode('utf-8'))
I did try to add some comments. I hope this solution help you!
To help you learn a better way to learn python, you should read the Style Guide for Python Code
I will suppose you can use the requests module (pip install requests).
requests is a simple yet powerful HTTP libraby for Python.
import json
import requests
my_data = json.dumps(list1)
headers = {
'Authorization': 'Basic {token}'.format(token=your_token)
}
response = requests.put(url, headers=headers, json=my_data)
print("Status code: {status}".format(status=response.status_code))
print("raw response: {raw_response}".format(
raw_response=response.text
)
print("json response: {json_response}".format(
json_response=response.json()
)
I have this code
try:
response = requests.post(url, data=json.dumps(payload))
except (ConnectionError, HTTPError):
msg = "Connection problem"
raise Exception(msg)
Now i want the following
if status_code == 401
login() and then try request again
if status_code == 400
then send respose as normal
if status_code == 500
Then server problem , try the request again and if not successful raise EXception
Now these are status codes , i donn't know how can i mix status codes with exceptions. I also don't know what codes will be covered under HttpError
requests has a call called raise_for_status available in your request object which will raise an HTTPError exception if any code is returned in the 400 to 500 range inclusive.
Documentation for raise_for_status is here
So, what you can do, is after you make your call:
response = requests.post(url, data=json.dumps(payload))
You make a call for raise_for_status as
response.raise_for_status()
Now, you are already catching this exception, which is great, so all you have to do is check to see which status code you have in your error. This is available to you in two ways. You can get it from your exception object, or from the request object. Here is the example for this:
from requests import get
from requests.exceptions import HTTPError
try:
r = get('http://google.com/asdf')
r.raise_for_status()
except HTTPError as e:
# Get your code from the exception object like this
print(e.response.status_code)
# Or you can get the code which will be available from r.status_code
print(r.status_code)
So, with the above in mind, you can now use the status codes in your conditional statements
https://docs.python.org/2/library/urllib2.html#urllib2.URLError
code
An HTTP status code as defined in RFC 2616. This numeric value
corresponds to a value found in the dictionary of codes as found in
BaseHTTPServer.BaseHTTPRequestHandler.responses.
You can get the error code from an HTTPError from its code member, like so
try:
# ...
except HTTPError as ex:
status_code = ex.code
I wanted to check if a certain website exists, this is what I'm doing:
user_agent = 'Mozilla/20.0.1 (compatible; MSIE 5.5; Windows NT)'
headers = { 'User-Agent':user_agent }
link = "http://www.abc.com"
req = urllib2.Request(link, headers = headers)
page = urllib2.urlopen(req).read() - ERROR 402 generated here!
If the page doesn't exist (error 402, or whatever other errors), what can I do in the page = ... line to make sure that the page I'm reading does exit?
You can use HEAD request instead of GET. It will only download the header, but not the content. Then you can check the response status from the headers.
For python 2.7.x, you can use httplib:
import httplib
c = httplib.HTTPConnection('www.example.com')
c.request("HEAD", '')
if c.getresponse().status == 200:
print('web site exists')
or urllib2:
import urllib2
try:
urllib2.urlopen('http://www.example.com/some_page')
except urllib2.HTTPError, e:
print(e.code)
except urllib2.URLError, e:
print(e.args)
or for 2.7 and 3.x, you can install requests
import requests
response = requests.get('http://www.example.com')
if response.status_code == 200:
print('Web site exists')
else:
print('Web site does not exist')
It's better to check that status code is < 400, like it was done here. Here is what do status codes mean (taken from wikipedia):
1xx - informational
2xx - success
3xx - redirection
4xx - client error
5xx - server error
If you want to check if page exists and don't want to download the whole page, you should use Head Request:
import httplib2
h = httplib2.Http()
resp = h.request("http://www.google.com", 'HEAD')
assert int(resp[0]['status']) < 400
taken from this answer.
If you want to download the whole page, just make a normal request and check the status code. Example using requests:
import requests
response = requests.get('http://google.com')
assert response.status_code < 400
See also similar topics:
Python script to see if a web page exists without downloading the whole page?
Checking whether a link is dead or not using Python without downloading the webpage
How do you send a HEAD HTTP request in Python 2?
Making HTTP HEAD request with urllib2 from Python 2
from urllib2 import Request, urlopen, HTTPError, URLError
user_agent = 'Mozilla/20.0.1 (compatible; MSIE 5.5; Windows NT)'
headers = { 'User-Agent':user_agent }
link = "http://www.abc.com/"
req = Request(link, headers = headers)
try:
page_open = urlopen(req)
except HTTPError, e:
print e.code
except URLError, e:
print e.reason
else:
print 'ok'
To answer the comment of unutbu:
Because the default handlers handle redirects (codes in the 300 range), and codes in the 100-299 range indicate success, you will usually only see error codes in the 400-599 range.
Source
There is an excellent answer provided by #Adem Öztaş, for use with httplib and urllib2. For requests, if the question is strictly about resource existence, then the answer can be improved upon in the case of large resource existence.
The previous answer for requests suggested something like the following:
def uri_exists_get(uri: str) -> bool:
try:
response = requests.get(uri)
try:
response.raise_for_status()
return True
except requests.exceptions.HTTPError:
return False
except requests.exceptions.ConnectionError:
return False
requests.get attempts to pull the entire resource at once, so for large media files, the above snippet would attempt to pull the entire media into memory. To solve this, we can stream the response.
def uri_exists_stream(uri: str) -> bool:
try:
with requests.get(uri, stream=True) as response:
try:
response.raise_for_status()
return True
except requests.exceptions.HTTPError:
return False
except requests.exceptions.ConnectionError:
return False
I ran the above snippets with timers attached against two web resources:
1) http://bbb3d.renderfarming.net/download.html, a very light html page
2) http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4, a decently sized video file
Timing results below:
uri_exists_get("http://bbb3d.renderfarming.net/download.html")
# Completed in: 0:00:00.611239
uri_exists_stream("http://bbb3d.renderfarming.net/download.html")
# Completed in: 0:00:00.000007
uri_exists_get("http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4")
# Completed in: 0:01:12.813224
uri_exists_stream("http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4")
# Completed in: 0:00:00.000007
As a last note: this function also works in the case that the resource host doesn't exist. For example "http://abcdefghblahblah.com/test.mp4" will return False.
I see many answers that use requests.get, but I suggest you this solution using only requests.head which is faster and also better for the webserver since it doesn't need to send back the body too.
import requests
def check_url_exists(url: str):
"""
Checks if a url exists
:param url: url to check
:return: True if the url exists, false otherwise.
"""
return requests.head(url, allow_redirects=True).status_code == 200
The meta-information contained in the HTTP headers in response to a HEAD request should be identical to the information sent in response to a GET request.
code:
a="http://www.example.com"
try:
print urllib.urlopen(a)
except:
print a+" site does not exist"
You can simply use stream method to not download the full file. As in latest Python3 you won't get urllib2. It's best to use proven request method. This simple function will solve your problem.
def uri_exists(url):
r = requests.get(url, stream=True)
if r.status_code == 200:
return True
else:
return False
def isok(mypath):
try:
thepage = urllib.request.urlopen(mypath)
except HTTPError as e:
return 0
except URLError as e:
return 0
else:
return 1
Try this one::
import urllib2
website='https://www.allyourmusic.com'
try:
response = urllib2.urlopen(website)
if response.code==200:
print("site exists!")
else:
print("site doesn't exists!")
except urllib2.HTTPError, e:
print(e.code)
except urllib2.URLError, e:
print(e.args)