Develop Reference

Writing a unit test for Python REST API function

I'm currently learning Python REST API (side project). I've been reading a lot of tutorials from RealPython, Python Requests documentation, etc. I found this post on how to write try/except properly in Python (Correct way to try/except using Python requests module?). One thing that still confuses me though is how to create a unit test for a function like this since it is not returning anything. Any help?
def google_do_something(blahblah):
url='http://www.google.com/' + blahblah
try:
r = requests.get(url,timeout=3)
r.raise_for_status()
except requests.exceptions.HTTPError as errh:
print (errh)
except requests.exceptions.ConnectionError as errc:
print (errc)
except requests.exceptions.Timeout as errt:
print (errt)
except requests.exceptions.RequestException as err:
print (err)
I could think of this but I don't know what to assert with.
def test_google_do_something():
g = google_do_something('blahblah')
# assert??

There are several unit test frameworks available in Python. Try/except blocks are good for error handling, but you still need a separate unit test around the call if you want to unit test it.
You do have something you can test, you can just return it and test that in your unit test.
Example Unit test using unittest:
import unittest
import requests
class RestCalls():
def google_do_something(blahblah):
url= blahblah
try:
r = requests.get(url,timeout=1)
r.raise_for_status()
return r.status_code
except requests.exceptions.Timeout as errt:
print (errt)
raise
except requests.exceptions.HTTPError as errh:
print (errh)
raise
except requests.exceptions.ConnectionError as errc:
print (errc)
raise
except requests.exceptions.RequestException as err:
print (err)
raise
class TestRESTMethods(unittest.TestCase):
def test_valid_url(self):
self.assertEqual(200,RestCalls.google_do_something('http://www.google.com/search'))
def test_exception(self):
self.assertRaises(requests.exceptions.Timeout,RestCalls.google_do_something,'http://localhost:28989')
if __name__ == '__main__':
unittest.main()
Executing should show (made some edits to this post, updated output included at bottom of post):
> python .\Tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.192s
OK
If you asserted a different response code from your request, it would fail (the request is just returning http response codes):
python .\Tests.py
F
======================================================================
FAIL: test_upper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\Tests.py", line 25, in test_upper
self.assertEqual(404,RestCalls.google_do_something('search'))
AssertionError: 404 != 200
----------------------------------------------------------------------
Ran 1 test in 0.245s
FAILED (failures=1)
Which is expected.
Edit: Included exception testing. You can test these by just including raise in the except block, which will show this after running:
> python .\Tests.py
HTTPConnectionPool(host='localhost', port=28989): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x03688598>, 'Connection to localhost timed out. (connect timeout=1)'))
..
----------------------------------------------------------------------
Ran 2 tests in 2.216s
OK
References:
Unit tests in Python
https://docs.python.org/3/library/unittest.html
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

I am not sure that your approach is such a good idea (just printing something in case of an error) but you could mock the print function to see if it was really called (and with what arguments):
https://docs.python.org/3/library/unittest.mock.html?highlight=mock#module-unittest.mock
Edit:
Working with mocks is a bit tricky as far as I remember. You would have to mock the print function in the current module. Perhaps something like this (not tested ...):
from unittest.mock import patch
from unittest import TestCase
class TestGoogleDoSomething(TestCase)
#patch("nameOfYourModule.print")
def test_google_do_something(self, print_mock): # the decorator will pass the mock object into the function
g = google_do_something('blahblah')
print_mock.assert_called_with("your error message here ...")

It seems that you are using print instead of all exception handlers. I don’t think that is a good practice. From my perspective, I prefer to raise those Exceptions out again if not sure how to deal with them right now.
With that said, when any error occurs, an exception will be thrown out; if there’re no exceptions, that means this function work well. Therefore you can design your unit test cases base on that.

python exception from module is not caught

I tried to catch an exception from vk_api module. I imported it:
import vk_api
then wrote some code, and then i screwed up my token on purpose so i can check if exception catch works:
try:
vk.method('wall.post', params)
except vk_api.exceptions.ApiError:
print('caught')
but it still gives me an error:
vk_api.exceptions.ApiError: [5] User authorization failed: invalid access_token (4).
What's the problem?

Your error comes from somewhere else in your code. You can have a better idea of where it comes from by using a wider try/except block.

try:
vk_session.auth(token_only=True)
except vk_api.AuthError as error_msg:
print(error_msg)
return
You can check it out an example here.
It seems the correct usage is vk_api.[ErrorName]
Your answer is
try:
vk.method('wall.post', params)
except vk_api.ApiError:
print('caught')

What's the best way to display Exception in Flask?

I'm a newbie in Flask and I am trying to display the Built-In Exceptions in python but I can't seem to have them display on my end.
NOTE:
set FLASK_DEBUG = 0
CODE:
def do_something:
try:
doing_something()
except Exception as err:
return f"{err}"
Expectation:
It will display one of the built-in exceptions:
KeyError
IndexError
NameError
Etc.
Reality:
It will return the line of code that didn't worked which is more ambiguous to the end user.
Also:
I have no problem seeing the errors when the debug mode is ON but that's not something that I want to do if I open them in public

Flask supplies you with a function that enables you to register an error handler throughout your entire app; you can do something as shown below:
def handle_exceptions(e):
# Log exception in your logs
# get traceback and sys exception info and log as required
# app.logger.error(getattr(e, 'description', str(e)))
# Print traceback
# return your response using getattr(e, 'code', 500) etc.
# Exception is used to catch all exceptions
app.register_error_handler(Exception, handle_exceptions)
In my honest opinion, this is the way to go. - Following the structure found in werkzeug.exceptions.HTTPException as an example is a solid foundation.
Having a unified exception handler that will standardise your Exception handling, visualisation and logging will make your life a tad better. :)

Try with this:
def do_something:
try:
doing_something()
except Exception as err:
return f"{err.__class__.__name__}: {err}"

DeadLink exception from Python2 to Python3

I found this code written in Python 2.7 to bypass a deadlink while reading a list of urls and retrieving their content:
for i in xrange(lines):
try:
t = urllib2.urlopen(urllib2.Request(lines[i]))
deadlinkfound = False
except:
deadlinkfound = True
if not(deadlinkfound):
urllib.urlretrieve(lines[i], "Images/imag" + "-%s" % i)
It worked fine in Python2 but I can't find the equivalent in Python3 because of the urllib2 merging.

You can do the exact same thing with urllib.request here. Don't catch every conceivable exception, only catch what is reasonably going to be thrown:
from urllib import request, error
from http.client import HTTPException
for i, url in enumerate(lines):
try:
t = request.urlopen(request.Request(url, method='HEAD'))
except (HTTPException, error.HTTPError):
continue
request.urlretrieve(url, 'Images/imag-{}'.format(i))
This code does the same, but more efficiently.

Print Python Exception Type (Raised in Fabric)

I'm using Fabric to automate, including the task of creating a directory. Here is my fabfile.py:
#!/usr/bin/env python
from fabric.api import *
def init():
try:
local('mkdir ./www')
except ##what exception?##:
#print exception name to put in above
Run fab fabfile.py and f I already have ./www created an error is raised, but I don't know what kind, so I don't know how to handle the error yet. Fabric only prints out the following:
mkdir: cannot create directory ‘./www’: File exists
Fatal error: local() encountered an error (return code 1) while executing 'mkdir ./www'
Aborting.
What I want to do is be able to find out the error type so that I can except my errors properly without blanket statements. It would be really helpful if an answer does not just tell me how to handle a mkdir exception, but print (or otherwise find the name to) any exception I may run into down the line (mkdir is just an example).
Thank you!

The issue is that fabric uses subprocess for doing these sorts of things. If you look at the source code for local you can see it doesn't actually raise an exception. It calls suprocess.Popen and uses communicate() to read stdout and stderr. If there is a non-zero return code then it returns a call to either warn or abort. The default is abort. So, to do what you want, try this:
def init():
with settings(warn_only=True):
local('mkdir ./www')
If you look at the source for abort, it looks like this:
10 def abort(msg):
21 from fabric.state import output
22 if output.aborts:
23 sys.stderr.write("\nFatal error: %s\n" % str(msg))
24 sys.stderr.write("\nAborting.\n")
25 sys.exit(1)
So, the exception would be a SystemExit exception. While you could catch this, the proper way to do it is outlined above using settings.

It is nothing to handle with exception, it is from the fabric api
try to set the entire script's warn_only setting to be true with
env.warn_only = True

Normally, when you get an uncaught exception, Python will print the exception type along with the error message:
>>> raise IOError("Error message.")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: Error message.
If that's not happening, you're probably not getting an exception at all.
If you really want to catch an arbitrary exception and print it, you want to catch Exception or BaseException. BaseException will include even things like KeyboardInterrupt, though, so be careful with that.
def init():
try:
local('mkdir ./www')
except BaseException as e:
print "local() threw a", type(e).__name__
raise # Reraise the exception

In general:
try:
some_code()
except Exception, e:
print 'Hit An Exception', e
raise
Will tell you what the exception was but if you are not planning on actually handling some of the exceptions then simply getting rid of the try: except: lines will have exactly the same effect.
Also if you run your code under a debugger then you can look at the exception(s) that you hit in more detail.

def init():
try:
local('mkdir ./www')
except Exception as e:
print e.__class__.__name__
That's all there is to it!
edit: Just re-read your question and realized that my code would only print "Fatal" in your case. It looks like fabric is throwing an error and returning their own error code so you would have to look at the documentation. I don't have any experience with fabric so I'd suggest to look here if you haven't already. Sorry if this isn't helpful!