Develop Reference

Mock two get responses with different result

I'am trying to mock two same get requests but with different result. My goal is to test second exception (except requests.exceptions.RequestException) but I can't make it work so that first requests.get pass and second requests.get fail to "connect" and therefore to reach second exception. Is that even possible? Thx!
try:
tenants = requests.get('https://hostname_1')
for tenant in tenants:
try:
a = requests.get('https://host_2')
try:
some_function(arguments)
except Exception as e:
print(e)
except requests.exceptions.RequestException as e:
print(e)
except Exception as e:
print(e)
here is what I tried:
#patch("argo_probe_poem.poem_cert.requests.get")
#patch("argo_probe_poem.poem_cert.requests.get")
def test_raise_request_exception(self, mock_requests1, mock_requests2):
mock_requests1.side_effect = pass_web_api
mock_requests2.side_effect = requests.exceptions.RequestException
with self.assertRaises(SystemExit) as e:
utils_func(self.arguments)
self.assertEqual(e.exception.code, 2)

You can make the Mock object return different values and/or raise different exceptions on different calls by specifying an iterable as the side_effect attribute.
An excerpt from the documentation:
If you pass in an iterable, it is used to retrieve an iterator which
must yield a value on every call. This value can either be an
exception instance to be raised, or a value to be returned from the
call to the mock...
So your test code should roughly look like:
#patch("argo_probe_poem.poem_cert.requests.get")
def test_raise_request_exception(self, mock_requests):
mock_requests.side_effect = pass_web_api, requests.exceptions.RequestException
with self.assertRaises(requests.exceptions.RequestException) as e:
utils_func(self.arguments)

Using nested Django transaction.atomic() to handle exceptions?

Django's docs say this about transaction.atomic() and exceptions:
https://docs.djangoproject.com/en/1.10/topics/db/transactions/#django.db.transaction.atomic
Avoid catching exceptions inside atomic!
...
The correct way to catch database errors is around an atomic block as shown above. If necessary, add an extra atomic block for this purpose. This pattern has another advantage: it delimits explicitly which operations will be rolled back if an exception occurs.
...
What does "If necessary, add an extra atomic block for this purpose." look like? Can I do this or does this cause "unexpected behavior"?
valid = True
errors = []
objects = MyModel.objects.all()
try:
with transaction.atomic():
for obj in objects:
try:
# Update and save obj here...
except:
errors.append("obj {} had errors".format(obj.pk))
valid = False
if not valid:
raise Exception('batch update failed.')
except Exception as ex:
# Handle it..
Do they mean to write it like this? If so, why is this different?
valid = True
errors = []
objects = MyModel.objects.all()
try:
with transaction.atomic():
for obj in objects:
try:
with transaction.atomic(): # Here's my 'extra atomic block'
# Update and save obj here...
except:
errors.append("obj {} had errors".format(obj.pk))
valid = False
if not valid:
raise Exception('batch update failed.')
except Exception as ex:
# Handle it..

Django triggers rolback only when transaction block catches DatabaseError (or its subclass), So you shouldn't catch it before it. If you add second transaction block (your 2nd example), error is caught, transaction is marked to rollbacked, and then you can do whatever you want.
I think that you should be good it yo re-raise exactly the same error, but this is just a guess.

Python Generic Exception to Catch Rest Of Errors

This is a python script that runs in a Django environment by Celery. I need to create a catch the 'rest of the errors' and raise an exception so Celery will send a email on that exception.
Would this be the best way?
for thread in thread_batch:
try:
obj = query_set.get(thread_id=thread['thread_id'])
for key, value in thread.iteritems():
setattr(obj, key, value)
obj.unanswered = True
except ThreadVault.DoesNotExist:
obj = ThreadVault(**thread)
except:
raise Exception("There has been a unknown error in the database")
obj.save()

Yes, and empty except will catch any exception different from ThreadVault.DoesNotExist (in this case). But you can improve your code a little more.
Try always to put the less code possible inside the try block. You code could be:
for thread in thread_batch:
try:
obj = query_set.get(thread_id=thread['thread_id'])
except ThreadVault.DoesNotExist:
obj = ThreadVault(**thread)
except:
raise Exception("There has been a unknown error in the database")
else: # Note we add the else statement here.
for key, value in thread.iteritems():
setattr(obj, key, value)
obj.unanswered = True
# Since save function also hits the database
# it should be within a try block as well.
try:
obj.save()
except:
raise Exception("There has been a unknown error in the database")

Which key failed in Python KeyError?

If I catch a KeyError, how can I tell what lookup failed?
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError:
# How can I tell what key ("FastestMan" or "FastestWoman") caused the error?
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key in JSON")

Take the current exception (I used it as e in this case); then for a KeyError the first argument is the key that raised the exception. Therefore we can do:
except KeyError as e: # One would do it as 'KeyError, e:' in Python 2.
cause = e.args[0]
With that, you have the offending key stored in cause.
Expanding your sample code, your log might look like this:
def poijson2xml(location_node, POI_JSON):
try:
man_json = POI_JSON["FastestMan"]
woman_json = POI_JSON["FastestWoman"]
except KeyError as e:
LogErrorMessage ("POIJSON2XML", "Can't find mandatory key '"
e.args[0]
"' in JSON")
It should be noted that e.message works in Python 2 but not Python 3, so it shouldn't be used.

Not sure if you're using any modules to assist you - if the JSON is coming in as a dict, one can use dict.get() towards a useful end.
def POIJSON2DOM (location_node, POI_JSON):
man_JSON = POI_JSON.get("FastestMan", 'No Data for fastest man')
woman_JSON = POI_JSON.get("FastestWoman", 'No Data for fastest woman')
#work with the answers as you see fit
dict.get() takes two arguments - the first being the key you want, the second being the value to return if that key does not exist.

If you import the sys module you can get exception info with sys.exc_info()
like this:
def POIJSON2DOM (location_node, POI_JSON):
try:
man_JSON = POI_JSON["FastestMan"]
woman_JSON = POI_JSON["FastestWoman"]
except KeyError:
# you can inspect these variables for error information
err_type, err_value, err_traceback = sys.exc_info()
REDI.LogErrorMessage ("POIJSON2DOM", "Can't find mandatory key in JSON")

Querying for multiple objects with Django-orm

RealEstateAgent is a model and doing RealEstateAgent.objects.filter(name = 'better homes') returns 5 objects.
I would like to use RealEstateAgent.objects.get(name='better homes') to catch the MultipleObjectsReturned exception.
Im trying this but the exception is not getting caught.
from django.core.exceptions import MultipleObjectsReturned
try:
RealEstateAgent.objects.get(name='bh')
except MultipleObjectsReturned, e:
print ''
this is the traceback:
DoesNotExist Traceback (most recent call last)
<ipython-input-49-9458986408df> in <module>()
1 try:
----> 2 RealEstateAgent.objects.get(name='better homes')
3 except MultipleObjectsReturned, e:
4 print ''
5
/home/dubizzle/webapps/django/src/django/django/db/models/manager.pyc in get(self, *args, **kwargs)
130
131 def get(self, *args, **kwargs):
--> 132 return self.get_query_set().get(*args, **kwargs)
133
134 def get_or_create(self, **kwargs):
/home/dubizzle/webapps/django/src/django/django/db/models/query.pyc in get(self, *args, **kwargs)
347 if not num:
348 raise self.model.DoesNotExist("%s matching query does not exist."
--> 349 % self.model._meta.object_name)
350 raise self.model.MultipleObjectsReturned("get() returned more than one %s -- it returned %s! Lookup parameters were %s"
351 % (self.model._meta.object_name, num, kwargs))
DoesNotExist: RealEstateAgent matching query does not exist.

It seems like there are some deeper issues with your code, but without more information those are difficult to debug. As far as your original question however, your line:
except MultipleObjectsReturned, e:
Will only catch exceptions of the type MultipleObjectsReturned. If you look at your traceback however, you see the actual exception being raised is a DoesNotExist exception.
If you change your except line (above) to :
except DoesNotExist, e:
It should properly catch that exception.
As for why the exception is raised in the first place, I'm willing to guess that you just don't have that object in your database. Are you inserting it anywhere? If you're looking for a backend that will automatically construct an entry for you when you try to access it, look into mongodb. Your current SQL database however, will error if you try to access an object that does not exist.
Based on what you said in the comments, it seems you misunderstand exactly what exception handling is. Specifically when you catch an exception, you are saying "This is an error that I am aware may happen and I have a contingency case for it".
The reason you might get a MultipleObjectsReturned exception is because the get method you used above is specifically for returning a single result. If you want to query for multiple entries you use a filter instead:
my_objects = RealEstateAgent.objects.filter(name='bh')
which will return a QuerySet filled with entries that match your query.
With that said, since a DoesNotExist exception is being raised, your assumption that there are five objects matching the query in your database appears to be incorrect.
To address your use case, you could pretty easily implement something along these lines:
my_objects = RealEstateAgent.objects.filter(name='bh')
if len(my_objects) > 1:
# Ask user to pick one