I am working with AWS Athena to get results. I have to initiate a query and then check to see if its completed.
I am now trying to write a unit test for the various states. Here is a sample code. I generate the athena connection from another function and hand it off to this function, as well as the execution ID.
def check_athena_status(athena, execution):
running = True
print('Checking Athena Execution Running State')
while running:
running_state = athena.get_query_execution(QueryExecutionId=execution)['QueryExecution']['Status']['State']
if running_state == 'SUCCEEDED':
print('Run SUCCEEDED')
running = False
elif running_state == 'RUNNING':
time.sleep(3)
print('Athena Query Still Running')
else:
raise RuntimeError('Athena Query Failed')
return True
I am basically trying to figure out is there a way where I can change the value of running_state from RUNNING to SUCCEEDED. I currently use this as the unit test for a successful run.
athena_succeed = mock.Mock()
execution_id = 'RandomID'
athena_succeed.get_query_execution.return_value = test_data.athena_succeeded
result = inventory_validator.check_athena_status(athena_succeed, execution_id)
assert result == True
where test_data.athena_succeeded is basically a dict
athena_succeed = {'QueryExecution': {
'Status': {'State': 'SUCCEEDED',
'SubmissionDateTime': '2021-08-08'}
}
}
I also have a "RUNNING" one.
athena_running = {'QueryExecution': {
'Status': {'State': 'RUNNING',
'SubmissionDateTime': '2021-08-08'}
}
}
I am trying to test branches so I want to go from running to succeed. I know I can change the while true value, but I want to change the actual "athena response" in the middle of the loop. I tried with PropertyMock but I am not sure thats the right use case.
This will fully test all branches of the code:
import pytest
from your_module import check_athena_status
def test_check_athena_status(mocker, capsys):
mock_sleep = mocker.patch("your_module.time.sleep")
mock_athena = mocker.Mock()
mock_athena.get_query_execution.side_effect = [
{"QueryExecution": {"Status": {"State": "RUNNING"}}},
{"QueryExecution": {"Status": {"State": "SUCCEEDED"}}},
]
result = check_athena_status(mock_athena, execution="RandomID")
assert result is True
mock_sleep.assert_called_once_with(3)
mock_athena.get_query_execution.assert_has_calls([
mocker.call(QueryExecutionId="RandomID"),
mocker.call(QueryExecutionId="RandomID"),
])
out, err = capsys.readouterr()
assert err == ""
assert out.splitlines() == [
"Checking Athena Execution Running State",
"Athena Query Still Running",
"Run SUCCEEDED",
]
def test_check_athena_status_error(mocker):
mock_athena = mocker.Mock()
other = {"QueryExecution": {"Status": {"State": "OTHER"}}}
mock_athena.get_query_execution.return_value = other
with pytest.raises(RuntimeError, match="^Athena Query Failed$"):
check_athena_status(mock_athena, execution="RandomID")
A couple of points to note:
time.sleep is mocked out so that the test runs immediately rather than taking 3 seconds, yet we assert that there was a delay intended between consecutive calls to the query.
the expected QueryExecutionId argument was asserted, and we check that it was called twice.
the status output text from the print statements is asserted.
The fixture mocker is provided by the plugin pytest-mock.
Use side_effect to change the return value on successive calls.
class TestCheckAthenaStatus(unittest.TestCase):
def test_check_athena_status_from_running_to_succeeded(self):
athena_running_succeeded = mock.Mock()
execution_id = 'RandomID'
athena_running_succeeded.get_query_execution.side_effect = (test_data.athena_running, test_data.athena_succeeded)
result = inventory_validator.check_athena_status(athena_running_succeeded, execution_id)
assert result == True
assert athena_running_succeeded.get_query_execution.call_count == 2
Related
I want to be able to mock a function that mutates an argument, and that it's mutation is relevant in order for the code to continue executing correctly.
Consider the following code:
def mutate_my_dict(mutable_dict):
if os.path.exists("a.txt"):
mutable_dict["new_key"] = "new_value"
return True
def function_under_test():
my_dict = {"key": "value"}
if mutate_my_dict(my_dict):
return my_dict["new_key"]
return "No Key"
def test_function_under_test():
with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock:
mutate_my_dict_mock.return_value = True
result = function_under_test()
assert result == "new_value"
**Please understand i know i can just mock os.path.exists in this case but this is just an example. I intentionally want to mock the function and not the external module.
**
I also read the docs here:
https://docs.python.org/3/library/unittest.mock-examples.html#coping-with-mutable-arguments
But it doesn't seem to fit in my case.
This is the test i've written so far, but it obviously doesn't work since the key changes:
def test_function_under_test():
with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock:
mutate_my_dict_mock.return_value = True
result = function_under_test()
assert result == "new_value"
Thanks in advance for all of your time :)
With the help of Peter i managed to come up with this final test:
def mock_mutate_my_dict(my_dict):
my_dict["new_key"] = "new_value"
return True
def test_function_under_test():
with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock:
mutate_my_dict_mock.side_effect = mock_mutate_my_dict
result = function_under_test()
assert result == "new_value"
How it works is that with a side effect you can run a function instead of the intended function.
In this function you need to both change all of the mutating arguments and return the value returned.
I have following python function in 'au.py' :
import os
def resolv_conf_audit():
ALT_PATH = "/etc/monitor/etc/resolv.conf.{}".format(os.uname()[1])
RES_PATH = "/data/bin/resolvconf"
if os.path.isfile(RES_PATH):
return "PASSED", "/data/bin/resolvconf is present"
elif os.path.isfile(ALT_PATH):
return "PASSED", "/etc/monitor/etc/resolv.conf. is present"
else:
return "FAILED"
I need to write a unit test with mock which can check the path exists or not
following is the unit test which I wrote
from au import resolv_conf_audit
import unittest
from unittest.mock import patch
class TestResolvConf(unittest.TestCase):
#patch('os.path.isfile.ALT_PATH')
def test_both_source_files_not(self, mock_os_is_file):
mock_os_is_file.return_value = False
assert resolv_conf_audit() == "FAILED"
but I am getting following error
AttributeError: <function isfile at 0x10bdea6a8> does not have the attribute 'ALT_PATH'
How do I mock to check the presence of ALT_PATH and RES_PATH so that I can validate the function. In future this unit test should have the capability to mock removal some files, before writing that I am testing this simple one
Thanks # Mauro Baraldi, as per your suggestion, I changed the code little bit and it works fine now
def test_both_source_files_not(self, mock_os_is_file):
mock_os_is_file.side_effect = [False , False]
assert resolv_conf_audit() == "FAILED"
Mocks by definition is a way to simulate beahvior of objects. You are trying to handle a variable (ALT_PATH) inside your function.
All you need is to mock just the os.path.isfile method.
class TestResolvConf(unittest.TestCase):
#patch('os.path.isfile')
def test_both_source_files_not(self, mock_os_is_file):
mock_os_is_file.return_value = False
assert resolv_conf_audit() == "FAILED"
#patch('os.path.isfile')
def test_both_source_files_exists(self, mock_os_is_file):
mock_os_is_file.return_value = True
assert resolv_conf_audit() == "PASSED"
Hi I am trying to get a respond from Alexa requesting on a backend using requests. I am using Python with these example: https://github.com/alexa/skill-sample-python-fact. However my backend is NodeJS.
From my Lambda:
URL = 'https://alexa-app-nikko.herokuapp.com/alexa'
def get_post_response():
r = requests.get(URL)
speech_output = str(r.text)
return response(speech_response(speech_output, True))
On my backend, it is routed to /alexa:
router.get('/', function(request, response) {
//console.log('Logged from Alexa.');
response.send('Hello World, Alexa!');
});
I tested it on the Lambda and works fine with these results:
{
"version": "1.0",
"response": {
"outputSpeech": {
"type": "PlainText",
"text": "Hello World, Alexa!"
},
"shouldEndSession": true
}
}
However I get a null on the Skill Output or this response from Alexa:
"There was a problem with the requested skill's response"
How do I debug from the Developer Console, because it seems the Lambda is fine.
Based on your own answer :
The problem is, when you invoke the LaunchIntent or other intents like AMAZON.StopIntent it doesn't have the key "slots" in them. And you were trying to access the value of slots which should throw a KeyError.
What you can do is, when you're sure of invocation of any particular intent which uses some slots, then you try to access them.
This is what I do :
def getSlotValue(intent, slot):
if 'slots' in intent:
if slot in intent['slots']:
if 'value' in intent['slots'][slot] and len(intent['slots'][slot]['value']) > 0:
return intent['slots'][slot]['value']
return -1
And try to access the slot values in your intent's function (in your get_post_response or get_power_response).
I do not know what is the connection of this problem to my request. It is now working.
def on_intent(request, session):
""" called on receipt of an Intent """
intent_name = request['intent']['name']
#intent_slots = request['intent']['slots']
# process the intents
if intent_name == "DebugIntent":
return get_debug_response()
elif intent_name == "PostIntent":
return get_post_response()
elif intent_name == "PowerIntent":
return get_power_response(request)
#return get_power_response(intent_slots)
# ----------- Amazon Built-in Intents -----------------
elif intent_name == "AMAZON.HelpIntent":
return get_help_response()
elif intent_name == "AMAZON.StopIntent":
return get_stop_response()
elif intent_name == "AMAZON.CancelIntent":
return get_stop_response()
elif intent_name == "AMAZON.FallbackIntent":
return get_fallback_response()
else:
print("invalid Intent reply with help")
return get_help_response()
I debugged it and what I got was a problem with the keyword 'slots' so I removed in my code the intent_slots = request['intent']['slots'] which I was also using to pass it to another function return get_power_response(intent_slots). I commented it out and replace or just placed the original request from def on_intent(request, session):.
I am trying to do a unit-test but don't quite get why these 2 dicts show up as not equal. I was wondering if someone could give me an explanation for this occurrence. My code is...
import unittest
class TestEmailValidator(unittest.TestCase):
def test(self):
known_dict = {
'debo#foobar.com': True,
'debo#gmail.com': False
}
result_dict = {}
for key in known_dict.keys():
result_dict[key] = is_email_valid(key)
# debugger results
# result_dict = {
# 'debo#foobar.com': True,
# 'debo#gmail.com': False
# }
if self.assertEqual(known_dict, result_dict):
print "is_email_valid passed"
else:
print "is_email_valid failed"
if __name__ == '__main__':
unittest.main()
I get the same result for assertEqual, assertEquals and assertDictEquals. I have tried assigning result_dict to known_dict before the test, but that did not pass either.
It would be great if someone could point me to why this could be happening.
You are misusing the assert. All the assertXYZ methods don't return a boolean value, they just raise an exception if the assertion fails. As these methods don't return anything, they implicitly return None. When evaluating None as a boolean it's treated as false, and hence your test prints is_email_valid failed, even though the test actually passes.
I have a need to poll a MSSQL database to watch the status of a running job. I want to run a status check every X seconds to see if the status=done. I am trying to use the threading module. I have tested the threading module with some simple print statements and it seems to work, but when I try inside my pymssql script it does not.
def watcher_query(cursor):
print 'Watching'
return cursor.execute(""" select *
from some_table' """)
def is_it_done(row):
if row['status'] == 'done':
return row['the_id'], True
else:
return row['the_id'], False
def d(cur):
watcher_query(cur)
for row in cur:
return is_it_done(row)[1]
threading.Timer(100, d).start()
def job_watch(server):
with pymssql.connect(server_info) as conn:
with conn.cursor(as_dict=True) as cur:
is_done = false
while is_done:
is_done = d(cur)
No matter what I set the threading.Timer to I see the 'Watching' statement print constantly. Is there a better way to set the polling timer perhaps?
I have also tried to use Twisted to set up a basic function which makes a function call every X sec until some condition is met. I haven't tried it with MSSQL yet though.
The way your code is written it doesn't seems to be in a working order:
It doesn't compile because of is_done = false,
If fixed to is_done = False, it skips the loop straight away,
Even if the loop is fixed in some reasonable way you never get to call threading.Timer(100, d).start() and don't examine any other rows as you return from d straight away after examining the first row using return is_it_done(row)[1]
It doesn't matter what the actual timed worker method does, prints to console or checks the database, should work just the same with the same timer.
What about something like this:
import threading
def is_it_done():
# get some dummy predictable results
if not hasattr(is_it_done, 'results'):
is_it_done.results = iter([False] * 3)
return next(is_it_done.results, True)
def job_watch():
is_done = False
def d():
is_done = is_it_done()
print('is_done: {}'.format(is_done))
timer = threading.Timer(3, d).start()
d()
job_watch()