Develop Reference

How do I Mock a method that makes multiple POST and GET request all requiring different response data?

I have looked at How to mock REST API and I have read the answers but I still can't seem to get my head around how I would go about dealing with a method that executes multiple GET and POST requests. Here is some of my code below.
I have a class, UserAliasGroups(). Its __init__() method executes requests.post() to login into the external REST API. I have in my unit test this code to handling the mocking of the login and it works as expected.
#mock.patch('aliases.user_alias_groups.requests.get')
#mock.patch('aliases.user_alias_groups.requests.post')
def test_user_alias_groups_class(self, mock_post, mock_get):
init_response = {
'HID-SessionData': 'token==',
'errmsg': '',
'success': True
}
mock_response = Mock()
mock_response.json.return_value = init_response
mock_response.status_code = status.HTTP_201_CREATED
mock_post.return_value = mock_response
uag = UserAliasGroups(auth_user='TEST_USER.gen',
auth_pass='FakePass',
groups_api_url='https://example.com')
self.assertEqual(uag.headers, {'HID-SessionData': 'token=='})
I also have defined several methods like obtain_request_id(), has_group_been_deleted(), does_group_already_exists() and others. I also define a method called create_user_alias_group() that calls obtain_request_id(), has_group_been_deleted(), does_group_already_exists() and others.
I also have code in my unit test to mock a GET request to the REST API to test my has_group_been_deleted() method that looks like this:
has_group_been_deleted_response = {
'error_code': 404,
'error_message': 'A group with this ID does not exist'
}
mock_response = Mock()
mock_response.json.return_value = has_group_been_deleted_response
mock_response.status_code = status.HTTP_404_NOT_FOUND
mock_get.return_value = mock_response
Now I can get to my question. Below is the pertinent part of my code.
class UserAliasGroups:
def __init__(
self,
auth_user=settings.GENERIC_USER,
auth_pass=settings.GENERIC_PASS,
groups_api_url=settings.GROUPS_API_URL
):
""" __init__() does the login to groups. """
self.auth_user = auth_user
self.auth_pass = auth_pass
self.headers = None
self.groups_api_url = groups_api_url
# Initializes a session with the REST API service. Each login session times out after 5 minutes of inactivity.
self.login_url = f'{self.groups_api_url}/api/login'
response = requests.post(self.login_url, json={}, headers={'Content-type': 'application/json'},
auth=(auth_user, auth_pass))
if response.status_code is not 201:
try:
json = response.json()
except:
json = "Could not decode json."
raise self.UserAliasGroupsException(f"Error: User {self.auth_user}, failed to login into "
f"{self.login_url} {json}")
response_json = response.json()
self.headers = {'HID-SessionData': response_json['HID-SessionData']}
def obtain_request_id(self, request_reason):
payload = {'request_reason': request_reason}
url = f'{self.groups_api_url}/api/v1/session/requests'
response = requests.post(url=url, json=payload, headers=self.headers)
if response.status_code is not status.HTTP_200_OK:
try:
json = response.json()
except:
json = "Could not decode json."
msg = f'obtain_request_id() Error url={url} {response.status_code} {json}.'
raise self.UserAliasGroupsException(msg)
request_id = response.json().get('request_id')
return request_id
def has_group_been_deleted(self, group_name):
url = f'{self.groups_api_url}/api/v1/groups/{group_name}/attributes/RESATTR_GROUP_DELETED_ON'
response = requests.get(url=url, headers=self.headers)
return response.status_code == status.HTTP_200_OK
def does_group_already_exists(self, group_name):
url = f'{self.groups_api_url}/api/v1/groups/{group_name}'
response = requests.get(url=url, headers=self.headers)
if response.status_code is status.HTTP_200_OK:
# check if the group has been "deleted".
return not self.has_group_been_deleted(group_name=group_name)
return False
def create_user_alias_group(
self,
... long list of params omitted for brevity ...
):
if check_exists:
# Check if group already exists or not.
if self.does_group_already_exists(group_name):
msg = f'Cannot create group {group_name}. Group already exists.'
raise self.UserAliasGroupsException(msg)
... more code omitted for brevity ...
My question is how do I write my unit test to deal with multiple calls to requests.post() and request.get() all resulting in different responses in my create_user_alias_group() method?
I want to call create_user_alias_group() in my unit test so I have to figure out how to mock multiple requests.get() and requests.post() calls.
Do I have use multiple decorators like this:
#mock.patch('aliases.user_alias_groups.obtain_request_id.requests.post')
#mock.patch('aliases.user_alias_groups.does_group_already_exists.requests.get')
#mock.patch('aliases.user_alias_groups.has_group_been_deleted.requests.get')
def test_user_alias_groups_class(self, mock_post, mock_get):
...
?
Thanks for looking my long question :)

You can use mock.side_effect which takes an iterable. Then different calls will return different values:
mock = Mock()
mock.side_effect = ['a', 'b', 'c']
This way the first call to mock returns "a", then the next one "b" and so on. (In your case, you'll set mock_get.side_effect).

Skip default model validation in Flask Restplus

Because of project specificity I have to write my own model validator for Flask-restplus API app. Simply speaking - when validation error occurs, its format and status code (400) is not proper. It should return JSON object with messages in particular format with status code 422.
What I do is more or less this:
ns = api.namespace('somenamespace', description='blabla')
class MyModel(MyBaseModel):
def __init__(self):
self.id = fields.Integer()
self.name = fields.String()
my_model = api.model('MyModel', MyModel())
#api.marshal_list_with(my_model, envelope='json')
#ns.route('/')
class SomeClass(Resource):
#api.expect(my_model, validate=False)
#api.doc(responses={
200: 'Success',
401: 'Authentication Error',
403: 'Requested resource unavailable',
409: 'Conflict, document already exists',
422: 'Validation Error'
})
def post(self):
"""
Save single document in the database.
:return:
"""
request_payload = json.loads(request.data)
validated_payload, payload_errors = some_validation(request_payload)
if payload_errors:
return jsonify(payload_errors), 422
else:
return jsonify({'response': 'ok})
Instance of `MyModel` behaves basically like a dict, so no problem in registration. Problem is that when I send data in `-d`, be it through `curl` from command line, or swagger, I constantly get `400` instead of `422`. I assume this is due to the default, built-in validation of input data based on `MyModel`. This is cool, but I have to omit it, and apply my own validation.

in the documentation and as #CloC says, one method is specifying the model as
your_model = ns.model('YourModel', {
'id': fields.Integer(
description='The user id'
)
'name': fields.String(
description='The user name'
)
})
... > profit
#ns.route('/', methods=["post"])
#ns.doc(params={
"id": "the user id (int)",
"name": "the user name (str)"
})
class SomeClass(Resource):
#ns.expect(your_model) # this defines the request
# #ns.marshal_list_with(your_model, envelope='json') # this defines the response
#ns.response(200, 'Success')
... > reponses
def post(self):
etc...
return <response with format as model in marshal>
you might want to re-define the response model though unless you will return something of the form that you put in.. also maybe specify marshal_with as you do not return a list?

TypeError with get_or_insert

Here is my code.
import webapp2
import json
from google.appengine.ext import ndb
class Email(ndb.Model):
email = ndb.StringProperty()
subscribed = ndb.BooleanProperty()
#staticmethod
def create(email):
ekey = ndb.Key("Email", email)
entity = Email.get_or_insert(ekey)
if entity.email: ###
# This email already exists
return None
entity.email = email
entity.subscribed = True
entity.put()
return entity
class Subscribe(webapp2.RequestHandler):
def post(self):
add = Email.create(self.request.get('email'))
success = add is not None
self.response.headers['Content-Type'] = 'application/json'
obj = {
'success': success
}
self.response.out.write(json.dumps(obj))
app = webapp2.WSGIApplication([
webapp2.Route(r'/newsletter/new', Subscribe),
], debug=True)
Here is my error.
File "/Users/nick/google-cloud-sdk/platform/google_appengine/google/appengine/ext/ndb/model.py", line 3524, in _get_or_insert_async
raise TypeError('name must be a string; received %r' % name) TypeError: name must be a string; received Key('Email', 'test#test.com')
What am I missing?

The error is caused by passing ekey (which is an ndb.Key) as arg to get_or_insert() (which expects a string):
ekey = ndb.Key("Email", email)
entity = Email.get_or_insert(ekey)
Since it appears you want to use the user's email as a unique key ID you should directly pass the email string to get_or_insert():
entity = Email.get_or_insert(email)

Flask RESTful Destroy User Token

I am setting up a Flask-RESTful service and have user authentication working. The method that I'm using is:
def generate_auth_token(username, expiration=600):
gen_serial = Serializer(secret_key, expires_in=expiration)
return gen_serial.dumps({'username': username})
I pass the token to the user as follows:
class token(Resource):
decorators = [auth.login_required]
def post(self):
username = g.user
return_token = generate_auth_token(username)
return {'token':return_token.decode()}, 200
And the token is then verified as such so that it does not need to be stored server side:
def verify_auth_token(auth_token):
serial = Serializer(secret_key)
try:
data = serial.loads(auth_token)
except SignatureExpired:
return None
except BadSignature:
return None
serial_user = data['username']
return serial_user
This seems to work well, however I am unsure how to logout the user before the expiration is expired without storing the token serverside. My thought was to pass back a garbage token when the user elects to logout, but I don't think this is an elegant or secure solution.
Any tips would be really helpful!

Rather than a garbage token simply encode no data:
def generate_auth_token(username=None, expiration=600):
gen_serial = Serializer(secret_key, expires_in=expiration)
data = {'username': username} if username is not None else {}
return gen_serial.dumps(data)
Then you can have an invalidate endpoint that requires a login and returns a token without a username:
def invalidate(self):
return_token = generate_auth_token()
return {'token':return_token.decode()}, 200
At that point, you can just handle the possibly missing username field:
def verify_auth_token(auth_token):
serial = Serializer(secret_key)
try:
data = serial.loads(auth_token)
except SignatureExpired:
return None
except BadSignature:
return None
serial_user = data.get('username')
return serial_user

Tornado RESTful PUT handler method not getting request arguments

I'm trying to unit test my RESTful API. Here's my API:
class BaseHandler(tornado.web.RequestHandler):
def __init__(self, *args, **kwargs):
tornado.web.RequestHandler.__init__(self, *args, **kwargs)
self.log = self.application.log
self.db = self.application.db
class ProductHandler(BaseHandler):
#tornado.web.removeslash
def put(self, id = None, *args, **kwargs):
try:
self.log.info("Handling PUT request")
if not id:
raise Exception('Object Id Required')
id = { '_id' : id }
new_values = dict()
name = self.get_argument('name', None)
description = self.get_argument('description', None)
if name:
new_values['name'] = name
if description:
new_values['description'] = description
self.db.products.update(id, new_values, safe = True)
except:
self.log.error("".join(tb.format_exception(*sys.exc_info())))
raise
class Application(tornado.web.Application):
def __init__(self, config_path, test = False, *args, **kwargs):
handlers = [
(r"/product/?(.*)", ProductHandler),
]
settings = dict(debug=True)
tornado.web.Application.__init__(self, handlers, **settings)
self.log = logging.getLogger(__name__)
self.config = ConfigParser()
self.config.read(config_path)
self.mongo_connection = Connection(
host = self.config.get('mongo','host'),
port = self.config.getint('mongo','port'),
)
if test:
db_name = self.config.get('test', 'mongo.db')
else:
db_name = self.config.get('mongo', 'db')
self.log.debug("Using db: %s" % db_name)
self.db = self.mongo_connection[db_name]
But, here's my problem: the handler isn't seeing the name or description arguments. :(
Any suggestions?

As a work-around, I found them in the request.body and parsed the encoded parameters manually. It was kindof annoying, but it works.
new_values = urlparse.parse_qs(self.request.body)
# values show as lists with only one item
for k in new_values:
new_values[k] = new_values[k][0]

Say if you are using jQuery to send this PUT request:
$.ajax({
type: "PUT",
url: "/yourURL",
data: JSON.stringify({'json':'your json here'),
dataType: 'json'
})
The data should not be like:
data: {'json': 'your json here'}, because it will automatically be encoded into query string, which needs to be parsed by parse_qs
Then in Tornado
def put(self, pid):
d = json.loads(self.request.body)
print d

put handler will parse request.body, if request had proper content-type header (application/x-www-form-urlencoded), for example if you are using tornado http client:
headers = HTTPHeaders({'content-type': 'application/x-www-form-urlencoded'})
http_client.fetch(
HTTPRequest(url, 'PUT', body=urllib.urlencode(body), headers=headers))

Have you tried using a get method instead? Because depending on how you test your program, if you test it via your browser like Firefox or Chrome, they might be able to do it. Doing a HTTP PUT from a browser
If I were you I would write get instead of put. Cause then you can definitely test it in your browser.
For example, instead of:
def put ...
Try:
def get ...
Or Actually in your:
name = self.get_argument('name', None)
description = self.get_argument('description', None)
Why is the None there? According to the documentation:
RequestHandler.get_argument(name, default=[], strip=True)
...
If default is not provided, the argument is considered to be required,
and we throw an HTTP 400 exception if it is missing.
So in your case because you are not providing a proper default, therefore your app is returning HTTP 400. Miss out the default! (i.e.)
name = self.get_argument('name')
description = self.get_argument('description')