Develop Reference

mock xmlrpc.client method python

I am in the process of learning unit testing, however I am struggling to understand how to mock functions for unit testing. I have reviewed many how-to's and examples but the concept is not transferring enough for me to use it on my code. I am hoping getting this to work on a actual code example I have will help.
In this case I am trying to mock isTokenValid.
Here is example code of what I want to mock.
<in library file>
import xmlrpc.client as xmlrpclib
class Library(object):
def function:
#...
AuthURL = 'https://example.com/xmlrpc/Auth'
auth_server = xmlrpclib.ServerProxy(AuthURL)
socket.setdefaulttimeout(20)
try:
if pull == 0:
valid = auth_server.isTokenValid(token)
#...
in my unit test file I have
import library
class Tester(unittest.TestCase):
#patch('library.xmlrpclib.ServerProxy')
def test_xmlrpclib(self, fake_xmlrpclib):
assert 'something'
How would I mock the code listed in 'function'? Token can be any number as a string and valid would be a int(1)

First of all, you can and should mock xmlrpc.client.ServerProxy; your library imports xmlrpc.client as a new name, but it is still the same module object so both xmlrpclib.ServerProxy in your library and xmlrpc.client.ServerProxy lead to the same object.
Next, look at how the object is used, and look for calls, the (..) syntax. Your library uses the server proxy like this:
# a call to create an instance
auth_server = xmlrpclib.ServerProxy(AuthURL)
# on the instance, a call to another method
valid = auth_server.isTokenValid(token)
So there is a chain here, where the mock is called, and the return value is then used to find another attribute that is also called. When mocking, you need to look for that same chain; use the Mock.return_value attribute for this. By default a new mock instance is returned when you call a mock, but you can also set test values.
So to test your code, you'd want to influence what auth_server.isTokenValid(token) returns, and test if your code works correctly. You may also want to assert that the correct URL is passed to the ServerProxy instance.
Create separate tests for different outcomes. Perhaps the token is valid in one case, not valid in another, and you'd want to test both cases:
class Tester(unittest.TestCase):
#patch('xmlrpc.client.ServerProxy')
def test_valid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response for a valid token
mock_auth_server.isTokenValid.return_value = 1
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
#patch('xmlrpc.client.ServerProxy')
def test_invalid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response; now testing for an invalid token instead
mock_auth_server.isTokenValid.return_value = 0
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')

There are many mock attributes to use, and you can change your patch decorator usage a little as follows:
class Tester(unittest.TestCase):
def test_xmlrpclib(self):
with patch('library.xmlrpclib.ServerProxy.isTokenValid') as isTokenValid:
self.assertEqual(isTokenValid.call_count, 0)
# your test code calling xmlrpclib
self.assertEqual(isTokenValid.call_count, 1)
token = isTokenValid.call_args[0] # assume this token is valid
self.assertEqual(isTokenValid.return_value, 1)
You can adjust the code above to satisfy your requirements.

Mock flask.request in python nosetests

I'm writing test cases for code that is called via a route under Flask. I don't want to test the code by setting up a test app and calling a URL that hits the route, I want to call the function directly. To make this work I need to mock flask.request and I can't seem to manage it. Google / stackoverflow searches lead to a lot of answers that show how to set up a test application which again is not what I want to do.
The code would look something like this.
somefile.py
-----------
from flask import request
def method_called_from_route():
data = request.values
# do something with data here
test_somefile.py
----------------
import unittest
import somefile
class SomefileTestCase(unittest.TestCase):
#patch('somefile.request')
def test_method_called_from_route(self, mock_request):
# want to mock the request.values here
I'm having two issues.
(1) Patching the request as I've sketched out above does not work. I get an error similar to "AttributeError: 'Blueprint' object has no attribute 'somefile'"
(2) I don't know how to exactly mock the request object if I could patch it. It doesn't really have a return_value since it isn't a function.
Again I can't find any examples on how to do this so I felt a new question was acceptable.

Try this
test_somefile.py
import unittest
import somefile
import mock
class SomefileTestCase(unittest.TestCase):
def test_method_called_from_route(self):
m = mock.MagicMock()
m.values = "MyData"
with mock.patch("somefile.request", m):
somefile.method_called_from_route()
unittest.main()
somefile.py
from flask import request
def method_called_from_route():
data = request.values
assert(data == "MyData")
This is going to mock the entire request object.
If you want to mock only request.values while keeping all others intact, this would not work.

A few years after the question was asked, but this is how I solved this with python 3.9 (other proposed solutions stopped working with python 3.8 see here). I'm using pytest and pytest-mock, but the idea should be the same across testing frameworks, as long as you are using the native unittest.mock.patch in some capacity (pytest-mock essentially just wraps these methods in an easier to use api). Unfortunately, it does require that you set up a test app, however, you do not need to go through the process of using test_client, and can just invoke the function directly.
This can be easily handled by using the Application Factory Design Pattern, and injecting application config. Then, just use the created app's .test_request_context as a context manager to mock out the request object. using .test_request_context as a context manager, gives everything called within the context access to the request object. Here's an example below.
import pytest
from app import create_app
#pytest.fixture
def request_context():
"""create the app and return the request context as a fixture
so that this process does not need to be repeated in each test
"""
app = create_app('module.with.TestingConfig')
return app.test_request_context
def test_something_that_requires_global_request_object(mocker, request_context):
"""do the test thing"""
with request_context():
# mocker.patch is just pytest-mock's way of using unittest.mock.patch
mock_request = mocker.patch('path.to.where.request.is.used')
# make your mocks and stubs
mock_request.headers = {'content-type': 'application/json'}
mock_request.get_json.return_value = {'some': 'json'}
# now you can do whatever you need, using mock_request, and you do not
# need to remain within the request_context context manager
run_the_function()
mock_request.get_json.assert_called_once()
assert 1 == 1
# etc.
pytest is great because it allows you to easily setup fixtures for your tests as described above, but you could do essentially the same thing with UnitTest's setUp instance methods. Happy to provide an example for the Application Factory design pattern, or more context, if necessary!

with help of Gabrielbertouinataa on this article: https://medium.com/#vladbezden/how-to-mock-flask-request-object-in-python-fdbc249de504:
code:
def print_request_data():
print(flask.request.data)
test:
flask_app = flask.Flask('test_flask_app')
with flask_app.test_request_context() as mock_context:
mock_context.request.data = "request_data"
mock_context.request.path = "request_path"
print_request_data()

Here is an example of how I dealt with it:
test_common.py module
import pytest
import flask
def test_user_name(mocker):
# GIVEN: user is provided in the request.headers
given_user_name = "Some_User"
request_mock = mocker.patch.object(flask, "request")
request_mock.headers.get.return_value = given_user_name
# WHEN: request.header.get method is called
result = common.user_name()
# THEN: user name should be returned
request_mock.headers.get.assert_called_once_with("USERNAME", "Invalid User")
assert result == given_user_name
common.py module
import flask
def user_name():
return flask.request.headers.get("USERNAME", "Invalid User")

What you're trying to do is counterproductive. Following the RFC 2616 a request is:
A request message from a client to a server includes, within the first line of that message, the method to be applied to the resource, the identifier of the resource, and the protocol version in use.
Mocking the Flask request you need to rebuild its structure, what certainly, you will not to want to do!
The best approach should be use something like Flask-Testing or use some recipes like this, and then, test your method.

Python mock patch doesn't work as expected for public method

I'm trying to patch a public method for my flask application but it doesn't seem to work.
Here's my code in mrss.feed_burner
def get_feed(env=os.environ):
return 'something'
And this is how I use it
#app.route("/feed")
def feed():
mrss_feed = get_feed(env=os.environ)
response = make_response(mrss_feed)
response.headers["Content-Type"] = "application/xml"
return response
And this is my test which it's not parsing.
def test_feed(self):
with patch('mrss.feed_burner.get_feed', new=lambda: '<xml></xml>'):
response = self.app.get('/feed')
self.assertEquals('<xml></xml>', response.data)

I believe your problem is that you're not patching in the right namespace. See where_to_patch documentation for unittest.mock.patch.
Essentially, you're patching the definition of get_feed() in mrss.feed_burner but your view handler feed() already has a reference to the original mrss.feed_burner.get_feed(). To solve this problem, you need to patch the reference in your view file.
Based on your usage of get_feed in your view function, I assume you're importing get_feed like so
view_file.py
from mrss.feed_burner import get_feed
If so, you should be patching view_file.get_feed like so:
def test_feed(self):
with patch('view_file.get_feed', new=lambda: '<xml></xml>'):
...

Passing a variable to a JSON web service

I am accessing the class from the code api_service.py, which can be found here. When I call the first function, I have no problem, because no variables are passed:
from api_service import ApiService
import json
def main():
api_key = *removed*
access_token = *removed*
calling = ApiService(api_key,access_token)
survey_list = calling.get_survey_list()
But when I use the same type of routine as above to call a function from ApiService that requires a variable, I'm told that I should pass an object.
survey_details = calling.get_survey_details("1234")
survey_details = json.loads(json.dumps(survey_details))
print survey_details
The specific error message:
{u'status': 3, u'errmsg': u"Value '1234' for field '_data' is not of type object"}
Details for the get_survey_details aspect of the SurveyMonkey API are here, although I think a python-guru can solve this without knowing about the API.

This is a javascript/json object:
{field:'value'}
You have passed a string which, doesn't count as an "object" for these purposes.
Note that the error message is being generated by the service you are accessing. This question would be better directed to the creator of the service.

How do I apply a monkeypatch to GAE?

Can you tell me how I apply this patch to google app engine as in where to put it? Thank you
def _user_init(self, email=None, _auth_domain=None,
_user_id=None, federated_identity=None, federated_provider=None):
if not _auth_domain:
_auth_domain = os.environ.get('AUTH_DOMAIN')
assert _auth_domain
if email is None and federated_identity is None:
email = os.environ.get('USER_EMAIL', email)
_user_id = os.environ.get('USER_ID', _user_id)
federated_identity = os.environ.get('FEDERATED_IDENTITY',
federated_identity)
federated_provider = os.environ.get('FEDERATED_PROVIDER',
federated_provider)
if not email and not federated_identity:
raise UserNotFoundError
self.__email = email
self.__federated_identity = federated_identity
self.__federated_provider = federated_provider
self.__auth_domain = _auth_domain
self.__user_id = _user_id or None
users.User.__init__ = _user_init

Just use it as-is: Put that code in a module that gets imported before you use the relevant User module or datastore functionality. I included the relevant line to patch the code (the last line) with the patch itself.

Overriding the constructor like this is not safe. If the internal implementation of the Users API changes in production, your application could break without warning.
What are you trying to accomplish here? I don't see any custom logic; it looks like you've just copied the constructor from the SDK verbatim. If you need to add custom logic, try subclassing UserProperty and/or wrapping the users API calls instead.

I think, this belongs to some application as a grep within the appengine sdk, for 'federated_identity' does not result any clues. BTW, you should be doing the same. Grep (or WinGrep) for terms like 'federated' to see if this partial patch can be applied to any source.
Thanks for the updated link. The patch can be added to the file google/appengine/api/users.py
You might just need to add the last line: users.User.__init__ = _user_init
I could figure this out after checking the latest code in the svn.