I am accessing the class from the code api_service.py, which can be found here. When I call the first function, I have no problem, because no variables are passed:
from api_service import ApiService
import json
def main():
api_key = *removed*
access_token = *removed*
calling = ApiService(api_key,access_token)
survey_list = calling.get_survey_list()
But when I use the same type of routine as above to call a function from ApiService that requires a variable, I'm told that I should pass an object.
survey_details = calling.get_survey_details("1234")
survey_details = json.loads(json.dumps(survey_details))
print survey_details
The specific error message:
{u'status': 3, u'errmsg': u"Value '1234' for field '_data' is not of type object"}
Details for the get_survey_details aspect of the SurveyMonkey API are here, although I think a python-guru can solve this without knowing about the API.
This is a javascript/json object:
{field:'value'}
You have passed a string which, doesn't count as an "object" for these purposes.
Note that the error message is being generated by the service you are accessing. This question would be better directed to the creator of the service.
Related
I am in the process of learning unit testing, however I am struggling to understand how to mock functions for unit testing. I have reviewed many how-to's and examples but the concept is not transferring enough for me to use it on my code. I am hoping getting this to work on a actual code example I have will help.
In this case I am trying to mock isTokenValid.
Here is example code of what I want to mock.
<in library file>
import xmlrpc.client as xmlrpclib
class Library(object):
def function:
#...
AuthURL = 'https://example.com/xmlrpc/Auth'
auth_server = xmlrpclib.ServerProxy(AuthURL)
socket.setdefaulttimeout(20)
try:
if pull == 0:
valid = auth_server.isTokenValid(token)
#...
in my unit test file I have
import library
class Tester(unittest.TestCase):
#patch('library.xmlrpclib.ServerProxy')
def test_xmlrpclib(self, fake_xmlrpclib):
assert 'something'
How would I mock the code listed in 'function'? Token can be any number as a string and valid would be a int(1)
First of all, you can and should mock xmlrpc.client.ServerProxy; your library imports xmlrpc.client as a new name, but it is still the same module object so both xmlrpclib.ServerProxy in your library and xmlrpc.client.ServerProxy lead to the same object.
Next, look at how the object is used, and look for calls, the (..) syntax. Your library uses the server proxy like this:
# a call to create an instance
auth_server = xmlrpclib.ServerProxy(AuthURL)
# on the instance, a call to another method
valid = auth_server.isTokenValid(token)
So there is a chain here, where the mock is called, and the return value is then used to find another attribute that is also called. When mocking, you need to look for that same chain; use the Mock.return_value attribute for this. By default a new mock instance is returned when you call a mock, but you can also set test values.
So to test your code, you'd want to influence what auth_server.isTokenValid(token) returns, and test if your code works correctly. You may also want to assert that the correct URL is passed to the ServerProxy instance.
Create separate tests for different outcomes. Perhaps the token is valid in one case, not valid in another, and you'd want to test both cases:
class Tester(unittest.TestCase):
#patch('xmlrpc.client.ServerProxy')
def test_valid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response for a valid token
mock_auth_server.isTokenValid.return_value = 1
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
#patch('xmlrpc.client.ServerProxy')
def test_invalid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response; now testing for an invalid token instead
mock_auth_server.isTokenValid.return_value = 0
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
There are many mock attributes to use, and you can change your patch decorator usage a little as follows:
class Tester(unittest.TestCase):
def test_xmlrpclib(self):
with patch('library.xmlrpclib.ServerProxy.isTokenValid') as isTokenValid:
self.assertEqual(isTokenValid.call_count, 0)
# your test code calling xmlrpclib
self.assertEqual(isTokenValid.call_count, 1)
token = isTokenValid.call_args[0] # assume this token is valid
self.assertEqual(isTokenValid.return_value, 1)
You can adjust the code above to satisfy your requirements.
I have a source file with following code which uses jsonrpclib module of python
server = jsonrpclib.Server(url)
try:
res = server.user_account_create(
EmailAddress=self.Email,
FirstName=self.FirstName,
LastName=self.LastName,
Password=self.Password
)
self.UserId = res['UserId']
except jsonrpclib.jsonrpc.ProtocolError:
er = json.loads(jsonrpclib.history.response)
jsonrpclib.history.clear()
return False
But when I looked through the source of jsonrpclib I haven't seen the function user_account_create.
Can somebody point what I missed ?
I want to check whether this function makes request to the relative url /accounts because that is what explained in the document.
user_account_create should be defined in the server side.
jsonrpclib.Server's __getattr__ will handle it: user_account_create is not defined in the jsonrpclib.Server; which cause the __getattr__ to be called; xmlrpclib._Method which represent rpc method is created dynamically.
I try to make a non blocking api calls for OpenWeatherMap, but my problem is:
When i was doing tests on the file, and run it, the global api was taking effect, but when importing the function, global dont work anymore, and api dident change: api = ""?
Just after declaring the function i put global api, and then when I use print 'The API link is: ' + api I get the exact api, but global dident took effect!
Here is the code: https://github.com/abdelouahabb/tornadowm/blob/master/tornadowm.py#L62
What am I doing wrong?
When I import the file:
from tornadowm import *
forecast('daily', q='london', lang='fr')
The API link is: http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london
api
Out[5]: ''
When executing the file instead of importing it:
runfile('C:/Python27/Lib/site-packages/tornadowm.py', wdir='C:/Python27/Lib/site-packages')
forecast('daily', q='london', lang='fr')
The API link is: http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london
api
Out[8]: 'http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london'
Edit: here is the code, if the Git got updated:
from tornado.httpclient import AsyncHTTPClient
import json
import xml.etree.ElementTree as ET
http_client = AsyncHTTPClient()
url = ''
response = ''
args = []
link = 'http://api.openweathermap.org/data/2.5/'
api = ''
result = {}
way = ''
def forecast(way, **kwargs):
global api
if way in ('weather', 'forecast', 'daily', 'find'):
if way == 'daily':
way = 'forecast/daily?'
else:
way += '?'
for i, j in kwargs.iteritems():
args.append('&{0}={1}'.format(i, j))
a = ''.join(set(args))
api = (link + way + a.replace(' ', '+')).replace('?&', '?')
print 'The API link is: ' + api
def handle_request(resp):
global response
if resp.error:
print "Error:", resp.error
else:
response = resp.body
http_client.fetch(api, handle_request)
else:
print "please put a way: 'weather', 'forecast', 'daily', 'find' "
def get_result():
global result
if response.startswith('{'):
print 'the result is JSON, stored in the variable result'
result = json.loads(response)
elif response.startswith('<'):
print 'the result is XML, parse the result variable to work on the nodes,'
print 'or, use response to see the raw result'
result = ET.fromstring(response)
else:
print '''Sorry, no valid response, or you used a parameter that is not compatible with the way!\n please check http://www.openweathermap.com/api for more informations''
It's the side effect of using global.
When you do from tornadowm import * your forecast() function is, we could say metaphorically, "on its own" and is not "hard-linked" to your global space anymore.
Why? Because any effect you make on your global api will "end" with your function, and the definition of api = "" in your global space will take precedence.
Also, as a side note, it's not considered a good practice to use from something import *. You should do from tornadowm import forecast or even better, import tornadown and then use tornadowm.forecast().
OR
Even better, I just noticed your forecast() function doesn't return anything. Which technically makes it not a function anymore, but a procedure (a procedure is like a function but it returns nothing, it just "does" stuff).
Instead of using a global, you should define api in this function and then return api from it. Like this:
def forecast(blablabla):
api = "something"
blablabla
return api
And then
import tornadowm
api = tornadown.forecast(something)
And you're done.
Globals are global only to the module they're defined in. So, normally, you would expect tornadowm.api to be changed when you call forecast, but not api in some other namespace.
The import * is contributing to your understanding of the problem. This imports api (among other names) into the importing namespace. This means that api and tornadowm.api initially point to the same object. But these two names are not linked in any way, and so calling forecast() changes only tornadowm.api and now the two names point to different objects.
To avoid this, don't use import *. It is bad practice anyway and this is just one of the reasons. Instead, import tornadowm and access the variable in the importing module as tornadowm.api.
I'm afraid this is because global is coupled within module, by the time you from tornadowm import * you have imported the api name, but the global api won't take any effects within another module.
I want to unit test my class, which is in another file named client_blogger.py.
My unit test file, is in the same directory. All of my other unit tests work, except when I try to mock one of my own methods.
## unit_test_client_blogger.py
import mock
import json
from client_blogger import BloggerClient, requests
Class TestProperties():
#pytest.fixture
def blog(self):
return BloggerClient(api_key='123', url='http://example.com')
#mock.patch('client_blogger._jload')
#mock.patch('client_blogger._send_request')
def test_gets_blog_info(self, mock_send, mock_jload):
""" Get valid blog info from API response. """
valid_blog_info = 'some valid json api response here'
parsed_response = json.loads(valid_blog_info)
correct_blog_id = '7488272653173849119'
mock_jload.return_value = valid_blog_info
id = self.blog().get_blog_info(parsed_response)
assert id == correct_blog_id
And here is the client_blogger.py file contents:
# client_blogger.py
import requests, json
class BloggerClient(object):
""" Client interface for Blogger API. """
def __init__(self, key, url):
# removed some code here for brevity
def _send_request(self, api_request):
""" Sends an HTTP get request to Blogger API.
Returns HTTP response in text format. """
# snip
def _jload(self, api_response):
""" Accepts text API response. Returns JSON encoded response. """
# snip
def get_blog_info(self):
""" Makes an API request. Returns Blog item information. """
request = '{b}/blogs/byurl?url={u}&key={k}'.format(b=self.base, u=self.url, k=self.key)
txt_response = self.send_request(request)
response = self._jload(txt_response)
return response['id']
I want to mock out self.send_request() and self._jload() method calls in the above method.
But Mock module complains: ImportError: No module named client_blogger.
The error must lie here:
#mock.patch('client_blogger._jload')
#mock.patch('client_blogger._send_request')
I've tried many variations in order to get mock.patch to find my module or class. But none of them have worked.
I've tried the following:
#mock.patch('client_blogger.BloggerClient._jload')
#mock.patch('BloggerClient._jload')
#mock.patch('._jload')
None of those work. Any idea how to mock.patch a method from my own module?
(It seems strange, because I can mock.patch other modules, just not my own :-s)
You want this:
#mock.patch('client_blogger.BloggerClient._jload')
#mock.patch('client_blogger.BloggerClient._send_request')
def test_gets_blog_info(self, mock_send, mock_jload):
""" Get valid blog info from API response. """
valid_blog_info = 'some valid json api response here'
parsed_response = json.loads(valid_blog_info)
correct_blog_id = '7488272653173849119'
mock_jload.return_value = valid_blog_info
id = self.blog().get_blog_info(parsed_response)
assert id == correct_blog_id
The BloggerClient implementation is coming from the client_blogger module, so you need to patch client_blogger.BloggerClient. You list that as one of the things you tried that doesn't work, but I just tried it, and it works fine for me. What issue did you have when you tried that?
You need to include the file name of the class in the path, before the object name.
e.g. if I have a method named foo in a class named Event in tools/event.py the path will be as follows:
patch("tools.event.Event.foo", boo)
For python3, the format is as follows:
from unittest.mock import patch
#patch('client_blogger.BloggerClient._jload')
.
.
.
Docs: https://docs.python.org/3/library/unittest.mock.html#patch
This is very, very important:
patch() is straightforward to use. The key is to do the patching in the right namespace. See the section where to patch.
I've been trying to unpickle some dictionaries from the database. I've reverted to using the marshal module, but was still wondering why pickle is having such a difficult time unserializing some data. Here is a command line python session showing essentially what I am trying to do:
>>> a = {'service': 'amazon', 'protocol': 'stream', 'key': 'lajdfoau09424jojf.flv'}
>>> import pickle; import base64
>>> pickled = base64.b64encode(pickle.dumps(a))
>>> pickled
'KGRwMApTJ3Byb3RvY29sJwpwMQpTJ3N0cmVhbScKcDIKc1Mna2V5JwpwMwpTJ2xhamRmb2F1MDk0MjRqb2pmLmZsdicKcDQKc1Mnc2VydmljZScKcDUKUydhbWF6b24nCnA2CnMu'
>>> unpickled = pickle.loads(base64.b64decode(pickled))
>>> unpickled
{'protocol': 'stream', 'service': 'amazon', 'key': 'lajdfoau09424jojf.flv'}
>>> unpickled['service']
'amazon'
This works all fine, but when I try this inside of a factory method for a class, it seems like the pickle.loads part errors out. The strings I am trying to load are pickled the same way as above. I've even tried copying the exact string that is pickled in the command line session above and just trying to unpickle that, but with no success. Here is the code for this latter attempt:
class Resource:
_service = 'unknown'
_protocol = 'unknown'
_key = 'unknown'
'''
Factory method that creates an appropriate instance of one of Resource’s subclasses based on
the type of data provided (the data being a serialized dictionary with at least the keys 'service',
'protocol', and 'key').
#param resource_data (string) -- the data used to create the new Resource instance.
'''
#staticmethod
def resource_factory(resource_data):
# Unpack the raw resource data and then create the appropriate Resource instance and return.
resource_data = "KGRwMApTJ3Byb3RvY29sJwpwMQpTJ3N0cmVhbScKcDIKc1Mna2V5JwpwMwpTJ2xhamRmb2F1MDk0MjRqb2pmLmZsdicKcDQKc1Mnc2VydmljZScKcDUKUydhbWF6b24nCnA2CnMu" #hack to just see if we can unpickle this string
logging.debug("Creating resource: " + resource_data)
unencoded = base64.b64decode(resource_data)
logging.debug("Unencoded is: " + unencoded)
unpacked = pickle.loads(unencoded)
logging.debug("Unpacked: " + unpacked)
service = unpacked['service']
protocol = unpacked['protocol']
key = unpacked['key']
if (service == 'amazon'):
return AmazonResource(service=service, protocol=protocol, key=key)
elif (service == 'fs'):
return FSResource(service=service, protocol=protocol, key=key)
Your code works. How are you testing it?
import logging
import base64
import pickle
class Resource:
#staticmethod
def resource_factory(resource_data):
resource_data = "KGRwMApTJ3Byb3RvY29sJwpwMQpTJ3N0cmVhbScKcDIKc1Mna2V5JwpwMwpTJ2xhamRmb2F1MDk0MjRqb2pmLmZsdicKcDQKc1Mnc2VydmljZScKcDUKUydhbWF6b24nCnA2CnMu" #hack to just see if we can unpickle this string
# logging.debug("Creating resource: " + resource_data)
unencoded = base64.b64decode(resource_data)
# logging.debug("Unencoded is: " + unencoded)
unpacked = pickle.loads(unencoded)
logging.debug("Unpacked: " + repr(unpacked))
service = unpacked['service']
protocol = unpacked['protocol']
key = unpacked['key']
logging.basicConfig(level=logging.DEBUG)
Resource.resource_factory('')
yields
# DEBUG:root:Unpacked: {'protocol': 'stream', 'service': 'amazon', 'key': 'lajdfoau09424jojf.flv'}
I was able to solve this after making some simplifications and then debugging in django. The main issue was that there were some errors in the Resource class itself that were preventing the correct completion of the resource_factory method. First, I was trying to concatenate a string and dictionary, which was throwing an error. I also had some errors elsewhere in the class where I was referring to the instance variables _service, _protocol, and key withouth the '' (typos).
In any case, the interesting thing was that when I used this code within Django's custom field infrastructure, the errors would get caught and I did not see any actual message indicating the problem. The debug statements suggested it was a problem with loads, but actually it was a problem with the debug statement itself and some code that came later. When I tried to implement this behavior using model properties instead of custom model fields for the data I was saving, the errors actually got printed out correctly and I was able to quickly debug.