Develop Reference

Ammend code interacting with Google translation service to return translated data in a list rather than priniting them

I am trying to find a way to interact with the Google translation service. I would like to be able to pass to the service a list with strings in a source language and get back a list with values strings in the target language.
Working from Python, I have found some relevant code here:
https://github.com/google/google-api-python-client/blob/master/samples/translate/main.py
I tried the code and it works. See below:
from __future__ import print_function
__author__ = 'jcgregorio#google.com (Joe Gregorio)'
from googleapiclient.discovery import build
def main():
# Build a service object for interacting with the API. Visit
# the Google APIs Console <http://code.google.com/apis/console>
# to get an API key for your own application.
service = build('translate', 'v2',
developerKey='AIzaSyCT4Ds5GuytOZnxGav492RAUvN508daobAVo')
print(service.translations().list(
source='el',
target='en',
q=['Είμαι πολύ ευτυχής που σας βλέπω σήμερα εδώ', 'Η κόρη μου θα χαρεί πολύ να σας δει']
).execute())
if __name__ == '__main__':
main()
When run this code printed the following output:
{'translations': [{'translatedText': 'I am very happy to see you here today'}, {'translatedText': 'My daughter will be very pleased to see you'}]}
However as I said, the code above prints the returned results to the screen. What I would like instead would be to store the translated text into an object that eventually I would make it a pandas object.
So I tried to ammend it in this way - see below:
from __future__ import print_function
__author__ = 'jcgregorio#google.com (Joe Gregorio)'
from googleapiclient.discovery import build
def main():
# Build a service object for interacting with the API. Visit
# the Google APIs Console <http://code.google.com/apis/console>
# to get an API key for your own application.
service = build('translate', 'v2',
developerKey='AIzaSyCT4Ds5GuytOZnxGav492RAUvN508daobAVo')
result = service.translations().list(
source='el',
target='en',
q=['Είμαι πολύ ευτυχής που σας βλέπω σήμερα εδώ', 'Η κόρη μου θα χαρεί πολύ να σας δει']
).execute()
return result
if __name__ == '__main__':
main()
However when I tried to display of the object "result" I got the following exception:
NameError: name 'result' is not defined

How are you trying to "display result"?
It looks like your code successfully creates result, but then main() returns it... to where? How are you calling main and using what it returns?
You can't access local function variables (e.g. result) outside the function that defines them.
Likely, what you want is for this code to be a function that returns the result to another function/program, that then uses the result returned.
E.g.
def your_main_program():
result = the_function_you_currently_have() #currently called main()
# do something with the result

mock xmlrpc.client method python

I am in the process of learning unit testing, however I am struggling to understand how to mock functions for unit testing. I have reviewed many how-to's and examples but the concept is not transferring enough for me to use it on my code. I am hoping getting this to work on a actual code example I have will help.
In this case I am trying to mock isTokenValid.
Here is example code of what I want to mock.
<in library file>
import xmlrpc.client as xmlrpclib
class Library(object):
def function:
#...
AuthURL = 'https://example.com/xmlrpc/Auth'
auth_server = xmlrpclib.ServerProxy(AuthURL)
socket.setdefaulttimeout(20)
try:
if pull == 0:
valid = auth_server.isTokenValid(token)
#...
in my unit test file I have
import library
class Tester(unittest.TestCase):
#patch('library.xmlrpclib.ServerProxy')
def test_xmlrpclib(self, fake_xmlrpclib):
assert 'something'
How would I mock the code listed in 'function'? Token can be any number as a string and valid would be a int(1)

First of all, you can and should mock xmlrpc.client.ServerProxy; your library imports xmlrpc.client as a new name, but it is still the same module object so both xmlrpclib.ServerProxy in your library and xmlrpc.client.ServerProxy lead to the same object.
Next, look at how the object is used, and look for calls, the (..) syntax. Your library uses the server proxy like this:
# a call to create an instance
auth_server = xmlrpclib.ServerProxy(AuthURL)
# on the instance, a call to another method
valid = auth_server.isTokenValid(token)
So there is a chain here, where the mock is called, and the return value is then used to find another attribute that is also called. When mocking, you need to look for that same chain; use the Mock.return_value attribute for this. By default a new mock instance is returned when you call a mock, but you can also set test values.
So to test your code, you'd want to influence what auth_server.isTokenValid(token) returns, and test if your code works correctly. You may also want to assert that the correct URL is passed to the ServerProxy instance.
Create separate tests for different outcomes. Perhaps the token is valid in one case, not valid in another, and you'd want to test both cases:
class Tester(unittest.TestCase):
#patch('xmlrpc.client.ServerProxy')
def test_valid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response for a valid token
mock_auth_server.isTokenValid.return_value = 1
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
#patch('xmlrpc.client.ServerProxy')
def test_invalid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response; now testing for an invalid token instead
mock_auth_server.isTokenValid.return_value = 0
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')

There are many mock attributes to use, and you can change your patch decorator usage a little as follows:
class Tester(unittest.TestCase):
def test_xmlrpclib(self):
with patch('library.xmlrpclib.ServerProxy.isTokenValid') as isTokenValid:
self.assertEqual(isTokenValid.call_count, 0)
# your test code calling xmlrpclib
self.assertEqual(isTokenValid.call_count, 1)
token = isTokenValid.call_args[0] # assume this token is valid
self.assertEqual(isTokenValid.return_value, 1)
You can adjust the code above to satisfy your requirements.

Django, store function in database

I'm trying to find a way to store a function in a Django Model.
My use case is an emailer system with planned and conditional sending.
"send this email tomorrow if John has not logged in".
Ideally I would like to create emails with a test function field.
def my_test(instance):
if (now() - instance.user.last_login) > timedelta(days=1):
return False
return True
Email(send_later_date=now() + timedelta(days=1),
conditional=my_test)
Here is what I'm thinking about
import importlib
Email(send_later_date=now() + timedelta(days=1),
conditional='my_app.views.my_test')
class Email(models.Model):
send_later_date = models.DateTimeField(null=True, blank=True)
conditional = models.TextField()
def send_email(self):
if self.send_later_date < now():
if not self.conditional:
function_send_email()
else:
function_string = self.conditional
mod_name, func_name = function_string.rsplit('.',1)
mod = importlib.import_module(mod_name)
func = getattr(mod, func_name)
if func():
function_send_email()
How would you do that? I was thinking of storing the function name in a Text field. How would you run it once needed? ImportLib seems interesting...

Storing the function name is a valid aproach, since it is actually the one used by the pickle module. This has the benefit (in your case) that if you update the code of a function, it will automatically apply to existing e-mails (but be careful with backward-compatibility).
For convenience and safety (especially if the function name could come from user inputs), you may want to keep all such functions in the same module, and store only the function name in DB, not the full import path. So you could simply import the function repository and read it with getattr. I imagine you will also want to give parameters to these functions. If you don't want to restrict yourself to a given number / order / type of arguments, you could store in DB a dictionary as a JSON string, and expand it with the ** operator.
import my_app.func_store as store
import json
func = getattr(store, self.conditional)
params = json.loads(self.conditional_params)
if func(**params):
function_send_email()

Python: import of Modules with mutual dependency

I have two files rest_api.py and Contact.py. Contact is similar to a domain object (contains Contact class), while rest_api has functions for setting up the application.
In rest_api I have the following lines:
from Contact import Contact
...
client = MongoClient('localhost',27017)
collection = client.crypto_database.test_collection
def dbcollection(){
return collection
}
...
api.add_resource(Contact,'/contact/<string:contact_id>')
In Contact I try to do the following:
from rest_api import dbcollection
class Contact(Resource):
def get(self,contact_id):
result = {}
result['data'] = dbcollection.find_one({'contact_id':contact_id})
result['code'] = 200 if result['data'] else 404
return make_response(dumps(result), result['code'],{"Content-type": "application/json"})
This fails with the following error:
ImportError: cannot import name Contact
What is the correct way of importing contact, so that it can also use variables/functions from rest_api?
p.s If I move the collection code to a different file, and import that file instead things work, but I assume there is some other way..

This is a circular import dependency, which cannot be solved as such. The problem is, that importing a python module really runs the code, which has to follow some order, one of the modules has to go first.
I would say that having the support code in a different file would be the proper way to go.
In this case however, the dbcollection is not actually needed at import time. Thus you can solve this by removing the import from the module level, into the get function. For example
class Contact(Resource):
def get(self,contact_id):
from rest_api import dbcollection
result = {}
result['data'] = dbcollection.find_one({'contact_id':contact_id})
result['code'] = 200 if result['data'] else 404
return make_response(dumps(result), result['code'],{"Content-type": "application/json"})
A similar approach would be the following:
import rest_api
class Contact(Resource):
def get(self,contact_id):
result = {}
result['data'] = rest_api.dbcollection.find_one({'contact_id':contact_id})
result['code'] = 200 if result['data'] else 404
return make_response(dumps(result), result['code'],{"Content-type": "application/json"})
This should work, as python does some effort to resolve circular import dependencies: When it start importing a module, it creates an empty module dict for that one. Then when it finds a nested import, it proceeds with that one. If that in turn imports a module that is already in the import process it just skips it. Thus at the time Contact.py is being loaded, the import rest_api just takes the module dict that is already there. Since it does not contain dbcollection yet, from rest_api import dbcollection fails. A simple import rest_api does work however, since it's member is only addressed after Contact.py finishes importing (unless you call Contact.get at module level from within).

Assuming you haven't a clue which modules are going to import which ever other ones, you can track that yourself and NOT do the import.
In your __init__.py, define these --
__module_imports__ = {}
def requires_module(name):
return name not in __module_imports__
def importing_module(name):
__module_imports__[name] = True
Then, at the top of each class file, where you define your classes add the following to this_module.py:
from my_modules import requires_module, importing_module
importing('ThisModule')
if requires_module('ThatModule')
from my_modules.that_module import ThatModule
class ThisModule:
""" Real Stuff Goes Here """
pass
and this to that_module.py:
from my_modules import requires_module, importing_module
importing('ThatModule')
if requires_module('ThisModule')
from my_modules.this_module import ThisModule
class ThatModule:
""" Real Stuff Goes Here """
pass
now you get your imports regardless of which gets imported first or whatever.

Passing a variable to a JSON web service

I am accessing the class from the code api_service.py, which can be found here. When I call the first function, I have no problem, because no variables are passed:
from api_service import ApiService
import json
def main():
api_key = *removed*
access_token = *removed*
calling = ApiService(api_key,access_token)
survey_list = calling.get_survey_list()
But when I use the same type of routine as above to call a function from ApiService that requires a variable, I'm told that I should pass an object.
survey_details = calling.get_survey_details("1234")
survey_details = json.loads(json.dumps(survey_details))
print survey_details
The specific error message:
{u'status': 3, u'errmsg': u"Value '1234' for field '_data' is not of type object"}
Details for the get_survey_details aspect of the SurveyMonkey API are here, although I think a python-guru can solve this without knowing about the API.

This is a javascript/json object:
{field:'value'}
You have passed a string which, doesn't count as an "object" for these purposes.
Note that the error message is being generated by the service you are accessing. This question would be better directed to the creator of the service.