Develop Reference

Convert string as module

is there a way to turn a string to an module object? i want to to do a function call after doing an API request but I cant convert the string I got from the request to a module object
I have seen some code that could actually do it however it is done by importing the module.
import controller.mipay.controller as mipay
#api.route('/redirect')
class Request(Resource):
parser = RequestChecker()
def post(self):
req = self.parser.parse_args() # <---- JSON/Dictionary
#method_to_call = getattr(mipay, req['PaymentEndpoint'])
method_to_call = getattr(req['PaymentMethod'], req['PaymentEndpoint'])
return method_to_call()
is there anyway to convert the string to a module object or anything I can do to the string so I could call my function

Using __import__ may be the solution:
module = __import__("module.path")
See importlib — The implementation of import

I assume req['PaymentMethod'] is the module and req['PaymentEndpoint'] is the method. What you can do is use importlib.
#import controller.mipay.controller as mipay
import importlib
my_mapping = {'mipay':'controller.mipay.controller'}
class Request(Resource):
parser = RequestChecker()
def post(self):
req = self.parser.parse_args() # <---- JSON/Dictionary
# method_to_call = getattr(mipay, req['PaymentEndpoint'])
mipay= importlib.import_module(my_mapping[req['PaymentMethod']])
method_to_call = getattr(mymodule, req['PaymentEndpoint'])
return method_to_call()

Python Json Config 'Extended Interpolation'

I am currently using the Python library configparser:
from configparser import ConfigParser, ExtendedInterpolation
I find the ExtendedInterpolation very useful because it avoids the risk of having to reenter constants in multiple places.
I now have a requirement to use a Json document as the basis of the configuration as it provides more structure.
import json
from collections import OrderedDict
def get_json_config(file):
"""Load Json into OrderedDict from file"""
with open(file) as json_data:
d = json.load(json_data, object_pairs_hook=OrderedDict)
return d
Does anyone have any suggestions as to the best way to implement configparser style ExtendedInterpolation?
For example if a node in the Json contains the value ${home_dir}/lumberjack this would copy root node home_dir and take value 'lumberjack'?

Try to use string.Template. But I'm not sure whether it's your need. There is one package can do this may be. Bellow is what i should do.
config.json
{
"home_dir": "/home/joey",
"class_path": "/user/local/bin",
"dir_one": "${home_dir}/dir_one",
"dir_two": "${home_dir}/dir_two",
"sep_path_list": [
"${class_path}/python",
"${class_path}/ruby",
"${class_path}/php"
]
}
python code:
import json
from string import Template
with open("config.json", "r") as config_file:
config_content = config_file.read()
config_template = Template(config_content)
mid_json = json.loads(config_content)
config = config_template.safe_substitute(mid_json)
print config
This can substitute the defined key in json file.

Came in very useful; however, I found "config" is a unicode string; I resolved with:
# proper code
return json.loads(config)

How to import own module for mocking? (import error: no module named my_module!)

I want to unit test my class, which is in another file named client_blogger.py.
My unit test file, is in the same directory. All of my other unit tests work, except when I try to mock one of my own methods.
## unit_test_client_blogger.py
import mock
import json
from client_blogger import BloggerClient, requests
Class TestProperties():
#pytest.fixture
def blog(self):
return BloggerClient(api_key='123', url='http://example.com')
#mock.patch('client_blogger._jload')
#mock.patch('client_blogger._send_request')
def test_gets_blog_info(self, mock_send, mock_jload):
""" Get valid blog info from API response. """
valid_blog_info = 'some valid json api response here'
parsed_response = json.loads(valid_blog_info)
correct_blog_id = '7488272653173849119'
mock_jload.return_value = valid_blog_info
id = self.blog().get_blog_info(parsed_response)
assert id == correct_blog_id
And here is the client_blogger.py file contents:
# client_blogger.py
import requests, json
class BloggerClient(object):
""" Client interface for Blogger API. """
def __init__(self, key, url):
# removed some code here for brevity
def _send_request(self, api_request):
""" Sends an HTTP get request to Blogger API.
Returns HTTP response in text format. """
# snip
def _jload(self, api_response):
""" Accepts text API response. Returns JSON encoded response. """
# snip
def get_blog_info(self):
""" Makes an API request. Returns Blog item information. """
request = '{b}/blogs/byurl?url={u}&key={k}'.format(b=self.base, u=self.url, k=self.key)
txt_response = self.send_request(request)
response = self._jload(txt_response)
return response['id']
I want to mock out self.send_request() and self._jload() method calls in the above method.
But Mock module complains: ImportError: No module named client_blogger.
The error must lie here:
#mock.patch('client_blogger._jload')
#mock.patch('client_blogger._send_request')
I've tried many variations in order to get mock.patch to find my module or class. But none of them have worked.
I've tried the following:
#mock.patch('client_blogger.BloggerClient._jload')
#mock.patch('BloggerClient._jload')
#mock.patch('._jload')
None of those work. Any idea how to mock.patch a method from my own module?
(It seems strange, because I can mock.patch other modules, just not my own :-s)

You want this:
#mock.patch('client_blogger.BloggerClient._jload')
#mock.patch('client_blogger.BloggerClient._send_request')
def test_gets_blog_info(self, mock_send, mock_jload):
""" Get valid blog info from API response. """
valid_blog_info = 'some valid json api response here'
parsed_response = json.loads(valid_blog_info)
correct_blog_id = '7488272653173849119'
mock_jload.return_value = valid_blog_info
id = self.blog().get_blog_info(parsed_response)
assert id == correct_blog_id
The BloggerClient implementation is coming from the client_blogger module, so you need to patch client_blogger.BloggerClient. You list that as one of the things you tried that doesn't work, but I just tried it, and it works fine for me. What issue did you have when you tried that?

You need to include the file name of the class in the path, before the object name.
e.g. if I have a method named foo in a class named Event in tools/event.py the path will be as follows:
patch("tools.event.Event.foo", boo)

For python3, the format is as follows:
from unittest.mock import patch
#patch('client_blogger.BloggerClient._jload')
.
.
.
Docs: https://docs.python.org/3/library/unittest.mock.html#patch
This is very, very important:
patch() is straightforward to use. The key is to do the patching in the right namespace. See the section where to patch.

Python config parser that supports section inheritance?

I'm looking for an ini style config parser in Python that supports section inheritance similar to what Zend_Config_Ini does in PHP.
Does such a module exist or will I need to roll my own?

Python's ConfigParser can load multiple files. Files read later on can
override settings from the first file.
For example, my application has database settings in its internal default
configuration file:
[database]
server = 127.0.0.1
port = 1234
...
I override these on a different server with a "environment.ini" file containing
the same section but different values:
[database]
server = 192.168.0.12
port = 2345
...
In Python:
import os
from ConfigParser import ConfigParser
dbconf = ConfigParser()
dbconf.readfp(open('default.ini'))
if os.path.exists('environment.ini'):
dbconf.readfp(open('environment.ini'))
dbconf.get('database', 'server') # Returns 192.168.0.12

Here's what I used. extended_get method is what you need - it supports hierarchical sections.
import re
import io
import ConfigParser
class ZendConfigParser(ConfigParser.ConfigParser):
def extended_get(self, section, key):
if self.has_option(section, key):
return self.get(section, key)
else:
orig_section, parent_section = self._get_orig_section(section)
if orig_section != None:
if self.has_option(orig_section,key):
return self.get(orig_section,key)
else:
return self.extended_get(parent_section,key)
else:
return None
def _get_orig_section(self, zend_section):
orig_section = None
parent_section = None
for section in self.sections():
if re.search(r'^[ \t]*' + zend_section + '\\b', section) != None:
orig_section = section
#look for a parent section
match = re.match(r'\w+[ \t]*:[ \t]*(\w+)$', section)
if match != None:
parent_section = match.groups()[0]
break
return (orig_section, parent_section)
config = ZendConfigParser()
config.read(file)
print(config.extended_get('production', 'database.params.host'))

I also did not find any ready solution. To solve it, I adapted the get function of ConfigParser to search in the child section and afterwards in the parent section:
config = SafeConfigParser()
config.read(filenames)
required_environment = "mysection"
# determine fallback requirement in case parameter is not found in required environment
fallback_environment = "default"
# loop through all sections of config files
for environment in config.sections():
# check whether we find an inheritance based on the required section
if re.search(required_environment + " *: *\w+", environment):
# found inheritance, take parent as fallback section
fallback_environment = re.sub(required_environment + r" : (\w+)", r"\1", environment)
# take this name as requested section
required_environment = environment
# override get method
_config_parser_get = config.get
def myConfigParserGet(id):
# check different sections for desired value
if config.has_option(required_environment, id):
return _config_parser_get(required_environment, id)
else:
return _config_parser_get(fallback_environment, id)
config.get = myConfigParserGet
Restrictions:
only read-only access to config supported
only one level of inheritance

How to decode a Google App Engine entity Key path str in Python?

In Google App Engine, an entity has a Key. A key can be made from a path, in which case str(key) is an opaque hex string. Example:
from google.appengine.ext import db
foo = db.Key.from_path(u'foo', u'bar', _app=u'baz')
print foo
gives
agNiYXpyDAsSA2ZvbyIDYmFyDA
if you set up the right paths to run the code.
So, how can one take the hex string and get the path back? I thought the answer would be in Key or entity group docs, but I can't see it.

from google.appengine.ext import db
k = db.Key('agNiYXpyDAsSA2ZvbyIDYmFyDA')
_app = k.app()
path = []
while k is not None:
path.append(k.id_or_name())
path.append(k.kind())
k = k.parent()
path.reverse()
print 'app=%r, path=%r' % (_app, path)
when run in a Development Console, this outputs:
app=u'baz', path=[u'foo', u'bar']
as requested. A shorter alternative is to use the (unfortunately, I believe, undocumented) to_path method of Key instances:
k = db.Key('agNiYXpyDAsSA2ZvbyIDYmFyDA')
_app = k.app()
path = k.to_path()
print 'app=%r, path=%r' % (_app, path)
with the same results. But the first, longer version relies only on documented methods.

Once you have the Key object (which can be created by passing that opaque identifier to the constructor), use Key.to_path() to get the path of a Key as a list. For example:
from google.appengine.ext import db
opaque_id = 'agNiYXpyDAsSA2ZvbyIDYmFyDA'
path = db.Key(opaque_id).to_path()