Develop Reference

How to test complicated functions which use requests?

I want to test my code that is based on the API created by someone else, but im not sure how should I do this.
I have created some function to save the json into file so I don't need to send requests each time I run test, but I don't know how to make it work in situation when the original (check) function takes an input arg (problem_report) which is an instance of some class provided by API and it has this
problem_report.get_correction(corr_link) method. I just wonder if this is a sign of bad written code by me, beacuse I can't write a test to this, or maybe I should rewrite this function in my tests file like I showed at the end of provided below code.
# I to want test this function
def check(problem_report):
corrections = {}
for corr_link, corr_id in problem_report.links.items():
if re.findall(pattern='detailCorrection', string=corr_link):
correction = problem_report.get_correction(corr_link)
corrections.update({corr_id: correction})
return corrections
# function serves to load json from file, normally it is downloaded by API from some page.
def load_pr(pr_id):
print('loading')
with open('{}{}_view_pr.json'.format(saved_prs_path, pr_id)) as view_pr:
view_pr = json.load(view_pr)
...
pr_info = {'view_pr': view_pr, ...}
return pr_info
# create an instance of class MyPR which takes json to __init__
#pytest.fixture
def setup_pr():
print('setup')
pr = load_pr('123')
my_pr = MyPR(pr['view_pr'])
return my_pr
# test function
def test_check(setup_pr):
pr = setup_pr
checked_pr = pr.check(setup_rft[1]['problem_report_pr'])
assert checker_pr
# rewritten check function in test file
#mock.patch('problem_report.get_correction', side_effect=get_corr)
def test_check(problem_report):
corrections = {}
for corr_link, corr_id in problem_report.links.items():
if re.findall(pattern='detailCorrection', string=corr_link):
correction = problem_report.get_correction(corr_link)
corrections.update({corr_id: correction})
return corrections
Im' not sure if I provided enough code and explanation to underastand the problem, but I hope so. I wish you could tell me if this is normal that some function are just hard to test, and if this is good practice to rewritte them separately so I can mock functions inside the tested function. I also was thinking that I could write new class with similar functionality but API is very large and it would be very long process.

I understand your question as follows: You have a function check that you consider hard to test because of its dependency on the problem_report. To make it better testable you have copied the code into the test file. You will test the copied code because you can modify this to be easier testable. And, you want to know if this approach makes sense.
The answer is no, this does not make sense. You are not testing the real function, but completely different code. Well, the code may not start being completely different, but in short time the copy and the original will deviate, and it will be a maintenance nightmare to ensure that the copy always resembles the original. Improving code for testability is a different story: You can make changes to the check function to improve its testability. But then, exactly the same resulting function should be used both in the test and the production code.
How to better test the function check then? First, are you sure that using the original problem_report objects really can not be sensibly used in your tests? (Here are some criteria that help you decide: What to mock for python test cases?). Now, lets assume that you come to the conclusion you can not sensibly use the original problem_report.
In that case, here the interface is simple enough to define a mocked problem_report. Keep in mind that Python uses duck typing, so you only have to create a class that has a links member which has an items() method. Plus, your mocked problem_report class needs a method get_correction(). Beyond that, your mock does not have to produce types that are similar to the types used by problem_report. The items() method can return simply a list of lists, like [["a",2],["xxxxdetailCorrectionxxxx",4]]. The same argument holds for get_correction, which could for example simply return its argument or a derived value, like, its negative.
For the above example (items() returning [["a",2],["xxxxdetailCorrectionxxxx",4]] and get_correction returning the negative of its argument) the expected result would be {4: -4}. No need to simulate real correction objects. And, you can create your mocked versions of problem_report without need to read data from files - the mocks can be setup completely from within the unit-testing code.

Try patching the problem_report symbol in the module. You should put your tests in a separate class.
#mock.patch('some.module.path.problem_report')
def test_check(problem_report):
problem_report.side_effect = get_corr
corrections = {}
for corr_link, corr_id in problem_report.links.items():
if re.findall(pattern='detailCorrection', string=corr_link):
correction = problem_report.get_correction(corr_link)
corrections.update({corr_id: correction})
return corrections

How to construct an instance of _pytest.pytester.Testdir

I'm attempting to do some debugging (specifically on pytest/testing/test_doctest.py) and I want to step through some code in IPython. I have experience with pytest, but I never do anything too fancy with it, so I've never delved to deep into the more "magic" things it does.
In the test that I want to step through (potentially introspecting some of the objects), there is an argument called testdir, but nowhere in this file does it reference what testdir is or how I could possibly construct one.
After doing some digging it seems this is some magic fixture that automatically gets constructed and send to your function as a parameter, when you execute pytest with the pytester plugin. When I tracked down that class, it is constructed again via some magic request param, where the code is massively unhelpful in telling you what that magic request is or how to make one.
To make this concrete I simply want to take a test like this one:
def test_reportinfo(self, testdir):
'''
Test case to make sure that DoctestItem.reportinfo() returns lineno.
'''
p = testdir.makepyfile(test_reportinfo="""
def foo(x):
'''
>>> foo('a')
'b'
'''
return 'c'
""")
items, reprec = testdir.inline_genitems(p, '--doctest-modules')
reportinfo = items[0].reportinfo()
assert reportinfo[1] == 1
and run its logic in IPython. Looking at what the testdir object does, it seems pretty cool. It automatically makes a file for you and runs pytest problematically instead of via the command line. How can I make one of these? Is there some documentation I missed that makes how to do this clear and seem less obfuscated?
If I wanted to use something like this is my tests is there a way I could make what the magic testdir parameter is slightly more explicit so the next coder that looks at it isn't pulling his/her hair out like I am?

After much agonizing, I've figured out how to instantiate a fixture value.
import _pytest
config = _pytest.config._prepareconfig(['-s'], plugins=['pytester'])
session = _pytest.main.Session(config)
_pytest.tmpdir.pytest_configure(config)
_pytest.fixtures.pytest_sessionstart(session)
_pytest.runner.pytest_sessionstart(session)
def func(testdir):
return testdir
parent = _pytest.python.Module('parent', config=config, session=session)
function = _pytest.python.Function(
'func', parent, callobj=func, config=config, session=session)
_pytest.fixtures.fillfixtures(function)
testdir = function.funcargs['testdir']
The main idea is to create a dummy pytest session. This is a bit tricky. Its critical that the ['-s'] is passed into _prepareconfig otherwise this will not print stdout, or crash when run in IPython.
Given a barebones config and session, the next step is to manually load in whatever fixture functionality you are going to use. This amounts to manually calling the hooks that pluggy usually takes care of for you. I found these by looking at the attribute error I got when trying to run code without them. Usually its just due to session or config lacking a required attribute. There may be a better way to go about doing this (aka automatically via pluggy).
Next, we create a function that requests the specific fixture we are interested in. Its up to you to know what these names are. Finally we setup a dummy module / function tree structure and call fillfixtures, which does the magic. The funcargs then contains a dictionary of these objects ready for use. Be careful if you expect some teardown functionality. I'm not sure if this covers that, but I don't really need it for what I'm doing.
Hope this helps someone else. Note: this talk helped me understand what was happening in pytest under the hood a bit better: https://www.youtube.com/watch?v=zZsNPDfOoHU

Self Modifying Python? How can I redirect all print statements within a function without touching sys.stdout?

I have a situation where I am attempting to port some big, complex python routines to a threaded environment.
I want to be able to, on a per-call basis, redirect the output from the function's print statement somewhere else (a logging.Logger to be specific).
I really don't want to modify the source for the code I am compiling, because I need to maintain backwards compatibility with other software that calls these modules (which is single threaded, and captures output by simply grabbing everything written to sys.stdout).
I know the best option is to do some rewriting, but I really don't have a choice here.
Edit -
Alternatively, is there any way I can override the local definition of print to point to a different function?
I could then define the local print = system print unless overwritten by a kwarg, and would only involve modify a few lines at the beginning of each routine.

In Python2.6 (and 2.7), you can use
from __future__ import print_function
Then you can change the code to use the print() function as you would for Python3
This allows you to create a module global or local function called print which will be used in preference to the builtin function
eg.
from __future__ import print_function
def f(x, print=print):
print(x*x)
f(5)
L=[]
f(6, print=L.append)
print(L)

Modifying the source code doesn't need to imply breaking backward compatibility.
What you need to do is first replace every print statement with a call to a function that does the same thing:
import sys
def _print(*args, **kw):
sep = kw.get('sep', ' ')
end = kw.get('end', '\n')
file = kw.get('file', sys.stdout)
file.write(sep.join(args))
file.write(end)
def foo():
# print "whatever","you","want"
_print("whatever","you","want")
Then the second step is to stop using the _print function directly and make it a keyword argument:
def foo(_print=_print):
...
and make sure to change all internal function calls to pass the _print function around.
Now all the existing code will continue to work and will use print, but you can pass in whatever _print function you want.
Note that the signature of _print is exactly that of the print function in more recent versions of Python, so as soon as you upgrade you can just change it to use print(). Also you may get away with using 2to3 to migrate the print statements in the existing code which should reduce the editing required.

Someone in the sixties had an idea about how to solve this but it requires a bit of alien technology. Unfortunately python has no "current environment" concept and this means you cannot provide context unless specifying it in calls as a parameter.
For handling just this specific problem what about replacing stdout with a file-like object that behaves depending on a thread-specific context ? This way the source code remains the same but for example you can get a separate log for each thread. It's even easy to do this on a specific per-call way... for example:
class MyFakeStdout:
def write(self, s):
try:
separate_logs[current_thread()].write(s)
except KeyError:
old_stdout.write(s)
and then having a function to set a logger locally to a call (with)
PS: I saw the "without touching stdout" in the title but I thought this was because you wanted only some thread to be affected. Touching it while still allowing other threads to work unaffected seems to me compatible with the question.

Is there a high-level profiling module for Python?

I want to profile my Python code. I am well-aware of cProfile, and I use it, but it's too low-level. (For example, there isn't even a straightforward way to catch the return value from the function you're profiling.)
One of the things I would like to do: I want to take a function in my program and set it to be profiled on the fly while running the program.
For example, let's say I have a function heavy_func in my program. I want to start the program and have the heavy_func function not profile itself. But sometime during the runtime of my program, I want to change heavy_func to profile itself while it's running. (If you're wondering how I can manipulate stuff while the program is running: I can do it either from the debug probe or from the shell that's integrated into my GUI app.)
Is there a module already written which does stuff like this? I can write it myself but I just wanted to ask before so I won't be reinventing the wheel.

It may be a little mind-bending, but this technique should help you find the "bottlenecks", it that's what you want to do.
You're pretty sure of what routine you want to focus on.
If that's the routine you need to focus on, it will prove you right.
If the real problem(s) are somewhere else, it will show you where they are.
If you want a tedious list of reasons why, look here.

I wrote my own module for it. I called it cute_profile. Here is the code. Here are the tests.
Here is the blog post explaining how to use it.
It's part of GarlicSim, so if you want to use it you can install garlicsim and do from garlicsim.general_misc import cute_profile.
If you want to use it on Python 3 code, just install the Python 3 fork of garlicsim.
Here's an outdated excerpt from the code:
import functools
from garlicsim.general_misc import decorator_tools
from . import base_profile
def profile_ready(condition=None, off_after=True, sort=2):
'''
Decorator for setting a function to be ready for profiling.
For example:
#profile_ready()
def f(x, y):
do_something_long_and_complicated()
The advantages of this over regular `cProfile` are:
1. It doesn't interfere with the function's return value.
2. You can set the function to be profiled *when* you want, on the fly.
How can you set the function to be profiled? There are a few ways:
You can set `f.profiling_on=True` for the function to be profiled on the
next call. It will only be profiled once, unless you set
`f.off_after=False`, and then it will be profiled every time until you set
`f.profiling_on=False`.
You can also set `f.condition`. You set it to a condition function taking
as arguments the decorated function and any arguments (positional and
keyword) that were given to the decorated function. If the condition
function returns `True`, profiling will be on for this function call,
`f.condition` will be reset to `None` afterwards, and profiling will be
turned off afterwards as well. (Unless, again, `f.off_after` is set to
`False`.)
`sort` is an `int` specifying which column the results will be sorted by.
'''
def decorator(function):
def inner(function_, *args, **kwargs):
if decorated_function.condition is not None:
if decorated_function.condition is True or \
decorated_function.condition(
decorated_function.original_function,
*args,
**kwargs
):
decorated_function.profiling_on = True
if decorated_function.profiling_on:
if decorated_function.off_after:
decorated_function.profiling_on = False
decorated_function.condition = None
# This line puts it in locals, weird:
decorated_function.original_function
base_profile.runctx(
'result = '
'decorated_function.original_function(*args, **kwargs)',
globals(), locals(), sort=decorated_function.sort
)
return locals()['result']
else: # decorated_function.profiling_on is False
return decorated_function.original_function(*args, **kwargs)
decorated_function = decorator_tools.decorator(inner, function)
decorated_function.original_function = function
decorated_function.profiling_on = None
decorated_function.condition = condition
decorated_function.off_after = off_after
decorated_function.sort = sort
return decorated_function
return decorator

Use of eval in Python?

There is an eval() function in Python I stumbled upon while playing around. I cannot think of a case when this function is needed, except maybe as syntactic sugar. Can anyone give an example?

eval and exec are handy quick-and-dirty way to get some source code dynamically, maybe munge it a bit, and then execute it -- but they're hardly ever the best way, especially in production code as opposed to "quick-and-dirty" prototypes &c.
For example, if I had to deal with such dynamic Python sources, I'd reach for the ast module -- ast.literal_eval is MUCH safer than eval (you can call it directly on a string form of the expression, if it's a one-off and relies on simple constants only, or do node = ast.parse(source) first, then keep the node around, perhaps munge it with suitable visitors e.g. for variable lookup, then literal_eval the node) -- or, once having put the node in proper shape and vetted it for security issues, I could compile it (yielding a code object) and build a new function object out of that. Far less simple (except that ast.literal_eval is just as simple as eval for the simplest cases!) but safer and preferable in production-quality code.
For many tasks I've seen people (ab-)use exec and eval for, Python's powerful built-ins, such as getattr and setattr, indexing into globals(), &c, provide preferable and in fact often simpler solutions. For specific uses such as parsing JSON, library modules such as json are better (e.g. see SilentGhost's comment on tinnitus' answer to this very question). Etc, etc...

The Wikipedia article on eval is pretty informative, and details various uses.
Some of the uses it suggests are:
Evaluating mathematical expressions
Compiler bootstrapping
Scripting (dynamic languages in general are very suitable to this)
Language tutors

You may want to use it to allow users to enter their own "scriptlets": small expressions (or even small functions), that can be used to customize the behavior of a complex system.
In that context, and if you do not have to care too much for the security implications (e.g. you have an educated userbase), then eval() may be a good choice.

In the past I have used eval() to add a debugging interface to my application. I created a telnet service which dropped you into the environment of the running application. Inputs were run through eval() so you can interactively run Python commands in the application.

In a program I once wrote, you had an input file where you could specify geometric parameters both as values and as python expressions of the previous values, eg:
a=10.0
b=5.0
c=math.log10(a/b)
A python parser read this input file and obtained the final data evaluating the values and the expressions using eval().
I don't claim it to be good programming, but I did not have to drive a nuclear reactor.

I use it as a quick JSON parser ...
r='''
{
"glossary": {
"title": "example glossary"
}
}
'''
print eval(r)['glossary']['title']

You can use eval in a decorator:
#this replaces the original printNumber with a lambda-function,
#which takes no arguments and which calls the old function with
#the number 10
#eval("lambda fun: lambda: fun(10)")
def printNumber(i: int) -> None:
print("The number is %i", i)
#call
printNumber()
while you cannot use complex expressions like
#lambda fun: lambda: fun(10)
def ...
nor
#(lambda fun: lambda: fun(10))
def ...
You cannot use a lambda-expression there, because the decorator should either be an identifier:
#myModule.functionWithOneArg
or a function call:
#functionReturningFunctionWithOneArg(any, "args")
You see that the call of the function eval with a string has valid syntax here, but the lambda-expression not. (-> https://docs.python.org/3/reference/compound_stmts.html#function-definitions)

eval() is not normally very useful. One of the few things I have used it for (well, it was exec() actually, but it's pretty similar) was allowing the user to script an application that I wrote in Python. If it were written in something like C++, I would have to embed a Python interpreter in the application.

Eval is a way to interact with the Python interpreter from within a program. You can pass literals to eval and it evaluates them as python expressions.
For example -
print eval("__import__('os').getcwd()")
would return the current working directory.
cheers

eval() is for single sentence, while exec() is for multiple ones.
usually we use them to add or visit some scripts just like bash shell.
because of they can run some byte scripts in the memory, if you have some important data or script you can decode and unzip your 'secret' then do everything you wanna.

I just came across a good use of eval. I was writing a test suite for some code, and created a Test class, where every method was a test to be run. I wanted a way so that I could run all the test methods without having to call each method individually. So, I wrote something rather dirty.
class Test:
def __init__(self, *args):
#bs
def test1(self):
#bs
def test2(self):
#bs
if __name__ == "__main__":
import argparse
#argparse bs
test = Test(*bs_args)
for func in (i for i in dir(test) if i[0] != '_' and i not in test.__dict__):
print(eval('test.{func}()'.format(func = func)))
Dynamic evaluation of arbitrary test cases is pretty cool. I just have to write the method, and after saving I can include the method in my test suite. As for the code, I basically just inspect the methods defined in the test object, and make sure they aren't default python "magic" methods or attributes to the Test object. After that I can assume they are methods and can be evaluated.

I used it to input variable values to the main program:
test.py var1=2 var2=True
...
var1=0
var2=False
for arg in sys.argv[1:]:
exec(arg)
A crude way to allow keyword args in the main program. If there's a better way let me know!

I had a case where I used eval in combination with an informix database. For some reason the query returned a string formed like this
query_result = "['1', '2', '3']"
I just used eval on the query result so python interpreted it as a list of strings.
[int(i) for i in eval(query_result)]
> [1,2,3]
I could not change the db so this was a quick (and dirty) way to get the integers.

I use exec to create a system of plugins in Python.
try:
exec ("from " + plugin_name + " import Plugin")
myplugin = Plugin(module_options, config=config)
except ImportError, message:
fatal ("No such module " + plugin_name + \
" (or no Plugin constructor) in my Python path: " + str(message))
except Exception:
fatal ("Module " + plugin_name + " cannot be loaded: " + \
str(sys.exc_type) + ": " + str(sys.exc_value) + \
".\n May be a missing or erroneous option?")
With a plugin like:
class Plugin:
def __init__ (self):
pass
def query(self, arg):
...
You will be able to call it like:
result = myplugin.query("something")
I do not think you can have plugins in Python without exec/eval.