I have a file named file.py containing the following script:
def b():
print("b")
def proc():
print("this is main")
b()
proc()
And I have another file named caller.py, which contains this script:
text = open('file.py').read()
exec(text)
When I run it, I get the expected output:
this is main
b
However, if I change caller.py to this:
def main():
text = open('file.py').read()
exec(text)
main()
I get the following error:
this is main
Traceback (most recent call last):
File "./caller.py", line 7, in <module>
main()
File "./caller.py", line 5, in main
exec(text)
File "<string>", line 10, in <module>
File "<string>", line 8, in main
NameError: global name 'b' is not defined
How is function b() getting lost? It looks to me like I'm not violating any scope rules. I need to make something similar to the second version of caller.py work.
exec(text) executes text in the current scope, but modifying that scope (as def b does via the implied assignment) is undefined.
The fix is simple:
def main():
text = open('file.py').read()
exec(text, {})
This causes text to run in an empty global scope (augmented with the default __builtins object), the same way as in a regular Python file.
For details, see the exec documentation. It also warns that modifying the default local scope (which is implied when not specifying any arguments besides text) is unsound:
The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.
Would it work for you if you imported and called the function instead?
myfile.py
def b():
print("b")
def proc():
print("this is main")
b()
caller.py
import myfile
myfile.proc()
Related
This is my code:
import os
if os.path.exists(r'C:\Genisis_AI'):
print("Main File path exists! Continuing with startup")
else:
createDirs()
def createDirs():
os.makedirs(r'C:\Genisis_AI\memories')
When I execute this, it throws an error:
File "foo.py", line 6, in <module>
createDirs()
NameError: name 'createDirs' is not defined
I made sure it's not a typo and I didn't misspell the function's name, so why am I getting a NameError?
You can't call a function unless you've already defined it. Move the def createDirs(): block up to the top of your file, below the imports.
Some languages allow you to use functions before defining them. For example, javascript calls this "hoisting". But Python is not one of those languages.
Note that it's allowable to refer to a function in a line higher than the line that creates the function, as long as chronologically the definition occurs before the usage. For example this would be acceptable:
import os
def doStuff():
if os.path.exists(r'C:\Genisis_AI'):
print("Main File path exists! Continuing with startup")
else:
createDirs()
def createDirs():
os.makedirs(r'C:\Genisis_AI\memories')
doStuff()
Even though createDirs() is called on line 7 and it's defined on line 9, this isn't a problem because def createDirs executes before doStuff() does on line 12.
I have a function func1() that is in production and cannot be modified. It calls a function ,function_to_be_mocked(), in another module. This takes input parameters.
I have another function func2() which calls func1().
I am writing unit tests to test func2(), and trying to mock function_to_be_mocked (as it depends on some keys I don't have (and should't have) on my local system). The only thing I can modify is test_func2().
I have a set up like the following (minimum example):
from othermodule import function_to_be_mocked
import pytest
import mock
def func1():
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
#mock.patch('othermodule.function_to_be_mocked', return_value = 3)
def test_func2(mocker):
func2()
And othermodule.py is:
def function_to_be_mocked(arg1):
if not arg1 == 'foo':
raise ValueError
My output:
Calling func2 directly:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/blah/temp.py", line 9, in func2
ret = func1()
File "/Users/blah/temp.py", line 6, in func1
function_to_be_mocked(None)
File "/Users/blah/othermodule.py", line 3, in function_to_be_mocked
raise ValueError
ValueError
Calling test_func2() which I would expect to be mocked:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/blah/venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/Users/blah/temp.py", line 14, in test_func2
func2()
File "/Users/blah/temp.py", line 9, in func2
ret = func1()
File "/Users/blah/temp.py", line 6, in func1
function_to_be_mocked(None)
File "/Users/blah/othermodule.py", line 3, in function_to_be_mocked
raise ValueError
ValueError
So the mock doesn't seem to be working. Does anyone have any thoughts how to achieve this?
============ Edited below this line ===========
It doesn't sound like I can do what I thought I could (as I cannot modify anything related to function 1, or 2 in fact. All I have control over is the test.
Let me pose the following problem then as perhaps more experienced eyes than mine can see a way forward.
I have a function:
def function_to_be_tested(args):
# Some processing steps
# Function call that works locally
function_that_returns_something_1()
# Some logic
# Function call that works locally
function_that_returns_something_2()
# Function that raises an exception when running locally,
# and since I need to test the logic after this function
# (and cannot edit this code here to bypass it) I would
# (naively) like to mock it.
function_I_would_like_to_mock()
# Much more logic that follows this function.
# And this logic needs to be unit tested.
return some_value_based_on_the_logic
Tests:
def test_function_to_be_tested():
assert function_to_be_tested(args) == some_expected_value
I can easily unit test anything before function_I_would_like_to_mock().
But since this function crashes locally (and I cannot edit the code to stop it crashing locally), I feel like the correct approach would be to mock it and force a sensible return value. So that I can unit tests the code paths beyond this.
What would you suggest as a good approach?
Please note, the only thing I can modify is the test function. I can't add even decorators to the main functions.
Option A)
The function you are willing to mock is loaded into func1. Therefore you have to apply the #patch decorator to func1
import pytest
from unittest import mock
#mock.patch('othermodule.function_to_be_mocked', return_value = 3)
def func1(mocker):
from othermodule import function_to_be_mocked
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
def test_func2():
func2()
test_func2()
=========Edit===========
Option B)
import pytest
from unittest import mock
def func1():
from othermodule import function_to_be_mocked
function_to_be_mocked(None)
def func2():
ret = func1()
print (ret)
def test_func2():
with mock.patch('othermodule.function_to_be_mocked', return_value = 3) as irrelevant:
func2()
test_func2()
The Where to patch section of the official "unittest.mock — mock object library" documentation explains this quite clearly:
a.py
-> Defines SomeClass
b.py
-> from a import SomeClass
-> some_function instantiates SomeClass
Now we want to test some_function but we want to mock out SomeClass using patch(). The problem is that when we import module b, which we will have to do then it imports SomeClass from module a. If we use patch() to mock out a.SomeClass then it will have no effect on our test; module b already has a reference to the real SomeClass and it looks like our patching had no effect.
The key is to patch out SomeClass where it is used (or where it is looked up ). In this case some_function will actually look up SomeClass in module b, where we have imported it. The patching should look like:
#patch('b.SomeClass')
So I think in your case the patching should look like:
#patch("module_of_func2.function_to_be_mocked_as_it_is_imported_there", return_value=3)
def test_func2():
...
If suppose you want to mock a function from module inside function in another module in python you can try this.
# application2.py
def app2_func(a):
print(a)
# application1.py
import application2
def app1_func(a):
application2.app2_func(a) # func to be mocked
the test file to test the function
# application_test.py
import application1
def test_app1_func(mocker):
app2_mocker = mocker.patch('application1.application2.app2_func')
application1.app1_func('mock call')
app2_mocker.assert_called_once() # to check if mocked function is called once
This is my code:
import os
if os.path.exists(r'C:\Genisis_AI'):
print("Main File path exists! Continuing with startup")
else:
createDirs()
def createDirs():
os.makedirs(r'C:\Genisis_AI\memories')
When I execute this, it throws an error:
File "foo.py", line 6, in <module>
createDirs()
NameError: name 'createDirs' is not defined
I made sure it's not a typo and I didn't misspell the function's name, so why am I getting a NameError?
You can't call a function unless you've already defined it. Move the def createDirs(): block up to the top of your file, below the imports.
Some languages allow you to use functions before defining them. For example, javascript calls this "hoisting". But Python is not one of those languages.
Note that it's allowable to refer to a function in a line higher than the line that creates the function, as long as chronologically the definition occurs before the usage. For example this would be acceptable:
import os
def doStuff():
if os.path.exists(r'C:\Genisis_AI'):
print("Main File path exists! Continuing with startup")
else:
createDirs()
def createDirs():
os.makedirs(r'C:\Genisis_AI\memories')
doStuff()
Even though createDirs() is called on line 7 and it's defined on line 9, this isn't a problem because def createDirs executes before doStuff() does on line 12.
The following executes without an error in Python 3:
code = """
import math
def func(x):
return math.sin(x)
func(10)
"""
_globals = {}
exec(code, _globals)
But if I try to capture the local variable dict as well, it fails with a NameError:
>>> _globals, _locals = {}, {}
>>> exec(code, _globals, _locals)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-9-aeda81bf0af1> in <module>()
----> 1 exec(code, {}, {})
<string> in <module>()
<string> in func(x)
NameError: name 'math' is not defined
Why is this happening, and how can I execute this code while capturing both global and local variables?
From the exec() documentation:
Remember that at module level, globals and locals are the same dictionary. If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition.
You passed in two separate dictionaries, but tried to execute code that requires module-scope globals to be available. import math in a class would produce a local scope attribute, and the function you create won't be able to access that as class scope names are not considered for function closures.
See Naming and binding in the Python execution model reference:
Class definition blocks and arguments to exec() and eval() are special in the context of name resolution. A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods[.]
You can reproduce the error by trying to execute the code in a class definition:
>>> class Demo:
... import math
... def func(x):
... return math.sin(x)
... func(10)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in Demo
File "<stdin>", line 4, in func
NameError: name 'math' is not defined
Just pass in one dictionary.
Windows 10, Python 3.5.1 x64 here.
This is weird... Let's say I have this script, called do.py. Please note the import string statement:
import string
# Please note that if the print statement is OUTSIDE 'main()', it works.
# It's like if 'main()' can't see the imported symbols from 'string'
def main():
print(string.ascii_lowercase)
main()
I want to run it from a "launcher script", in a subthread, like this (launcher.py):
import sys
import threading
sys.argv.append('do.py')
def run(script, filename):
exec(compile(script, filename, 'exec'))
with open(sys.argv[1], 'rb') as _:
script = _.read()
# But this WORKS:
# exec(compile(script, sys.argv[1], 'exec'))
thread = threading.Thread(name='Runner', target=run, args=(script, sys.argv[1]))
thread.start()
thread.join()
It dies with the following error:
Exception in thread Runner:
Traceback (most recent call last):
File "C:\Program Files\Python35\lib\threading.py", line 914, in _bootstrap_inner
self.run()
File "C:\Program Files\Python35\lib\threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "tmpgui.py", line 7, in run
exec(compile(script, filename, 'exec'))
File "do.py", line 6, in <module>
main()
File "do.py", line 4, in main
print(string.ascii_lowercase)
NameError: name 'string' is not defined
That is, the exec'ed code is not importing string properly or something like that, and within main() the string module is not visible.
This is not the full code of my project, which is too big to post here, but the bare minimum I've created which mimics the problem.
Just in case someone is curious, I'm rewriting an old program of mine which imported the main() function of a script and ran that function with the standard output streams redirected to a tkinter text box. Instead of importing a function from the script, I want to load the script and run it. I don't want to use subprocess for a whole variety of reasons, I prefer to run the "redirected" code in a thread and communicate with the main thread which is the one handling the GUI. And that part works perfectly, the only problem I have is this and I can't understand why is happening!
My best bet: I should be passing something in globals or locals dictionaries to exec, but I'm at a lost here...
Thanks a lot in advance!
exec(thing) is equivalent to exec(thing, globals(), locals()).
Thus,
the local symbol table of do.py is the local symbol table of the run function
the global symbol table of do.py is the global symbol table of launcher.py
import string imports the module and binds it to the variable in the local space, which is the local space of the run function. You can verify this:
def run(script, filename):
try:
exec(compile(script, filename, 'exec'))
finally:
assert 'string' in locals(), "won't fail because 'import' worked properly"
main has a separate local scope, but it shares the global symbol table with do.py and, consequently, with launcher.py.
Python tried to find the variable named string inside both local (it's empty) and global symbol tables of main, but failed, and raised the NameError.
Pass one empty dictionary in a call to exec:
def run(script, filename):
exec(compile(script, filename, 'exec'), {})