from module import _dict -> _dict['new_key'] = 1 - and now, the same import in another file imports _dict with a previously non-existent key. This is a problem with pytest that runs several test*.py files, each mutating _dict - so e.g. test_b imports _dict modified by test_a.
A workaround is _dict = copy.deepcopy(_dict) before mutating - but suppose that isn't desired. importlib.reload(module) will not reload _dict - this said, is there any way to ensure the original module._dict is always imported?
Example (also runnable code, without pytest).
# configs.py
_dict = {'a': 1, 'b': 2}
# non_test.py
from configs import _dict
class SomeClass():
def __init__(self, a=None, b=None):
self.a = a or _dict['a']
self.b = b or _dict['b']
del _dict['a']
# test_a.py
def test_class():
SomeClass()
# test_b.py
def test_class():
SomeClass()
Each test*.py has the following 'header' & 'footer':
import pytest
from non_test import SomeClass
# test_*()
if __name__ == '__main__':
pytest.main([__file__, "-s"])
Note: the example isn't reflective of the actual context, within which I have a lot less flexibility. I'm not asking for a solution to 'the problem' itself - what I ask is right in the question's title. If it's "impossible" or there's nothing close to it, then that is the answer.
You can use the importlib.reload function to reload a module. Since _dict from configs is imported into module non_test, which is then imported into test_b, you should reload both non_test and configs to be able to both re-instantiate a new configs._dict and have it re-imported into non_test:
# test_b.py
import non_test
import configs
import importlib
importlib.reload(non_test)
importlib.reload(configs)
def test_class():
SomeClass()
Demo: https://repl.it/#blhsing/SaneDesertedMode
Related
I have 3 scripts: params.py (it defines a configuration class), foo.py (it uses that configuration) and main.py (it initializes the configuration and calls foo).
params.py:
class Config:
def __init__(self, x=0):
self.val = x
global config
config = Config()
foo.py:
from params import config
def foo():
return config.val + 5
main.py:
from params import config
from foo import foo
config = Config(10)
print(foo())
But instead of print 15, it prints 5. How can I fix it? It occurs because when foo.py does the import, it initializes config with 0. But, what can I do to modify from main the config value and read the new value from all other scripts?
Thank you!
Conceptually, you need to separate an object like Config() from the variables that may be referencing it at any given time. When params.py does config = Config(), it creates a Config object and assigns it to a variable in the params module namespace. It is params.config.
When main.py does from params import config, it adds a reference to this Config object to its own namespace. Now there are two references to the same object, one in params.config and another in main.config. So far, so good. from X import Y adds a binding to X.Y into the current namespace. Since params.config is a mutable class instance, main could change the values in that single Config object and it would be seen by all other referrers to that same object. config.val = 10 would be seen by all.
Now things go off the rails. When main does config = Config(10), it creates a new Config object and reassigns that variable to the main namespace. Now params.config references the first object and main references the second. That means changes made to the second object are not seen by the first.
If you want everyone to see the same object, you need to keep the namespace qualification. The scripts would change to
foo.py:
import params
def foo():
return params.config.val + 5
main.py:
import params
from foo import foo
params.config = Config(10)
print(foo())
Now, all of the scripts are using the one variable params.config and see any changes made to that object. This is kindof fragile as you've seen. If anybody does from params import config, reassiging params.config doesn't work.
global only marks a name in a local scope as being global; it has no affect in a global scope, in that it is already global.
What you want isn't really possible, as global namespaces are specific to an individual module, not the process as a whole.
If the value is defined in params.py, you will need to access it via params from all other modules, include the __main__ module created by your script.
params.py:
class Config:
def __init__(self, x=0):
self.val = x
config = Config()
foo.py:
import params
def foo():
return params.config.val + 5
main.py:
import params
from foo import foo
params.config = params.Config(10)
print(foo())
If you simply modified the existing configuration, you could use
params.py (same as above):
class Config:
def __init__(self, x=0):
self.val = x
config = Config()
foo.py (same as your original foo.py):
from params import config
def foo():
return config.val + 5
main.py
from params import config
from foo import foo
config.val = 10
print(foo())
In general, I don't think this is a good idea, as you're essentially creating a global state that can change from any file that imports the configuration file. This is known as action at a distance.
The best answer is to avoid this pattern altogether. For example, come up with a way to use the configuration file in a read-only manner.
That being said, if you really want to do this, make the variable class-level rather than instance-level, so that there exists only one val shared across the entire program.
class Config:
val = 0
def __init__(self, x=0):
Config.val = x
global config
config = Config()
Then, running main.py will print 15.
I am trying to dynamically import modules and get it as global variable.
I am using maya 2020 python interpreter (Python 2.7)
I have a test module called "trigger_test_script.py" under "/home/arda.kutlu/Downloads/" folder.
When I dont import any custom class and run this:
###########################################################################[START]
import sys
import imp
class TestClass(object):
def __init__(self):
self.filePath = None
self.asName = None
def action(self):
exec("global %s" % self.asName, globals())
foo = "imp.load_source('%s', '/home/arda.kutlu/Downloads/trigger_test_script.py')" %self.asName
cmd = "{0}={1}".format(self.asName, foo)
exec(cmd, globals())
###########################################################################[END]
test = TestClass()
test.filePath = "/home/arda.kutlu/Downloads/trigger_test_script.py"
test.asName = "supposed_to_be_global"
test.action()
print(supposed_to_be_global)
I get the exact result that I want:
<module 'trigger_test_script' from '/home/arda.kutlu/Downloads/trigger_test_script.pyc'>
However, when I save TestClass (the part between hashes) into a file and import it like this:
import testClass
test = testClass.TestClass()
test.filePath = "/home/arda.kutlu/Downloads/trigger_test_script.py"
test.asName = "supposed_to_be_global"
test.action()
print(supposed_to_be_global)
the variable which 'supposed_to_be_global' is not becoming global and I get the NameError.
I always assumed that these two usages should return the same result but clearly I am missing something.
I appreciate any help, thanks.
I don't quite understand your last comment about having several modules with different action() methods is a problem. So ignoring that, here's how to make what's in your question work, The part in the hashes will work both in-line or if put in a separate module and imported.
###########################################################################[START]
import imp
class TestClass(object):
def __init__(self):
self.filePath = None
self.asName = None
def action(self):
foo = imp.load_source(self.asName, self.filePath)
return foo
###########################################################################[END]
#from testclass import TestClass
test = TestClass()
test.filePath = "/home/arda.kutlu/Downloads/trigger_test_script.py"
test.asName = "supposed_to_be_global"
supposed_to_be_global = test.action()
print(supposed_to_be_global)
Basic Setup
Suppose I want to create a class named Foo. I may create a file like so:
foo.py:
class Foo:
def __init__(self):
self.data = "world"
def print(self):
print("Hello, " + self.data)
To utilize this class in my main script:
main.py
import foo
test = foo.Foo()
test.print()
Having to type foo.Foo() every time I instantiate the class already feels ridiculous enough, but it gets worse when I want to organize my code by separating my classes into a package:
classes/__init__.py
# Empty
classes/foo.py
# Copy of foo.py, above
main.py
import classes.foo
test = classes.foo.Foo()
test.print()
Simple Answer
I know I can clean this up somewhat by using from X import Y like so:
from classes.foo import Foo
test = Foo()
Preferred Answer
Because the file foo.py contains only one member whose name matches the file, I would prefer if I could do something like the following:
from classes import Foo
# Or:
import classes.Foo as Foo
test = Foo()
Is there a way to do this? Maybe with some code in my __init__.py?
In classes/__init__.py, put:
from .foo import Foo
Now you can write from classes import Foo.
I am currently attempting to write unit tests for my Main.py's main() function
Here is a simplified version of my Main.py:
from Configuration import Configuration # Configuration.py is a file in the same dir
def main():
try:
Configuration('settings.ini')
except:
sys.exit(1) # Test path1
sys.exit(0) # Test path2
if __name__ == '__main__':
main()
In my Unit Tests\MainUnitTests.py I want to import ..\Main.py and fake the Configuration class in such a way that I can hit Test path1 and Test path2
I found that i can assert sys.exit() with the following:
with self.assertRaises(SystemExit) as cm:
main()
self.assertEqual(cm.exception.code, 1)
but I am having trouble overriding the from Configuration import Configuration
Thoughts?
So far I have tried the following within Unit Tests\MainUnitTests.py:
class FakeFactory(object):
def __init__(self, *a):
pass
sys.modules['Configuration'] = __import__('FakeFactory')
class Configuration(FakeFactory):
pass
Another example for demonstration:
foo.py:
from bar import a,b
x = a()
class a(object):
def __init__(self):
self.q = 2
y = a()
print x.q, y.q # prints '1 2' as intended
b() # I want this to print 2 without modifying bar.py
bar.py:
class a(object):
def __init__(self):
self.q = 1
def b():
t = a()
print t.q
when you use the import
from bar import a
it import the name directly into the module, so monkeypatching bar won't help, you need to override a directly in the main file:
def fake_a():
print("OVERRIDEN!")
main.a = fake_a
Do know that unittest has helper functions for this in the mock subpackage, I believe you could do something like:
from unittest.mock import patch
...
with patch("main.a", fake_a) as mock_obj: #there are additional things you can do with the mock_obj
do_stuff()
This would work in your first example with configuration since the class that needs to be patched is not used in the global scope although foo uses bar.a as soon as it is loaded so you would need to patch it before even loading foo:
from unittest.mock import patch
...
with patch("bar.a", fake_a) as mock_obj: #there are additional things you can do with the mock_obj
import foo #now when it loads it will be loaded with the patched name
However in this case foo.a would not be reverted at the end of the with block because it can't be caught by unittest... I really hope your actual use case doesn't use the stuff to be patched at module level.
I'm curious if it's possible to access the name of the parent module that some other module is imported into.
For instance, if I have a module (moduleA) and a parent is module, foo.py, into which it will be imported into, is it possible for moduleA to know where foo is located ?
ModuleA
def print_parent_module():
os.path.asbpath(#somehow access filename of parent module)
foo.py
import moduleA
print moduleA.print_parent_module()
>>> "foo.py"
I ran into a similar issue. You could make use of __name__:
parent_name = '.'.join(__name__.split('.')[:-1])
Or if you try to access the module directly (not the OP's question but related), see my answer in Is there a way to access parent modules in Python
No. Imported modules do not hold any form of state which stores data related to how they are imported.
I think the better question is, why would you even want to do it this way? You're aware that if foo.py is indeed your __main__ then you can easily get the name foo.py out of it (by using sys.argv)? And if foo.py is an imported module instead, then you obviously already know the name of foo.py and its location, etc. at the time that you import it.
Here is something I came with to store a parent's method name and variables to be recalled later in the class with a decorator.
import inspect
from functools import wraps
def set_reuse_vars(method):
#wraps(method)
def _impl(self, *method_args, **method_kwargs):
func_current = inspect.currentframe()
self.recall_func = dict()
self.recall_func['method_kwargs'] = func_current.f_locals['method_kwargs']
self.recall_func['method_name'] = func_current.f_locals['method'].__name__
return method(self, *method_args, **method_kwargs)
return _impl
class APIData(object):
def __init__(self):
self.client = None
self.response = None
self.recall_func = None
def get_next_page(self):
# Takes a pageToken to return the next result
get_func = getattr(self, self.recall_func['method_name'])
self.recall_func['method_kwargs']['pageToken'] = self.response['nextPageToken']
return get_func(**self.recall_func['method_kwargs'])
#set_reuse_vars
def search_list_by_keyword(self, **kwargs):
self.response = self.client.search().list(
**kwargs
).execute()
return self.response
script to run it.
api_data = APIData()
results = api_data.search_list_by_keyword(
part='snippet',
maxResults=25,
order='date')
print(results)
resultsTwo = api_data.get_next_page()
print(resultsTwo)
Figured it out!
A little import statement inside of the function can give access to the module name.
moduleA
def print_module_name():
import sys
return sys.argv[0]
Then, inside the "parent" module
# foo.py
import os
import moduleA
if __name__ == '__main__':
print os.path.split(moduleA.print_module_name())[-1]
Gives:
>>> 'foo.py'