I have a global object which uses a class, and that class uses the global object. How can i put the imports correct in a clean way
I now have:
run.py (The file i run)
from global_class import Global_class
global_object = Global_class()
global_object.create_some_object()
global_class.py
from some_class import Some_class
class Global_class:
def __init__(self):
self.name = 'my_name'
def create_some_object(self):
self.some_object = Some_class()
some_class.py
class Some_class:
def __init__(self):
print(global_object.name)
How can i now access global_object in Some_class? If i put:
from run import global_object
It creates a circular dependency, and it crashes. A possible method i thought of was putting the some_class import in the Global_class::create_some_object() method, but it seems to me as unclean code. Is there any better way
Any python import module or from module import Class statement runs corresponding module line by line and loads all the objects in the module namespace in the memory. However each of the names in the modules reside separately (that is the purpose of the modules after all). So a global_object in some_class.py is completely separate from the global_object in run.py. When the interpreter sees this name in some_class.py it will look in the local and global namespace (using the LEGB rule which stands for local, enclosed, global and builtins). But there is no reference to global_object exists here, it does in the calling module. Your suggestion of putting the some_class import statement inside the method create_object() will also not work for the same reason. As you have found out you cannot import global_object in some_class as it will again need to run the run.py thus creating a loop.
If you want to maintain this setup then one option would be to explicitly pass the global_object which is self in Global_class to the Some_class() constructor like below
#run.py
from global_class import Global_class
global_object = Global_class()
global_object.create_some_object()
#global_class.py
from some_class import Some_class
class Global_class:
def __init__(self):
self.name = 'my_name'
def create_some_object(self):
self.some_object = Some_class(self) #self -> global_object
#some_class.py
class Some_class:
def __init__(self, global_object):
print(global_object.name)
$ python run.py
my_name
Related
I'm testing a script that mainly consists of an object Foo, which imports a library bar.
import bar
class Foo():
*do stuff*
f = Foo()
*do stuff*
Now I create a unittest class to test Foo.
import foo
class FooTest(unittest.TestCase):
def setUp(self):
self.foo = foo.Foo()
*do tests*
Later, it has a line like:
bar.method(...)
This fails with 'bar not defined', but shouldn't it be imported via import foo? How do automatically load whatever is required by the script under test?
The importer model indeed imports everything from the other file, but under the imported model name as a namespace. See an example:
so_imported.py
import json
class Test:
def __init__(self) -> None:
self.text = "A phrase"
so_import.py
import so_imported
class Test2:
def __init__(self) -> None:
self.inner = {"test": so_imported.Test().text}
print(so_imported.json.dumps(self.inner))
Test2()
So you see the json library imported under the module's namespace ?
If you want to import everything from the imported file without any namespaces, you can use: from foo import *. Then the objects will be merged into the importer namespace.
I have 3 scripts: params.py (it defines a configuration class), foo.py (it uses that configuration) and main.py (it initializes the configuration and calls foo).
params.py:
class Config:
def __init__(self, x=0):
self.val = x
global config
config = Config()
foo.py:
from params import config
def foo():
return config.val + 5
main.py:
from params import config
from foo import foo
config = Config(10)
print(foo())
But instead of print 15, it prints 5. How can I fix it? It occurs because when foo.py does the import, it initializes config with 0. But, what can I do to modify from main the config value and read the new value from all other scripts?
Thank you!
Conceptually, you need to separate an object like Config() from the variables that may be referencing it at any given time. When params.py does config = Config(), it creates a Config object and assigns it to a variable in the params module namespace. It is params.config.
When main.py does from params import config, it adds a reference to this Config object to its own namespace. Now there are two references to the same object, one in params.config and another in main.config. So far, so good. from X import Y adds a binding to X.Y into the current namespace. Since params.config is a mutable class instance, main could change the values in that single Config object and it would be seen by all other referrers to that same object. config.val = 10 would be seen by all.
Now things go off the rails. When main does config = Config(10), it creates a new Config object and reassigns that variable to the main namespace. Now params.config references the first object and main references the second. That means changes made to the second object are not seen by the first.
If you want everyone to see the same object, you need to keep the namespace qualification. The scripts would change to
foo.py:
import params
def foo():
return params.config.val + 5
main.py:
import params
from foo import foo
params.config = Config(10)
print(foo())
Now, all of the scripts are using the one variable params.config and see any changes made to that object. This is kindof fragile as you've seen. If anybody does from params import config, reassiging params.config doesn't work.
global only marks a name in a local scope as being global; it has no affect in a global scope, in that it is already global.
What you want isn't really possible, as global namespaces are specific to an individual module, not the process as a whole.
If the value is defined in params.py, you will need to access it via params from all other modules, include the __main__ module created by your script.
params.py:
class Config:
def __init__(self, x=0):
self.val = x
config = Config()
foo.py:
import params
def foo():
return params.config.val + 5
main.py:
import params
from foo import foo
params.config = params.Config(10)
print(foo())
If you simply modified the existing configuration, you could use
params.py (same as above):
class Config:
def __init__(self, x=0):
self.val = x
config = Config()
foo.py (same as your original foo.py):
from params import config
def foo():
return config.val + 5
main.py
from params import config
from foo import foo
config.val = 10
print(foo())
In general, I don't think this is a good idea, as you're essentially creating a global state that can change from any file that imports the configuration file. This is known as action at a distance.
The best answer is to avoid this pattern altogether. For example, come up with a way to use the configuration file in a read-only manner.
That being said, if you really want to do this, make the variable class-level rather than instance-level, so that there exists only one val shared across the entire program.
class Config:
val = 0
def __init__(self, x=0):
Config.val = x
global config
config = Config()
Then, running main.py will print 15.
I have classes which require dependencies in order to be instantiated but are otherwise optional. I'd like to lazily import the dependencies and fail to instantiate the class if they aren't available. Note that these dependencies are not required at the package level (otherwise they'd be mandatory via setuptools). I currently have something like this:
class Foo:
def __init__(self):
try:
import module
except ImportError:
raise ModuleNotFoundError("...")
def foo(self):
import module
Because this try/except pattern is common, I'd like to abstract it into a lazy importer. Ideally if module is available, I won't need to import it again in Foo.foo so I'd like module to be available once it's been imported in __init__. I've tried the following, which populates globals() and fails to instantiate the class if numpy isn't available, but it pollutes the global namespace.
def lazy_import(name, as_=None):
# Doesn't handle error_msg well yet
import importlib
mod = importlib.import_module(name)
if as_ is not None:
name = as_
# yuck...
globals()[name] = mod
class NeedsNumpyFoo:
def __init__(self):
lazy_import("numpy", as_="np")
def foo(self):
return np.array([1,2,])
I could instantiate the module outside the class and point to the imported module if import doesn't fail, but that is the same as the globals() approach. Alternatively lazy_import could return the mod and I could call it whenever the module is needed, but this is tantamount to just importing it everywhere as before.
Is there a better way to handle this?
Pandas actually has a function import_optional_dependency which may make a good example (link GitHub) as used in SQLAlchemyEngine (link GitHub)
However, this is only used during class __init__ to get a meaningful error (raise ImportError(...) by default!) or warn about absence or old dependencies (which is likely a more practical use of it, as older or newer dependencies may import correctly anywhere if they exist, but not work or be explicitly tested against or even be an accidental local import)
I'd consider doing similarly, and either not bother to have special handling or only do it in the __init__ (and then perhaps only for a few cases where you're interested in the version, etc.) and otherwise simply import where needed
class Foo():
def __init__(self, ...):
import bar # only tests for existence
def usebar(self, value):
import bar
bar.baz(value)
Plausibly you could assign to a property of the class, but this may cause some trouble or confusion (as the import should already be available in globals once imported)
class Foo():
def __init__(self, ...):
import bar
self.bar = bar
def usebar(self, value):
self.bar.baz(value)
Gave it a quick test with a wrapper, seems to work fine:
def requires_math(fn):
def wrapper(*args, **kwargs):
global math
try:
math
except NameError:
import math
return fn(*args, **kwargs)
return wrapper
#requires_math
def func():
return math.ceil(5.5)
print(func())
Edit: More advanced one that works with any module, and ensures it is a module in case it's been set to something else.
from types import ModuleType
def requires_import(*mods):
def decorator(fn):
def wrapper(*args, **kwargs):
for mod in mods:
if mod not in globals() or not isinstance(globals()[mod], ModuleType):
globals()[mod] = __import__(mod)
return fn(*args, **kwargs)
return wrapper
return decorator
#requires_import('math', 'random')
def func():
return math.ceil(random.uniform(0, 10))
print(func())
I have three modules:
in_mod.py
class IN(object):
def __init__(self):
print("i am the original IN")
module1.py
from in_mod import IN
class C(object):
def __init__(self):
cl = IN()
and module2.py
from module1 import C
class IN2(object):
def __init__(self):
print("I am the new IN")
C()
import in_mod
in_mod.IN = IN2
C()
import module1
module1.IN = IN2
C()
I get the desired behaviour of monkey-patching out the IN class and replacing it with the IN2 class when I use module1.IN = IN2.
I would like to understand what the underlying difference between in_mod.IN = IN2 and module1.IN = IN2 is in this context.
I am referencing a related post.
from module import IN creates a local variable that references module.IN; module.IN and IN are two separate references to the same class. IN = IN2 changes the local reference, but module.IN (which is used by module.C) continues to reference the same class.
UPDATE
In your edit, module1.IN is a global variable in the module1 namespace that initially refers to a class in in_mod, but is distinct from the global in_mod.IN in the module namespace. Changing its value doesn't affect in_mod at all. There's no way to directly access in_mod's namespace via module1, because you don't import the entire module, just one value from the module.
I have a package with testing modules and inside the init file I have a setUp method with some operations. These operations are executed correctly before any unit test in the package's modules run. Inside the setUp method I'd like to initialize a global variable and then access it from other modules of the package. But this doesn't work.
# TestPackage/__init__.py
def setUp():
global spec_project
core_manager = get_core_manager()
spec_project = core_manager.get_spec()
#TestPackage/test_module.py
from TestPackage import spec_project
import unittest
class TestRules(unittest.TestCase):
def setUp(self):
spec_project.get_new_device()
Like this I get an
ImportError: cannot import name spec_project
If I initialize the spec_project variable outside of the setUp method in the init file I can have access to it but its content is not changed after the operations in the setUp method.
# TestPackage/__init__.py
spec_project = None
def setUp():
global spec_project
core_manager = get_core_manager()
spec_project = core_manager.get_spec()
#TestPackage/test_module.py
from TestPackage import spec_project
import unittest
class TestRules(unittest.TestCase):
def setUp(self):
spec_project.get_new_device()
Like this I get an
AttributeError: 'NoneType' object has no attribute 'get_new_device'
How can initialize the spec_project variable inside the setUp method of the init file and still have access to it from other module in the package?
It looks like setUp() isn't being called, but if you are certain that it is, then it could be the way that you are importing TestPackage. Try importing like this:
#TestPackage/test_module.py
import TestPackage
import unittest
class TestRules(unittest.TestCase):
def setUp(self):
TestPackage.spec_project.get_new_device()
The setUp() method has to be called before you use the global. This same thing should apply to the second way you tried. But again, that is assuming that setUp is run. You can alias TestPackage if you feel it is necessary, or you should be able to import it if it is defined outside the method.
Since you are explicitly importing it, it is likely trying to make a copy of it, which isn't possible, since it is inside of the setUp() body.