My code is structured as follows:
main.py
utils.py
blah.py
The main module uses argparse to read in the location of a configurations yaml file which is then loaded as a dictionary. Is there a way for utils and blah to import this built-up dictionary?
Edit: I tried using from main import config (config being the dictionary I built) but I get ImportError: cannot import name 'config' from 'main'
Edit2: Main imports the other 2 modules - apologies for leaving out this very important detail
I would recommend making another file, say, globals.py. Import this in main, utils, and blah, and set properties in it to be recalled by the other modules. For example:
globals.py
configs = {}
main.py
import .globals
...
user_configs = yaml.load('user/entered/path.yml')
globals.configs.update(user_configs) # modifies the global `configs` variable
utils.py
import .globals
...
# need to use one of the configs for something:
try:
relevant_config = globals.configs['relevant_config']
except KeyError:
print("User did not input the config field 'relevant_config'")
All modules will be able to see the same globals instance, thus allowing you to use what are effectively global variables across your program.
You could simply save configs as a gobal variable in main.py and have utils.py and blah.py import .main, but having a designated module for this is cleaner and clearer than to have other modules importing the main module.
Just do
import main
and use it as
main.dictionary
That should do it!
Related
Consider the following:
a.py
foo = 1
b.py
bar = 2
c.py
import a
kik = 3
d.py
import a
import c
def main():
import b
main()
main()
How many times is a.py loaded?
How many times is b.py loaded?
More generally, I would like to know how is Python handling imported files and functions/variables?
Both a and b are loaded once. When you import a module, its content is cached so when you load the same module again, you're not calling upon the original script for the import, done using a "finder":
https://www.python.org/dev/peps/pep-0451/#finder
https://docs.python.org/3/library/importlib.html#importlib.abc.MetaPathFinder
This works across modules so if you had a d.py of which import b, it will bind to the same cache as an import within c.py.
Some interesting builtin modules can help understand what happens during an import:
https://docs.python.org/3/reference/import.html#importsystem
When a module is first imported, Python searches for the module and if found, it creates a module object 1, initializing it.
Notably here the first import, all imports after follow the __import__. Internal caches of finders are stored at sys.meta_path.
https://docs.python.org/3/library/functions.html#import
You can leverage the import system to invalidate those caches for example:
https://docs.python.org/3/library/importlib.html#importlib.import_module
If you are dynamically importing a module that was created since the interpreter began execution (e.g., created a Python source file), you may need to call invalidate_caches() in order for the new module to be noticed by the import system.
The imp (and importlib py3.4+) allows the recompilation of a module after import:
import imp
import a
imp.reload(a)
https://docs.python.org/3/library/importlib.html#importlib.reload
Python module’s code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module’s dictionary by reusing the loader which originally loaded the module.
https://docs.python.org/3/library/imp.html
I am developping a program which runs on two different plattforms. Depending on which plattform I want to run it, the import directories and the names of the libraries change. For this reason i set a variable called RUN_ON_PC to True or False.
I want to implement a helper which sets the paths correctly and imports the libraries with the correct name depending of the platform and gives an interface with the same name of the libraries to the main program. The module myimporthelper is either in the "/mylib" or in the "/sd/mylib" directory. The other module names in these directories differ.
I try to do the following which is not working, since the imported modules from myimporthelper.py are not visible to main.py:
main.py:
RUN_ON_PC = True
import sys
if RUN_ON_PC:
sys.path.append("/mylib1")
else:
sys.path.append("/sd/mylib1")
import myimporthelper
myimporthelper.importall(RUN_ON_PC)
a = moduleA.ClassA() -> produces NameError: name not defined
myimporthelper.py:
import sys
def importall(run_on_pc):
if (run_on_pc == True):
sys.path.append("C:\\Users\\.....\\mylib")
import module1 as moduleA
else:
sys.path.append("/sd/mylib")
import module_a as moduleA
I want to keep the main.py short and want to outsource the platform dependent importing stuff to other module. I was not able to find a solution for this and would aprecciate any help.
Thanks a lot in advance.
You just have to qualify the name with the helper module name
a = myimporthelper.moduleA.ClassA()
But the moduleA name has to be accessible. If you import it inside a function in the helper it won't be, because of scope, unless you assign it to a name you previously declared as global in the helper module function.
I am trying to set up a library in python. I have created a setup.py file and in a folder under that I have a library folder, and then I tried to create an sample and test folder (for sample code and tests that I would include)
Folders:
- setup.py
- cvImageUtils # this is the library
- __init__.py # this includs ColorUtils
- ColorUtils.py # this includes a class called ColorUtils
- examples
- color.py # this is shown below
init.py in ColorUtils folder
from . import ColorUtils
ColorUtils.py
class ColorUtils:
def __...
Color.py
from cvImageUtils import ColorUtils
m1 = cv2.imread(os.path.join(image_folder, "bb.jpeg"), 1) # 1 = load color
cv2.imshow('grayscale', ColorUtils.convert_rbg_to_grayscale(m1))
At first, it said, unable to find the module, so I added to the top of the file the following based on another SO solution:
import sys
sys.path.append('../')
Now that seems broken to me already, but it did get me past the no module found, but now it says ColorUtils has no method convert_rbg_to_grayscale. So Then I had to change it to ColorUtils.ColorUtils.convert_rbg_to_grayscale
cv2.imshow('grayscale', ColorUtils.ColorUtils.convert_rbg_to_grayscale(m1))
How can I setup the folder so that it allows me to include the library without sys, and call it without declaring ColorUtils twice.
change your __init__.py:
from cvImageUtils.ColorUtils import ColorUtils
I don't think you'll need to import sys anymore, and you don't have import ColorUtils twice. but just like you have to instantiate an object, you should create a ColorUtils object.
my personal preference would be not creating a Class for Utils.
you might have done this already, but if you want to use a method straight from a class like you did in python, you might want to declare it static.
class ColorUtils:
#staticmethod
def util_method():
pass
then you can just do:
ColorUtils.util_method()
Update:
you can read more about relative/absolute import from here as well.
to fix your actual problem though, you can do:
color.py
remove your import sys and sys call from color.py
change: import cvImageUtils.ColorUtils as ct
to: from cvImageUtils.ColorUtils import *
remove all your ct reference instead just use the actual functions.
cvImageUtils/__init__.py
change: from . import ColorUtils
to __all__=['ColorUtils']
I was able to run color.py to get all the images printed out on screen.
a image.png was also generated locally as well.
Every directory that you want to expose in module search(we usually hide test.py) in python need a init.py file. That should be rule of thumb, by using sys module you can add the module to your "module search path".
After having init.py in your directories, you need to import packages/modules/funcitons you want to use:-
import cvImageUtils.ColorUtils.convert_rbg_to_grayscale
You can execute following code in python to see, what have included in your sys path(used by python to search for modules/packages)
import sys
sys.path
Look into below links for more detailed explations
https://www.programiz.com/python-programming/package
https://www.programiz.com/python-programming/modules#search
Let's say I have file main.py :
import math
import mymodule.py
print(math.ceil(5/3))
and then mymodule.py :
print(math.ceil(10/3))
mymodule.py gives an error that math is not defined, even though its parent module has it imported.
Considering both main.py and mymodule.py need to use the math lib, do I need to import it twice? It just seems non-optimal. What's the most pythonic way to solve this issue?
I know it's a dumb example, but I'm trying to fragment a code I made into several modules for organization, and this issue appeared multiple times in several levels
mymodule.py is parent for main.py since you are importing mymodule within main.
You need to import math within mymodule so that it gets inherited in main.
Then there won't be a need to import within main.
mymodule.py
import math
main.py
import mymodule
print mymodule.math.pow(10,2)
Result:
>>>
100.0
>>>
This is really very basic. If you have something in a separate file, like mymodule.py, then you can import that function in any python file easily in the same directory.
two files:
mymodule.py:
import math
def aFunc():
return math.ceil(10/3)
# We could also just use this file as a standalone
if __name__ == "__main__":
print(aFunc())
main.py:
import mymodule
print(mymodule.aFunc())
You could also specifically call out the function you want to import.
main.py (alternative):
from mymodule import aFunc
print(aFunc())
I faced this problem while i was expirementing with my project.My project is designed so it has multipule submoudles.
Now each submodule needs a position of an the same item on the screen,which is random each time the program ran.I thought it was a good idea to check once where is the position of the item (It does not change in runtime) , and let all modules access the position.(Since checking the position takes a long time)
This is what i did:
main.py
import Config
import sub_module
main()
def main():
position = get_pos()
Config.pos = position
sub_module.click_on_item()
Config.py
pos = None
I tried using this code , so when i ran the program , it sets the config.py module pos variable , to the position of the item on screen.
This code works fine but when i try to use it in the submodules like this:
sub_module.py
import Config
def click_on_item():
click(Config.pos)
It resets the value to None , since it reruns the Config module code.
Is there an elegant solution for this problem ? I can't let the position as an argument for the , since my case is much more complex.
One solution that i can think of is writing to the disk , and rereading but it's not fast.
Thanks for the help.
2 EDIT:
My project is multi-package , and there is the problem
this is the directory tree:
mainPackage:
__init__.py
Config.py
main.py
b.py
Packages:
Package_A:
__init__.py
a.py
main.py
import Config
from Packages.Package_A import a
import b
def main():
Config.Position = (124, 586)
a.print_val()
b.print_val()
if __name__ == '__main__':
main()
Config
Position = None
b.py
import Config
def print_val():
print Config.Position
a.py
from mainPackage import Config
def print_val():
print Config.Position
output
None -> from Package_A
(124, 586) -> from b.py
Sorry for the inconvenience since i didn't know what was causing the problem , my interest is in multi package global variable.
The problem is in a.py: from mainPackage import Config. You did an absolute import from a package and ended up importing Config twice. Python only imports a module once but in your case you used two different names (Config in some files, mainPackage.Config in others) which confused python and it imported the module twice - once by a package relative import and once by an absolute import. Add print "Importing Config" to Config.py and you will see it printed twice.
Just change a.py to import Config and all will be well.
Config.py will only be initialized on import once in your application and pos will be initially None.
main.py then sets it, and all other modules can access pos with Config.pos.
Config.py
# Initial value
pos = None
main.py
import Config
import sub_module
def main():
position = get_pos()
Config.pos = position
sub_module.click_on_item()
The rest of your files can be kept as is.
UPDATE
As #tdelaney mentioned in his answer.
Change
from mainPackage import Config
to
import Config
to avoid another Config.
For each of your module, define your functions/class with a 'pos' argument.
Module A.py
def foo(pos):
click(pos)
Module main.py
def main():
position = get_pos()
A.foo(position)
In general, do not use global var.