I am having problems to manage my modules and pacakges dependencies in Python. I am in Windows 7 and using Python 3.3
I want to have a basic structrure like this:
/
myTask.py
-->utils
--> utils1.py
--> utils2.py
-->processes
--> process1.py
--> process2.py
MyTask will run process1. Process1 imports process2 and imports the utils1 and utils2 in the utils package
So in process1 I have
import sys
sys.path.append('..path..\\utils\\')
from utils1 import *
from utils2 import *
from process2 import *
if __name__ == '__main__':
my_method()
def my_method():
####CODE
I can run Process1 as script and works fine.
MyTask.py looks like:
import processes.process1
process1.my_method()
When I run it I recieve and error that says:
ImportError: No module named 'process2'
Is my package/module structure correct? Why do I receive that error?
There are several errors in the structure you have created.
First of all, in python package must contain __init__.py file at it's root. It's ordinary python file and it might be completely empty. It's required to mark folder as python package
Next, you shouldn't mess with sys.path excplicitly, especially using relative paths (unless you are sure what you are doing, but this time you definitely not).
So, the correct folder structure would be
/MyTask.py
/processes
__init__.py
process1.py
process2.py
/utils
__init__.py
utils1.py
utils2.py
And process1.py:
from utils import utils1, utils2 # relative import
import process2 # import from same package
# everything else
MyTask.py:
import processes.process1
process1.my_method()
More on packaging in python docs
Related
below the folder structure of my software:
below the code of all the .py files:
run.py:
import modules.module_01.aa as a
a.test()
# test:
if __name__=="__main__":
pass
aa.py (module 1):
import libraries.qq as q
import libraries.zz as z
def test():
q.qq_fun()
z.zz_fun()
print("ciao")
qq.py (library used by aa.py):
def qq_fun():
pass
zz.py (library used by aa.py):
def zz_fun():
pass
my question is really simple, why when I run "run.py" Python say to me:
why "aa.py" can't import the module "qq.py" and "zz.py"? how can I fix this issue?
run.py
In run.py, the Python interpreter thinks you're trying to import module_01.aa from a module named module. To import aa.py, you'll need to add this code to the top of your file, which adds the directory aa.py is in to the system path, and change your import statement to import aa as a.
import sys
sys.path.insert(0, "./modules/module_01/")
aa.py
The same problem occurs in aa.py. To fix the problem in this file, you'll need to add this code to the top of aa.py, which adds the directory qq.py and zz.py are in, and remove the libraries. from both of your import statements.
import sys
sys.path.insert(0, "./modules/module_01/libraries")
I have a Python project in which I have the following folder structure:
> root
> download_module
> __init__.py
> downloadProcess.py
> sharedFunctions.py
> someHelper.py
> useSharedFunction.py
The download_module/__init__.py has the following code:
from .sharedFunctions import stringArgumentToDate
from .downloadProcess import downloadProcessMethod
The sharedFunctions.py file contains the following function:
def stringArgumentToDate(arg):
dateformat = "%m/%d/%Y"
date = None
if arg.isnumeric():
date = datetime.fromtimestamp(int(arg))
if date == None:
date = datetime.strptime(arg, dateformat)
return date
Then on the useSharedFunction.py I try to import the shared function and use it like this.
from download_module import stringArgumentToDate
from download_module import downloadProcessMethod
def main():
arg = '03/14/2022'
dateArg = stringArgumentToDate(arg)
if __name__ == '__main__':
main()
When I try to run this by using python3 useSharedFunction.py I got the following error:
Traceback (most recent call last):
File "useSharedFunction.py", line 4, in <module>
from download_module import stringArgumentToDate
File "/Users/jacobo/Documents/project/download_module/__init__.py", line 2, in <module>
from .download_module import downloadAndProcessMethod
File "/Users/jacobo/Documents/project/download_module/downloadProcess.py", line 10, in <module>
from sharedFunctions import stringArgumentToDate, otherFunction
ModuleNotFoundError: No module named 'sharedFunctions'
I do believe the error is in downloadProcess since at the beggining of the file we got this import:
from sharedFunctions import stringArgumentToDate, otherFunction
from someHelper import Helper
Which refers to sibling files.
However I'm unsure what will be a proper fix to allow to run the downloadProcess.py main independently but also, being able to call it one of its method from a root or any other file out of the module.
Consider this structure:
┬ module
| ├ __init__.py
| ├ importing_submodule.py
| └ some_submodule.py
├ __main__.py
├ some_submodule.py
└ module_in_parent_dir.py
with content:
__main__.py
import module
/module/__init__.py
from . import importing_submodule
/module/importing_submodule.py
from some_submodule import SomeClass
/module/some_submodule.py
print("you imported from module")
class SomeClass:
pass
/some_submodule.py
print("you imported from root")
class SomeClass:
pass
/module_in_parent_dir.py
class SomeOtherClass:
pass
How sibling import works
(skip this section if you know already)
Now lets run __main__.py and it will say "you imported from root".
But if we change code a bit..
/module/importing_submodule.py
from module.some_submodule import SomeClass
It now says "You imported from module" as we wanted, probably with scary red line in IDE saying "Unresolved reference" if you didn't config working directory in IDE.
How this happen is simple: script root(Current working directory) is decided by main script(first script that's running), and python uses namespaces.
Python's import system uses 2 import method, and for convenience let's call it absolute import and relative import.
Absolute import: Import from dir listed in sys.path and current working directory
Relative import: Import relative to the very script that called import
And what decide the behavior is whether we use . at start of module name or not.
Since we imported by from some_submodule without preceeding dot, python take it as 'Absolute import'(the term we decided earlier).
And then when we also specified module name like from module.some_submodule python looks for module in path list or in current working directory.
Of course, this is never a good idea; script root can change via calls like os.chdir() then submodules inside module folder may get lost.
Therefore, the best practices for sibling import is using relative import inside module folder.
/module/importing_submodule.py
from .some_submodule import SomeClass
Making script that work in both way
To make submodule import it's siblings when running as main script, yet still work as submodule when imported by other script, then use try - except and look for ImportError.
For importing_submodule.py as an example:
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
Importing modules from parent directory is a bit more tricky.
Since submodule is now main script, relative import to parent level directory doesn't work.
So we need to add the parent directory to sys.path, when the script is running as main script.
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
# now since we don't have parent package, we just append the path.
from sys import path
import pathlib
path.append(pathlib.Path(__file__).parent.parent.as_posix())
print("Importing module_in_parent_dir from sys.path")
else:
print("Importing module_in_parent_dir from working directory")
# Now either case we have parent directory of `module_in_parent_dir`
# in working dir or path, we can import it
# might need to suppress false IDE warning this case.
# noinspection PyUnresolvedReferences
from module_in_parent_dir import SomeOtherClass
Output:
"C:\Program Files\Python310\python.exe" .../module/importing_module.py
you imported from module
Importing module_in_parent_dir from sys.path
Process finished with exit code 0
"C:\Program Files\Python310\python.exe" .../__main__.py
you imported from module
Importing module_in_parent_dir from working directory
Process finished with exit code 0
using pyscafold in order to build a module I get an structure as follows
scr
---module
------__init__.py
------file.py (containing func inside)
tests
---fileTest.py
How is the right way to import file.py in the test file fileTest.py?
So far many of this variations dont work:
import pytest
from ../src/module import func
including this in init.py does not help:
import os, sys
sys.path.append(os.path.dirname(os.path.realpath(__file__)))
These do not help:
from .src/module/myfile import func
from ..src/module/myfile import func
There should not be forward slashes in your imports.
Also:
add a __init__.py file in tests directory
in fileTest.py, you should have:
import pytest
from src.module.file import func
Then, make sure the current working directory is the parent directory of src and run pytest .
I tried to include file detect_simple.py, but it doesn't work.
Capture screnshoot: https://imgur.com/a/Rku8O1f
My code: in file app.py
__import__("Face_recognition/detect_simple")
from flask import Flask
app = Flask(__name__)
#app.route("/")
def main():
return "OK";
#app.route('/api1')
def api1():
return "OK"
if __name__ == "__main__":
app.run()
run command: python app.py
return error: `Import by filename is not supported.`
So for instance if your app.py is in the top directory and you need to import a module thats in a folder below that directory just use the from subfolder import mymodule.
For example if your directory looks like this:
- FlaskApp
- app.py
- Face_recognition
- detect_simple.py
Then use:
from Face_recognition import detect_simple
To expand on #Jab's answer: your Face_recognition directory must contain a (potentially empty) __init__.py file (with two leading and two trailing underscores) to be recognized by the import system.
Once your tree looks like this:
FlaskApp
|_ app.py
|_ Face_detection/
|_ __init__.py
|_ detect_simple.py
You'll be able to do a
from Face_detection.detect_simple import <whatever>
# or
import Face_detection.detect_simple
# or
from Face_detecting import detect_simple
Keep in mind that the import system is case-sensitive.
On a separate note, please avoid using the __import__ function in almost any case. Prefer the import statement if you're just importing stuff normally, or use importlib.import_module if you have to do programmatic imports (although this shouldn't be the default in most cases).
I am running a python project like this:
project
Test.py
COMMON.py
SYSTEM.py
PTEST1
Hello.py
when run the code "Test.py" it will show NameError, I am not sure why?
But if I replaced the "from SYSTEM import *" with "from COMMON import *" in Test.py and PTEST1/Hello.py, it works as expect.
#Test.py is like this:
from SYSTEM import *
myvalue.Hello.printf()
# COMMON.py is like this:
myvalue = lambda: None
from PTEST1.Hello import Hello
myvalue.Hello = Hello
# SYSTEM.py is like this:
from COMMON import *
#PTEST1/Hello.py
from SYSTEM import *
class Hello():
#staticmethod
def printf():
print("Hello1")
print(vars(myvalue))
I expect there is no "NameError" by not changing import code. BTW, my python is 3.6+
Good practice is to give filenames in lowercase.
It looks like you are creating a Python project under project/. Any directory needs to have a file __init__.py in every directory in order for it to be discovered in Python.
You then need to refer to modules by their full name (not relative naming).
So the directory structure should be:
project/
__init__.py
test.py
common.py
system.py
ptest1/
__init__.py
hello.py
Every time you refer to file you should give the full path.
# import everything from hello.py
from project.ptest1.hello import *
# import everything from common.py
from project.common import *
# import everything from system.py
from project.system import *