I'm working on migrating an exsting Python 2.7 project to Python 3.9. I'm facing a directory structure-related issue in Python 3.
My current project directory structure is:
├───project
│ ├───core
| +--__init__.py
| +--main.py
| +--settings.py
│ ├───jobs
| +--job.py
main.py:
import settings
class Main:
def __init__(self, a=None, b=settings.B):
self.a = a
self.b = b
def start(self):
print(self.a, self.b)
job.py:
import sys
# sys.path.insert(0, '../core/')
from core.main import Main
from core import settings
main = Main(settings.A)
main.start()
There is no issues with this structure when Python 2.7 interpreter is used, but in Python 3.9 I see the following error when job.py is executed:
File "project\core\main.py", line 1, in <module>
import settings
ModuleNotFoundError: No module named 'settings'
The issue is fixable by uncommenting the code on line #2 of the job.py script, but I would like to avoid hardcoding folder values like that. I would appreciate if someone could provide an alternative approach and an explanation why it's behaving this way in the newer Python version.
Due to ambiguity absolute import was removed in python3 for such use case. (This explains it very well: Changes in import statement python3)
You can use relative import for this use case maybe - https://docs.python.org/3/reference/import.html#package-relative-imports
Related
below the folder structure of my software:
below the code of all the .py files:
run.py:
import modules.module_01.aa as a
a.test()
# test:
if __name__=="__main__":
pass
aa.py (module 1):
import libraries.qq as q
import libraries.zz as z
def test():
q.qq_fun()
z.zz_fun()
print("ciao")
qq.py (library used by aa.py):
def qq_fun():
pass
zz.py (library used by aa.py):
def zz_fun():
pass
my question is really simple, why when I run "run.py" Python say to me:
why "aa.py" can't import the module "qq.py" and "zz.py"? how can I fix this issue?
run.py
In run.py, the Python interpreter thinks you're trying to import module_01.aa from a module named module. To import aa.py, you'll need to add this code to the top of your file, which adds the directory aa.py is in to the system path, and change your import statement to import aa as a.
import sys
sys.path.insert(0, "./modules/module_01/")
aa.py
The same problem occurs in aa.py. To fix the problem in this file, you'll need to add this code to the top of aa.py, which adds the directory qq.py and zz.py are in, and remove the libraries. from both of your import statements.
import sys
sys.path.insert(0, "./modules/module_01/libraries")
I have a Python project in which I have the following folder structure:
> root
> download_module
> __init__.py
> downloadProcess.py
> sharedFunctions.py
> someHelper.py
> useSharedFunction.py
The download_module/__init__.py has the following code:
from .sharedFunctions import stringArgumentToDate
from .downloadProcess import downloadProcessMethod
The sharedFunctions.py file contains the following function:
def stringArgumentToDate(arg):
dateformat = "%m/%d/%Y"
date = None
if arg.isnumeric():
date = datetime.fromtimestamp(int(arg))
if date == None:
date = datetime.strptime(arg, dateformat)
return date
Then on the useSharedFunction.py I try to import the shared function and use it like this.
from download_module import stringArgumentToDate
from download_module import downloadProcessMethod
def main():
arg = '03/14/2022'
dateArg = stringArgumentToDate(arg)
if __name__ == '__main__':
main()
When I try to run this by using python3 useSharedFunction.py I got the following error:
Traceback (most recent call last):
File "useSharedFunction.py", line 4, in <module>
from download_module import stringArgumentToDate
File "/Users/jacobo/Documents/project/download_module/__init__.py", line 2, in <module>
from .download_module import downloadAndProcessMethod
File "/Users/jacobo/Documents/project/download_module/downloadProcess.py", line 10, in <module>
from sharedFunctions import stringArgumentToDate, otherFunction
ModuleNotFoundError: No module named 'sharedFunctions'
I do believe the error is in downloadProcess since at the beggining of the file we got this import:
from sharedFunctions import stringArgumentToDate, otherFunction
from someHelper import Helper
Which refers to sibling files.
However I'm unsure what will be a proper fix to allow to run the downloadProcess.py main independently but also, being able to call it one of its method from a root or any other file out of the module.
Consider this structure:
┬ module
| ├ __init__.py
| ├ importing_submodule.py
| └ some_submodule.py
├ __main__.py
├ some_submodule.py
└ module_in_parent_dir.py
with content:
__main__.py
import module
/module/__init__.py
from . import importing_submodule
/module/importing_submodule.py
from some_submodule import SomeClass
/module/some_submodule.py
print("you imported from module")
class SomeClass:
pass
/some_submodule.py
print("you imported from root")
class SomeClass:
pass
/module_in_parent_dir.py
class SomeOtherClass:
pass
How sibling import works
(skip this section if you know already)
Now lets run __main__.py and it will say "you imported from root".
But if we change code a bit..
/module/importing_submodule.py
from module.some_submodule import SomeClass
It now says "You imported from module" as we wanted, probably with scary red line in IDE saying "Unresolved reference" if you didn't config working directory in IDE.
How this happen is simple: script root(Current working directory) is decided by main script(first script that's running), and python uses namespaces.
Python's import system uses 2 import method, and for convenience let's call it absolute import and relative import.
Absolute import: Import from dir listed in sys.path and current working directory
Relative import: Import relative to the very script that called import
And what decide the behavior is whether we use . at start of module name or not.
Since we imported by from some_submodule without preceeding dot, python take it as 'Absolute import'(the term we decided earlier).
And then when we also specified module name like from module.some_submodule python looks for module in path list or in current working directory.
Of course, this is never a good idea; script root can change via calls like os.chdir() then submodules inside module folder may get lost.
Therefore, the best practices for sibling import is using relative import inside module folder.
/module/importing_submodule.py
from .some_submodule import SomeClass
Making script that work in both way
To make submodule import it's siblings when running as main script, yet still work as submodule when imported by other script, then use try - except and look for ImportError.
For importing_submodule.py as an example:
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
Importing modules from parent directory is a bit more tricky.
Since submodule is now main script, relative import to parent level directory doesn't work.
So we need to add the parent directory to sys.path, when the script is running as main script.
/module/importing_submodule.py
try:
from .some_submodule import SomeClass
except ImportError:
# attempted relative import with no known parent package
# because this is running as main script, there's no parent package.
from some_submodule import SomeClass
# now since we don't have parent package, we just append the path.
from sys import path
import pathlib
path.append(pathlib.Path(__file__).parent.parent.as_posix())
print("Importing module_in_parent_dir from sys.path")
else:
print("Importing module_in_parent_dir from working directory")
# Now either case we have parent directory of `module_in_parent_dir`
# in working dir or path, we can import it
# might need to suppress false IDE warning this case.
# noinspection PyUnresolvedReferences
from module_in_parent_dir import SomeOtherClass
Output:
"C:\Program Files\Python310\python.exe" .../module/importing_module.py
you imported from module
Importing module_in_parent_dir from sys.path
Process finished with exit code 0
"C:\Program Files\Python310\python.exe" .../__main__.py
you imported from module
Importing module_in_parent_dir from working directory
Process finished with exit code 0
I am developping a new little project which need to run on Windows and Linux. To explain my problem I will use 3 files.
parser/__init__.py
from .toto import Parser as TotoParser
parser/toto.py
class Variable(object):
def __str__(self):
return "totoVariable"
class Parser(object):
#staticmethod
def parse(data):
return Variable()
main.py
#!/usr/bin/env python3
from parser import TotoParser
def main():
print(TotoParser.parse(""))
if __name__ == '__main__':
main()
In this project. I create several modules(file) into different packages(directory). The thing is I need to change the name of module imported. To do that I use aliasing into __init__ files.
My project run perfectly on Lunix but when I tried it on Windows this problem occurs !
ImportError: cannot import name 'TotoParser'
Sorry for my English, I am learning it...
Please rename init.py to __init__.py, I believe that it is work, case already named as __init__.py ignore this anwser...
I'm trying to make some code Python 3 compatible. To my understanding, in Python 3, importing a module from the same package must be relative. So if this is the directory structure:
/package
/modA
/modB
there's a lot of existing code that looks like this (in modA):
import modB
modB.some_function()
Is there a way to change the imports so I can keep calling the methods/properties the same way?
I know you can do this:
from .modB import some_function
some_function()
But I'd like to avoid changing all the calls if possible. When I call it like so:
import .modB
it gives an error
The syntax to relative-import a module modB from the same package would be
from . import modB
This extends the answer of #user2357112
In order to work in both python2 and python3 you have to:
Structure:
/package
__init__.py
modA.py
modB.py
content of __init__.py
from . import modA, modB
content of modA.py
# from . import modB ## works in py3 but not in py2
def testA():
""" call a function from modB """
from . import modB
modB.printB()
def printA():
print('I am in modA')
content of modB.py
# from . import modA ## works in py3 not in py2
def testB():
""" call a function from modA """
from . import modA
modA.printA()
def printB():
print('I am in modB')
Then you can do:
import package
package.modA.testA()
package.modB.testB()
you can use importlib
import importlib
and then
modB = importlib.import_module('package.modB')
... granted it's not optimal but still a working option if you're having persistent issues with relative path imports (never run into issues on POSIX and OSx, but on NT platform I have)
I'm having an issue with the import statement in Python 3. I'm following a book (Python 3 Object Oriented) and am having the following structure:
parent_directory/
main.py
ecommerce/
__init__.py
database.py
products.py
payments/
__init__.py
paypal.py
authorizenet.py
In paypal.py, I'm trying to use the Database class from database.py. So I tried this:
from ecommerce.database import Database
I get this error:
ImportError: No module named 'ecommerce'
so I try with both of these import statements:
from .ecommerce.database import Database
from ..ecommerce.database import Database
and I get this error:
SystemError: Parent module '' not loaded, cannot perform relative import
What am I doing wrong or missing?
Thank you for your time!
Add your parent_directoryto Python's search path. For example so:
import sys
sys.path.append('/full/path/to/parent_directory')
Alternatively, you can add parent_directory to the environmental variable PYTHONPATH.