My Directory is structured as follows:
superParent
│ app.py
├───custom_package
file1.py
file2.py
file3.py
file4.py
__init__.py
I am running app.py
I want to import functions from file1.py and file2.py into app.py. I also import functions from file3.py and file4.py in file1.py and file2.py.
Now when I include
from custom_package.file1 import func_abc
in app.py it throws an ModuleNotFound error. However,
from custom_package import file1
func_abc = file1.func_abc
works perfectly.
Similarly, when I am trying to import a function from file3.py into file1.py using:
from file3 import func_efg
in file1.py it throws an ModuleNotFound error. However,
from custom_package import file3
func_efg = file3.func_efg
works perfectly in file.py.
I have found a lot of similar questions in StackOverFlow, however, none of them addressed this peculiar (for me - not for everyone) behaviour. A lot of them suggest making changes to the PYTHONPATH or sys.append('path\to\custom_package') (while strongly arguing that we should not resort to this). How do we solve these import-related issues to import particular functions from custom-defined local packages from a relative directory (preferably without messing with sys.path at all)?
Additional Context (not sure if this is necessary):
I will be containerizing this and deploying the container to AKS.
Some additional StackOverflow questions which have addressed similar issues:
Can't import my own modules in Python
Importing files from different folder
ImportError: No module named <something>
Importing files from different folder
Importing modules from parent folder
How to do relative imports in Python?
relative import in Python 3
I would suggest to use the leading dot format for package relative imports
And also to add your submodules (file1, file2, etc) to custom_package/__init__.py as it is showed here: submodules
Related
I have a big project:
main
golang
src
file1.go
python
src
file1.py
file2.py
__init__.py
java
src
file1.java
scripts
script.py
validator.sh
venv
bin
pip
python3
pyyaml
dateutil
Python project will use interpreter from:
ven/bin/python3
So anywhere inside
file1.py
file2.py
I can use imports:
import pyyaml
import dateutil
And this will work, by running from CLI:
venv/bin/python3 python/src/file1.py
However I wish to use some functions from file1.py inside file2.py
And have "relative reference" like this (inside file2.py)
from src.file1 import some_function
But having this kind of import and running the same way as before from CLI fails with error:
ModuleNotFoundError: No module named 'src'
What should I do? Pay attention that I have init.py file.
When you do from src.file1 this is relative to your sys.path.
Usually your current working directory is the first element in sys.path.
Thus you need to cd to main/python and then run ../venv/bin/python3 src/file1.py to make the imports work.
I've had similar issues and thus I have created an experimental, new import library: ultraimport.
It gives you more control over your imports and allows file system based imports.
You could then write in file2.py:
import ultraimport
file1 = ultraimport('__dir__/file1.py)'
This will always work, no matter how you run your code
I can't make this work..
My structure is:
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
thetest.py:
from ..src.Process.thefile.py import sth
Running: pytest ./tests/thetest.py from program_name gives :
ValueError: attempted relative import beyond top-level package
I tried also other approaches but i am receiving various errors.
But I would expect for the above to work.
ValueError: Attempted relative import in non-package
States that you're trying to use relative import in the module, which are to be used for packages i.e. to make it a package add __init__.py and call the thetest.py from some file outside the package.
Directly running thetest.py from interpreter won't work.
Relative imports require that the module which uses them is being
imported itself either as package module.
Suggestion 1:
The current tests directory has a __init__.py file but that doesn't allow you to run it as a module (via shell) - to make your current (relative) import work, you need to import it in an external (to package) file/module - let's create a main.py (can name it anything you like):
main.py
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
src/Process/thefile.py:
s = 'Hello world'
tests/thetest.py:
from ..src.Process.thefile import s
print s
main.py:
from program_name.tests.thetest import s
Executing main.py:
[nahmed#localhost ~]$ python main.py
Hello world
Suggestion 2:
Execute the file just above root dir i.e. one level up the program_name/ , in the following fashion:
[nahmed#localhost ~]$ python -m program_name.tests.thetest
Hell World
P.S. relative imports are for packages, not modules.
Just solved a similar problem with a lot of googling.
Here's two solutions without changing the existing file structor:
1
The way to import module from parent folder from ..src.Process.thefile.py import sth is called "relative import".
It's only supported when launching as a package from the top-level package. In your case, that is launching command line from the directory which contains program_name/ and type (for win environment)
python -m program_name.tests.thetest
or simply (useful for many pytest files):
python -m pytest
2
Otherwise -- when trying to run a script alone or from a non top-level package --
you could manually add directory to the PYTHONPATH at run time.
import sys
from os import path
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
from src.Process.thefile import s
Try the first one first see if it's compatiable with the pytest framework. Otherwise the second one should always solve the problem.
Reference (How to fix "Attempted relative import in non-package" even with __init__.py)
When importing a file, Python only searches the current directory, the directory that the entry-point script is running from.
you can use sys.path to include different locations
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import thefile
Suppose I have a module rules with the following structure:
rules
├── conditions.py
├── __init__.py
In the script conditions.py, I'd like to import a class called RuleParserError defined in __init__.py. However, I haven't been able to figure out how to do this. (Following Python: How to import from an __init__.py file? I've tried
from . import RuleParserError
but this leads to an ImportError: cannot import name RuleParserError when trying to run conditions.py as __main__).
I see 'import from parent module' as an anti-pattern in Python. Imports should be the other way around. Importing from modules's __init__.py is especially problematic. As you noticed, importing module foo.bar from foo/bar.py involves importing foo/__init__.py first, and you may end up with a circular dependency. Adding a print("Importing", __name__) to your init files helps see the sequence and understand the problem.
I'd suggest that you moved the code you want to import in conditions.py from __init__.py to a separate lower-level module, and just import some names from that module in __init__.py to expose it at higher level.
Let's suppose that you had some class Bar in your __init__.py. I'd reorganize it the following way.
__init__.py:
from bar import Bar # exposed at the higher level, as it used to be.
bar.py:
class Bar(object): ...
conditions.py:
from . import Bar # Now it works.
Ideally an __init__.py should contain nothing but imports from lower-level modules, or nothing at all.
If you get
ImportError: attempted relative import with no known parent package
when you do like
import . from something
especially from the script executed, just try
from __init__ import something
Even though it could be problematic when there are many __init__.pys in sys.path, it would be helpful in some simple situaions.
adding __package__ on top the script
__package__ = "pkg.dir.dir"
resolving the issue
in this case
__package__ = "rules"
I am using PyCharm Community Edition 2016.3.1 but when I have two+ python files on the same directory, if I import one of them into the other it gets underlined saying:
# main.py
import MyClass1
No module named MyClass1 less... (Ctrl+F1) This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level items are supported better than instance items.
but when I execute my main.py it works properly.
Other regular imports like import sys don't get underlined.
If those two python files are under the same directory eg. MyDirectory, you need to import the classes using the MyDirectory as the root. So for example if you have the below project structure:
└── MyDirectory
├── file1.py (MyClass1)
└── file2.py (MyClass2)
To import the MyClass1 into the file2.py you can do it as below:
from MyDirectory.file1 import MyClass1
The reason you are getting this error is because you are not importing correctly.
Python imports follow this syntax.
import filename
This means you need to have a file name filename.py in the current directory. You can also import a specific class from that file like so.
from filename import MyCalss
This question already has answers here:
Python3 correct way to import relative or absolute?
(2 answers)
Closed 2 years ago.
I have the following directory:
mydirectory
├── __init__.py
├── file1.py
└── file2.py
I have a function f defined in file1.py.
If, in file2.py, I do
from .file1 import f
I get the following error:
SystemError: Parent module '' not loaded, cannot perform relative
import
Why? And how to make it work?
Launching modules inside a package as executables is a bad practice.
When you develop something you either build a library, which is intended to be imported by other programs and thus it doesn't make much sense to allow executing its submodules directly, or you build an executable in which case there's no reason to make it part of a package.
This is why in setup.py you distinguish between packages and scripts. The packages will go under site-packages while the scripts will be installed under /usr/bin (or similar location depending on the OS).
My recommendation is thus to use the following layout:
/
├── mydirectory
| ├── __init__.py
| ├── file1.py
└── file2.py
Where file2.py imports file1.py as any other code that wants to use the library mydirectory, with an absolute import:
from mydirectory.file1 import f
When you write a setup.py script for the project you simply list mydirectory as a package and file2.py as a script and everything will work. No need to fiddle with sys.path.
If you ever, for some reason, really want to actually run a submodule of a package, the proper way to do it is to use the -m switch:
python -m mydirectory.file1
This loads the whole package and then executes the module as a script, allowing the relative import to succeed.
I'd personally avoid doing this. Also because a lot of people don't even know you can do this and will end up getting the same error as you and think that the package is broken.
Regarding the currently accepted answer, which says that you should just use an implicit relative import from file1 import f because it will work since they are in the same directory:
This is wrong!
It will not work in python3 where implicit relative imports are disallowed and will surely break if you happen to have installed a file1 module (since it will be imported instead of your module!).
Even if it works the file1 will not be seen as part of the mydirectory package. This can matter.
For example if file1 uses pickle, the name of the package is important for proper loading/unloading of data.
When launching a python source file, it is forbidden to import another file, that is in the current package, using relative import.
In documentation it is said:
Note that relative imports are based on the name of the current module. Since the name of the main module is always "__main__", modules intended for use as the main module of a Python application must always use absolute imports.
So, as #mrKelley said, you need to use absolute import in such situation.
since file1 and file2 are in the same directory, you don't even need to have an __init__.py file. If you're going to be scaling up, then leave it there.
To import something in a file in the same directory, just do like this
from file1 import f
i.e., you don't need to do the relative path .file1 because they are in the same directory.
If your main function, script, or whatever, that will be running the whole application is in another directory, then you will have to make everything relative to wherever that is being executed.
myproject/
mypackage
├── __init__.py
├── file1.py
├── file2.py
└── file3.py
mymainscript.py
Example to import from one file to another
#file1.py
from myproject import file2
from myproject.file3 import MyClass
Import the package example to the mainscript
#mymainscript.py
import mypackage
https://docs.python.org/3/tutorial/modules.html#packages
https://docs.python.org/3/reference/import.html#regular-packages
https://docs.python.org/3/reference/simple_stmts.html#the-import-statement
https://docs.python.org/3/glossary.html#term-import-path
The variable sys.path is a list of strings that determines the interpreter’s search path for modules. It is initialized to a default path taken from the environment variable PYTHONPATH, or from a built-in default if PYTHONPATH is not set. You can modify it using standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
sys.path.insert(0, '/ufs/guido/myhaxxlib/python')
Inserting it at the beginning has the benefit of guaranteeing that the path is searched before others (even built-in ones) in the case of naming conflicts.