If I have
a.py
b.py
In b.py I can import a
But if I have
c.py
m
a.py
b.py
and in c.py do import m.b, suddenly in b.py I get ModuleNotFoundError: No module named 'a'
What's the problem? I don't see how the second case is any different from the first
So... the modules are searched in the directory of the module that was started initially. I just don't understand the reasoning.
I'm not asking how to fix the problem. But rather asking why there's a problem in the first place...
(tested on Python 3.8.8)
It's based what file you run python.
If you run with python c.py
file c.py use:
from m import a, b
file b.py also use:
from m import a
If you go python b.py you just need import a to use in b.py
so depend what file you run it's will be the parent for relative import.
Let's start with why your first one works.
Assuming you have a file structure such that:
c.py
m - |
|
__init__.py
a.py
b.py
Let's say your current directory is within the m folder and you run the command.
python b.py
If within your b.py you place something like:
import a
import sys
print(sys.path)
it will work just fine. But let's take a look at the output from print(sys.path).
On windows, this is will look something like ['C:\\path\\to\\myprogram\\m', '<path to python installation>', etc.]
The important thing is that the m folder is on your sys.path and that is how import a gets resolved.
However if we go up one level and run:
python c.py
you should immediately notice that m is no longer on your sys.path and instead is replaced by 'C:\\path\\to\\myprogram'.
That is why it fails. Python automatically includes your current working directory in sys.path and changing out of it means it no longer knows where to look for the import.
This is an example of an absolute import. You can manipulate where python looks for these imports by modifying sys.path to include the path of the file you want to import.
sys.path.append('C:\\path\\to\\myprogram\\m')
But there's a better way to do it. If m is a package or sub-package (includes an __init__.py) you can use a relative import.
from . import a
However, there is a small caveat to this.
You can only use relative imports when the file using them is being run as a module or package.
So running them directly as a top level script
python b.py
will produce
ImportError: attempted relative import with no known parent package
Python luckily has the ability to run a script as a module built in.
if you cd one level up so as to encapsulate your m package, you can run
python -m m.b
and have it run just fine.
Additionally, since b.py is being treated as a module in c.py because of the import, the relative import will work in this case as well.
python c.py
Related
The directory I have looks like this:
repository
/src
/main.py
/a.py
/b.py
/c.py
I run my program via python ./main.py and within main.py there's an important statement from a import some_func. I'm getting a ModuleNotFoundError: No module named 'a' every time I run the program.
I've tried running the Python shell and running the commands import b or import c and those work without any errors. There's nothing particularly special about a either, it just contains a few functions.
What's the problem and how can I fix this issue?
repository/
__init__.py
/src
__init__.py
main.py
a.py
b.py
c.py
In the __init__.py in repository, add the following line:
from . import repository
In the __init__.py in src, add the following line:
from . import main
from . import a
from . import b
from . import c
Now from src.a import your_func is going to work on main.py
Maybe you could try using a relative import, which allows you to import modules from other directories relative to the location of the current file.
Note that you will need to add a dot (.) before the module name when using a relative import, this indicates that the module is in the same directory as the current file:
from . import a
Or try running it from a different directory and appending the /src path like this:
import sys
sys.path.append('/src')
You could also try using the PYTHONPATH (environment variable) to add a directory to the search path:
Open your terminal and navigate to the directory containing the main.py file (/src).
Set the PYTHONPATH environment variable to include the current directory, by running the following command
export PYTHONPATH=$PYTHONPATH:$(pwd)
At last you could try to use the -m flag inside your command, so that Python knows to look for the a module inside the /src directory:
python -m src.main
I've had similar problems in the past. Imports in Python depend on a lot of things like how you run your program, as a script or as a module and what is your current working directory.
Thus I've created a new import library: ultraimport It gives the programmer more control over imports and lets you do file system based, relative imports.
Your main.py could look like this:
import ultraimport
a = ultraimport('__dir__/a.py')
This will always work, no matter how you run your code, no matter what is your sys.path and also no init files are necessary.
When defining multiple python 3 packages in a project, I cannot determine how to configure the environment (using VS Code) such that I can run a file in any package with intra-package imports without breaking extra-package imports.
Example with absolute paths:
src/
foo/
a.py
b.py
goo/
c.py
# b.py contents
from foo.a import *
# c.py contents
from foo.b import *
where PYTHONPATH="${workspaceFolder}\src", so it should be able to see the foo and goo directories.
I can run c.py, but running b.py gives a ModuleNotFoundError: "No module named 'foo'".
Modifying b.py to use relative paths:
# modified b.py contents
from a import *
Then allows me to run b.py, but attempting to then run c.py gives a ModuleNotFoundError: "No module named 'b'".
I have also added __init__.py files to foo/ and goo/ directories, but the errors still occurs.
I did set the cwd in the launch.json file to be the ${workspaceFolder} directory, if that is relevant.
A solution I was trying to avoid, but does work in-case there are no better solutions, is putting each individual package-dependency onto the PYTHONPATH environment variable.
E.g., "${workspaceFolder}\src;${workspaceFolder}\foo" when running c.py.
But this is not an easily scalable solution and is also something that needs to be tracked across project changes, so I do not think it is a particularly good or pythonic solution.
In fact, you can find it didn't work if you try to print the current path. It just add ".." to the path instead of the parent directory.
import sys
print(sys.path)
sys.path.append('..')
print(sys.path)
So, what we have to do is add the path of module we used to "sys" in path.
Here we have three ways:
Put the written .py file into the directory that has been added to the system environment variable;
Create a new .pth file under \Python\Python310\Lib\site-packages.
Write the path of the module in this new .pth file , one path per line.
Using the pythonpath environment variable as upstairs said.
Then we can use this two ways to import :
sys. path. append ('a').
sys. path. insert (0, 'a')
I have a directory structure:
/somedir
/x
A.py
B.py
/anotherdir
/y
C.py
B imports A. This works when I run B.py from somedir/x/. However, in C.py when I try to import B.py and then run
$> python C.py
It complains about A not being found. I can add somedir/x/ to the pythonpath so that when I run C.py, it can find B and also A. But, I am concerned if there will be another A in a different directory (diffdir/z/A) I use which might conflict with the pythonpath that includes somedir/x/A.
I am guessing the basic issue you are getting is because of the change to intra-package references between Python 2.x and Python 3.x. From documentation -
6.4.2. Intra-package References
When packages are structured into subpackages (as with the sound package in the example), you can use absolute imports to refer to submodules of siblings packages. For example, if the module sound.filters.vocoder needs to use the echo module in the sound.effects package, it can use from sound.effects import echo .
Basically, if you are importing B and B imports a sibling, in Python 3.x you would need to use absolute path to import A in B . From what you said , that you are importing B as -
import somedir.x.B
Then in B you would need to import A as -
import somedir.x.A
Or you can also try -
from . import A
Imagine the directory structure:
/
a/
__init__.py
b.py
c.py
c.py
File /a/b.py looks like:
import c
should_be_absolute = c
All the other files (including __init__) are empty.
When running a test script (using python 2.7):
import a.b
print a.b.should_be_absolute
with PYTHONPATH=/ from an empty directory (so nothing is added to PYTHONPATH from current directory) I get
<module 'a.c' from '/a/c.py'>
where according to PEP 328 and the statement import <> is always absolute I would expect:
<module 'c' from '/c.py'>
The output is as expected when I remove the /a/c.py file.
What am I missing? And if this is the correct behavior - how to import the c module from b (instead of a.c)?
Update:
According to python dev mailing list it appears to be a bug in the documentation. The imports are not absolute by default in python27.
you need to add from __future__ import absolute_import or use importlib.import_module('c') on Python 2.7
It is default on Python 3.
There was a bug in Python: __future__.py and its documentation claim absolute imports became mandatory in 2.7, but they didn't.
If you are only adding / to your PYTHONPATH, then the search order could still be looking for c in the current directory. It would be a lot better if you placed everything under a root package, and referred to it absolutely:
/myPackage
a/
__init__.py
b.py
c.py
__init__.py
c.py
And a PYTHONPATH like: export PYTHONPATH=/:$PYTHONPATH
So in your a.c you would do and of these:
from myPackage import c
from myPackage.c import Foo
import myPackage.c
This way, it is always relative to your package.
"Absolute" doesn't mean the one that you think it does; instead, it means that the "usual" package resolving procedure takes place: first, it looks in the directory of the package, then in all the elements of sys.path; which includes the elements from PYTHONPATH.
If you really want to, you can use tools like the imp module, but I'd recommend against it for something like this. Because in general, you shouldn't ever have to create a module with the same name as one in the standard Python distribution.
I have the following directory structure:
test1/
test1/a.py
test1/test2/b.py
b.py needs to import a class in a.py. So I can add the following line to b.py before importing a.
sys.path.append(os.path.dirname(sys.argv[0]) + "/..")
This works and I can invoke b.py from any directory and it is able to import a.
But this fails when I write a script in another directory to invoke this file using execfile().
I tried relative imports but I get a "Attempted Relative Import in Non-Package error"
from ..a import someclass as cls
I have __init__.py in both test1, test2
Does someone have an idea how to make it work?
Is PYTHONPATH the way to go?
The problem is that execfile will evaluate the file you are calling as pure python code. Every relative import statement inside of b.py (and any package module imported by it) will have to remain true to your calling script.
One solution is to not use any relative import paths within the package. Make sure test1 package is on your PYTHONPATH as well.
b.py
from test1 import a
With test1 in your PYTHONPATH, the import of a should success in your execfile
>>> import sys
>>> sys.path.append('/path/to/parent/of_test1')
>>> execfile('/path/to/parent/of_test1/test1/test2/b.py')