Following console output explains the issue
$ ls -r *
b:
y.py __init__.py
a:
x.pyc x.py __init__.pyc __init__.py
L-IDC42NDV7M-M:pytest rohit$ python
>>> import a.x as xx
>>> print xx.v
1
$ python b/y.py
Traceback (most recent call last):
File "b/y.py", line 1, in <module>
import a.x as xx
ImportError: No module named a.x
If python looks at current working directory for modules shouldn't 'python b/y.py' find a/x.py?
In Java sibling modules can be imported. I am looking for something
similar in python.
Any ideas?
Python does not search for current directory (.). But it search for the directory that containing the input script (./b). See The Module Search Path.
If you want current directory to be search, you can use PYTHONPATH=.
PYTHONPATH=. python b/y.py
or using -m option:
python -m b.y
The directory of the main program is automatically added to Python path. The module name of the main program is __main__, not b.y
You can either write a simple main program that is in the parent directory of a and b, or you can add that parent directory to PYTHONPATH.
If you call python very/long/path/to/the/main/program.py it does not mean that the current directory is the root of a deep namespace. I think, in Java is the main program relative deep in the namespace and therefore everything is different.
Related
In python I have a folder with three files
- __init__.py
- module.py
- test_module.py
in which the module module.py is imported inside the file test_module.py as follows:
from . import module
Of course, when I just run test_module.py I get an error
> python test_module.py
Traceback (most recent call last):
File "test_module.py", line 4, in <module>
from . import module
ImportError: attempted relative import with no known parent package
But as I set the PYTHONPATH to the absolute path to where I am working in
export PYTHONPATH=`pwd`
I expect the import to work (as I did set the PYTHONPATH). But to my surprise I get the same error!
So can I fix the relative import error without any code change?
Since the directory you describe (let's call it thatdirectory) is a package, as marked by an __init__ file, you can "fix" that by cding a directory higher up, then running
python -m thatdirectory.test_module
since running modules with -m fixes up sys.path in a way that makes this particular configuration work.
(In general, any code that messes with sys.path manually, or requires changes to PYTHONPATH to work, is broken in my eyes...)
When I try to $> python ./tools/test.py I get an import error that I cannot import a module that exists in the directory from which I am invoking python. However, I can import this module, $> python -c "import mod" works.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
What is python doing to the python path when I run the interpreter on a script that exists in a different directory? Is there a way to "lock" the current working directory so that I can get the import to work?
My setup:
./mod.py :
x = 5 # just for demonstration purposes
./tools/test.py :
from mod import x
# ... snip ... actual content
I am invoking python from the directory that contains mod.py and tools/:
$> python -c "from mod import x" # works fine
$> python tools/test.py
Traceback (most recent call last):
File "tools/test.py", line 1, in <module>
from mod import x
ModuleNotFoundError: No module named 'mod'
Note that the current directory, which contains mod.py and tools is not on my PYTHONPATH.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
It's not. It's not on PYTHONPATH, and it's not on sys.path. When you run a script by file path, it's the script's directory that gets added to sys.path. You can see what gets added to sys.path for each way of specifying a program to run in the Python command line docs.
I read that to use a python file as a module for import, I need to put __init__.py in the directory.
I have the following directory structure:
data_load
-- __init__.py
-- rand_data.py
etc
-- __init__.py
-- test.py
In test.py I import a class defined in rand_data and I get the error:
python test.py
Traceback (most recent call last):
File "test.py", line 8, in <module>
from data_load.rand_data import RandData
ModuleNotFoundError: No module named 'data_load'
change to the parent directory of etc and data_load and type
python -m etc.test
This should do the job.
Here a small test case (Assuming you're on a linux machine)
## create the test case
mkdir -p import_issue/data_load import_issue/etc
touch import_issue/data_load/__init__.py import_issue/etc/__init__.py
echo 'print("I am", __name__)' > import_issue/etc/test.py
echo 'from data_load.rand_data import RandData' >> import_issue/etc/test.py
echo 'print("Randdata = ", RandData)' >> import_issue/etc/test.py
echo "class RandData:" > import_issue/data_load/rand_data.py
echo ' pass' >> import_issue/data_load/rand_data.py
#
# now perform the test
cd import_issue
python -m etc.test
The reason why things in your initial example didn't work out as expected is, that you present working directory was probably etc
and if you load a python script in etc, then it tries to import load_data relative to the present working directory (etc) and below etc there is no directory named rand_data, that has a file __init__.py in it.
My suggestion to fix is to go up to the common parent directory (This will now be your present working directory) and import etc/test as a module.
The reason is, that test.py is in a directory etc with an init.py so you should import it as etc.test and not call it directly.
Calling a file, that is a module directly with etc/test.py is not really recommended and can provoke some rare confusing situations.
instead of running command in etc directory python test.py run the python script in parent directory of etc and data_load.
ie what you need to do in terminal is (considering you are still in etc directory in terminal)
1. `cd ..`
2. `python etc/test.py`
this will work.
why are you facing error is because in test.py python is looking in the current directory for data_load module, and since there is no data_load module in etc directory it is giving this error.
how to solve above problem
append the sys.path in code
run the script from project main folder which contains all the package and python module and scripts.
Suppose I have:
src/
__init__.py
a.py
b.py
Suppose __init__.py is an empty file, and a.py is just one line:
TESTVALUE = 5
Suppose b.py is:
from src import a
print(a.TESTVALUE)
Now in both Python 2.7 and Python 3.x, running b.py gives the result (5).
However, if I delete the file __init__.py, b.py still works in Python 3.x, but in Python 2.7, I get the error:
Traceback (most recent call last):
File "b.py", line 5, in <module>
from src import a
ImportError: No module named src
Why does Python 2.7 exhibit different behaviour in this situation?
Python 3 supports namespace packages that work without an __init__.py file.
Furthermore, these packages can be distribute over several directories. This means all directories on your sys.path that contain *.py files will be recognized as packages.
This breaks backwards compatibility in Python 3 in terms of imports. A typical problem is a directory in your current working directory that has a name like a library such as numpy and that contains Python files. While Python 2 ignores this directory, Python 3 will find it first and tries to import the library from there. This has bitten me several times.
I have a Python script, say myscript.py, that uses relative module imports, ie from .. import module1, where my project layout is as follows:
project
+ outer_module
- __init__.py
- module1.py
+ inner_module
- __init__.py
- myscript.py
- myscript.sh
And I have a Bash script, say myscript.sh, which is a wrapper for my python script, shown below:
#!/bin/bash
python -m outer_module.inner_module.myscript $#
This works to execute myscript.py and forwards the arguments to my script as desired, but it only works when I call ./outer_module/inner_module/myscript.sh from within the project directory shown above.
How can I make this script work from anywhere? For example, how can I make this work for a call like bash /root/to/my/project/outer_module/inner_module/myscript.sh?
Here are my attempts:
When executing myscript.sh from anywhere else, I get the error: No module named outer_module.inner_module. Then I tried another approach to execute the bash script from anywhere, by replacing myscript.sh with:
#!/bin/bash
scriptdir=`dirname "$BASH_SOURCE"`
python $scriptdir/myscript.py $#
When I execute myscript.sh as shown above, I get the following:
Traceback (most recent call last):
File "./inner_module/myscript.py", line 10, in <module>
from .. import module1
ValueError: Attempted relative import in non-package
Which is due to the relative import on the first line in myscript.py, as mentioned earlier, which is from .. import module1.
As the error message says:
ValueError: Attempted relative import in non-package
The solution to this is to create a package, and have your script execute with that package in its path.
You already have a package, as you have the __init__.py files in those directories; but you only have that package in your path when you are calling it from the project directory, as you mentioned; that's because . is in your Python path by default.
To fix this, just add the project directory to your Python path, and then invoke it with python -m outer_module.inner_module.myscript:
#!/bin/bash
export PYTHONPATH=$PYTHONPATH:$(dirname "$BASH_SOURCE")/../..
python -m outer_module.inner_module.myscript $#
You need to include the path to the outer module's parent directory in the PYTHONPATH environmental variable, then you can use the same command you used in the first script from anywhere.
The PYTHONPATH is where python searches for any modules you try to import:
#!/bin/bash
export PYTHONPATH=$PYTHONPATH:PATH/TO/MODULE/PARENTDIR
python -m outer_module.inner_module.myscript $#