I read that to use a python file as a module for import, I need to put __init__.py in the directory.
I have the following directory structure:
data_load
-- __init__.py
-- rand_data.py
etc
-- __init__.py
-- test.py
In test.py I import a class defined in rand_data and I get the error:
python test.py
Traceback (most recent call last):
File "test.py", line 8, in <module>
from data_load.rand_data import RandData
ModuleNotFoundError: No module named 'data_load'
change to the parent directory of etc and data_load and type
python -m etc.test
This should do the job.
Here a small test case (Assuming you're on a linux machine)
## create the test case
mkdir -p import_issue/data_load import_issue/etc
touch import_issue/data_load/__init__.py import_issue/etc/__init__.py
echo 'print("I am", __name__)' > import_issue/etc/test.py
echo 'from data_load.rand_data import RandData' >> import_issue/etc/test.py
echo 'print("Randdata = ", RandData)' >> import_issue/etc/test.py
echo "class RandData:" > import_issue/data_load/rand_data.py
echo ' pass' >> import_issue/data_load/rand_data.py
#
# now perform the test
cd import_issue
python -m etc.test
The reason why things in your initial example didn't work out as expected is, that you present working directory was probably etc
and if you load a python script in etc, then it tries to import load_data relative to the present working directory (etc) and below etc there is no directory named rand_data, that has a file __init__.py in it.
My suggestion to fix is to go up to the common parent directory (This will now be your present working directory) and import etc/test as a module.
The reason is, that test.py is in a directory etc with an init.py so you should import it as etc.test and not call it directly.
Calling a file, that is a module directly with etc/test.py is not really recommended and can provoke some rare confusing situations.
instead of running command in etc directory python test.py run the python script in parent directory of etc and data_load.
ie what you need to do in terminal is (considering you are still in etc directory in terminal)
1. `cd ..`
2. `python etc/test.py`
this will work.
why are you facing error is because in test.py python is looking in the current directory for data_load module, and since there is no data_load module in etc directory it is giving this error.
how to solve above problem
append the sys.path in code
run the script from project main folder which contains all the package and python module and scripts.
Related
I have a certain project structure:
- azima
- .vscode
- core
- project_setup.py
- helper
- log_helper
- venv
In project_setup.py:
import os
import json
import numpy as np
import pandas as pd
import random
from helper.log_helper import log
if __name__ == "__main__":
print('hello world')
Running this file in terminal:
(venv) rmali#rakeshmali:~/git/azima$ /home/rmali/git/azima/venv/bin/python /home/rmali/git/azima/core/project_setup.py
Traceback (most recent call last):
File "/home/rmali/git/azima/core/project_setup.py", line 6, in <module>
from helper.log_helper import log
ModuleNotFoundError: No module named 'helper'
I get this error. What am I doing wrong? Am I missing something?
But running like this python -m core.project_setup works.
Reason:
The path of folder azima does not in the sys.path(PYTHONPATH).
Solution:
You can do this to modify the PYTHONPATH:
Add these in the settings.json file to Modify the PYTHONPATH in the terminal:
"terminal.integrated.env.windows": {
"PYTHONPATH": "xxx/site-packages"
}
Create a .env file under your workspace, and add these settings in it to modify the PYTHONPATH for the extension and debugger:
PYTHONPATH=xxx/site-packages
You can refer to here to understand the effects of these two configurations.
Modify it directly in the python file. Add these codes in the b.py file.
import sys; sys.path.append("xxx/Project/src")
The reason that running
(venv) rmali#rakeshmali:~/git/azima$ python ./core/project_setup.py
fails while
(venv) rmali#rakeshmali:~/git/azima$ python -m core.project_setup
succeeds is that when running python -m <module-name, Python adds the current directory to the start of sys.path, which allows modules in that directory such as helper to be imported as top level modules, i.e. with import helper. Running python <script> does not add the current directory to the start of sys.path. Instead, Python adds the directory containing the script to the start of sys.path.
Here are the relevant sections of the docs.
The -m switch
As with the -c option, the current directory will be added to the start of sys.path.
and the docs for the -c option add
the current directory will be added to the start of sys.path (allowing modules in that directory to be imported as top level modules)
Docs for python <script>
If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.
If the script name refers to a directory or zipfile, the script name is added to the start of sys.path and the __main__.py file in that location is executed as the __main__ module.
When I try to $> python ./tools/test.py I get an import error that I cannot import a module that exists in the directory from which I am invoking python. However, I can import this module, $> python -c "import mod" works.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
What is python doing to the python path when I run the interpreter on a script that exists in a different directory? Is there a way to "lock" the current working directory so that I can get the import to work?
My setup:
./mod.py :
x = 5 # just for demonstration purposes
./tools/test.py :
from mod import x
# ... snip ... actual content
I am invoking python from the directory that contains mod.py and tools/:
$> python -c "from mod import x" # works fine
$> python tools/test.py
Traceback (most recent call last):
File "tools/test.py", line 1, in <module>
from mod import x
ModuleNotFoundError: No module named 'mod'
Note that the current directory, which contains mod.py and tools is not on my PYTHONPATH.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
It's not. It's not on PYTHONPATH, and it's not on sys.path. When you run a script by file path, it's the script's directory that gets added to sys.path. You can see what gets added to sys.path for each way of specifying a program to run in the Python command line docs.
I'm trying to get a Python script running on my Ubuntu server. I have the following directory structure:
/home/pythontest
|_ __init__.py
|_ main.py
|_ module_a.py
inside module_a.py:
def print_a():
print('a')
inside main.py:
from pythontest.module_a import print_a
def execute():
print_a()
execute()
When I run main.py in PyCharm on my Windows machine, it prints a as expected, on my Linux machine, when I call python3 main.py I get a
Traceback (most recent call last):
File "main.py", line 1, in <module>
from pythontest.module_a import print_a
ModuleNotFoundError: No module named 'pythontest'
The __init__.py exists (and is completely empty) and I have added the directory /home/pythontest to the PYTHONPATH with the following command:
export PYTHONPATH="${PYTHONPATH}:/home/pythontest"
(testing this with echo $PYTHONPATHalso yields the correct path)
Additional Notes:
- The python3 version on my machine is Python 3.6.9
- My Server runs Ubuntu 18.04
- All those files are written in PyCharm on Windows and copied over via SSH
You're importing pythontest(.module_a), which is located in /home. That's what you're supposed to add to PYTHONPATH:
export PYTHONPATH=${PYTHONPATH}:/home
More details on [Python.Docs]: The import system.
Or you could not reference the package name from within it (consider relative imports):
from .module_a import print_a
Might also want to check [SO]: How PyCharm imports differently than system command prompt (Windows) (#CristiFati's answer), to see why does it work from PyCharm.
You need to change your import in main.py to:
from module_a import print_a
since module_a is a module that exists in the path that you exported.
I have a Python script, say myscript.py, that uses relative module imports, ie from .. import module1, where my project layout is as follows:
project
+ outer_module
- __init__.py
- module1.py
+ inner_module
- __init__.py
- myscript.py
- myscript.sh
And I have a Bash script, say myscript.sh, which is a wrapper for my python script, shown below:
#!/bin/bash
python -m outer_module.inner_module.myscript $#
This works to execute myscript.py and forwards the arguments to my script as desired, but it only works when I call ./outer_module/inner_module/myscript.sh from within the project directory shown above.
How can I make this script work from anywhere? For example, how can I make this work for a call like bash /root/to/my/project/outer_module/inner_module/myscript.sh?
Here are my attempts:
When executing myscript.sh from anywhere else, I get the error: No module named outer_module.inner_module. Then I tried another approach to execute the bash script from anywhere, by replacing myscript.sh with:
#!/bin/bash
scriptdir=`dirname "$BASH_SOURCE"`
python $scriptdir/myscript.py $#
When I execute myscript.sh as shown above, I get the following:
Traceback (most recent call last):
File "./inner_module/myscript.py", line 10, in <module>
from .. import module1
ValueError: Attempted relative import in non-package
Which is due to the relative import on the first line in myscript.py, as mentioned earlier, which is from .. import module1.
As the error message says:
ValueError: Attempted relative import in non-package
The solution to this is to create a package, and have your script execute with that package in its path.
You already have a package, as you have the __init__.py files in those directories; but you only have that package in your path when you are calling it from the project directory, as you mentioned; that's because . is in your Python path by default.
To fix this, just add the project directory to your Python path, and then invoke it with python -m outer_module.inner_module.myscript:
#!/bin/bash
export PYTHONPATH=$PYTHONPATH:$(dirname "$BASH_SOURCE")/../..
python -m outer_module.inner_module.myscript $#
You need to include the path to the outer module's parent directory in the PYTHONPATH environmental variable, then you can use the same command you used in the first script from anywhere.
The PYTHONPATH is where python searches for any modules you try to import:
#!/bin/bash
export PYTHONPATH=$PYTHONPATH:PATH/TO/MODULE/PARENTDIR
python -m outer_module.inner_module.myscript $#
Following console output explains the issue
$ ls -r *
b:
y.py __init__.py
a:
x.pyc x.py __init__.pyc __init__.py
L-IDC42NDV7M-M:pytest rohit$ python
>>> import a.x as xx
>>> print xx.v
1
$ python b/y.py
Traceback (most recent call last):
File "b/y.py", line 1, in <module>
import a.x as xx
ImportError: No module named a.x
If python looks at current working directory for modules shouldn't 'python b/y.py' find a/x.py?
In Java sibling modules can be imported. I am looking for something
similar in python.
Any ideas?
Python does not search for current directory (.). But it search for the directory that containing the input script (./b). See The Module Search Path.
If you want current directory to be search, you can use PYTHONPATH=.
PYTHONPATH=. python b/y.py
or using -m option:
python -m b.y
The directory of the main program is automatically added to Python path. The module name of the main program is __main__, not b.y
You can either write a simple main program that is in the parent directory of a and b, or you can add that parent directory to PYTHONPATH.
If you call python very/long/path/to/the/main/program.py it does not mean that the current directory is the root of a deep namespace. I think, in Java is the main program relative deep in the namespace and therefore everything is different.