I have a certain project structure:
- azima
- .vscode
- core
- project_setup.py
- helper
- log_helper
- venv
In project_setup.py:
import os
import json
import numpy as np
import pandas as pd
import random
from helper.log_helper import log
if __name__ == "__main__":
print('hello world')
Running this file in terminal:
(venv) rmali#rakeshmali:~/git/azima$ /home/rmali/git/azima/venv/bin/python /home/rmali/git/azima/core/project_setup.py
Traceback (most recent call last):
File "/home/rmali/git/azima/core/project_setup.py", line 6, in <module>
from helper.log_helper import log
ModuleNotFoundError: No module named 'helper'
I get this error. What am I doing wrong? Am I missing something?
But running like this python -m core.project_setup works.
Reason:
The path of folder azima does not in the sys.path(PYTHONPATH).
Solution:
You can do this to modify the PYTHONPATH:
Add these in the settings.json file to Modify the PYTHONPATH in the terminal:
"terminal.integrated.env.windows": {
"PYTHONPATH": "xxx/site-packages"
}
Create a .env file under your workspace, and add these settings in it to modify the PYTHONPATH for the extension and debugger:
PYTHONPATH=xxx/site-packages
You can refer to here to understand the effects of these two configurations.
Modify it directly in the python file. Add these codes in the b.py file.
import sys; sys.path.append("xxx/Project/src")
The reason that running
(venv) rmali#rakeshmali:~/git/azima$ python ./core/project_setup.py
fails while
(venv) rmali#rakeshmali:~/git/azima$ python -m core.project_setup
succeeds is that when running python -m <module-name, Python adds the current directory to the start of sys.path, which allows modules in that directory such as helper to be imported as top level modules, i.e. with import helper. Running python <script> does not add the current directory to the start of sys.path. Instead, Python adds the directory containing the script to the start of sys.path.
Here are the relevant sections of the docs.
The -m switch
As with the -c option, the current directory will be added to the start of sys.path.
and the docs for the -c option add
the current directory will be added to the start of sys.path (allowing modules in that directory to be imported as top level modules)
Docs for python <script>
If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.
If the script name refers to a directory or zipfile, the script name is added to the start of sys.path and the __main__.py file in that location is executed as the __main__ module.
Related
When I try to $> python ./tools/test.py I get an import error that I cannot import a module that exists in the directory from which I am invoking python. However, I can import this module, $> python -c "import mod" works.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
What is python doing to the python path when I run the interpreter on a script that exists in a different directory? Is there a way to "lock" the current working directory so that I can get the import to work?
My setup:
./mod.py :
x = 5 # just for demonstration purposes
./tools/test.py :
from mod import x
# ... snip ... actual content
I am invoking python from the directory that contains mod.py and tools/:
$> python -c "from mod import x" # works fine
$> python tools/test.py
Traceback (most recent call last):
File "tools/test.py", line 1, in <module>
from mod import x
ModuleNotFoundError: No module named 'mod'
Note that the current directory, which contains mod.py and tools is not on my PYTHONPATH.
I'm relying on the fact that ./ is (in effect) on the PYTHONPATH.
It's not. It's not on PYTHONPATH, and it's not on sys.path. When you run a script by file path, it's the script's directory that gets added to sys.path. You can see what gets added to sys.path for each way of specifying a program to run in the Python command line docs.
I'm trying to get a Python script running on my Ubuntu server. I have the following directory structure:
/home/pythontest
|_ __init__.py
|_ main.py
|_ module_a.py
inside module_a.py:
def print_a():
print('a')
inside main.py:
from pythontest.module_a import print_a
def execute():
print_a()
execute()
When I run main.py in PyCharm on my Windows machine, it prints a as expected, on my Linux machine, when I call python3 main.py I get a
Traceback (most recent call last):
File "main.py", line 1, in <module>
from pythontest.module_a import print_a
ModuleNotFoundError: No module named 'pythontest'
The __init__.py exists (and is completely empty) and I have added the directory /home/pythontest to the PYTHONPATH with the following command:
export PYTHONPATH="${PYTHONPATH}:/home/pythontest"
(testing this with echo $PYTHONPATHalso yields the correct path)
Additional Notes:
- The python3 version on my machine is Python 3.6.9
- My Server runs Ubuntu 18.04
- All those files are written in PyCharm on Windows and copied over via SSH
You're importing pythontest(.module_a), which is located in /home. That's what you're supposed to add to PYTHONPATH:
export PYTHONPATH=${PYTHONPATH}:/home
More details on [Python.Docs]: The import system.
Or you could not reference the package name from within it (consider relative imports):
from .module_a import print_a
Might also want to check [SO]: How PyCharm imports differently than system command prompt (Windows) (#CristiFati's answer), to see why does it work from PyCharm.
You need to change your import in main.py to:
from module_a import print_a
since module_a is a module that exists in the path that you exported.
I read that to use a python file as a module for import, I need to put __init__.py in the directory.
I have the following directory structure:
data_load
-- __init__.py
-- rand_data.py
etc
-- __init__.py
-- test.py
In test.py I import a class defined in rand_data and I get the error:
python test.py
Traceback (most recent call last):
File "test.py", line 8, in <module>
from data_load.rand_data import RandData
ModuleNotFoundError: No module named 'data_load'
change to the parent directory of etc and data_load and type
python -m etc.test
This should do the job.
Here a small test case (Assuming you're on a linux machine)
## create the test case
mkdir -p import_issue/data_load import_issue/etc
touch import_issue/data_load/__init__.py import_issue/etc/__init__.py
echo 'print("I am", __name__)' > import_issue/etc/test.py
echo 'from data_load.rand_data import RandData' >> import_issue/etc/test.py
echo 'print("Randdata = ", RandData)' >> import_issue/etc/test.py
echo "class RandData:" > import_issue/data_load/rand_data.py
echo ' pass' >> import_issue/data_load/rand_data.py
#
# now perform the test
cd import_issue
python -m etc.test
The reason why things in your initial example didn't work out as expected is, that you present working directory was probably etc
and if you load a python script in etc, then it tries to import load_data relative to the present working directory (etc) and below etc there is no directory named rand_data, that has a file __init__.py in it.
My suggestion to fix is to go up to the common parent directory (This will now be your present working directory) and import etc/test as a module.
The reason is, that test.py is in a directory etc with an init.py so you should import it as etc.test and not call it directly.
Calling a file, that is a module directly with etc/test.py is not really recommended and can provoke some rare confusing situations.
instead of running command in etc directory python test.py run the python script in parent directory of etc and data_load.
ie what you need to do in terminal is (considering you are still in etc directory in terminal)
1. `cd ..`
2. `python etc/test.py`
this will work.
why are you facing error is because in test.py python is looking in the current directory for data_load module, and since there is no data_load module in etc directory it is giving this error.
how to solve above problem
append the sys.path in code
run the script from project main folder which contains all the package and python module and scripts.
I have a module that I use in a number of scripts. Its path is C:\PYTHONprojects\utilities\utility_module.py There is an (empty) file called __init__.py in the same folder.
If I import the module inside the PyCharm environment, it works fine:
import utility_module as um but if do the same from the Python command line, I get:
ModuleNotFoundError: No module named 'utility_module'
But the folder is in the PATH:
From the windows command line:
In: PATH
Out:
......
C:\PYTHONprojects\utilities; C:\PYTHONprojects
.....
or from the Python Command Line:
In: import sys
sys.path
Out:
......
'C:\\PYTHONprojects', 'C:\\PYTHONprojects\\utilities'
In order to load your module, it needs to be part of your PYTHONPATH system env (not PATH) or sys.path.
Your pycharm loads the working directory into your path for you. When you do that on command line, you need to take care for that.
So either add your project folder to system env PYTHONPATH or before you load the module, execute this:
import sys
sys.path.append(r'C:\PYTHONprojects\utilities')
I have the following directory structure
project/
bin/
script
stuff/
__init__.py
mainfile.py
Inside of script, I have the following to enable command line execution:
#!/usr/bin/env python
from stuff import mainfile
This works, but I would have expected needing to jump up one level...
from ..stuff import mainfile
or
from project.stuff import mainfile
What am I missing here?
Actually none of your examples should work out of the box.
Let's modify bin/script.py somewhat:
#! /usr/bin/env python
import sys
print sys.path
from stuff import mainfile
This should yield something like
['.../project/bin', ...]
Traceback (most recent call last):
File "bin/script.py", line 6, in <module>
from stuff import mainfile
ImportError: No module named stuff
Only the directory of the script (not the current directory) is added to sys.path automatically:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter.* If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
Hence there's no stuff module on sys.path. I'm not sure about your environment setup, but this is the canonical result when no additional parameters are set up (e.g. PYTHONPATH).
Similarily,
from ..stuff import mainfile
will result in the classic ValueError: Attempted relative import in non-package. You are informed of the fact that you can only do relative imports relative to actual modules. Since inside script .. does not refer to a module (because the script itself is the top level module so to say), a relative import does not work here. In a manner of speaking from Python's perspective, there is no module above the script and thus .. does not refer to something tangible when used in the context of the script.
Note that this also means that it does not help to only make project and project/bin into modules themselves by dropping __init__.py marker files. Relative imports to a parent of the script are only possible if the parent of the script is actually something python has a concept of.
This is one of the reasons, why the -m command line switch exists making it possible to run a module from the command line. For example, given the above relative import
python -m project.bin.script
does the trick, but only if executed from the proper directory (projectdir/..).
That problem is even worse with
from project.stuff import mainfile
because the project directory is only automatically on sys.path when you start a script from the directory above project and do not specify a main script to run:
cd <projectdir>/..
python -m project.bin.script
# works
cd <projectdir>
python -m bin.script
# does not work, because `sys.path` starts with <projectdir>
# and it's `stuff.mainfile` now, not `project.stuff.mainfile`.
If you want to import modules from project in your script, fix up sys.path to your needs:
import sys
import os
sys.path.insert(0, os.path.dirname(sys.path[0]))
from stuff import mainfile
You need to first add the parent directory into sys.path. Try the following:
# This is a file in bin/ directory.
this_file_path = os.path.dirname(__file__)
sys.path.append(os.path.join(this_file_path, '..'))
import stuff.mainfile