Normally I have to do in every file something like this :
import os, sys
sys.path.extend(['lib', '../lib'])
Is there a easy way. So I dont have to do this in every file ?
So that I can simply do :
from abc import *
cant understand how this work... https://docs.python.org/2/library/pkgutil.html#module-pkgutil
but probably some __init__.py trick may work !
You can modify the environment variable PYTHONPATH. See the documentation.
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
Related
I ran my project and received the following error:
File "/home/nguyentv/schoollink/web/views/apis.py", line 10, in <module>
from util.redis.redis_client import Redis
ImportError: No module named util.redis.redis_client
How do I properly import this library?
The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory). PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended. See section Standard Modules for more information.
Basically, the interpreter is going to perform a lookup in your current working directory, then it will search through the system defined library directories.
The issue you are facing could be that your code is looking for a module that does not exist, you are calling the script from an incorrect directory, or the sys.path is setup incorrectly.
I could help more if you showed how you instantiated the interpreter, pwd output, and tree output.
You are trying to import Redis from a package named util. Unless this package is part of your application, it does not exist.
According to python-redis' documentation, here is how to import it:
import redis
# then use redis.Redis(...)
or, equivalently:
from redis import Redis
# then use Redis(...)
I am using Python 2.7. I have the following directory structure:
alogos
- resources
- __init__.py
- test.py
- lib
- __init__.py
- utils.py
- common
- config
- config.json
My utils.py is the following:
def read_json_data(filename):
with open(filename) as data:
json_data=json.load(data)
return json_data
My test.py has the following:
from lib.utils import read_json_data
running_data = read_json_data('common/config/config.json')
print running_data
when I try to run python test.py from the resources directory, I get the following error:
ImportError: No module named lib.utils
What is the correct way to access files and modules
Your lib.utils module is not present in the current directory (and apparently not anywhere else import checks), and so the import fails.
The Python doc details the module search path:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
* the directory containing the input script (or the current directory).
* PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
* the installation-dependent default.
After initialization, Python programs can modify sys.path. The
directory containing the script being run is placed at the beginning
of the search path, ahead of the standard library path. This means
that scripts in that directory will be loaded instead of modules of
the same name in the library directory. This is an error unless the
replacement is intended. See section Standard Modules for more
information.
While this is certainly not the only way, what I would do would be to have your lib.utils module as a separate module, stored in a local Pypi server (Artifactory is one example, but there are others, such as devpi) where you can install it just like any other module, just from a different index URL from the regular Pypi. That way, any of your scripts can use it just like any other module, and it obviates the need to play assorted path-related games that can add unnecessary complexity.
This questions is detailing a behavior that I can't explain to myself.
src/package/__init__.py is empty but present.
src/package/subpackage/__init__.py:
pink = 'It works'
src/package/test/test.py:
import package.subpackage as subpackage
# I also tried `import package.subpackage as subpackage
print subpackage.pink
Calling from src: python package/test/test.py just fails with ImportError: No module named subpackage. Please note that import package doesn't work either.
NB: (Running an interpreter from src and typing the import statement works perfectly well.
Should I understand that I'm not suppose to call subfile of a package? In my project it's a test file so it sounds logical for me have it here.
Why the current working directory is not in the import path?
Many thanks for those who reads and those who answers.
Because you package is not in $PYTHONPATH. If you what to call test.py, you can move your test.py file to src/ directory, or add src to $PYTHONPATH
PYTHONPATH="/path/to/src:$PYTHONPATH"
export PYTHONPATH
From Documentation
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path
>>> import sys
>>> sys.path
The output is like this
['.', '/usr/bin', ...
This means that the current directory is in sys.path as well. If you want to import a module, please make sure that the module path is in sys.path, by adding your package directory to the environment variable PYTHONPATH, or changing your current directory or script directory to the package directory.
On python package/test/test.py fails, it's also ran from src:
when you starts a intepreter from src, '' is in sys.path, so path of src could be found;
when you run python package/test/test.py from src, '' is missing from sys.path, although os.path.abspath('.') shows current dir is "<xxx>\\src", "<xxx>\\src" is not in sys.path, while "<xxx>\\src\\package\\test" is in sys.path. That's saying, python adds path of the file to sys.path, not the path where you run the script.
see what the docs says:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
I wasn't clear how to correctly name this question.
Case 1
Assume that I have the following directory structure.
foo
|
+- bar/__init__.py
|
+- bar.py
If I have
from foo import bar
How do I know which bar (bar.py or bar/__init__.py) is being imported? Is there any easy way to automatically detect this from occurring?
Case 2
foo
|
+- foo.py
|
+- other.py
If other.py has the line
import foo
How do I know which foo (foo or foo.foo) is being imported? Again, is tehre any easy way to automatically detect this from occurring?
TLDR; a package takes precedence over a module of the same name if they are in the same directory.
From the docs:
"When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH. This has the same syntax as the shell variable PATH, that is, a list of directory names."
This is a bit misleading because the interpreter will also look for a package called spam (a directory called spam containing an __init__.py file). Since the directory entries are sorted before searching, packages take precedence over modules with the same name if they are in the same directory because spam comes before spam.py.
Note that "current directory" is relative to the main script path (the one where __name__ == '__main__' is True). So if you are at /home/billg calling /foo/bar.py, "current directory" refers to /foo.
from a python shell:
from foo import bar
print bar.__file__
should tell you which file has been imported
Rob
Packages (directories with __init__.py) take precedence over modules. The documentation of this fact is difficult to find but you can see this in the source: python 2.7, python 3.6 (thanks #qff for the find).
You will also need a __init__.py within the foo directory for your example to work.
If other.py is inside of foo/ then it will load foo.py (not the directory foo/) because it will look in the current directory first (unless you've played with PYTHONPATH or sys.path).
I would like to complement the accepted answer. For Python 3.3+, namespace packages have been introduced and the import order according to PEP 420 follows:
During import processing, the import machinery will continue to iterate over each directory in the parent path as it does in Python 3.2. While looking for a module or package named "foo", for each directory in the parent path:
If <directory>/foo/__init__.py is found, a regular package is imported and returned.
If not, but <directory>/foo.{py,pyc,so,pyd} is found, a module is imported and returned. The exact list of extension varies by platform and whether the -O flag is specified. The list here is representative.
If not, but <directory>/foo is found and is a directory, it is recorded and the scan continues with the next directory in the parent path.
Otherwise the scan continues with the next directory in the parent path.
If the scan completes without returning a module or package, and at least one directory was recorded, then a namespace package is created.
in the first case you're trying to import the function bar from file 'foo.py'
In the second you're trying to import the file 'foo.py'
I have this kind of path architecture :
>main_path/
__init__.py
config/
__init__.py
common.py
app_1/
__init__.py
config.py
index.py
>
I'd like to be able to do so in config.py :
>from main_path.config import common
>
Though it does not work. Python tells me :
> $> pwd
..../main_path/app_1
$> python index.py
[...]
ImportError: No module named main_path.config
>
As far as I understand, this would be possible if i loaded everything up from the main_path, though the aim is to have multiple apps with a common config file.
I tried to add the parent directory to the __path__ in the app_1/__init__.py but it changed nothing.
My next move would be to have a symbolic link, though I don't really like this "solution", so if you have any idea to help me out, this would be much appreciated !
Thanks in advance !
According to the Modules documentation a module has to be in your PYTHONPATH environment variable to be imported. You can modify this within your program with something like:
import sys
sys.path.append('PATH_TO/config')
import common
For more information, you may want to see Modifying Python's Search Path in Installing Python Modules.
If you want to modify python's search path without having to set PYTHONPATH each time, you can add a path configuration file (.pth file) to a directory that is already on the python path.
This document describes it in detail: http://docs.python.org/install/#inst-search-path
The most convenient way is to add a
path configuration file to a directory
that’s already on Python’s path,
usually to the .../site-packages/
directory. Path configuration files
have an extension of .pth, and each
line must contain a single path that
will be appended to sys.path.
You can tweak your PYTHONPATH environment variable or edit sys.path.