I ran my project and received the following error:
File "/home/nguyentv/schoollink/web/views/apis.py", line 10, in <module>
from util.redis.redis_client import Redis
ImportError: No module named util.redis.redis_client
How do I properly import this library?
The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory). PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended. See section Standard Modules for more information.
Basically, the interpreter is going to perform a lookup in your current working directory, then it will search through the system defined library directories.
The issue you are facing could be that your code is looking for a module that does not exist, you are calling the script from an incorrect directory, or the sys.path is setup incorrectly.
I could help more if you showed how you instantiated the interpreter, pwd output, and tree output.
You are trying to import Redis from a package named util. Unless this package is part of your application, it does not exist.
According to python-redis' documentation, here is how to import it:
import redis
# then use redis.Redis(...)
or, equivalently:
from redis import Redis
# then use Redis(...)
Related
I created a py file called orderbook.py which contains a class called OrderbookLoader. Everything is in a directory called loaders. I added that path to the ipythonpath, so the following: /home/ec2-user/SageMaker/research/data_manager/loaders
In my directory I have multiple other files as well:
But when I go my root directory and try the following:
from loaders.orderbook import OrderbookLoader it throws the following error message: ModuleNotFoundError: No module named 'loaders.orderbook'; 'loaders' is not a package
But I can do the following:
from loaders import OrderbookLoader which is a class from the loaders.py file. Do you have any idea what I did wrong? thanks!
This has to do with interactions between your work directory (not specified in your question), the Python path (also not specified), and the files in the supposed package directory.
To make a directory a package, you should:
Ensure that is resides directly a directory that is in the Python path (sys.path).
Ensure that it includes a file named __init__.py (which you did). That file may be empty, or contain legal Python code.
What is probably happening here is that you made the directory loaders, itself, part of the Python path. If you added its parent directory (as you should have), it was probably placed after loaders. What happens next is that when Python tries to import loaders, it imports the file loaders.py inside the loaders directory, before finding the loaderspackage directory inside its parent. This means that loaders, specified to the import mechanism, refers to the module, rather than the package, hence the results of your two import attempts.
This placement of a module inside a package, both with the same name, is not a good idea, as it may lead to the situation you encounter here.
I am currently working in the following directory ( my script and associated files are here)
/scratch/conte/v/vprithiv/Gmsh
While I give this line , it throws me that error mentioned
from fenpy.sirah.Sira import Sira
Sira is located in the following path
/home/vprithiv/Fen/Utils/fenpy/sirah
If I am running my script from /home/vprithiv/Fen/Utils
I am able to obtain the output, But can I do it from my current working directory and somehow import sira??
When you try and import a module (eg import foo) in Python, the interpreter first checks the list of built in modules, and then checks the list of directories given in the sys.path variable for a file called foo.py.
sys.path typically contains your current working directory as it's first entry, and then a list of standard package locations on your system.
If you are only going to be working on your system, and you know the path to your package is going to stay constant, then you can just add your package directory to sys.path.
import sys
sys.path.append('/home/vprithiv/Fen/Utils')
from fendpy.sirah.Sira import Sira
I am using Python 2.7. I have the following directory structure:
alogos
- resources
- __init__.py
- test.py
- lib
- __init__.py
- utils.py
- common
- config
- config.json
My utils.py is the following:
def read_json_data(filename):
with open(filename) as data:
json_data=json.load(data)
return json_data
My test.py has the following:
from lib.utils import read_json_data
running_data = read_json_data('common/config/config.json')
print running_data
when I try to run python test.py from the resources directory, I get the following error:
ImportError: No module named lib.utils
What is the correct way to access files and modules
Your lib.utils module is not present in the current directory (and apparently not anywhere else import checks), and so the import fails.
The Python doc details the module search path:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
* the directory containing the input script (or the current directory).
* PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
* the installation-dependent default.
After initialization, Python programs can modify sys.path. The
directory containing the script being run is placed at the beginning
of the search path, ahead of the standard library path. This means
that scripts in that directory will be loaded instead of modules of
the same name in the library directory. This is an error unless the
replacement is intended. See section Standard Modules for more
information.
While this is certainly not the only way, what I would do would be to have your lib.utils module as a separate module, stored in a local Pypi server (Artifactory is one example, but there are others, such as devpi) where you can install it just like any other module, just from a different index URL from the regular Pypi. That way, any of your scripts can use it just like any other module, and it obviates the need to play assorted path-related games that can add unnecessary complexity.
This questions is detailing a behavior that I can't explain to myself.
src/package/__init__.py is empty but present.
src/package/subpackage/__init__.py:
pink = 'It works'
src/package/test/test.py:
import package.subpackage as subpackage
# I also tried `import package.subpackage as subpackage
print subpackage.pink
Calling from src: python package/test/test.py just fails with ImportError: No module named subpackage. Please note that import package doesn't work either.
NB: (Running an interpreter from src and typing the import statement works perfectly well.
Should I understand that I'm not suppose to call subfile of a package? In my project it's a test file so it sounds logical for me have it here.
Why the current working directory is not in the import path?
Many thanks for those who reads and those who answers.
Because you package is not in $PYTHONPATH. If you what to call test.py, you can move your test.py file to src/ directory, or add src to $PYTHONPATH
PYTHONPATH="/path/to/src:$PYTHONPATH"
export PYTHONPATH
From Documentation
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path
>>> import sys
>>> sys.path
The output is like this
['.', '/usr/bin', ...
This means that the current directory is in sys.path as well. If you want to import a module, please make sure that the module path is in sys.path, by adding your package directory to the environment variable PYTHONPATH, or changing your current directory or script directory to the package directory.
On python package/test/test.py fails, it's also ran from src:
when you starts a intepreter from src, '' is in sys.path, so path of src could be found;
when you run python package/test/test.py from src, '' is missing from sys.path, although os.path.abspath('.') shows current dir is "<xxx>\\src", "<xxx>\\src" is not in sys.path, while "<xxx>\\src\\package\\test" is in sys.path. That's saying, python adds path of the file to sys.path, not the path where you run the script.
see what the docs says:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
I have a Python script that uses built-in modules but also imports a number of custom modules that exist in the same directory as the main script itself.
For example, I would call
python agent.py
and agent.py has a number of imports, including:
import checks
where checks is in a file in the same directory as agent.py
agent/agent.py
agent/checks.py
When the current working directory is agent/ then everything is fine. However, if I call agent.py from any other directory, it is obviously unable to import checks.py and so errors.
How can I ensure that the custom modules can be imported regardless of where the agent.py is called from e.g.
python /home/bob/scripts/agent/agent.py
Actually your example works because checks.py is in the same directory as agent.py, but say checks.py was in the preceeding directory, eg;
agent/agent.py
checks.py
Then you could do the following:
path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if not path in sys.path:
sys.path.insert(1, path)
del path
Note the use of __file__.
You should NOT need to fiddle with sys.path. To quote from the Python 2.6 docs for sys.path:
As initialized upon program startup, the first item of this list,
path[0], is the directory containing the script that was used to
invoke the Python interpreter. If the script directory is not
available (e.g. if the interpreter is invoked interactively or if the
script is read from standard input), path[0] is the empty string,
which directs Python to search modules in the current directory first.
Notice that the script directory is inserted before the entries
inserted as a result of PYTHONPATH.
=== amod.py ===
def whoami():
return __file__
=== ascript.py ===
import sys
print "sys.argv", sys.argv
print "sys.path", sys.path
import amod
print "amod __file__", amod.whoami()
=== result of running ascript.py from afar ===
C:\somewhere_else>\python26\python \junk\timport\ascript.py
sys.argv ['\\junk\\timport\\ascript.py']
sys.path ['C:\\junk\\timport', 'C:\\WINDOWS\\system32\\python26.zip', SNIP]
amod __file__ C:\junk\timport\amod.py
and if it's re-run, the last line changes of course to ...amod.pyc. This appears not to be a novelty, it works with Python 2.1 and 1.5.2.
Debug hints for you: Try two simple files like I have. Try running Python with -v and -vv. Show us the results of your failing tests, including full traceback and error message, and your two files. Tell us what platform you are running on, and what version of Python. Check the permissions on the checks.py file. Is there a checks.something_else that's causing interference?
You need to add the path to the currently executing module to the sys.path variable. Since you called it on the command line, the path to the script will always be in sys.argv[0].
import sys
import os
sys.path.append(os.path.split(sys.argv[0])[0])
Now when import searches for the module, it will also look in the folder that hosts the agent.py file.
There are several ways to add things to the PYTHONPATH.
Read http://docs.python.org/library/site.html
Set the PYTHONPATH environment variable prior to running your script.
You can do this python -m agent to run agent.py from your PYTHONPATH.
Create .pth files in your lib/site-packages directory.
Install your modules in lib/site-packages.
I think you should consider making the agent directory into a proper Python package. Then you place this package anywhere on the python path, and you can import checks as
from agent import checks
See http://docs.python.org/tutorial/modules.html
If you know full path to check.py use this recipe (http://code.activestate.com/recipes/159571/)
If you want to add directory to system path -- this recipe (http://code.activestate.com/recipes/52662/). In this case I have to determine application directory (sys.argv[0]) an pass this value to AddSysPath function. If you want to look at production sample please leave a comment on this thread so I post it later.
Regards.
To generalize my understanding of your goal, you want to be able to import custom packages using import custom_package_name no matter where you are calling python from and no matter where your python script is located.
A number of answers mention what I'm about to describe, but I feel like most of the answers assume a lot of previous knowledge. I'll try to be as explicit as I can.
To achieve the goal of allowing custom packages to be imported via the import statement, they have to be discoverable somewhere through the path that python uses to search for packages. Python actually uses multiple paths, but we'll only focus on the one that can be found by combining the output of sys.prefix (in your python interpreter) with /lib/pythonX.Y/site-packages (or lib/site-packages if you're using windows) where X.Y is your python version.
Concretely, find the path that your python uses by running:
import sys
your_path = sys.prefix + '/lib/pythonX.Y/site-packages'
print(your_path)
This path should look something like /usr/local/lib/python3.5/site-packages if you're using python 3.5, but it could be much different depending on your setup.
Python uses this path (and a few others) to find the packages that you want to import. So what you can do is put your custom packages in the /usr/local/lib/python3.5/site-packages folder. Don't forget to add an init.py file to the folder.
To be concrete again, in terminal type:
cd your_path
cp path_to_custom_package/custom_package ./
Now you should be able to import everything your custom package like you would if the package was located in the same directory (i.e. import package.subpackage for each subpackage file in your package should work).