I created a py file called orderbook.py which contains a class called OrderbookLoader. Everything is in a directory called loaders. I added that path to the ipythonpath, so the following: /home/ec2-user/SageMaker/research/data_manager/loaders
In my directory I have multiple other files as well:
But when I go my root directory and try the following:
from loaders.orderbook import OrderbookLoader it throws the following error message: ModuleNotFoundError: No module named 'loaders.orderbook'; 'loaders' is not a package
But I can do the following:
from loaders import OrderbookLoader which is a class from the loaders.py file. Do you have any idea what I did wrong? thanks!
This has to do with interactions between your work directory (not specified in your question), the Python path (also not specified), and the files in the supposed package directory.
To make a directory a package, you should:
Ensure that is resides directly a directory that is in the Python path (sys.path).
Ensure that it includes a file named __init__.py (which you did). That file may be empty, or contain legal Python code.
What is probably happening here is that you made the directory loaders, itself, part of the Python path. If you added its parent directory (as you should have), it was probably placed after loaders. What happens next is that when Python tries to import loaders, it imports the file loaders.py inside the loaders directory, before finding the loaderspackage directory inside its parent. This means that loaders, specified to the import mechanism, refers to the module, rather than the package, hence the results of your two import attempts.
This placement of a module inside a package, both with the same name, is not a good idea, as it may lead to the situation you encounter here.
Related
This is a python newbie question:
I have the following directory structure:
test
-- test_file.py
a
-- b
-- module.py
where test, a and b are folders. Both test and a are on the same level.
module.py has a class called shape, and I want to instantiate an instance of it in test_file.py. How can I do so?
I have tried:
from a.b import module
but I got:
ImportError: No module named a.b
What you want is a relative import like:
from ..a.b import module
The problem with this is that it doesn't work if you are calling test_file.py as your main module. As stated here:
Note that both explicit and implicit relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application should always use absolute imports.
So, if you want to call test_file.py as your main module, then you should consider changing the structure of your modules and using an absolute import, else just use the relative import from above.
The directory a needs to be a package. Add an __init__.py file to make it a package, which is a step up from being a simple directory.
The directory b also needs to be a subpackage of a. Add an __init__.py file.
The directory test should probably also be a package. Hard to say if this is necessary or not. It's usually a good idea for every directory of Python modules to be a formal package.
In order to import, the package needs to be on sys.path; this is built from the PYTHONPATH environment variable. By default the installed site-packages and the current working directory are (effectively) the only two places where a package can be found.
That means that a must either be installed, or, your current working directory must also be a package one level above a.
OR, you need to set your PYTHONPATH environment variable to include a.
http://docs.python.org/tutorial/modules.html#the-module-search-path
http://docs.python.org/using/cmdline.html#envvar-PYTHONPATH
Also, http://docs.python.org/library/site.html for complete information on how sys.path is built.
The first thing to do would be to quickly browse the official docs on this.
To make a directory a package, you'll have to add a __init__.py file. This means that you'll have such a file in the a and b directories. Then you can directly do an
import a.b.module
But you'll have to refer to it as a.b.module which is tedious so you can use the as form of the import like so
import a.b.module as mod #shorter name
and refer to it as mod.
Then you can instantiate things inside mod using the regular conventions like mod.shape().
There are a few other subtleties. Please go through the docs for details.
I have a large repository with some fixed structure and I have extended it by some folders and python scripts to add extra functionality to it as a whole. The structure looks as follows:
toplevelfolder
featureA
someModuleA.py
__ init __.py
featureB
someModuleB.py
__ init __.py
application
__ init __.py
app.py
Now someModuleA.py and someModuleB.py can be invoked via app.py but at the same time also have be able to be invoked directly, however this invocation must come from the toplevelfolder for the relative paths in the file to resolve correctly, i.e. via python ./featureA/someModuleA.py.
This all works well, but now I need some function definitions from someModuleB in someModuleA and hence I want to import this module. I have tried both absolute and relative imports, but both fail with different errors, the absolute import with
from toplevelfolder.featureA import someModuleA as A
# ModuleNotFoundError: No module named 'toplevelfolder'
and the relative import with
from toplevelfolder.featureA import someModuleA as A
# ImportError: attempted relative import with no known parent package
Now I can see that the relative import would cause problems when python is invoked from the toplevelfolder, as .. would represent the latter's parent directory, rather than the parent directory of featureA. However, I cannot get a hold of the first error message, especially since toplevelfolder should not be a module but a package.
Is there another way to import in Python that I'm not aware of, if possibly without modifying PYTHONPATH or sys.path or something like that?
Not 100% sure on what the goal is here. My advice would be:
Identify clearly what you want your top level modules and packages to be.
Make all imports absolute.
Either:
make your project a real installable project, so that those top level modules and packages are installed in the environment's site-packages directory;
or make sure that the current working directory is the one containing the top level modules and packages.
Make sure to call your code via the executable module or package method instead of the script method, if the "entry point" you want to execute is part of a package
DO (executable module or package):
path/to/pythonX.Y -m toplevelpackage.module
path/to/pythonX.Y -m toplevelpackage.subpackage (assuming there is a toplevelpackage/subpackage/__main__.py file)
DON'T (script within a package):
path/to/pythonX.Y toplevelpackage/module.py
(Optional) Later on, once it all works well and everything is under control, you might decide to change some or all imports to relative. (If things are done right, I believe it could be possible to make it so that it is possible to call the executable modules from any level within the directory structure as the current working directory.)
References:
Old reference, possibly outdated, but assuming I interpreted it right, it says that running scripts that live in a package is an anti pattern, and one should use python -m package.module instead: https://mail.python.org/pipermail/python-3000/2007-April/006793.html -- https://www.python.org/dev/peps/pep-3122/
Don't include the top level directory. In featureB.someModuleB:
from featureA.someModuleA import two
Sample directory.
Try pasting this above your import:
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
Then you should be able to import a file from the parent-folder.
I ran my project and received the following error:
File "/home/nguyentv/schoollink/web/views/apis.py", line 10, in <module>
from util.redis.redis_client import Redis
ImportError: No module named util.redis.redis_client
How do I properly import this library?
The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory). PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended. See section Standard Modules for more information.
Basically, the interpreter is going to perform a lookup in your current working directory, then it will search through the system defined library directories.
The issue you are facing could be that your code is looking for a module that does not exist, you are calling the script from an incorrect directory, or the sys.path is setup incorrectly.
I could help more if you showed how you instantiated the interpreter, pwd output, and tree output.
You are trying to import Redis from a package named util. Unless this package is part of your application, it does not exist.
According to python-redis' documentation, here is how to import it:
import redis
# then use redis.Redis(...)
or, equivalently:
from redis import Redis
# then use Redis(...)
I am currently working in the following directory ( my script and associated files are here)
/scratch/conte/v/vprithiv/Gmsh
While I give this line , it throws me that error mentioned
from fenpy.sirah.Sira import Sira
Sira is located in the following path
/home/vprithiv/Fen/Utils/fenpy/sirah
If I am running my script from /home/vprithiv/Fen/Utils
I am able to obtain the output, But can I do it from my current working directory and somehow import sira??
When you try and import a module (eg import foo) in Python, the interpreter first checks the list of built in modules, and then checks the list of directories given in the sys.path variable for a file called foo.py.
sys.path typically contains your current working directory as it's first entry, and then a list of standard package locations on your system.
If you are only going to be working on your system, and you know the path to your package is going to stay constant, then you can just add your package directory to sys.path.
import sys
sys.path.append('/home/vprithiv/Fen/Utils')
from fendpy.sirah.Sira import Sira
I read various answer (relative import), but none work to import a simple class. I have this structure:
__init__.py (empty)
my_script.py
other_script.py
my_class.py
I want to use my_class.py in both script (my_script.py and other_script.py). Like all answer suggest, I just use:
from .my_class import My_class
but I always get
SystemError: Parent module '' not loaded, cannot perform relative import
I am using Python 3.5 and PyCharm 2016.1.2. Does I need to configure the __init__.py? How can I import a simple class?
Edit
All the files are in the working directory. I just use the Pycharm to run and I wasn't having problem until try to import the class.
Ensure that your current working directory is not the one that contains these files. For example, if the path to __init__.py is spam/eggs/__init__.py, make sure you are working in directory spam—or alternatively, that /path/to/spam is in sys.path and you are working in some third place. Either way, do not attempt to work in directory eggs. To test your code, you say import eggs.
The reasoning is as follows. Since you're using __init__.py you clearly want to treat this collection of files as a package. To work as a package, a directory must fulfill both the following criteria:
it contains a file called __init__.py
its parent directory is part of the search path (or the parent directory is your current working directory); in effect, a package directory must masquerade as a file, i.e. be findable in exactly the same way that a single-file module might be found.
If you're working inside the package directory you may not be fulfilling criterion 2. You can do a straightforward import my_class from there, certainly, but the "relative import" thing you're trying is a feature that only a fully-working package will support.