I have a question. I have a directory setup like this:
folder/
main.py
/stuff/
__init__.py
function.py
/items/
__init__.py
class.py
My question is how would I import the class.py into the function.py? This setup is very specific and is unable to be changed. What would I need to put in order for this to work?
Your current directory structure seems ideal, so long as the application is started via main.py.
Python will always automatically add the parent directory of the main script to the start of sys.path (i.e. folder in your example). This means that the import machinery will give that directory priority when searching for modules and packages that are not part of the standard libarary.
Given this, you can import the classes.py module into function.py, like so:
from items import classes
(Note that I have renamed the module, because class is a python keyword).
If you later added another module to stuff, and wanted to import it into functions.py, you would do:
from stuff import another
and if a sub-package was added to items, and you wanted to import a module from that, you would do:
from items.subpackage import module
Imports specified in this top-down way can be used from any module within the application, because they are always relative to the parent directory of the main script, which has priority.
Related
Consider this folder structure:
main.py
module_a/
aa.py
bb.py
__init__.py
In main.py, I import aa as:
from module_a import aa
aa.yyy()
Then in aa.py, I import bb and include its functions as:
import bb
bb.xxx()
However, when I run main.py, python says "No module named 'bb'".
May I know why this happens. What is the correct way to import bb.
Thanks!!!
I have tried to write aa.py as:
import .bb
bb.xxx()
But it still does not work.
why this happens
Because the aa folder is not a place that Python is searching for modules.
Python's imports are absolute by default. They only look in specific places determined by sys.path. In main.py, import module_a.aa works because the root folder of the project happens to be on the sys.path; that folder contains a module_a folder; and that folder contains aa.py.
What is the correct way to import bb.
Please use relative imports between files in your package. In this case, the necessary import in aa.py looks like:
from . import bb
Absolute imports are error-prone; a project that uses two packages whose contents have overlapping names will run into namespace collisions. (Sadly, the standard library uses absolute imports in most places, such that projects need to ban certain module names for safety.) Relative imports will also require much less maintenance, should you later rename a sub-package.
The only thing relative imports require is that the package gets loaded, which typically will happen automatically with the first (yes, absolute) import of any of the package contents. When the package is loaded, it automatically sets a __package__ attribute on the modules in that package, which Python can use to resolve the relative imports. It's important to note that relative imports are relative to the package hierarchy, not the directory structure, which is why this is necessary; imports like from .. import example do not work by figuring out the current file location and then going up a level in the directory hierarchy. Instead, they check the __package__ to figure out what the containing package is, then check the file/folder location for that, and work from there.
If the "driver" script is within the package, run it as a module, using the -m switch for Python. For example, from the root folder, if module_a/aa.py is the driver, use python -m module_a.aa. This instructs Python that module_a is the containing package for aa.py, and ensures it gets loaded even though no import in the code has loaded it.
Contrary to what many people will wrongly tell you, it is almost never required to manipulate sys.path; there are popular Python projects on GitHub, running hundreds of thousands of lines of code, which either do not use it at all or use it only once in an ancillary role (perhaps because of a special requirement for a documentation tool). Just don't do it.
Also contrary to what many people will wrongly tell you, __init__.py files are not required to create packages in Python. They are simply a place where additional code can be placed for package initialization - for example, to create aliases for sub-package contents, or to limit what will be imported with a *-import (by setting __all__).
import .bb
This is just invalid. Relative imports only use the from syntax. See above for the correct syntax.
Suppose the same file structure as stated in OP, and:
main.py:
import module_a.aa
module_a.aa.thisFile()
module_a.aa.module_a.bb.thisFile()
aa.py:
import module_a.bb
def thisFile():
print("aa")
bb.py:
def thisFile():
print("bb")
Then, this will print
aa
bb
like you would expect. The main difference here, is that bb.py is imported in aa.py via module_a.bb. Running main.py gives no problem, however, running aa.py does not work in this way. That is why you might want to add folders to your path, such that you can call a function from a different file without this trouble. This is done via:
import os, inspect, sys
current_folder = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parent_folder = os.path.dirname(current_folder)
sys.path.insert(0,parent_folder)
Then you can import your files such as import file. If you consider this option, I would suggest to do some reading about how this works. Tip: make sure you avoid cyclic import problems.
This is a python newbie question:
I have the following directory structure:
test
-- test_file.py
a
-- b
-- module.py
where test, a and b are folders. Both test and a are on the same level.
module.py has a class called shape, and I want to instantiate an instance of it in test_file.py. How can I do so?
I have tried:
from a.b import module
but I got:
ImportError: No module named a.b
What you want is a relative import like:
from ..a.b import module
The problem with this is that it doesn't work if you are calling test_file.py as your main module. As stated here:
Note that both explicit and implicit relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application should always use absolute imports.
So, if you want to call test_file.py as your main module, then you should consider changing the structure of your modules and using an absolute import, else just use the relative import from above.
The directory a needs to be a package. Add an __init__.py file to make it a package, which is a step up from being a simple directory.
The directory b also needs to be a subpackage of a. Add an __init__.py file.
The directory test should probably also be a package. Hard to say if this is necessary or not. It's usually a good idea for every directory of Python modules to be a formal package.
In order to import, the package needs to be on sys.path; this is built from the PYTHONPATH environment variable. By default the installed site-packages and the current working directory are (effectively) the only two places where a package can be found.
That means that a must either be installed, or, your current working directory must also be a package one level above a.
OR, you need to set your PYTHONPATH environment variable to include a.
http://docs.python.org/tutorial/modules.html#the-module-search-path
http://docs.python.org/using/cmdline.html#envvar-PYTHONPATH
Also, http://docs.python.org/library/site.html for complete information on how sys.path is built.
The first thing to do would be to quickly browse the official docs on this.
To make a directory a package, you'll have to add a __init__.py file. This means that you'll have such a file in the a and b directories. Then you can directly do an
import a.b.module
But you'll have to refer to it as a.b.module which is tedious so you can use the as form of the import like so
import a.b.module as mod #shorter name
and refer to it as mod.
Then you can instantiate things inside mod using the regular conventions like mod.shape().
There are a few other subtleties. Please go through the docs for details.
I read various answer (relative import), but none work to import a simple class. I have this structure:
__init__.py (empty)
my_script.py
other_script.py
my_class.py
I want to use my_class.py in both script (my_script.py and other_script.py). Like all answer suggest, I just use:
from .my_class import My_class
but I always get
SystemError: Parent module '' not loaded, cannot perform relative import
I am using Python 3.5 and PyCharm 2016.1.2. Does I need to configure the __init__.py? How can I import a simple class?
Edit
All the files are in the working directory. I just use the Pycharm to run and I wasn't having problem until try to import the class.
Ensure that your current working directory is not the one that contains these files. For example, if the path to __init__.py is spam/eggs/__init__.py, make sure you are working in directory spam—or alternatively, that /path/to/spam is in sys.path and you are working in some third place. Either way, do not attempt to work in directory eggs. To test your code, you say import eggs.
The reasoning is as follows. Since you're using __init__.py you clearly want to treat this collection of files as a package. To work as a package, a directory must fulfill both the following criteria:
it contains a file called __init__.py
its parent directory is part of the search path (or the parent directory is your current working directory); in effect, a package directory must masquerade as a file, i.e. be findable in exactly the same way that a single-file module might be found.
If you're working inside the package directory you may not be fulfilling criterion 2. You can do a straightforward import my_class from there, certainly, but the "relative import" thing you're trying is a feature that only a fully-working package will support.
What exactly is the use of __init__.py? Yes, I know this file makes a directory into an importable package. However, consider the following example:
project/
foo/
__init__.py
a.py
bar/
b.py
If I want to import a into b, I have to add following statement:
sys.path.append('/path_to_foo')
import foo.a
This will run successfully with or without __init__.py. However, if there is not an sys.path.append statement, a "no module" error will occur, with or without __init__.py. This makes it seem lik eonly the system path matters, and that __init__.py does not have any effect.
Why would this import work without __init__.py?
__init__.py has nothing to do with whether Python can find your package. You've run your code in such a way that your package isn't on the search path by default, but if you had run it differently or configured your PYTHONPATH differently, the sys.path.append would have been unnecessary.
__init__.py used to be necessary to create a package, and in most cases, you should still provide it. Since Python 3.3, though, a folder without an __init__.py can be considered part of an implicit namespace package, a feature for splitting a package across multiple directories.
During import processing, the import machinery will continue to
iterate over each directory in the parent path as it does in Python
3.2. While looking for a module or package named "foo", for each directory in the parent path:
If <directory>/foo/__init__.py is found, a regular package is imported and returned.
If not, but <directory>/foo.{py,pyc,so,pyd} is found, a module is imported and returned. The exact list of extension varies by platform
and whether the -O flag is specified. The list here is
representative.
If not, but <directory>/foo is found and is a directory, it is recorded and the scan continues with the next directory in the parent
path.
Otherwise the scan continues with the next directory in the parent path.
If the scan completes without returning a module or package, and at
least one directory was recorded, then a namespace package is created.
If you really want to avoid __init__.py for some reason, you don't sys.path. Rather, create a module object and set its __path__ to a list of directories.
if I want to import a into b, I have to add following statement:
No! You'd just say: import foo.a. All this is provided you run the entire package at once using python -m main.module where main.module is the entry point to your entire application. It imports all other modules, and the modules that import more modules will try to look for them from the root of this project. For instance, foo.bar.c will import as foo.bar.b
Then it seems that only the system path matters and init.py does not have any effect.
You need to modify sys.path only when you are importing modules from locations that are not in your project, or the places where python looks for libraries. __init__.py not only makes a folder look like a package, it also does a few more things like "export" objects to outside world (__all__)
When you import something it has to either:
Retrieve an already loaded module or
Load the module that was imported
When you do import foo and python finds a folder called foo in a folder on your sys.path then it will look in that folder for an __init__.py to be considered the top level module.
(Note that if the package is not on your sys.path then you would need to append it's location to be able to import it.)
If that is not present it will look for a __init__.pyc version possibly in the __pycache__ folder, if that is also missing then that folder foo is not considered a loadable python package. If no other options for foo are found then an ImportError is raised.
If you try deleting the __init__.pyc file as well you will see that the the initializer script for a package is indeed necessary.
I have an application with a heirarchy of packages. There are a fair number of modules that reference other modules higher up in the package heirarchy. As exemplified below, I can use relative imports to solve this problem. However, running the module directly for testing fails with an Attempted relative import in non-package exception.
Is there a better way to organize my application or my import statements so that modules can both be executed individually for testing and imported from other modules?
Example
\spam
\morespam
child.py
base.py
\eggs
user.py
base.py
class Base(object):
def hello(self):
print 'Um, wot?'
child.py
from ..base import Base # references the parent package correctly,
# but fails when this module is executed individually
class Child(Base):
def hello(self):
print 'Hello, world!'
if __name__ == '__main__':
import unittest
# ... unit test code ...
user.py
from spam.morespam.child import Child
print Child().hello()
Existing solution
I've found that I can add the following header added to the top of a module that needs to reference modules higher in the hierarchy:
if __name__ == '__main__':
import sys, os
sys.path.append(os.path.abspath(os.path.join(sys.path[0], '..')))
The downside is that I need to add this code all over the place, and it isn't constant: the '..' relative path varies based on the depth of the package in the hierarchy.
Other possible solutions...
I could create a .pth file with my application root. Unfortunately, this must be added to the site-packages folder of my Python installation, so it's not very application-specific.
I could temporarily add my application root to the PYTHONPATH, and make all my modules reference packages from the root. This eliminates the simplicity of running a script by simply calling python.exe whatever.py, though.
I could write some sort of module that, when imported, would look for a marker file in the root folder of my application. This generic module could then be installed to my local Python installation and imported by every module in my app, thus guaranteeing that the root is in PYTHONPATH, regardless of which module I'm executing.
You need to add __init__.py as empty files in folders that you'd want to do imports from. Doing this will cause python to treat the directories as packages, allowing you to use them in import statements:
\spam
\morespam
__init__.py
child.py
__init__.py
base.py
\eggs
__init__.py
user.py
Once you do this and establish your PYTHONPATH to the base of this directory, you can then do imports:
base.py:
from morespam.child import Child
from ..eggs import user
While a __init__.py file only needs to be present, you can also define the variable __ALL__ in this file as a list of modules in that directory that you'd want to import if a script tries to use import *.
I solved this with PyCharm, which automagically adds my source root to the PYTHONPATH every time it runs my application. As far as I can tell, PyCharm basically does option #2 for me.