I'm wondering about the preferred way to import packages in a Python application. I have a package structure like this:
project.app1.models
project.app1.views
project.app2.models
project.app1.views imports project.app1.models and project.app2.models. There are two ways to do this that come to mind.
With absolute imports:
import A.A
import A.B.B
or with explicit relative imports, as introduced in Python 2.5 with PEP 328:
# explicit relative
from .. import A
from . import B
What is the most pythonic way to do this?
Python relative imports are no longer strongly discouraged, but using absolute_import is strongly suggested in that case.
Please see this discussion citing Guido himself:
"Isn't this mostly historical? Until the new relative-import syntax
was implemented there were various problems with relative imports. The
short-term solution was to recommend not using them. The long-term
solution was to implement an unambiguous syntax. Now it is time to
withdraw the anti-recommendation. Of course, without going overboard
-- I still find them an acquired taste; but they have their place."
The OP correctly links the PEP 328 that says:
Several use cases were presented, the most important of which is being
able to rearrange the structure of large packages without having to
edit sub-packages. In addition, a module inside a package can't easily
import itself without relative imports.
Also see almost duplicate question When or why to use relative imports in Python
Of course it still stands as a matter of taste. While it's easier to move code around with relative imports, that might also unexpectedly break things; and renaming the imports is not that difficult.
To force the new behaviour from PEP 328 use:
from __future__ import absolute_import
In this case, implicit relative import will no longer be possible (eg. import localfile will not work anymore, only from . import localfile). For clean and future proof behaviour, using absolute_import is advisable.
An important caveat is that because of PEP 338 and PEP 366, relative imports require the python file to be imported as a module - you cannot execute a file.py that has a relative import or you'll get a ValueError: Attempted relative import in non-package.
This limitation should be taken into account when evaluating the best approach. Guido is against running scripts from a module in any case:
I'm -1 on this and on any other proposed twiddlings of the __main__ machinery.
The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern.
To make me change my mind you'd have to convince me that it isn't.
Exhaustive discussions on the matter can be found on SO; re. Python 3 this is quite comprehensive:
Relative imports in Python 3
Absolute imports. From PEP 8:
Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.
Explicit relative imports are a nice language feature (I guess), but they're not nearly as explicit as absolute imports. The more readable form is:
import A.A
import A.B.B
especially if you import several different namespaces. If you look at some well written projects/tutorials that include imports from within packages, they usually follow this style.
The few extra keystrokes you take to be more explicit will save others (and perhaps you) plenty of time in the future when they're trying to figure out your namespace (especially if you migrate to 3.x, in which some of the package names have changed).
Relative imports not only leave you free to rename your package later without changing dozens of internal imports, but I have also had success with them in solving certain problems involving things like circular imports or namespace packages, because they do not send Python "back to the top" to start the search for the next module all over again from the top-level namespace.
Related
I'm wondering about the preferred way to import packages in a Python application. I have a package structure like this:
project.app1.models
project.app1.views
project.app2.models
project.app1.views imports project.app1.models and project.app2.models. There are two ways to do this that come to mind.
With absolute imports:
import A.A
import A.B.B
or with explicit relative imports, as introduced in Python 2.5 with PEP 328:
# explicit relative
from .. import A
from . import B
What is the most pythonic way to do this?
Python relative imports are no longer strongly discouraged, but using absolute_import is strongly suggested in that case.
Please see this discussion citing Guido himself:
"Isn't this mostly historical? Until the new relative-import syntax
was implemented there were various problems with relative imports. The
short-term solution was to recommend not using them. The long-term
solution was to implement an unambiguous syntax. Now it is time to
withdraw the anti-recommendation. Of course, without going overboard
-- I still find them an acquired taste; but they have their place."
The OP correctly links the PEP 328 that says:
Several use cases were presented, the most important of which is being
able to rearrange the structure of large packages without having to
edit sub-packages. In addition, a module inside a package can't easily
import itself without relative imports.
Also see almost duplicate question When or why to use relative imports in Python
Of course it still stands as a matter of taste. While it's easier to move code around with relative imports, that might also unexpectedly break things; and renaming the imports is not that difficult.
To force the new behaviour from PEP 328 use:
from __future__ import absolute_import
In this case, implicit relative import will no longer be possible (eg. import localfile will not work anymore, only from . import localfile). For clean and future proof behaviour, using absolute_import is advisable.
An important caveat is that because of PEP 338 and PEP 366, relative imports require the python file to be imported as a module - you cannot execute a file.py that has a relative import or you'll get a ValueError: Attempted relative import in non-package.
This limitation should be taken into account when evaluating the best approach. Guido is against running scripts from a module in any case:
I'm -1 on this and on any other proposed twiddlings of the __main__ machinery.
The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern.
To make me change my mind you'd have to convince me that it isn't.
Exhaustive discussions on the matter can be found on SO; re. Python 3 this is quite comprehensive:
Relative imports in Python 3
Absolute imports. From PEP 8:
Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.
Explicit relative imports are a nice language feature (I guess), but they're not nearly as explicit as absolute imports. The more readable form is:
import A.A
import A.B.B
especially if you import several different namespaces. If you look at some well written projects/tutorials that include imports from within packages, they usually follow this style.
The few extra keystrokes you take to be more explicit will save others (and perhaps you) plenty of time in the future when they're trying to figure out your namespace (especially if you migrate to 3.x, in which some of the package names have changed).
Relative imports not only leave you free to rename your package later without changing dozens of internal imports, but I have also had success with them in solving certain problems involving things like circular imports or namespace packages, because they do not send Python "back to the top" to start the search for the next module all over again from the top-level namespace.
Are there any rules or guidelines concerning when to use relative imports in Python? I see them in use all the time, such as in the Flask web framework. When searching for this topic, I only see articles on how to use relative imports, but not why.
So is there some special benefit to using:
from . import x
rather than:
from package import x
Moreover, I noticed that a related SO post mentions that relative imports are discouraged. Yet people still continue to use them.
Check out PEP 328's section on relative imports
The rationale seems to be as written:
Several use cases were presented, the most important of which is being able to rearrange the structure of large packages without having to edit sub-packages. In addition, a module inside a package can't easily import itself without relative imports.
I've recently created a Python package, and within it, used only relative imports to access functions stored in other methods.
Now, in Numpy, I see a lot of files that make heavy use of absolute imports, e.g. this file. It has a lot of statements like from numpy.core import overrides.
I don't see a disadvantage in using relative imports. Why are they doing it like that, instead of from ..core import overrides? Doesn't the absolute import require numpy to be already installed?
Absolute vs Relative Imports in Python
Absolute Import
Absolute imports are preferred because they are quite clear and straightforward. It is easy to tell exactly where the imported resource is, just by looking at the statement. Additionally, absolute imports remain valid even if the current location of the import statement changes. In fact, PEP 8 explicitly recommends absolute imports.
Sometimes, however, absolute imports can get quite verbose, depending on the complexity of the directory structure.
Relative Import
One clear advantage of relative imports is that they are quite succinct.
Unfortunately, relative imports can be messy, particularly for shared
projects where directory structure is likely to change. Relative
imports are also not as readable as absolute ones, and it’s not easy
to tell the location of the imported resources.
PEP8: about Imports
Just after standard pythonmodule imports?
If I postpone it to the main function and do my specific module imports before it, it gives error (which is quite obvious). Python Style guide no where mentions the correct location for it.
It should go before the import or from statements that need it (which as you say is obvious). So for example a module could start with:
import sys
import os
import math
try:
import foo
except ImportError:
if 'foopath' in sys.path: raise
sys.path.append('foopath')
import foo
Note that I've made the append conditional (on the import failing and the specific module's path not being on sys.path yet) to avoid the risk of sys.path ending up with dozens of occurrences of string foopath, which would not be particularly helpful;-).
One reason this isn't mentioned in PEP 8 or other good Python style guides is that modifying sys.path isn't something you want to do in a real program; it makes your program less robust and portable. A better solution might be to put your package somewhere that would already be in sys.path or to define PYTHONPATH systemwide to include your package.
I often use a shell script to launch my python applications: I put my sys.path.insert (append) statement just after the "standard" python module imports in my "launch python script".
Using sys.path.insert(0, ...) gets your "imports" in priority in the path list.
I generally do it before importing anything. If you're worried that your module names might conflict with the Python stdlib names, then change your module names!
I think it's a matter of taste. But most people tend to put it behind the import sys :-)
I prefer wrapping it in an extra function:
def importmod_abs(name):
sys.path.append() ..
__import__ ...
sys.path.pop()
... this way sys.path remains clean. Of course that's only applicable to certain module structures. Anyways, I'd import everything that works without altering sys.path first.
This is related to a previous question of mine.
I understand how to store and read configuration files. There are choices such as ConfigParser and ConfigObj.
Consider this structure for a hypothetical 'eggs' module:
eggs/
common/
__init__.py
config.py
foo/
__init__.py
a.py
'eggs.foo.a' needs some configuration information. What I am currently doing is, in 'a', import eggs.common.config. One problem with this is that if 'a' is moved to a deeper level in the module tree, the relative imports break. Absolute imports don't, but they require your module to be on your PYTHONPATH.
A possible alternative to the above absolute import is a relative import. Thus, in 'a',
import .common.config
Without debating the merits of relative vs absolute imports, I was wondering about other possible solutions?
edit- Removed the VCS context
"imports ... require your module to be on your PYTHONPATH"
Right.
So, what's wrong with setting PYTHONPATH?
require statement from pkg_resources maybe what you need.
As I understand it from this and previous questions you only need one path to be in sys.path. If we are talking about git as VCS (mentioned in previous question) when only one branch is checked out at any time (single working directory). You can switch, merge branches as frequently as you like.
I'm thinking of something along the lines of a more 'push-based' kind of solution. Instead of importing the shared objects (be they for configuration, or utility functions of some sort), have the top-level init export it, and each intermediate init import it from the layer above, and immediately re-export it.
I'm not sure if I've got the python terminology right, please correct me if I'm wrong.
Like this, any module that needs to use the shared object(which in the context of this example represents configuration information) simply imports it from the init at its own level.
Does this sound sensible/feasible?
You can trick the import mechanism, by adding each subdirectory to egg/__init__.py:
__path__.append(__path__[0]+"\\common")
__path__.append(__path__[0]+"\\foo")
then, you simply import all modules from the egg namespace; e.g. import egg.bar (provided you have file egg/foo/bar.py).
Note that foo and common should not be a package - in other words, they should not contain __init__.py file.
This solution completely solves the issue of eventually moving files around; however it flattens the namespace and therefore it may not be as good, especially in big projects - personally, I prefer full name resolution.