I'm wondering about the preferred way to import packages in a Python application. I have a package structure like this:
project.app1.models
project.app1.views
project.app2.models
project.app1.views imports project.app1.models and project.app2.models. There are two ways to do this that come to mind.
With absolute imports:
import A.A
import A.B.B
or with explicit relative imports, as introduced in Python 2.5 with PEP 328:
# explicit relative
from .. import A
from . import B
What is the most pythonic way to do this?
Python relative imports are no longer strongly discouraged, but using absolute_import is strongly suggested in that case.
Please see this discussion citing Guido himself:
"Isn't this mostly historical? Until the new relative-import syntax
was implemented there were various problems with relative imports. The
short-term solution was to recommend not using them. The long-term
solution was to implement an unambiguous syntax. Now it is time to
withdraw the anti-recommendation. Of course, without going overboard
-- I still find them an acquired taste; but they have their place."
The OP correctly links the PEP 328 that says:
Several use cases were presented, the most important of which is being
able to rearrange the structure of large packages without having to
edit sub-packages. In addition, a module inside a package can't easily
import itself without relative imports.
Also see almost duplicate question When or why to use relative imports in Python
Of course it still stands as a matter of taste. While it's easier to move code around with relative imports, that might also unexpectedly break things; and renaming the imports is not that difficult.
To force the new behaviour from PEP 328 use:
from __future__ import absolute_import
In this case, implicit relative import will no longer be possible (eg. import localfile will not work anymore, only from . import localfile). For clean and future proof behaviour, using absolute_import is advisable.
An important caveat is that because of PEP 338 and PEP 366, relative imports require the python file to be imported as a module - you cannot execute a file.py that has a relative import or you'll get a ValueError: Attempted relative import in non-package.
This limitation should be taken into account when evaluating the best approach. Guido is against running scripts from a module in any case:
I'm -1 on this and on any other proposed twiddlings of the __main__ machinery.
The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern.
To make me change my mind you'd have to convince me that it isn't.
Exhaustive discussions on the matter can be found on SO; re. Python 3 this is quite comprehensive:
Relative imports in Python 3
Absolute imports. From PEP 8:
Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.
Explicit relative imports are a nice language feature (I guess), but they're not nearly as explicit as absolute imports. The more readable form is:
import A.A
import A.B.B
especially if you import several different namespaces. If you look at some well written projects/tutorials that include imports from within packages, they usually follow this style.
The few extra keystrokes you take to be more explicit will save others (and perhaps you) plenty of time in the future when they're trying to figure out your namespace (especially if you migrate to 3.x, in which some of the package names have changed).
Relative imports not only leave you free to rename your package later without changing dozens of internal imports, but I have also had success with them in solving certain problems involving things like circular imports or namespace packages, because they do not send Python "back to the top" to start the search for the next module all over again from the top-level namespace.
Related
Are there any rules or guidelines concerning when to use relative imports in Python? I see them in use all the time, such as in the Flask web framework. When searching for this topic, I only see articles on how to use relative imports, but not why.
So is there some special benefit to using:
from . import x
rather than:
from package import x
Moreover, I noticed that a related SO post mentions that relative imports are discouraged. Yet people still continue to use them.
Check out PEP 328's section on relative imports
The rationale seems to be as written:
Several use cases were presented, the most important of which is being able to rearrange the structure of large packages without having to edit sub-packages. In addition, a module inside a package can't easily import itself without relative imports.
I've recently created a Python package, and within it, used only relative imports to access functions stored in other methods.
Now, in Numpy, I see a lot of files that make heavy use of absolute imports, e.g. this file. It has a lot of statements like from numpy.core import overrides.
I don't see a disadvantage in using relative imports. Why are they doing it like that, instead of from ..core import overrides? Doesn't the absolute import require numpy to be already installed?
Absolute vs Relative Imports in Python
Absolute Import
Absolute imports are preferred because they are quite clear and straightforward. It is easy to tell exactly where the imported resource is, just by looking at the statement. Additionally, absolute imports remain valid even if the current location of the import statement changes. In fact, PEP 8 explicitly recommends absolute imports.
Sometimes, however, absolute imports can get quite verbose, depending on the complexity of the directory structure.
Relative Import
One clear advantage of relative imports is that they are quite succinct.
Unfortunately, relative imports can be messy, particularly for shared
projects where directory structure is likely to change. Relative
imports are also not as readable as absolute ones, and it’s not easy
to tell the location of the imported resources.
PEP8: about Imports
This may be a dumb question, but charging boldly ahead anyway.
I have a library of about a dozen Python modules I maintain for general use. Recently, after advice found here on SO, I changed all of the modules so they are imported in the import x as y style instead of from x import *. This solved several problems and made the code easier to manage.
However, there was an unintended side effect of this. Many of the modules use Python builtin modules like sys or os to do whatever, and the way the code was previously set up, if I typed import sys in module x, and used from x import * in module y, I didn't have to import sys in module y. As a result, I took this for granted quite a lot (terrible practice, I know). When I switched to import x, this caused a lot of broken functions, as you can imagine.
So here's the main issue: because Python is an interpreted language, errors about missing modules in a function won't show up until a function is actually run. And because this is just a general-use library, some of these errors could persist for months undetected, or longer.
I'm completely prepared to write a unit test for each module (if __name__ == "__main__" and all that), but I wanted to ask first: is there an automated way of checking every function in a module for import/syntax errors, or any other error that is not dependent on input? Things that a compiler would catch in C or another language. A brief Google and SO search didn't turn up anything. Any suggestions are welcome and appreciated.
Yes. PyFlakes will warn you about those most basic errors, and you should make sure it's integrated with your favorite text editor, so it tells you about missing imports or unused imports whenever you save the file.
PyFlakes will, amgonst other things, tell you about
syntax errors
undefined names
missing imports
unused imports
To just run PyFlakes on all your files in a directory, you can just do:
pyflakes /path/to/dir
One big advantage that PyFlakes has over more advanced linting tools like PyLint is that it does static analysis - which means, it doesn't need to import your code (which can be a pain if you've got some complex dependencies). It just analyses the abstract syntax tree of your Python source, and therefore catches the most basic of errors - those that usually prevent your script from having even a chance of running.
I should also mention that there is a related tool, flake8, which combines PyFlakes with PEP8 convention checks and McCabe code complexity analysis.
There's PyFlakes integrations for every editor (or IDE) I know. Here's just a couple (in no particular order):
Sublime Text
vim
emacs
TextMate
Eclipse / PyDev
I'm wondering about the preferred way to import packages in a Python application. I have a package structure like this:
project.app1.models
project.app1.views
project.app2.models
project.app1.views imports project.app1.models and project.app2.models. There are two ways to do this that come to mind.
With absolute imports:
import A.A
import A.B.B
or with explicit relative imports, as introduced in Python 2.5 with PEP 328:
# explicit relative
from .. import A
from . import B
What is the most pythonic way to do this?
Python relative imports are no longer strongly discouraged, but using absolute_import is strongly suggested in that case.
Please see this discussion citing Guido himself:
"Isn't this mostly historical? Until the new relative-import syntax
was implemented there were various problems with relative imports. The
short-term solution was to recommend not using them. The long-term
solution was to implement an unambiguous syntax. Now it is time to
withdraw the anti-recommendation. Of course, without going overboard
-- I still find them an acquired taste; but they have their place."
The OP correctly links the PEP 328 that says:
Several use cases were presented, the most important of which is being
able to rearrange the structure of large packages without having to
edit sub-packages. In addition, a module inside a package can't easily
import itself without relative imports.
Also see almost duplicate question When or why to use relative imports in Python
Of course it still stands as a matter of taste. While it's easier to move code around with relative imports, that might also unexpectedly break things; and renaming the imports is not that difficult.
To force the new behaviour from PEP 328 use:
from __future__ import absolute_import
In this case, implicit relative import will no longer be possible (eg. import localfile will not work anymore, only from . import localfile). For clean and future proof behaviour, using absolute_import is advisable.
An important caveat is that because of PEP 338 and PEP 366, relative imports require the python file to be imported as a module - you cannot execute a file.py that has a relative import or you'll get a ValueError: Attempted relative import in non-package.
This limitation should be taken into account when evaluating the best approach. Guido is against running scripts from a module in any case:
I'm -1 on this and on any other proposed twiddlings of the __main__ machinery.
The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern.
To make me change my mind you'd have to convince me that it isn't.
Exhaustive discussions on the matter can be found on SO; re. Python 3 this is quite comprehensive:
Relative imports in Python 3
Absolute imports. From PEP 8:
Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.
Explicit relative imports are a nice language feature (I guess), but they're not nearly as explicit as absolute imports. The more readable form is:
import A.A
import A.B.B
especially if you import several different namespaces. If you look at some well written projects/tutorials that include imports from within packages, they usually follow this style.
The few extra keystrokes you take to be more explicit will save others (and perhaps you) plenty of time in the future when they're trying to figure out your namespace (especially if you migrate to 3.x, in which some of the package names have changed).
Relative imports not only leave you free to rename your package later without changing dozens of internal imports, but I have also had success with them in solving certain problems involving things like circular imports or namespace packages, because they do not send Python "back to the top" to start the search for the next module all over again from the top-level namespace.
Just after standard pythonmodule imports?
If I postpone it to the main function and do my specific module imports before it, it gives error (which is quite obvious). Python Style guide no where mentions the correct location for it.
It should go before the import or from statements that need it (which as you say is obvious). So for example a module could start with:
import sys
import os
import math
try:
import foo
except ImportError:
if 'foopath' in sys.path: raise
sys.path.append('foopath')
import foo
Note that I've made the append conditional (on the import failing and the specific module's path not being on sys.path yet) to avoid the risk of sys.path ending up with dozens of occurrences of string foopath, which would not be particularly helpful;-).
One reason this isn't mentioned in PEP 8 or other good Python style guides is that modifying sys.path isn't something you want to do in a real program; it makes your program less robust and portable. A better solution might be to put your package somewhere that would already be in sys.path or to define PYTHONPATH systemwide to include your package.
I often use a shell script to launch my python applications: I put my sys.path.insert (append) statement just after the "standard" python module imports in my "launch python script".
Using sys.path.insert(0, ...) gets your "imports" in priority in the path list.
I generally do it before importing anything. If you're worried that your module names might conflict with the Python stdlib names, then change your module names!
I think it's a matter of taste. But most people tend to put it behind the import sys :-)
I prefer wrapping it in an extra function:
def importmod_abs(name):
sys.path.append() ..
__import__ ...
sys.path.pop()
... this way sys.path remains clean. Of course that's only applicable to certain module structures. Anyways, I'd import everything that works without altering sys.path first.