I noticed Flask was using Werkzeug to __import__ a module, and I was a little confused. I went and checked out the docs on it and saw that it seems to give you more control somehow in terms of where it looks for the module, but I'm not sure exactly how and I have zero idea how it's different from importlib.import_module.
The odd thing in the Werkzeug example is that it just says __import__(import_name), so I don't see how that's any different from just using the import statement, since it's ignoring the optional extra parameters.
Can anyone explain? I looked at other people having asked similar questions on SO previously but they weren't very clearly phrased questions and the answers didn't address this at all.
__import__ is a low-level hook function that's used to import modules; it can be used to import a module dynamically by giving the module name to import as a variable, something the import statement won't let you do.
importlib.import_module() is a wrapper around that hook* to produce a nice API for the functionality; it is a very recent addition to Python 2, and has been more fleshed out in Python 3. Codebases that use __import__ generally do so because they want to remain compatible with older Python 2 releases, e.g. anything before Python 2.7.
One side-effect of using __import__ can be that it returns the imported module and doesn't add anything to the namespace; you can import with it without having then to delete the new name if you didn't want that new name; using import somename will add somename to your namespace, but __import__('somename') instead returns the imported module, which you can then ignore. Werkzeug uses the hook for that reason in one location.
All other uses are to do with dynamic imports. Werkzeug supports Python 2.6 still so cannot use importlib.
* importlib is a Pure-Python implementation, and import_module() will use that implementation, whist __import__ will use a C-optimised version. Both versions call back to importlib._bootstrap._find_and_load() so the difference is mostly academic.
__import__(import_name), so I don't see how that's any different from
just using the import statement
Both __import__() and importlib.import_module() allow you to import a module when you have the module name as a string. You cannot write:
x = 're'
import x
or you'll get:
File "1.py", line 3, in <module>
import x ImportError: No module named x
Related
from module import * VS import module
What I know
I know the difference between the 2, the difference is when you are using from module import *, you can just refer the classes, functions etc. in the module just like they are defined in the file they are imported in itself.
But when you are just usingimport module, you have to use module. before the name of the object to refer it.
The problem
So what I don’t know is why is it sometimes considered bad practice to use from module import * instead of import module?
PEP 8 states that
Wildcard imports (from <module> import *) should be avoided, as they
make it unclear which names are present in the namespace, confusing
both readers and many automated tools. There is one defensible use
case for a wildcard import, which is to republish an internal
interface as part of a public API (for example, overwriting a pure
Python implementation of an interface with the definitions from an
optional accelerator module and exactly which definitions will be
overwritten isn't known in advance).
I have an excel sheet having thousands of import statements.
eg.,
from XYZ.loghelper import LogHelper,
import os,
from models import CustomUser, VerticalApp,
from django.http import HttpResponse,
Some of them are built-in and some are user defined.
Now I have to find whether they are user defined or built-in.
How can I do that?
I assume that by builtin you mean "part of Python's stdlib" (python's "builtin" features, being "builtin", don't need to be imported at all). The definition of "user defined" is much more vague - is a 3rd part package like Django "builtin" or "user-defined" ?
But anyway: the short answer is that technically you CAN NOT tell just from the import statement. Modules are looked up in sys.path and the first matching module will be selected, so if you have a module named "os.py" in a local directory that comes before your Python installation's stdlib directory in sys.path then "import os" will indeed import your own "os.py" module instead of the stdlib's one. IOW, you need to use the exact same environment, import the module, and check the module's __file__ attribute to find out where it's been imported from.
Now most python devs try to avoid shadowing stdlib's module names (for obvious reasons) so you can also just build a list (technically you want a set for better perfs but anyway) of Python's stdlibs modules names, parse your imports statements, and check if the name of the module to be imported belongs to the set of stdlib's names. This should yield correct results in most cases, but it's not garanteed to be 100% accurate.
Basically there is a file called 8puzzle.py and I want to import the file into another file (in the same folder and I cannot change the file name as the file is provided). Is there anyway to do this in Python? I tried usual way from 8puzzle import *, it gives me an error.
Error is:
>>> import 8puzzle
File "<input>", line 1
import 8puzzle
^
SyntaxError: invalid syntax
>>>
You could do
puzzle = __import__('8puzzle')
Very interesting problem. I'll remember not to name anything with a number.
If you'd like to import * -- you should check out this question and answer.
The above answers are correct, but as for now, the recommended way is to use import_module function:
importlib.import_module(name, package=None)
Import a module. The name
argument specifies what module to import in absolute or relative terms
(e.g. either pkg.mod or ..mod). If the name is specified in relative
terms, then the package argument must be set to the name of the
package which is to act as the anchor for resolving the package name
(e.g. import_module('..mod', 'pkg.subpkg') will import pkg.mod).
The import_module() function acts as a simplifying wrapper around
importlib.__import__(). This means all semantics of the function are
derived from importlib.__import__(). The most important difference
between these two functions is that import_module() returns the
specified package or module (e.g. pkg.mod), while __import__() returns
the top-level package or module (e.g. pkg).
If you are dynamically importing a module that was created since the
interpreter began execution (e.g., created a Python source file), you
may need to call invalidate_caches() in order for the new module to be
noticed by the import system.
__import__ is not recommended now.
importlib.__import__(name, globals=None, locals=None, fromlist=(), level=0)
An implementation of the built-in __import__() function.
Note Programmatic importing of modules should use import_module() instead of this function.
The file directory structure is as follows:
daily
-- 20210504
permutations.py
__init__.py
__init__.py
You can import the permutations module by __import__ or importlib.import_module.
The official documentation recommends using importlib.import_module.
import(name, globals=None, locals=None, fromlist=(), level=0) -> module
Import a module. Because this function is meant for use by the Python interpreter and not for general use, it is better to
useimportlib.import_module()to programmatically import a module.
What is the difference?
If implemented using __import__.
For example:
res = __import__('daily.20210504.permutations')
The result of res is the daily module.
So, if you want to get the permutations module, you need to provide the fromlist parameter, which is written as follows.
res = __import__('daily.20210504.permutations', fromlist=('daily.20210504'))
The result of res can be seen now as
That's the right result.
What if I use importlib.import_module?
res = importlib.import_module('daily.20210504.permutations')
this allows you to get the permutations module directly.
Don't use the .py extension in your imports.
Does from 8puzzle import * work?
For what it's worth, from x import * is not a preferred Python pattern, as it bleeds that module's namespace into your current context.
In general, try to import things you specifically want from that module. Any global from the other module can be imported.
e.g., if you have 8puzzle.foo you could do `from 8puzzle import
Edit:
While my .py message is correct, it isn't sufficient.
The other poster's __import__('8puzzle') suggestion is correct. However, I highly recommend avoiding this pattern.
For one, it's reserved an internal, private Python method. You are basically breaking the fundamental assumptions of what it means to be able to import a module. Simply renaming the file to something else, like puzzle8, will remedy this.
This will frustrate the hell out of experienced Python programmers who are expecting to know what your imports are at the top and are expecting code to (try to) conform to PEP8.
I'm successfully able to create import hooks to load files directly from memory in python2.7. The example I used was the accepted response to this question:
python:Import module from memory
However; when applying this code on pypy; i get an import error. I have also tried other import hook examples that work with regular python but not with pypy, such as this:
python load zip with modules from memory
Does anyone know why import hooks do not work in pypy? Is there something I am missing?
The problem is that in both of the examples you point to, load_module() does not add the loaded module to sys.modules. Normally, it should do so (and then PyPy works like CPython).
If load_module() does not add the module to sys.modules, then every single import a will call load_module() again and return a new copy of the module. For example, in the example from python:Import module from memory:
import a as a1
import a as a2
print a1 is a2 # False!
a1.foo = "foo"
print a2.foo # AttributeError
This is documented in https://www.python.org/dev/peps/pep-0302/#id27. The load_module() method is responsible for doing more checks than these simple examples show. In particular, note this line (emphasis in the original):
Note that the module object must be in sys.modules before the loader executes the module code.
So, the fact that PyPy behaves differently than CPython in this case could be understood as a behavior difference that follows from code that fails to respect the docs.
But anyway, my opinion is that it should be fixed. I've created an issue at https://bitbucket.org/pypy/pypy/issues/2382/sysmeta_path-not-working-like-cpythons.
Basically there is a file called 8puzzle.py and I want to import the file into another file (in the same folder and I cannot change the file name as the file is provided). Is there anyway to do this in Python? I tried usual way from 8puzzle import *, it gives me an error.
Error is:
>>> import 8puzzle
File "<input>", line 1
import 8puzzle
^
SyntaxError: invalid syntax
>>>
You could do
puzzle = __import__('8puzzle')
Very interesting problem. I'll remember not to name anything with a number.
If you'd like to import * -- you should check out this question and answer.
The above answers are correct, but as for now, the recommended way is to use import_module function:
importlib.import_module(name, package=None)
Import a module. The name
argument specifies what module to import in absolute or relative terms
(e.g. either pkg.mod or ..mod). If the name is specified in relative
terms, then the package argument must be set to the name of the
package which is to act as the anchor for resolving the package name
(e.g. import_module('..mod', 'pkg.subpkg') will import pkg.mod).
The import_module() function acts as a simplifying wrapper around
importlib.__import__(). This means all semantics of the function are
derived from importlib.__import__(). The most important difference
between these two functions is that import_module() returns the
specified package or module (e.g. pkg.mod), while __import__() returns
the top-level package or module (e.g. pkg).
If you are dynamically importing a module that was created since the
interpreter began execution (e.g., created a Python source file), you
may need to call invalidate_caches() in order for the new module to be
noticed by the import system.
__import__ is not recommended now.
importlib.__import__(name, globals=None, locals=None, fromlist=(), level=0)
An implementation of the built-in __import__() function.
Note Programmatic importing of modules should use import_module() instead of this function.
The file directory structure is as follows:
daily
-- 20210504
permutations.py
__init__.py
__init__.py
You can import the permutations module by __import__ or importlib.import_module.
The official documentation recommends using importlib.import_module.
import(name, globals=None, locals=None, fromlist=(), level=0) -> module
Import a module. Because this function is meant for use by the Python interpreter and not for general use, it is better to
useimportlib.import_module()to programmatically import a module.
What is the difference?
If implemented using __import__.
For example:
res = __import__('daily.20210504.permutations')
The result of res is the daily module.
So, if you want to get the permutations module, you need to provide the fromlist parameter, which is written as follows.
res = __import__('daily.20210504.permutations', fromlist=('daily.20210504'))
The result of res can be seen now as
That's the right result.
What if I use importlib.import_module?
res = importlib.import_module('daily.20210504.permutations')
this allows you to get the permutations module directly.
Don't use the .py extension in your imports.
Does from 8puzzle import * work?
For what it's worth, from x import * is not a preferred Python pattern, as it bleeds that module's namespace into your current context.
In general, try to import things you specifically want from that module. Any global from the other module can be imported.
e.g., if you have 8puzzle.foo you could do `from 8puzzle import
Edit:
While my .py message is correct, it isn't sufficient.
The other poster's __import__('8puzzle') suggestion is correct. However, I highly recommend avoiding this pattern.
For one, it's reserved an internal, private Python method. You are basically breaking the fundamental assumptions of what it means to be able to import a module. Simply renaming the file to something else, like puzzle8, will remedy this.
This will frustrate the hell out of experienced Python programmers who are expecting to know what your imports are at the top and are expecting code to (try to) conform to PEP8.