I have a basic understanding of python, but somewhere I have read that when we import a module using following syntax, it doesn't import attributes defined in specified module which starts with _ (single underscore). Can anybody tell me how it is happening and why it is like that ?
from module.submodule import *
It's by design. Variables starting with an underscore are regarded as for internal use only (not the same as private in other languages). They can still be accessed on the module directly, but they arn't imported on a * import.
From the documentation about * imports:
This imports all names except those beginning with an underscore (_). In most cases Python programmers do not use this facility since it introduces an unknown set of names into the interpreter, possibly hiding some things you have already defined.
This is also to tell you that it's discouraged to use a * import, better to explicitly import the things you need. The exception are modules that are designed to be used via * import, that means they have an __all__ attribute (a list containing containing the names of everything the module wants to export).
Related
from module import * VS import module
What I know
I know the difference between the 2, the difference is when you are using from module import *, you can just refer the classes, functions etc. in the module just like they are defined in the file they are imported in itself.
But when you are just usingimport module, you have to use module. before the name of the object to refer it.
The problem
So what I don’t know is why is it sometimes considered bad practice to use from module import * instead of import module?
PEP 8 states that
Wildcard imports (from <module> import *) should be avoided, as they
make it unclear which names are present in the namespace, confusing
both readers and many automated tools. There is one defensible use
case for a wildcard import, which is to republish an internal
interface as part of a public API (for example, overwriting a pure
Python implementation of an interface with the definitions from an
optional accelerator module and exactly which definitions will be
overwritten isn't known in advance).
This question already has answers here:
Does Python have “private” variables in classes?
(15 answers)
Closed 7 years ago.
(I've checked out Does Python have “private” variables in classes? -- it asks about classes rather than modules. As such, answers there don't cover import which is what I'm interested in.)
Consider, there is a module called X with variable y. If any other module tries to import the module X, how to avoid loading the variable y in Python?
For example:
# x.py
y=10
We then use this in some other module:
import x
print x.y
How to avoid accessing x.y from the module x.py ?
If you do import module, there's no way to hide any global members, and that's intentional: the module object returned is the "true" one, the very same that's used by the module's members. Dirty hacks like __getattr__ are also prohibited for modules.
One way is to mark "internal" entities with a leading underscore to hint the user they are not intended for external use. This isn't necessary for references to other modules imported by yours since the guidelines explicitly discourage external use of them (the only exception is if the referenced module is inaccessible the normal way).
When doing from module import *, however, you don't get a module reference but import things from it directly into the current namespace. By default, everything except names starting from an underscore is imported. You can override this by defining the __all__ module attribute.
In normal use import foo only imports the module once; all other imports see that it has been imported, and doesn't load it again.
Quoth https://docs.python.org/3/reference/import.html#the-module-cache:
During import, the module name is looked up in sys.modules and if
present, the associated value is the module satisfying the import, and
the process completes. However, if the value is None, then an
ImportError is raised. If the module name is missing, Python will
continue searching for the module.
This has been long been a feature of the interpreter going back at least to version 2.7 and probably earlier.
Speaking to your specific question, there is no variable y, there is x.y because that's what you imported. If you do the highly unrecommended from module import * you can end up with a y, but you shouldn't do that.
From the perspective of an external user of the module, are both necessary?
From my understanding, by correctly prefix hidden functions with an underscore it essentially does the same thing as explicitly define __all__, but I keep seeing developers doing both in their code. Why is that?
When importing from a module with from modulename import * names starting with underscores are indeed skipped.
However, a module rarely contains only public API objects. Usually you've made imports to support the code as well, and those names are global in the module as well. Without __all__, those names would be part of the import too.
In other words, unless you want to 'export' os in the following example you should use __all__:
import os
from .implementation import some_other_api_call
_module_path = os.path.dirname(os.path.abspath(__file__))
_template = open(os.path.join(_module_path, 'templates/foo_template.txt')).read()
VERSION = '1.0.0'
def make_bar(baz, ham, spam):
return _template.format(baz, ham, spam)
__all__ = ['some_other_api_call', 'make_bar']
because without the __all__ list, Python cannot distinguish between some_other_api_call and os here and divine which one should not be imported when using from ... import *.
You could work around this by renaming all your imports, so import os as _os, but that'd just make your code less readable.
And an explicit export list is always nice. Explicit is better than implicit, as the Zen of Python tells you.
I also use __all__: that explictly tells module users what you intend to export. Searching the module for names is tedious, even if you are careful to do, e.g., import os as _os, etc. A wise man once wrote "explicit is better than implicit" ;-)
Defining all will overide the default behaviour. There is actually might be one reason to define __all__
When importing a module, you might want that doing from mod import * will import only a minimal amount of things. Even if you prefix everything correctly, there could be reasons not to import everything.
The other problem that I had once was defining a gettext shortcut. The translation function was _ which would not get imported. Even if it is prefixed "in theory" I still wanted it to get exported.
One other reason as stated above is importing a module that imports a lot of thing. As python cannot make the difference between symbols created by imports and the one defined in the actual module. It will automatically reexport everything that can be reexported. For that reason, it could be wise to explicitely limit the thing exported to the things you want to export.
With that in mind, you might want to have some prefixed symbols exported by default. Usually, you don't redefine __all__. Whenever you need it to do something unusual then it may make sense to do it.
Got a slight problem with some appengine code that I can't work out (may just be because it's late)
I have a folder called modules that has the following items:
-modules
: __init__.py (blank)
: checklogin.py
: customhandlers.py
: datastoretools.py
: emailtools.py
In my code I use all of these, therefore I'm doing this to import them:
from modules import *
Everything works bar emailtools. This:
emailtools.sendNotificationEmail('assignee',report,True)
Results in this error:
File "/home/tom/dev/ad-project/handlers/reporterhandler.py", line 42, in get
emailtools.sendNotificationEmail('assignee',report,True)
NameError: global name 'emailtools' is not defined
This happens wherever I try to use it, but the other three work perfectly fine, any ideas why? Or should I just import them all rather than using *?
This shouldn't work at all. It seems that this wildcard is not the only one You have used. Maybe other modules came from other wildcards.
In any cases wildcard is undesirable
according to PEP 8:
Wildcard imports (from import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).
P.S. I assume You use windows otherwise Modules can't be imported as modules I think.
I tried:
__all__ = ['SpamPublicClass']
But, of course that's just for:
from spammodule import *
Is there a way to block importing of a class. I'm worried about confusion on the API level of my code that somebody will write:
from spammodule import SimilarSpamClass
and it'll cause debugging mayhem.
The convention is to use a _ as a prefix:
class PublicClass(object):
pass
class _PrivateClass(object):
pass
The following:
from module import *
Will not import the _PrivateClass.
But this will not prevent them from importing it. They could still import it explicitly.
from module import _PrivateClass
Start the names of private classes with and underscore, so that it will be clear just by the name that it is not for public use. That will not actually prevent anybody from importing the class, but it shouldn't happen by accident. It's a well established convention that names starting with an underscore are "internal".
There is no way to actually block access to the contents of a module or the contents of a class for that matter in Python. This sort of thing is handled by convention name your class _SimilarSpamClass (with a leading underscore) to indicate to callers that this is an implementation detail of your module and not part of the published API.
To mark something as "private" in Python properly document your public API so other developers know how to use your module correctly and follow the standard naming conventions so that users of your module easily notice when they have strayed from your API to your implementation.