Develop Reference

“from module import *” VS “import module”

from module import * VS import module
What I know
I know the difference between the 2, the difference is when you are using from module import *, you can just refer the classes, functions etc. in the module just like they are defined in the file they are imported in itself.
But when you are just usingimport module, you have to use module. before the name of the object to refer it.
The problem
So what I don’t know is why is it sometimes considered bad practice to use from module import * instead of import module?

PEP 8 states that
Wildcard imports (from <module> import *) should be avoided, as they
make it unclear which names are present in the namespace, confusing
both readers and many automated tools. There is one defensible use
case for a wildcard import, which is to republish an internal
interface as part of a public API (for example, overwriting a pure
Python implementation of an interface with the definitions from an
optional accelerator module and exactly which definitions will be
overwritten isn't known in advance).

import vs __import__( ) vs importlib.import_module( )?

I noticed Flask was using Werkzeug to __import__ a module, and I was a little confused. I went and checked out the docs on it and saw that it seems to give you more control somehow in terms of where it looks for the module, but I'm not sure exactly how and I have zero idea how it's different from importlib.import_module.
The odd thing in the Werkzeug example is that it just says __import__(import_name), so I don't see how that's any different from just using the import statement, since it's ignoring the optional extra parameters.
Can anyone explain? I looked at other people having asked similar questions on SO previously but they weren't very clearly phrased questions and the answers didn't address this at all.

__import__ is a low-level hook function that's used to import modules; it can be used to import a module dynamically by giving the module name to import as a variable, something the import statement won't let you do.
importlib.import_module() is a wrapper around that hook* to produce a nice API for the functionality; it is a very recent addition to Python 2, and has been more fleshed out in Python 3. Codebases that use __import__ generally do so because they want to remain compatible with older Python 2 releases, e.g. anything before Python 2.7.
One side-effect of using __import__ can be that it returns the imported module and doesn't add anything to the namespace; you can import with it without having then to delete the new name if you didn't want that new name; using import somename will add somename to your namespace, but __import__('somename') instead returns the imported module, which you can then ignore. Werkzeug uses the hook for that reason in one location.
All other uses are to do with dynamic imports. Werkzeug supports Python 2.6 still so cannot use importlib.
* importlib is a Pure-Python implementation, and import_module() will use that implementation, whist __import__ will use a C-optimised version. Both versions call back to importlib._bootstrap._find_and_load() so the difference is mostly academic.

__import__(import_name), so I don't see how that's any different from
just using the import statement
Both __import__() and importlib.import_module() allow you to import a module when you have the module name as a string. You cannot write:
x = 're'
import x
or you'll get:
File "1.py", line 3, in <module>
import x ImportError: No module named x

Should I define __all__ even if I prefix hidden functions and variables with underscores in modules?

From the perspective of an external user of the module, are both necessary?
From my understanding, by correctly prefix hidden functions with an underscore it essentially does the same thing as explicitly define __all__, but I keep seeing developers doing both in their code. Why is that?

When importing from a module with from modulename import * names starting with underscores are indeed skipped.
However, a module rarely contains only public API objects. Usually you've made imports to support the code as well, and those names are global in the module as well. Without __all__, those names would be part of the import too.
In other words, unless you want to 'export' os in the following example you should use __all__:
import os
from .implementation import some_other_api_call
_module_path = os.path.dirname(os.path.abspath(__file__))
_template = open(os.path.join(_module_path, 'templates/foo_template.txt')).read()
VERSION = '1.0.0'
def make_bar(baz, ham, spam):
return _template.format(baz, ham, spam)
__all__ = ['some_other_api_call', 'make_bar']
because without the __all__ list, Python cannot distinguish between some_other_api_call and os here and divine which one should not be imported when using from ... import *.
You could work around this by renaming all your imports, so import os as _os, but that'd just make your code less readable.
And an explicit export list is always nice. Explicit is better than implicit, as the Zen of Python tells you.

I also use __all__: that explictly tells module users what you intend to export. Searching the module for names is tedious, even if you are careful to do, e.g., import os as _os, etc. A wise man once wrote "explicit is better than implicit" ;-)

Defining all will overide the default behaviour. There is actually might be one reason to define __all__
When importing a module, you might want that doing from mod import * will import only a minimal amount of things. Even if you prefix everything correctly, there could be reasons not to import everything.
The other problem that I had once was defining a gettext shortcut. The translation function was _ which would not get imported. Even if it is prefixed "in theory" I still wanted it to get exported.
One other reason as stated above is importing a module that imports a lot of thing. As python cannot make the difference between symbols created by imports and the one defined in the actual module. It will automatically reexport everything that can be reexported. For that reason, it could be wise to explicitely limit the thing exported to the things you want to export.
With that in mind, you might want to have some prefixed symbols exported by default. Usually, you don't redefine __all__. Whenever you need it to do something unusual then it may make sense to do it.

from import syntax discards attributes starts with _ (underscore)

I have a basic understanding of python, but somewhere I have read that when we import a module using following syntax, it doesn't import attributes defined in specified module which starts with _ (single underscore). Can anybody tell me how it is happening and why it is like that ?
from module.submodule import *

It's by design. Variables starting with an underscore are regarded as for internal use only (not the same as private in other languages). They can still be accessed on the module directly, but they arn't imported on a * import.
From the documentation about * imports:
This imports all names except those beginning with an underscore (_). In most cases Python programmers do not use this facility since it introduces an unknown set of names into the interpreter, possibly hiding some things you have already defined.
This is also to tell you that it's discouraged to use a * import, better to explicitly import the things you need. The exception are modules that are designed to be used via * import, that means they have an __all__ attribute (a list containing containing the names of everything the module wants to export).

Module name different than directory name?

Let's assume I have a python package called bestpackage.
Convention dictates that bestpacakge would also be a directory on sys.path that contains an __init__.py to make the interpreter assume it can be imported from.
Is there any way I can set a variable for the package name so the directory could be named something different than the directive I import it with? Is there any way to make the namespacing not care about the directory name and honor some other config instead?
My super trendy client-side devs are just so much in love with these sexy something.otherthing.js project names for one of our smaller side projects!
EDIT:
To clarify, the main purpose of my question was to allow my client side guys continue to call the directories in their "projects" (the one we all have added to our paths) folder using their existing convention (some.app.js), even though in some cases they are in fact python packages that will be on the path and sourced to import statements internally. I realize this is in practice a pretty horrible thing and so I ask more out of curiosity. So part of the big problem here is circumventing the fact that the . in the directory name (and thereby the assumed package name) implies attribute access. It doesn't really surprise me that this cannot be worked around, I was just curious if it was possible deeper in the "magic" behind import.
There's some great responses here, but all rely on doing a classical import of some kind where the attribute accessor . will clash with the directory names.

A directory with a __init__.py file is called a package.
And no, the package name is always the same as the directory. That's how Python can discover packages, it matches it against directory names found on the search path, and if there is a __init__.py file in that directory it has found a match and imports the __init__.py file contained.
You can always import something into your local module namespace under a shorter, easier to use name using the from module import something or the import module as alias syntax:
from something.otherthing.js import foo
from something.otherthing import js as bar
import something.otherthing.js as hamspam

There is one solution wich needs one initial import somewhere
>>> import sys
>>> sys.modules['blinot_existing_blubb'] = sys
>>> import blinot_existing_blubb
>>> blinot_existing_blubb
<module 'sys' (built-in)>
Without a change to the import mechanism you can not import from an other name. This is intended, I think, to make Python easier to understand.
However if you want to change the import mechanism I recommend this: Getting the Most Out of Python Imports

Well, first I would say that Python is not Java/Javascript/C/C++/Cobol/YourFavoriteLanguageThatIsntPython. Of course, in the real world, some of us have to answer to bosses who disagree. So if all you want is some indirection, use smoke and mirrors, as long as they don't pay too much attention to what's under the hood. Write your module the Python way, then provide an API on the side in the dot-heavy style that your coworkers want. Ex:
pythonic_module.py
def func_1():
pass
def func_2():
pass
def func_3():
pass
def func_4():
pass
indirection
/dotty_api_1/__init__.py
from pythonic_module import func_1 as foo, func_2 as bar
/dotty_api_2/__init__.py
from pythonic_module import func_3 as foo, func_4 as bar
Now they can dot to their hearts' content, but you can write things the Pythonic way under the hood.

Actually yes!
you could do a canonical import Whatever or newmodulename = __import__("Whatever")
python keeps track of your modules and you can inspect that by doing:
import sys
print sys.modules
See this article more details
But that's maybe not your problem? let's guess: you have a module in a different path, which your current project can't access because it's not in the sys-path?
well the just add:
import sys
sys.path.append('path_to_the_other_package_or_module_directory')
prior to your import statement or see this SO-post for a more permanent solution.

I was looking for this to happen with setup.py at sdist and install time, rather than runtime, and found the directive package_dir:
https://docs.python.org/3.5/distutils/setupscript.html#listing-whole-packages