I am importing lots of functions from a module
Is it better to use
from my_module import function1, function2, function3, function4, function5, function6, function7
which is a little messy, but avoids flooding the current namespace with everything from that module or
from my_module import *
Which looks tidy but will fill the namespace with everything from that module.
Can't find anything in PEP8 about what the limit for how much you should import by name is. Which is better and why?
If you really need that many functions, you are already polluting your namespace.
I would suggest:
import my_module
Or, if my_module has a long name use an alias:
import my_long_module as m
If it's between one or the other, use
from my_module import function1, function2, function3, function4, function5, function6, function7
See "Explicit is better than implicit." in import this.
If you just want a shorter name than my_module.function1, there is always import my_module as mod.
For the few functions you use many times (either type many times so you want a short name or in a loop so access speed is important), there is
func1 = my_module.function1
With a little bit of management you can control what import * imports. Say your my_module has function1..function8 but you only want to make functions 1 through 6 available. In your my_module, reassign the __all__ attribute:
my_module.py:
__all__ = ['function1', 'function2', 'function3' ...]
def function1():
...
# etc...
Now if you use from my_module import *, you'll only import those functions and variables you defined in the __all__ attribute from my_module.py.
Not sure if this is new, but now you can do:
from my_module import (
function1,
function2,
function3,
function4
)
At least this doesn't go off the page and is easier to read IMO.
Related
I have a module foo.py, which I am importing in my file main.py, I have imports at the top of foo.py such as import numpy as np, etc.
Now, if I'm only calling a certain function fun(arg1, arg2, arg3), do the imports at the top of foo.py take place or do I have to add the imports inside the function definition of fun?
Also, does from foo import fun make a difference then import foo in this regard?
File foo.py (to be imported)
import numpy as np
def fun(arg1, arg2, arg3):
x = np.argsort(arg1)
return x
File main.py
import foo
call = fun([2, 34, 0, -1], 4, 5])
Or should I go with this in foo.py?
def fun(arg1, arg2, arg3):
import numpy as np
x = np.argsort(arg1)
return x
No you don't need to import it again inside fun. You can test this out, your code should work even if your import numpy only once at the top of foo.py.
The two ways don't make any difference except that if you import as import foo you have to refer to fun as foo.fun. If you do from foo import fun instead, you can just use fun.
When you import a module, python will execute all the statements in the module file. So, when you import foo, in either of the above two ways, it will run import numpy as np and update the private symbol table for foo. All statements, functions defined inside foo can use symbols in this table without any qualification. In your case, fun will have access to numpy as np.
What happens to numpy import itself is more interesting.
Case 1
from foo import fun
You are only importing fun nothing else. Whatever code in fun will run because of the above reasons, but np itself will be invisible to code in main.
Case 2
import foo
Here you will refer to fun as foo.fun like I said before, but np can also be used as foo.np but this is absolutely not recommended.
It's always best to import modules again if you are going to use them in the current file, don't rely on indirect imports from other files. Since, python caches imports you needn't worry about performance or circular imports.
Read about the import system to understand all this fully.
When the module is loaded for the first time, all the lines in it are run. imports, defs, regular assignments, etc. All these lines initialize a namespace that is the module object. The namespace of foo will have a variable np that points to the loaded numpy module, and a variable fun that points to your function object.
Functions are first class objects in python. In particular, they have a __globals__ (look under "Callable Types" in the linked docs) attribute, which points to the namespace of the module they were defined in. No matter what you do to the reference of foo.fun, the namenp will be available in the function until you delete it from foo itself.
It is not recommended that you import anything inside your function, unless you have good reason to do so, like avoiding a global name. When you import a module, the interpreter will first look into sys.modules. If it is found, the import will not take much longer than a lookup into the global dictionary. However, if the module hasn't been loaded yet, it will be right there and then. You may not want to incur that overhead at an arbitrary point in your program, especially one that may be time-sensitive.
As far as import form, the differences are mostly aesthetic, but they do have practical consequences as well. from foo import fun creates a name fun in your namespace, referring directly to the function object of interest. It contaminates your local namespace with an extra name, but saves you a lookup through foo's namespace dictionary every time you access the function. import foo, on the other hand, keeps everything bundled nicely since you have to call foo.fun, but that requires an extra lookup.
TL;DR
You should put all your imports at the top of your file. It doesn't really matter how you do it.
Don't need to import it again inside function fun().
For further, please check this
Which is a better practice - global import or local import
I have a small application that I would like to split into modules so that my code is better structured and readable. The drawback to this has been that for every module that I import using:
import module
I then have to use module.object for any object that I want to access from that module.
In this case I don't want to pollute the global namespace, I want to fill it with the proper module names so that I don't have to use
from module import *
in order to call an object without the module prepend.
Is there a means to do this that isn't consider to be poor use of from import or import?
Two reasonable options are to import with a shorter name to prepend. E.g.
import module as m
m.foo()
Or explicitly import names that you plan to use:
from module import (foo,bar)
foo()
You should avoid using an asterisk in your imports always. So to answer your question, I would say no, there isn't a better way than just:
import module
module.method()
OR
import really_long_module_name as mm
mm.method()
Take a look here at the pep8 guide "Imports" section:
https://www.python.org/dev/peps/pep-0008/#imports
Wildcard imports ( from import * ) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).
Specific is safer than globbing and I try to only import what I need if I can help it. When I'm learning a new module I'll import the whole thing and then once it's in a good state I go back and refactor by specifically importing the methods I need:
from module import method
method()
I would have to say that you should use the module's name. It's a better way of usage, and makes your code free of namespace confusions and also very understandable.
To make your code more beautiful, you could use the as import:
import random as r
# OR
from random import randint as rint
One solution that I think is not very beautiful, but works, that comes to mind, in case you don't want to pollute the global namespace, you can try to use the import statements, for example, inside functions.
For example:
>>> def a():
... from random import randint
... x = randint(0,2)
... print x
...
>>> a()
1
>>> randint(0,2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'randint' is not defined
This way, the local namespace of the specific function is filled with the values from the module but the global one is clean.
I've literally been trying to understand Python imports for about a year now, and I've all but given up programming in Python because it just seems too obfuscated. I come from a C background, and I assumed that import worked like #include, yet if I try to import something, I invariably get errors.
If I have two files like this:
foo.py:
a = 1
bar.py:
import foo
print foo.a
input()
WHY do I need to reference the module name? Why not just be able to write import foo, print a? What is the point of this confusion? Why not just run the code and have stuff defined for you as if you wrote it in one big file? Why can't it work like C's #include directive where it basically copies and pastes your code? I don't have import problems in C.
To do what you want, you can use (not recommended, read further for explanation):
from foo import *
This will import everything to your current namespace, and you will be able to call print a.
However, the issue with this approach is the following. Consider the case when you have two modules, moduleA and moduleB, each having a function named GetSomeValue().
When you do:
from moduleA import *
from moduleB import *
you have a namespace resolution issue*, because what function are you actually calling with GetSomeValue(), the moduleA.GetSomeValue() or the moduleB.GetSomeValue()?
In addition to this, you can use the Import As feature:
from moduleA import GetSomeValue as AGetSomeValue
from moduleB import GetSomeValue as BGetSomeValue
Or
import moduleA.GetSomeValue as AGetSomeValue
import moduleB.GetSomeValue as BGetSomeValue
This approach resolves the conflict manually.
I am sure you can appreciate from these examples the need for explicit referencing.
* Python has its namespace resolution mechanisms, this is just a simplification for the purpose of the explanation.
Imagine you have your a function in your module which chooses some object from a list:
def choice(somelist):
...
Now imagine further that, either in that function or elsewhere in your module, you are using randint from the random library:
a = randint(1, x)
Therefore we
import random
You suggestion, that this does what is now accessed by from random import *, means that we now have two different functions called choice, as random includes one too. Only one will be accessible, but you have introduced ambiguity as to what choice() actually refers to elsewhere in your code.
This is why it is bad practice to import everything; either import what you need:
from random import randint
...
a = randint(1, x)
or the whole module:
import random
...
a = random.randint(1, x)
This has two benefits:
You minimise the risks of overlapping names (now and in future additions to your imported modules); and
When someone else reads your code, they can easily see where external functions come from.
There are a few good reasons. The module provides a sort of namespace for the objects in it, which allows you to use simple names without fear of collisions -- coming from a C background you have surely seen libraries with long, ugly function names to avoid colliding with anybody else.
Also, modules themselves are also objects. When a module is imported in more than one place in a python program, each actually gets the same reference. That way, changing foo.a changes it for everybody, not just the local module. This is in contrast to C where including a header is basically a copy+paste operation into the source file (obviously you can still share variables, but the mechanism is a bit different).
As mentioned, you can say from foo import * or better from foo import a, but understand that the underlying behavior is actually different, because you are taking a and binding it to your local module.
If you use something often, you can always use the from syntax to import it directly, or you can rename the module to something shorter, for example
import itertools as it
When you do import foo, a new module is created inside the current namespace named foo.
So, to use anything inside foo; you have to address it via the module.
However, if you use from from foo import something, you don't have use to prepend the module name, since it will load something from the module and assign to it the name something. (Not a recommended practice)
import importlib
# works like C's #include, you always call it with include(<path>, __name__)
def include(file, module_name):
spec = importlib.util.spec_from_file_location(module_name, file)
mod = importlib.util.module_from_spec(spec)
# spec.loader.exec_module(mod)
o = spec.loader.get_code(module_name)
exec(o, globals())
For example:
#### file a.py ####
a = 1
#### file b.py ####
b = 2
if __name__ == "__main__":
print("Hi, this is b.py")
#### file main.py ####
# assuming you have `include` in scope
include("a.py", __name__)
print(a)
include("b.py", __name__)
print(b)
the output will be:
1
Hi, this is b.py
2
Say I have two Python modules:
module1.py:
import module2
def myFunct(): print "called from module1"
module2.py:
def myFunct(): print "called from module2"
def someFunct(): print "also called from module2"
If I import module1, is it better etiquette to re-import module2, or just refer to it as module1.module2?
For example (someotherfile.py):
import module1
module1.myFunct() # prints "called from module1"
module1.module2.myFunct() # prints "called from module2"
I can also do this: module2 = module1.module2. Now, I can directly call module2.myFunct().
However, I can change module1.py to:
from module2 import *
def myFunct(): print "called from module1"
Now, in someotherfile.py, I can do this:
import module1
module1.myFunct() # prints "called from module1"; overrides module2
module1.someFunct() # prints "also called from module2"
Also, by importing *, help('module1') shows all of the functions from module2.
On the other hand, (assuming module1.py uses import module2), I can do:
someotherfile.py:
import module1, module2
module1.myFunct() # prints "called from module1"
module2.myFunct() # prints "called from module2"
Again, which is better etiquette and practice? To import module2 again, or to just refer to module1's importation?
Quoting the PEP 8 style guide:
When importing a class from a class-containing module, it's usually okay to spell this:
from myclass import MyClass
from foo.bar.yourclass import YourClass
If this spelling causes local name clashes, then spell them
import myclass
import foo.bar.yourclass
Emphasis mine.
Don't use module1.module2; you are relying on the internal implementation details of module1, which may later change what imports it is using. You can import module2 directly, so do so unless otherwise documented by the module author.
You can use the __all__ convention to limit what is imported from a module with from modulename import *; the help() command honours that list as well. Listing the names you explicitly export in __all__ helps clean up the help() text presentation:
The public names defined by a module are determined by checking the module’s namespace for a variable named __all__; if defined, it must be a sequence of strings which are names defined or imported by that module. The names given in __all__ are all considered public and are required to exist. If __all__ is not defined, the set of public names includes all names found in the module’s namespace which do not begin with an underscore character ('_'). __all__ should contain the entire public API. It is intended to avoid accidentally exporting items that are not part of the API (such as library modules which were imported and used within the module).
Just import module2. Re-importing is relatively costless, since Python caches module objects in sys.modules.
Moreover, chaining dots as in module1.module2.myFunct is a violation of the Law of Demeter. Perhaps some day you will want to replace module1 with some other module module1a which does not import module2. By using import module2, you will avoid having to rewrite all occurrences of module1.module2.myFunct.
from module2 import * is generally a bad practice since it makes it hard to trace where variables come from. And mixing module namespaces can create variable-name conflicts. For example, from numpy import * is a definite no-no, since doing so would override Python's builtin sum, min, max, any, all, abs and round.
I am developing a Python package for dealing with some scientific data. There are multiple frequently-used classes and functions from other modules and packages, including numpy, that I need in virtually every function defined in any module of the package.
What would be the Pythonic way to deal with them? I have considered multiple variants, but every has its own drawbacks.
Import the classes at module-level with from foreignmodule import Class1, Class2, function1, function2
Then the imported functions and classes are easily accessible from every function. On the other hand, they pollute the module namespace making dir(package.module) and help(package.module) cluttered with imported functions
Import the classes at function-level with from foreignmodule import Class1, Class2, function1, function2
The functions and classes are easily accessible and do not pollute the module, but imports from up to a dozen modules in every function look as a lot of duplicate code.
Import the modules at module-level with import foreignmodule
Not too much pollution is compensated by the need to prepend the module name to every function or class call.
Use some artificial workaround like using a function body for all these manipulations and returning only the objects to be exported... like this
def _export():
from foreignmodule import Class1, Class2, function1, function2
def myfunc(x):
return function1(x, function2(x))
return myfunc
myfunc = _export()
del _export
This manages to solve both problems, module namespace pollution and ease of use for functions... but it seems to be not Pythonic at all.
So what solution is the most Pythonic? Is there another good solution I overlooked?
Go ahead and do your usual from W import X, Y, Z and then use the __all__ special symbol to define what actual symbols you intend people to import from your module:
__all__ = ('MyClass1', 'MyClass2', 'myvar1', …)
This defines the symbols that will be imported into a user's module if they import * from your module.
In general, Python programmers should not be using dir() to figure out how to use your module, and if they are doing so it might indicate a problem somewhere else. They should be reading your documentation or typing help(yourmodule) to figure out how to use your library. Or they could browse the source code yourself, in which case (a) the difference between things you import and things you define is quite clear, and (b) they will see the __all__ declaration and know which toys they should be playing with.
If you try to support dir() in a situation like this for a task for which it was not designed, you will have to place annoying limitations on your own code, as I hope is clear from the other answers here. My advice: don't do it! Take a look at the Standard Library for guidance: it does from … import … whenever code clarity and conciseness require it, and provides (1) informative docstrings, (2) full documentation, and (3) readable code, so that no one ever has to run dir() on a module and try to tell the imports apart from the stuff actually defined in the module.
One technique I've seen used, including in the standard library, is to use import module as _module or from module import var as _var, i.e. assigning imported modules/variables to names starting with an underscore.
The effect is that other code, following the usual Python convention, treats those members as private. This applies even for code that doesn't look at __all__, such as IPython's autocomplete function.
An example from Python 3.3's random module:
from warnings import warn as _warn
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin
from os import urandom as _urandom
from collections.abc import Set as _Set, Sequence as _Sequence
from hashlib import sha512 as _sha512
Another technique is to perform imports in function scope, so that they become local variables:
"""Some module"""
# imports conventionally go here
def some_function(arg):
"Do something with arg."
import re # Regular expressions solve everything
...
The main rationale for doing this is that it is effectively lazy, delaying the importing of a module's dependencies until they are actually used. Suppose one function in the module depends on a particular huge library. Importing the library at the top of the file would mean that importing the module would load the entire library. This way, importing the module can be quick, and only client code that actually calls that function incurs the cost of loading the library. Further, if the dependency library is not available, client code that doesn't need the dependent feature can still import the module and call the other functions. The disadvantage is that using function-level imports obscures what your code's dependencies are.
Example from Python 3.3's os.py:
def get_exec_path(env=None):
"""[...]"""
# Use a local import instead of a global import to limit the number of
# modules loaded at startup: the os module is always loaded at startup by
# Python. It may also avoid a bootstrap issue.
import warnings
Import the module as a whole: import foreignmodule. What you claim as a drawback is actually a benefit. Namely, prepending the module name makes your code easier to maintain and makes it more self-documenting.
Six months from now when you look at a line of code like foo = Bar(baz) you may ask yourself which module Bar came from, but with foo = cleverlib.Bar it is much less of a mystery.
Of course, the fewer imports you have, the less of a problem this is. For small programs with few dependencies it really doesn't matter all that much.
When you find yourself asking questions like this, ask yourself what makes the code easier to understand, rather than what makes the code easier to write. You write it once but you read it a lot.
For this situation I would go with an all_imports.py file which had all the
from foreignmodule import .....
from another module import .....
and then in your working modules
import all_imports as fgn # or whatever you want to prepend
...
something = fgn.Class1()
Another thing to be aware of
__all__ = ['func1', 'func2', 'this', 'that']
Now, any functions/classes/variables/etc that are in your module, but not in your modules's __all__ will not show up in help(), and won't be imported by from mymodule import * See Making python imports more structured? for more info.
I would compromise and just pick a short alias for the foreign module:
import foreignmodule as fm
It saves you completely from the pollution (probably the bigger issue) and at least reduces the prepending burden.
I know this is an old question. It may not be 'Pythonic', but the cleanest way I've discovered for exporting only certain module definitions is, really as you've found, to globally wrap the module in a function. But instead of returning them, to export names, you can simply globalize them (global thus in essence becomes a kind of 'export' keyword):
def module():
global MyPublicClass,ExportedModule
import somemodule as ExportedModule
import anothermodule as PrivateModule
class MyPublicClass:
def __init__(self):
pass
class MyPrivateClass:
def __init__(self):
pass
module()
del module
I know it's not much different than your original conclusion, but frankly to me this seems to be the cleanest option. The other advantage is, you can group any number of modules written this way into a single file, and their private terms won't overlap:
def module():
global A
i,j,k = 1,2,3
class A:
pass
module()
del module
def module():
global B
i,j,k = 7,8,9 # doesn't overwrite previous declarations
class B:
pass
module()
del module
Though, keep in mind their public definitions will, of course, overlap.