I've inherited some code with imports in each function and using underscores for each module imported as below
def my_func():
from foo import bar as _bar
from spam import meat as _meat
# Do some work
What is the point in the _bar? All imports are done like this.
If the actual names are things that exist as a part of the built in commands in python, this is done as a way to avoid shadowing those built in functions (for example - from mymodule import open would make the built in open which returns file handles inaccessble). Otherwise, it's simply convention for the original author.
I believe functions with name starting with a single underscore can't be imported using this line :
from module import *
for example this module :
def _some_function_1():
pass
def some_function_2():
pass
if you imported this module, you will be able to access only some_function_2()
main:
import fileb
favouritePizza = "pineapple"
fileb.eatPizza
fileb:
from main import favouritePizza
def eatPizza():
print("favouritePizza")
This is what I tried, however, I get no such attribute. I looked at other problems and these wouldn't help.
This is classic example of circular dependency. main imports fileb, while fileb requires main.
Your case is hard (impossible?) to solve even in theory. In reality, python import machinery does even less expected thing — every time you import anything from some module, whole module is read and imported into global(per process) module namespace. Actually, from module import function is just a syntax sugar that gives you ability to not litter your namespace with everything form particular module (from module import *), but behind the scenes, its the almost the same as import module; module.function(...).
From composition/architecture point of view, basic program structure is:
top modules import bottom
bottom modules get parameters from top when being called
You probably want to use favouritePizza variable somehow in fileb, i.e. in eatPizza functions. Good thing to do is to make this function accept parameter, that will be passed from any place that uses it:
# fileb.py
def eatPizza(name):
print(name)
And call it with
# main.py
import fileb
favouritePizza = "pineapple"
fileb.eatPizza(favouritePizza)
I was writing a code in python and got stuck with a doubt. Seems irrelevant but can't get over it. The thing is when I import a module and use it as below:
import math
print math.sqrt(9)
Here I see math(module) as a class which had a method sqrt(). If that is the case then how can I directly use the class without creating an object of it. I am basically unable to understand here the abstraction between class and and object.
Modules are more like objects, not like classes. You don't "instantiate" a module, there's only one instance of each module and you can access it using the import statement.
Specifically, modules are objects of type 'module':
>>> import math
>>> type(math)
<type 'module'>
Each module is going to have a different set of variables and methods.
Modules are instantiated by Python, whenever they are first imported. Modules that have been instantiated are stored in sys.modules:
>>> import sys
>>> 'math' in sys.modules
False
>>> import math
>>> 'math' in sys.modules
True
>>> sys.modules['math'] is math
True
AFAIK all python modules (like math and million more) are instantiated when you have imported it. How many times are they instantiated you ask ? Just once! All modules are singletons.
Just saying the above statement isn't enough so let's dive deep into it.
Create a python module ( module is basically any file ending with ".py" extension ) say "p.py" containing some code as follows:
In p.py
print "Instantiating p.py module. Please wait..."
# your good pythonic optimized functions, classes goes here
print "Instantiating of p.py module is complete."
and in q.py try importing it
import p
and when you run q.py you will see..
Instantiating p.py module. Please wait...
Instantiating of p.py module is complete.
Now have you created an instance of it ? NO! But still you have it up and running ready to be used.
In your case math is not a class. When you import math the whole module math is imported. You can see it like the inclusion of a library (the concept of it).
If you want to avoid to import the whole module (in order to not have everything included in your program), you can do something like this:
from math import sqrt
print sqrt(9)
This way only sqrt is imported and not everything from the math module.
Here I see math(module) as a class which had a method sqrt(). If that is the case then how can I directly use the class without creating an object of it. I am basically unable to understand here the abstraction between class and and object.
When you import a module, the module object is created. Just like when you use open('file.txt') a file object will be created.
You can use a class without creating an object from it by referencing the class name:
class A:
value = 2 + 2
A.value
class A is an object of class type--the built-in class used to create classes. Everything in Python is an object.
When you call the class A() that's how you create an object. *Sometimes objects are created by statements like import creates a module object, def creates a function object, classcreates a class object that creates other objects and many other statements...
I see that there are other questions related to cross module variables, but they don't really fully answer my question.
I have an application that I have split into 3 modules + 1 main application, mainly for ease of readability and maintainability.
2 of these modules have threads with variables that need to be modified from other modules and other module threads.
Whilst I can modify a module's variable from the main code, I don't appear to be able to modify one module's variable from another module unless I import every module into every other module.
The example below where a&b are imported into main and a module a needs to access a variable in module b:
main
module a
var a
module b
var a
main
a.a = 1
b.a = 2
module a
b.a = 3
module b
a.a = 0
without importing module a into module b and importing module b into module a, can this be achieved globally through the main program ?
If I do have to import a and b into main, and then import a into b and b into a, what are the implications in terms of memory and resource usage / speed etc ?
I tried the suggestion from #abarnert:
#moda
vara = 10
#modb
print(str(vara))
#main
import moda
from moda import vara
import modb
however I get "name error vara is not defined"
If the code in the modules are defined as classes, and the main program creates instances of these classes, the main program can pass an instance of one module class to another, and changes to that instance will be reflected everywhere. There would be no need to import a or b into each other, because they would simply have references to each other.
If I do have to import a and b into main, and then import a into b and b into a, what are the implications in terms of memory and resource usage / speed etc ?
Absolutely none for memory—every module that imports a will get a reference to the exact same a module object. All you're doing is increasing its refcount, not creating new objects.
For speed, the time to discover that you're trying to import a module that already exists is almost nothing (it's just looking up the module name in a dictionary). It is slightly slower to access a.a than to just access a. But this is very rarely an issue. If it is, you're almost certainly going to want to copy that value into the locals of whatever function is accessing it over and over, at which point it won't matter which globals it came from.
without importing module a into module b and importing module b into module a, can this be achieved globally through the main program ?
Sure. All you have to do is import (with from a import a or import a.a as aa or whatever) or copy the variables from module a into main.
Note that just makes a new name for each value; it doesn't make references to the variables. There is no such thing as a reference to a variable in Python.
This works if the variables are holding constants, or if they're holding mutable values that you modify. It just doesn't do anything useful if the variables are names that you want to rebind to new values. (If you do need to do that, just wrap the values in something mutable—e.g., turn each variable into a 1-item list, so you can rebind a[0] instead of a, which means anyone else who has a reference to a can see your new a[0] value.)
If you insist on a "true global", even that isn't impossible. See builtins for details. But you almost certainly don't want this.
If you want to be able to modify a module-level variable from a different module then yes, you will need to import the other module. I would question why you need to do this. Perhaps you should be breaking your code into classes instead of separate modules.
For example you could choose to encapsulate the variables that need to be modified by both modules inside a separate class and pass a single instance of that class to all classes (or modules but you should really use classes) that need it.
See Circular (or cyclic) imports in Python for more information about cyclical imports.
I am developing a Python package for dealing with some scientific data. There are multiple frequently-used classes and functions from other modules and packages, including numpy, that I need in virtually every function defined in any module of the package.
What would be the Pythonic way to deal with them? I have considered multiple variants, but every has its own drawbacks.
Import the classes at module-level with from foreignmodule import Class1, Class2, function1, function2
Then the imported functions and classes are easily accessible from every function. On the other hand, they pollute the module namespace making dir(package.module) and help(package.module) cluttered with imported functions
Import the classes at function-level with from foreignmodule import Class1, Class2, function1, function2
The functions and classes are easily accessible and do not pollute the module, but imports from up to a dozen modules in every function look as a lot of duplicate code.
Import the modules at module-level with import foreignmodule
Not too much pollution is compensated by the need to prepend the module name to every function or class call.
Use some artificial workaround like using a function body for all these manipulations and returning only the objects to be exported... like this
def _export():
from foreignmodule import Class1, Class2, function1, function2
def myfunc(x):
return function1(x, function2(x))
return myfunc
myfunc = _export()
del _export
This manages to solve both problems, module namespace pollution and ease of use for functions... but it seems to be not Pythonic at all.
So what solution is the most Pythonic? Is there another good solution I overlooked?
Go ahead and do your usual from W import X, Y, Z and then use the __all__ special symbol to define what actual symbols you intend people to import from your module:
__all__ = ('MyClass1', 'MyClass2', 'myvar1', …)
This defines the symbols that will be imported into a user's module if they import * from your module.
In general, Python programmers should not be using dir() to figure out how to use your module, and if they are doing so it might indicate a problem somewhere else. They should be reading your documentation or typing help(yourmodule) to figure out how to use your library. Or they could browse the source code yourself, in which case (a) the difference between things you import and things you define is quite clear, and (b) they will see the __all__ declaration and know which toys they should be playing with.
If you try to support dir() in a situation like this for a task for which it was not designed, you will have to place annoying limitations on your own code, as I hope is clear from the other answers here. My advice: don't do it! Take a look at the Standard Library for guidance: it does from … import … whenever code clarity and conciseness require it, and provides (1) informative docstrings, (2) full documentation, and (3) readable code, so that no one ever has to run dir() on a module and try to tell the imports apart from the stuff actually defined in the module.
One technique I've seen used, including in the standard library, is to use import module as _module or from module import var as _var, i.e. assigning imported modules/variables to names starting with an underscore.
The effect is that other code, following the usual Python convention, treats those members as private. This applies even for code that doesn't look at __all__, such as IPython's autocomplete function.
An example from Python 3.3's random module:
from warnings import warn as _warn
from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType
from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin
from os import urandom as _urandom
from collections.abc import Set as _Set, Sequence as _Sequence
from hashlib import sha512 as _sha512
Another technique is to perform imports in function scope, so that they become local variables:
"""Some module"""
# imports conventionally go here
def some_function(arg):
"Do something with arg."
import re # Regular expressions solve everything
...
The main rationale for doing this is that it is effectively lazy, delaying the importing of a module's dependencies until they are actually used. Suppose one function in the module depends on a particular huge library. Importing the library at the top of the file would mean that importing the module would load the entire library. This way, importing the module can be quick, and only client code that actually calls that function incurs the cost of loading the library. Further, if the dependency library is not available, client code that doesn't need the dependent feature can still import the module and call the other functions. The disadvantage is that using function-level imports obscures what your code's dependencies are.
Example from Python 3.3's os.py:
def get_exec_path(env=None):
"""[...]"""
# Use a local import instead of a global import to limit the number of
# modules loaded at startup: the os module is always loaded at startup by
# Python. It may also avoid a bootstrap issue.
import warnings
Import the module as a whole: import foreignmodule. What you claim as a drawback is actually a benefit. Namely, prepending the module name makes your code easier to maintain and makes it more self-documenting.
Six months from now when you look at a line of code like foo = Bar(baz) you may ask yourself which module Bar came from, but with foo = cleverlib.Bar it is much less of a mystery.
Of course, the fewer imports you have, the less of a problem this is. For small programs with few dependencies it really doesn't matter all that much.
When you find yourself asking questions like this, ask yourself what makes the code easier to understand, rather than what makes the code easier to write. You write it once but you read it a lot.
For this situation I would go with an all_imports.py file which had all the
from foreignmodule import .....
from another module import .....
and then in your working modules
import all_imports as fgn # or whatever you want to prepend
...
something = fgn.Class1()
Another thing to be aware of
__all__ = ['func1', 'func2', 'this', 'that']
Now, any functions/classes/variables/etc that are in your module, but not in your modules's __all__ will not show up in help(), and won't be imported by from mymodule import * See Making python imports more structured? for more info.
I would compromise and just pick a short alias for the foreign module:
import foreignmodule as fm
It saves you completely from the pollution (probably the bigger issue) and at least reduces the prepending burden.
I know this is an old question. It may not be 'Pythonic', but the cleanest way I've discovered for exporting only certain module definitions is, really as you've found, to globally wrap the module in a function. But instead of returning them, to export names, you can simply globalize them (global thus in essence becomes a kind of 'export' keyword):
def module():
global MyPublicClass,ExportedModule
import somemodule as ExportedModule
import anothermodule as PrivateModule
class MyPublicClass:
def __init__(self):
pass
class MyPrivateClass:
def __init__(self):
pass
module()
del module
I know it's not much different than your original conclusion, but frankly to me this seems to be the cleanest option. The other advantage is, you can group any number of modules written this way into a single file, and their private terms won't overlap:
def module():
global A
i,j,k = 1,2,3
class A:
pass
module()
del module
def module():
global B
i,j,k = 7,8,9 # doesn't overwrite previous declarations
class B:
pass
module()
del module
Though, keep in mind their public definitions will, of course, overlap.