I understand that below are the scoping levels in python (listed in highest to lowest order)
Local(L): Defined inside function/class
Enclosed(E): Defined inside enclosing functions(Nested function concept)
Global(G): Defined at the uppermost level
Built-in(B): Reserved names in Python builtin modules
I understand this concept when we have a single file. I am trying to understand how this concept of namespace, local, enclosed, global, built-in works when using import in python.
I understand we can import using foll. approaches:
import modulename
import modulename.function(function/variable)
import package.module.function(function/variable)
from module import function(function/variable)
from package import module
from package.module import function(function/variable)
from module import *
from package import *
I know that in each of the above the entire module is loaded in the sys modules dictionary. But I am confused with respect to namespace, scope, global, local. I want to understand LEGB principle, namespace, global, localwith respect to the method of import vs from import. I have read that in case of point 1 (import modulename), the modulename is loaded in its own namespace and reference is available to the calling module with which it can change the variable values globally. Does this also mean it is in global namespace? Any if so then global with respect to what? Also I have read that from module import function/variable or from module import * brings function(s)/variable(s) in the calling modules namespace and any changes are not global. Does this mean that it is in local namespace? Local with respect to what?
Example:
Say module foo has a variable bar=10. When we use [import foo] - this brings foo in current namespace and allows us to change the value of bar via foo.bar=1000 and this change is visible to every piece of code that makes use of [import foo print(foo.bar)] after the above assignment was made. Where as in case of [from foo import bar], any change made to bar is only visible to this module and not any other module irrespective of whether they do an [import foo] or [from foo import bar] they would see value of bar as 10.
Any reference/links in this area will be very helpful. I want to understand the basics of how this works internally. Any information will be helpful to understand this. Articles that I found on this topic explain what is import vs from xx import yy. But they don't explain why/how the changes made become global (in case of import) vs local (in case of from import).
Some more examples:
In below code example, change made to a is only seen in the main.py file and does not affect the value of a in module1.py and subsequent modules that import module1.py will still see the value of a as 10.
module1.py:
a=10
main.py:
from module1 import a
print(a)
a=100
Whereas if we used import module1 instead of from module1 import a then the assignment would have changed value of a in module1.py and any subsequent import of module1.py would show the value of a as 100.
Note: Similarly we could also have functions inside the module.py that get and set the value of variable a. And using this approach, once we do a from module1 import getfn, setfn in main.py, we can use these functions which when called make the change to variable a and this change is globally visible.
You're over complicating things. From the point of view of imports, the only thing to know is that import foo brings foo into the current namespace, and from foo import bar brings bar into the current namespace.
Python has only 3 scopes: global local and builtin.
They are relative to the position in the code
Every variable whose value you can change is local
(note that changing global variable results in creating a new local variable instead,
so there are local and global variable of the same name)
Enclosed you speak of are local to that function
Every variable with unchangeable content whose value you can get is global
Builtin variables are exactly like global so they can even be considered global
to see what variables are global and what are local, put this line in your code
print("global:", globals(), "\n\nlocal:", locals())
Related
I am currently trying to ship some bigger project. That is the reason why I decided to use submodules. Take a look at the project structure:
/sandbox
__init.py__
constants.py
/sub1
__init__.py
foo.py
In my constants.py file, I have declared a single global variable:
MYGLOBAL = 42
I want to use its value in foo.py. This file has 2 functions for testing:
def foofunc():
return 'I am foo.'
def constfunc():
return 'I am {MYGLOBAL}.'
Also, I put this code into /sandbox/sub1/__init__.py:
from .foo import *
from sandbox.constants import *
Now, when I use my interpreter, I try to use both functions like this:
>> import sandbox.sub1
>> sandbox.sub1.foofunc()
'I am foo.'
>> sandbox.sub1.MYGLOBAL
42
>> sandbox.sub1.constfunc()
NameError: name 'MYGLOBAL' is not defined
To my understanding, the global variable is in the same namespace as the functions, but somehow the function does not see it.
How can I access it? Python version is 3.6.
Thanks!
"Global" variables are not really global in Python, and are only available in the namespace of the modules in which they're defined.
You should import constants in foo.py so that MYGLOBAL can be made available as an attribute of the constants module object:
from sandbox import constants
def constfunc():
return f'I am {constants.MYGLOBAL}.'
A late answer to this very nice question (upvoted).
Here is one thing that will give you a better understanding of what is going on:
>>> sandbox.sub1.constfunc.__module__
'sandbox.sub1.foo'
>>> sandbox.sub1.constfunc.__globals__['MYGLOBAL']
KeyError: 'MYGLOBAL'
>>> sandbox.sub1.foo.MYGLOBAL = 5
>>> sandbox.sub1.constfunc.__globals__['MYGLOBAL']
5
>>> sandbox.sub1.constfunc()
'I am 5'
In other words,
"which module does this function actually belong in, and thus looks at to decide what module-global variables are available to it?"
You erroneously assumed it would be sandbox.sub1, but it's actually sandbox.sub1.foo. The reason is because the function's "parent module" is carried with the function object, even when this function object is copied to another module's "workspace".
With this understanding, there are several things you can do. One is as per the other answer here. An alternative solution might be to do this in __init__.py:
from . import foo
from .. import constants
foo.MYGLOBAL = constants.MYGLOBAL
etc.
Interesting things to note:
One thing to note is that the from X import Y as Z syntax is effectively equivalent to saying Z = X.Y. The reason this is an important insight is because it helps you realise that Z is not a 'reference' per se: if you change X.Y, Z will not automatically get updated.
If you change the Y in X to another value, and you want to update Z, you need to reimport it with the "from X import Y as Z" syntax.
Interestingly, you cannot simply do:
sandbox.sub1.constfunc.__module__ = sandbox.constants
and simply expect to 'replace' the function's module-global variables to those of sandbox.constants. There reason for this seems to be that the constfunc.__module__ attribute is simply a 'copy' of the original parent module, and changing it does not affect the actual parent module bound to the function (i.e., it does not dynamically 'rebind' the function to another module; though, since this is an internal variable, it may change in future python versions. Who knows. Don't rely on it.).
Also, the __module__ attribute is not used to create the constfunc.__globals__ dictionary for the function; this always seems to reflect the global dictionary of the 'parent module' directly. So simply replacing the sandbox.sub1.constfunc.__module__ variable doesn't automatically replace the constfunc.__globals__ dictionary, but updating sandbox.sub1.foo does.
It's pretty well-known (and documented in PEP8) that import statements should be placed at the top of a Python module, which is what I usually do. However, in my current case, I need to import a module (call it module_a) that gets created when another module (call it module_b) gets run, so it won't work to have import module_a at the top of module_b (which needs to create module_a before importing it).
The solution that I have implemented to solve this is to import module_a inside of a function, after declaring global module_a. As far as I can tell, this works as I want it to.
# In module_b
def create_module_a():
# Creates module_a.py
def import_module_a():
global module_a
import module_a
Part 1 of my question, then, is: Is there a way to do what I want to do and not have an import in the function? Is there a way to adhere to PEP8? One potential solution that I've thought of so far is to have another module, module_c, create module_a before module_b is run and imports module_a... but I'm not even sure that this would even work in my situation, and it would most likely complicate things, at least in the short term.
Part 2 of my question has to do with this statement found in the Python documentation:
Names listed in a global statement must not be defined as formal parameters or in a for loop control target, class definition, function definition, or import statement.
Does the last part of this statement mean that a global name must not be imported, or does it mean that one must not do something like import some_module as module_a, where module_a is declared global?
If the former is true, then what is an alternative to having import_module_a() in module_b?
I'm new to Python and there's something that's been bothering me for quite some time. I read in "Learning Python" by Mark Lutz that when we use a from statement to import a name present in a module, it first imports the module, then assigns a new name to it (i.e. the name of the function, class, etc. present in the imported module) and then deletes the module object with the del statement. However what happens if I try to import a name using from that references a name in the imported module that itself is not imported? Consider the following example in which there are two modules mod1.py and mod2.py:
#mod1.py
from mod2 import test
test('mod1.py')
#mod2.py
def countLines(name):
print len(open(name).readlines())
def countChars(name):
print len(open(name).read())
def test(name):
print 'loading...'
countLines(name)
countChars(name)
print '-'*10
Now see what happens when I run or import mod1:
>>>import mod1
loading...
3
44
----------
Here when I imported and ran the test function, it ran successfully although I didn't even import countChars or countLines, and the from statement had already deleted the mod2 module object.
So I basically need to know why this code works even though considering the problems I mentioned it shouldn't.
EDIT: Thanx alot to everyone who answered :)
Every function have a __globals__ attribute which holds a reference for the environment where it search for global variables and functions.
The test function is then linked to the global variables of mod2. So when it calls countLines the interpreter will always find the right function even if you wrote a new one with the same name in the module importing the function.
I think you're wrestling with the way python handles namespaces. when you type from module import thing you are bringing thing from module into your current namespace. So, in your example, when mod1 gets imported, the code is evaluated in the following order:
from mod2 import test #Import mod2, bring test function into current module namespace
test("mod1.py") #run the test function (defined in mod2)
And now for mod2:
#create a new function named 'test' in the current (mod2) namespace
#the first time this module is imported. Note that this function has
#access to the entire namespace where it is defined (mod2).
def test(name):
print 'loading...'
countLines(name)
countChars(name)
print '-'*10
The reason that all of this is important is because python lets you choose exactly what you want to pull into your namespace. For example, say you have a module1 which defines function cool_func. Now you are writing another module (module2) and it makes since for module2 to have a function cool_func also. Python allows you to keep those separate. In module3 you could do:
import module1
import module2
module1.cool_func()
module2.cool_func()
Or, you could do:
from module1 import cool_func
import module2
cool_func() #module1
module2.cool_func()
or you could do:
from module1 import cool_func as cool
from module2 import cool_func as cooler
cool() #module1
cooler() #module2
The possibilities go on ...
Hopefully my point is clear. When you import an object from a module, you are choosing how you want to reference that object in your current namespace.
The other answers are better articulated than this one, but if you run the following you can see that countChars and countLines are actually both defined in test.__globals__:
from pprint import pprint
from mod2 import test
pprint(test.__globals___)
test('mod1')
You can see that importing test brings along the other globals defined in mod2, letting you run the function without worrying about having to import everything you need.
Each module has its own scope. Within mod1, you cannot use the names countLines or countChars (or mod2).
mod2 itself isn't affected in the least by how it happens to be imported elsewhere; all names defined in it are available within the module.
If the webpage you reference really says that the module object is deleted with the del statement, it's wrong. del only removes names, it doesn't delete objects.
From A GUIDE TO PYTHON NAMESPACES,
Even though modules have their own global namespaces, this doesn’t mean that all names can be used from everywhere in the module. A scope refers to a region of a program from where a namespace can be accessed without a prefix. Scopes are important for the isolation they provide within a module. At any time there are a number of scopes in operation: the scope of the current function you’re in, the scope of the module and then the scope of the Python builtins. This nesting of scopes means that one function can’t access names inside another function.
Namespaces are also searched for names inside out. This means that if there is a certain name declared in the module’s global namespace, you can reuse the name inside a function while being certain that any other function will get the global name. Of course, you can force the function to use the global name by prefixing the name with the ‘global’ keyword. But if you need to use this, then you might be better off using classes and objects.
An import statement loads the whole module in memory so that's why the test() function ran successfully.
But as you used from statement that's why you can't use the countLines and countChars directly but test can surely call them.
from statement basically loads the whole module and sets the imported function, variable etc to the global namespace.
for eg.
>>> from math import sin
>>> sin(90) #now sin() is a global variable in the module and can be accesed directly
0.89399666360055785
>>> math
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
math
NameError: name 'math' is not defined
>>> vars() #shows the current namespace, and there's sin() in it
{'__builtins__': <module '__builtin__' (built-in)>, '__file__': '/usr/bin/idle', '__package__': None, '__name__': '__main__', 'main': <function main at 0xb6ac702c>, 'sin': <built-in function sin>, '__doc__': None}
consider a simple file, file.py:
def f1():
print 2+2
def f2():
f1()
import only f2:
>>> from file import f2
>>> f2()
4
though I only imported f2() not f1() but it ran f1() succesfully it's because the module is loaded in memory but we can only access f2(), but f2() can access other parts of the module.
Suppose I have a package named bar, and it contains bar.py:
a = None
def foobar():
print a
and __init__.py:
from bar import a, foobar
Then I execute this script:
import bar
print bar.a
bar.a = 1
print bar.a
bar.foobar()
Here's what I expect:
None
1
1
Here's what I get:
None
1
None
Can anyone explain my misconception?
You are using from bar import a. a becomes a symbol in the global scope of the importing module (or whatever scope the import statement occurs in).
When you assign a new value to a, you are just changing which value a points too, not the actual value. Try to import bar.py directly with import bar in __init__.py and conduct your experiment there by setting bar.a = 1. This way, you will actually be modifying bar.__dict__['a'] which is the 'real' value of a in this context.
It's a little convoluted with three layers but bar.a = 1 changes the value of a in the module called bar that is actually derived from __init__.py. It does not change the value of a that foobar sees because foobar lives in the actual file bar.py. You could set bar.bar.a if you wanted to change that.
This is one of the dangers of using the from foo import bar form of the import statement: it splits bar into two symbols, one visible globally from within foo which starts off pointing to the original value and a different symbol visible in the scope where the import statement is executed. Changing a where a symbol points doesn't change the value that it pointed too.
This sort of stuff is a killer when trying to reload a module from the interactive interpreter.
One source of difficulty with this question is that you have a program named bar/bar.py: import bar imports either bar/__init__.py or bar/bar.py, depending on where it is done, which makes it a little cumbersome to track which a is bar.a.
Here is how it works:
The key to understanding what happens is to realize that in your __init__.py,
from bar import a
in effect does something like
a = bar.a
# … where bar = bar/bar.py (as if bar were imported locally from __init__.py)
and defines a new variable (bar/__init__.py:a, if you wish). Thus, your from bar import a in __init__.py binds name bar/__init__.py:a to the original bar.py:a object (None). This is why you can do from bar import a as a2 in __init__.py: in this case, it is clear that you have both bar/bar.py:a and a distinct variable name bar/__init__.py:a2 (in your case, the names of the two variables just happen to both be a, but they still live in different namespaces: in __init__.py, they are bar.a and a).
Now, when you do
import bar
print bar.a
you are accessing variable bar/__init__.py:a (since import bar imports your bar/__init__.py). This is the variable you modify (to 1). You are not touching the contents of variable bar/bar.py:a. So when you subsequently do
bar.foobar()
you call bar/bar.py:foobar(), which accesses variable a from bar/bar.py, which is still None (when foobar() is defined, it binds variable names once and for all, so the a in bar.py is bar.py:a, not any other a variable defined in another module—as there might be many a variables in all the imported modules). Hence the last None output.
Conclusion: it is best to avoid any ambiguity in import bar, by not having any bar/bar.py module (since bar.__init__.py makes directory bar/ a package already, that you can also import with import bar).
To put another way:
Turns out this misconception is very easy to make.
It is sneakily defined in the Python language reference: the use of object instead of symbol. I would suggest that the Python language reference make this more clear and less sparse..
The from form does not bind the module name: it goes through the
list of identifiers, looks each one of them up in the module found in
step (1), and binds the name in the local namespace to the object thus
found.
HOWEVER:
When you import, you import the current value of the imported symbol and add it to your namespace as defined. You are not importing a reference, you are effectively importing a value.
Thus, to get the updated value of i, you must import a variable that holds a reference to that symbol.
In other words, importing is NOT like an import in JAVA, external declaration in C/C++ or even a use clause in PERL.
Rather, the following statement in Python:
from some_other_module import a as x
is more like the following code in K&R C:
extern int a; /* import from the EXTERN file */
int x = a;
(caveat: in the Python case, "a" and "x" are essentially a reference to the actual value: you're not copying the INT, you're copying the reference address)
I'm new to Python and programming in general (a couple of weeks at most).
Concerning Python and using modules, I realise that functions can imported using from a import *.
So instead of typing
a.sayHi()
a.sayBye()
I can say
sayHi()
sayBye()
which I find simplifies things a great deal. Now, say I have a bunch of variables that I want to use across modules and I have them all defined in one python module. How can I, using a similar method as mentioned above or an equally simple one, import these variables. I don't want to use import a and then be required to prefix all my variables with a..
The following situation would by ideal:
a.py
name = "Michael"
age = 15
b.py
some_function
if name == "Michael":
if age == 15:
print("Simple!")
Output:
Simple!
You gave the solution yourself: from a import * will work just fine. Python does not differentiate between functions and variables in this respect.
>>> from a import *
>>> if name == "Michael" and age == 15:
... print('Simple!')
...
Simple!
Just for some context, most linters will flag from module import * with a warning, because it's prone to namespace collisions that will cause headaches down the road.
Nobody has noted yet that, as an alternative, you can use the
from a import name, age
form and then use name and age directly (without the a. prefix). The from [module] import [identifiers] form is more future proof because you can easily see when one import will be overriding another.
Also note that "variables" aren't different from functions in Python in terms of how they're addressed -- every identifier like name or sayBye is pointing at some kind of object. The identifier name is pointing at a string object, sayBye is pointing at a function object, and age is pointing at an integer object. When you tell Python:
from a import name, age
you're saying "take those objects pointed at by name and age within module a and point at them in the current scope with the same identifiers".
Similarly, if you want to point at them with different identifiers on import, you can use the
from a import sayBye as bidFarewell
form. The same function object gets pointed at, except in the current scope the identifier pointing at it is bidFarewell whereas in module a the identifier pointing at it is sayBye.
Like others have said,
from module import *
will also import the modules variables.
However, you need to understand that you are not importing variables, just references to objects. Assigning something else to the imported names in the importing module won't affect the other modules.
Example: assume you have a module module.py containing the following code:
a= 1
b= 2
Then you have two other modules, mod1.py and mod2.py which both do the following:
from module import *
In each module, two names, a and b are created, pointing to the objects 1 and 2, respectively.
Now, if somewhere in mod1.py you assign something else to the global name a:
a= 3
the name a in module.py and the name a in mod2.py will still point to the object 1.
So from module import * will work if you want read-only globals, but it won't work if you want read-write globals. If the latter, you're better off just importing import module and then either getting the value (module.a) or setting the value (module.a= …) prefixed by the module.
You didn't say this directly, but I'm assuming you're having trouble with manipulating these global variables.
If you manipulate global variables from inside a function, you must declare them global
a = 10
def x():
global a
a = 15
print a
x()
print a
If you don't do that, then a = 15 will just create a local variable and assign it 15, while the global a stays 10