Develop Reference

What is the difference between import X and from X import *

The common explanation given (as in 12270954 and 710551) is that from X import *
clutters up the namespace, and hence is not recommended. However, the following example
shows that there is something more to it than just cluttering the namespace.
Consider the following code:
x1.py:
g_c = 5
class TestClass():
def run(self):
global g_c
g_c = 1
print(g_c) # prints 1
x2.py:
from x1 import *
t = TestClass()
t.run()
print(g_c) # prints 5, why?
x3.py:
import x1
t = x1.TestClass()
t.run()
print(x1.g_c) # prints 1
The results for running x2.py and x3.py are different (on python 3.6.8). Can someone please explain why
the two imports are behaving differently?
Extra Notes: (to demonstrate the note mentioned by #tdelaney about lists)
Change the assignments in x1.py like this:
g_c = [5]
...
g_c.append(1)
And now, both x2.py and x3.py give the same answer. It is only when an atomic type is being used that the problem arises.

You can imagine from x1 import * doing something approximately equal to
import m1
globals.update(m1.__dict__)
del m1
In other words it imports the module and then copies all the globals into your namespace. The function and classes inside the module however will keep referring to their original globals, not to the copies in your namespace.
Note the "implementation" above is not completely correct as not all names are copied and there are other subtleties, but it's I think an easy way to understand the general behavior.
The difference becomes evident when the module changes its globals but you don't see the change reflected in your global namespace.
After executing from x1 import * the variables sys.modules["x1"].g_c and g_c are two distinct variables with the same value; code in the module is referring to the one inside the module, not to your copy.

from X import * rebinds the objects in X's global namespace to the current module's namespace. If one of these variables is reassigned, the reassignment is private to the current namespace.
The import doesn't change the imported objects, though, it just adds a new reference. So for instance, the global namespace for class TestClass is still the x1.py module. TestClass.run always changes x1.g_c (its global namespace) regardless of which module calls it.
In x2.py, you import the original 5 referenced by x1.g_c to x2.g_c. But reassignment of x1.g_c doesn't affect x2.g_c which is a variable in a different namespace.
Suppose conversely that x1.g_c was a list and your code was appending to the list. In that case, all modules would see the appends because you are modifying the object referenced by x1.g_c, not reassigning it.

When you import X, you will have to use X to call a function inside X, ie:
X.func1()
But when you use from X import *, you import all names in the module's namespace that don't begin with _ or all names in module.__all__ , and you don't require to use X to call the function, ie:
func1()
In x2.py when you did:
from x1 import *
you just imported every thing from x1 , ie , including the variables. This means that your altered the namespace of the current script (x2). So when you call:
print(g_c)
it prints :
5
which is called from the current scripts namespace.
On the run of the class TestClass() in x2 , it changed the value of g_c in x1, but not in x2.
This indicates the Disadvantages of using Global Variables.
When variables are declared as global, then they remain in the memory till program execution is completed. That is, your global declaration is only valid inside x1. The declaration is valid inside x1's namespace.
And when you did:
import x1
t = x1.TestClass()
t.run()
This import x1 as a module and everything inside isin't imported unless you call it. And you did call the class TestClass(), ran it, resulting g_c inside the class being made a global variable. So then for x1, g_c is 1 since it was globally defined and the current script doesn't know x1 has a g_c = 5 since it wasn't imported/called.
On print(x1.g_c), it calls for the value of g_c in x1's namespace, which is 1.

The keys of this behavior is following:
Python makes namespaces for each files
global X makes variable X refer to the object that lives in the namespace where global X is declared
When t.run() is executed in x2.py, global g_c is evaluated and this makes g_c refers to the object in the namespace for x1.py not x2.py.
Trying the following snippet would be help.
x2_.py
from x1 import *
t = TestClass()
t.run()
print(g_c) # prints 5
import x1
print(x1.g_c) # this print 1

Let us take an example where you are importing the module suman in Python. It brings the entire module into your workspace i.e. In the import suman statement it does not give you access to any methods or function associated with like for example if their is a function display_name() in the suman module then you have to import it like suman.display_name()
In from suman import * it brings all of the names inside suman module (like display() for example) into your module. Now you can access those names without a prefix of name of your module like this:
from suman import *
print(display_name())

Concept of namespace, global, local with respect to import

I understand that below are the scoping levels in python (listed in highest to lowest order)
Local(L): Defined inside function/class
Enclosed(E): Defined inside enclosing functions(Nested function concept)
Global(G): Defined at the uppermost level
Built-in(B): Reserved names in Python builtin modules
I understand this concept when we have a single file. I am trying to understand how this concept of namespace, local, enclosed, global, built-in works when using import in python.
I understand we can import using foll. approaches:
import modulename
import modulename.function(function/variable)
import package.module.function(function/variable)
from module import function(function/variable)
from package import module
from package.module import function(function/variable)
from module import *
from package import *
I know that in each of the above the entire module is loaded in the sys modules dictionary. But I am confused with respect to namespace, scope, global, local. I want to understand LEGB principle, namespace, global, localwith respect to the method of import vs from import. I have read that in case of point 1 (import modulename), the modulename is loaded in its own namespace and reference is available to the calling module with which it can change the variable values globally. Does this also mean it is in global namespace? Any if so then global with respect to what? Also I have read that from module import function/variable or from module import * brings function(s)/variable(s) in the calling modules namespace and any changes are not global. Does this mean that it is in local namespace? Local with respect to what?
Example:
Say module foo has a variable bar=10. When we use [import foo] - this brings foo in current namespace and allows us to change the value of bar via foo.bar=1000 and this change is visible to every piece of code that makes use of [import foo print(foo.bar)] after the above assignment was made. Where as in case of [from foo import bar], any change made to bar is only visible to this module and not any other module irrespective of whether they do an [import foo] or [from foo import bar] they would see value of bar as 10.
Any reference/links in this area will be very helpful. I want to understand the basics of how this works internally. Any information will be helpful to understand this. Articles that I found on this topic explain what is import vs from xx import yy. But they don't explain why/how the changes made become global (in case of import) vs local (in case of from import).
Some more examples:
In below code example, change made to a is only seen in the main.py file and does not affect the value of a in module1.py and subsequent modules that import module1.py will still see the value of a as 10.
module1.py:
a=10
main.py:
from module1 import a
print(a)
a=100
Whereas if we used import module1 instead of from module1 import a then the assignment would have changed value of a in module1.py and any subsequent import of module1.py would show the value of a as 100.
Note: Similarly we could also have functions inside the module.py that get and set the value of variable a. And using this approach, once we do a from module1 import getfn, setfn in main.py, we can use these functions which when called make the change to variable a and this change is globally visible.

You're over complicating things. From the point of view of imports, the only thing to know is that import foo brings foo into the current namespace, and from foo import bar brings bar into the current namespace.

Python has only 3 scopes: global local and builtin.
They are relative to the position in the code
Every variable whose value you can change is local
(note that changing global variable results in creating a new local variable instead,
so there are local and global variable of the same name)
Enclosed you speak of are local to that function
Every variable with unchangeable content whose value you can get is global
Builtin variables are exactly like global so they can even be considered global
to see what variables are global and what are local, put this line in your code
print("global:", globals(), "\n\nlocal:", locals())

How to access 'global' variable from subpackage?

I am currently trying to ship some bigger project. That is the reason why I decided to use submodules. Take a look at the project structure:
/sandbox
__init.py__
constants.py
/sub1
__init__.py
foo.py
In my constants.py file, I have declared a single global variable:
MYGLOBAL = 42
I want to use its value in foo.py. This file has 2 functions for testing:
def foofunc():
return 'I am foo.'
def constfunc():
return 'I am {MYGLOBAL}.'
Also, I put this code into /sandbox/sub1/__init__.py:
from .foo import *
from sandbox.constants import *
Now, when I use my interpreter, I try to use both functions like this:
>> import sandbox.sub1
>> sandbox.sub1.foofunc()
'I am foo.'
>> sandbox.sub1.MYGLOBAL
42
>> sandbox.sub1.constfunc()
NameError: name 'MYGLOBAL' is not defined
To my understanding, the global variable is in the same namespace as the functions, but somehow the function does not see it.
How can I access it? Python version is 3.6.
Thanks!

"Global" variables are not really global in Python, and are only available in the namespace of the modules in which they're defined.
You should import constants in foo.py so that MYGLOBAL can be made available as an attribute of the constants module object:
from sandbox import constants
def constfunc():
return f'I am {constants.MYGLOBAL}.'

A late answer to this very nice question (upvoted).
Here is one thing that will give you a better understanding of what is going on:
>>> sandbox.sub1.constfunc.__module__
'sandbox.sub1.foo'
>>> sandbox.sub1.constfunc.__globals__['MYGLOBAL']
KeyError: 'MYGLOBAL'
>>> sandbox.sub1.foo.MYGLOBAL = 5
>>> sandbox.sub1.constfunc.__globals__['MYGLOBAL']
5
>>> sandbox.sub1.constfunc()
'I am 5'
In other words,
"which module does this function actually belong in, and thus looks at to decide what module-global variables are available to it?"
You erroneously assumed it would be sandbox.sub1, but it's actually sandbox.sub1.foo. The reason is because the function's "parent module" is carried with the function object, even when this function object is copied to another module's "workspace".
With this understanding, there are several things you can do. One is as per the other answer here. An alternative solution might be to do this in __init__.py:
from . import foo
from .. import constants
foo.MYGLOBAL = constants.MYGLOBAL
etc.
Interesting things to note:
One thing to note is that the from X import Y as Z syntax is effectively equivalent to saying Z = X.Y. The reason this is an important insight is because it helps you realise that Z is not a 'reference' per se: if you change X.Y, Z will not automatically get updated.
If you change the Y in X to another value, and you want to update Z, you need to reimport it with the "from X import Y as Z" syntax.
Interestingly, you cannot simply do:
sandbox.sub1.constfunc.__module__ = sandbox.constants
and simply expect to 'replace' the function's module-global variables to those of sandbox.constants. There reason for this seems to be that the constfunc.__module__ attribute is simply a 'copy' of the original parent module, and changing it does not affect the actual parent module bound to the function (i.e., it does not dynamically 'rebind' the function to another module; though, since this is an internal variable, it may change in future python versions. Who knows. Don't rely on it.).
Also, the __module__ attribute is not used to create the constfunc.__globals__ dictionary for the function; this always seems to reflect the global dictionary of the 'parent module' directly. So simply replacing the sandbox.sub1.constfunc.__module__ variable doesn't automatically replace the constfunc.__globals__ dictionary, but updating sandbox.sub1.foo does.

How to get the same scope for imported classes in a Python script? [duplicate]

This question already has answers here:
Visibility of global variables in imported modules
(9 answers)
Closed 7 years ago.
It appears that a class defined in a script has a different scope to one that is imported into the script. For example:
In a file foo.py:
class foo(object):
def __init__(self):
print globals()
In my main file:
from foo import foo
class bar(object):
def __init__(self):
print globals()
classimport = foo()
classinternal = bar()
The list of globals returned from foo and bar are different - why is this?
It is making life difficult as any class that needs to access the main globals() has to reside in the main file. How do I ensure that the imported class has the same global scope? Some things that I have tried after reading other posts here and here include:
module = __import__("foo", fromlist="foo")
globals()["foo"] = getattr(module, "foo")
and
__builtin__.foo = foo
Any help appreciated!
[EDIT] ---
So per the link above, this is answered in a duplicate article. It turns out that scope is not shared across modules. It mentions several ways around this, but in my case I need to actually create / read / write global variables. So I created a routine in the main script and pass it as an object when foo and bar are initialized. For example:
def PrintGlobals():
print globals()
class bar(object):
def __init__(self, PrintGlobals):
self.PrintGlobals = PrintGlobals
self.PrintGlobals()
classinternal = bar(PrintGlobals)
(Not my choice of how all this should work, its a hack until I get some time with the application devs :-)

Here's what the Python 3 FAQ has to say:
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a value anywhere within the function’s body, it’s assumed to be a local unless explicitly declared as global.
Though a bit surprising at first, a moment’s consideration explains this. On one hand, requiring global for assigned variables provides a bar against unintended side-effects. On the other hand, if global was required for all global references, you’d be using global all the time. You’d have to declare as global every reference to a built-in function or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for identifying side-effects.
To see globals in various scopes, try doing a print(globals()) at various points during your execution. For example: at the top-level module before any code is run, then in __init__.py if you have any code in there (because you import foo), at foo's module level, within each function, and before/after you modify any variables passed to the function.
This answer further explains:
I think the key thing you're missing here is that each module has its own "global" namespace. This can be a bit confusing at first, because in languages like C, there's a single global namespace shared by all external variables and functions. But once you get past that assumption, the Python way makes perfect sense.
Note however that all names assigned in a package __init__.py file are available in the package namespace when you import the package or a module in the package.

How to make a cross-module variable?

The __debug__ variable is handy in part because it affects every module. If I want to create another variable that works the same way, how would I do it?
The variable (let's be original and call it 'foo') doesn't have to be truly global, in the sense that if I change foo in one module, it is updated in others. I'd be fine if I could set foo before importing other modules and then they would see the same value for it.

If you need a global cross-module variable maybe just simple global module-level variable will suffice.
a.py:
var = 1
b.py:
import a
print a.var
import c
print a.var
c.py:
import a
a.var = 2
Test:
$ python b.py
# -> 1 2
Real-world example: Django's global_settings.py (though in Django apps settings are used by importing the object django.conf.settings).

I don't endorse this solution in any way, shape or form. But if you add a variable to the __builtin__ module, it will be accessible as if a global from any other module that includes __builtin__ -- which is all of them, by default.
a.py contains
print foo
b.py contains
import __builtin__
__builtin__.foo = 1
import a
The result is that "1" is printed.
Edit: The __builtin__ module is available as the local symbol __builtins__ -- that's the reason for the discrepancy between two of these answers. Also note that __builtin__ has been renamed to builtins in python3.

I believe that there are plenty of circumstances in which it does make sense and it simplifies programming to have some globals that are known across several (tightly coupled) modules. In this spirit, I would like to elaborate a bit on the idea of having a module of globals which is imported by those modules which need to reference them.
When there is only one such module, I name it "g". In it, I assign default values for every variable I intend to treat as global. In each module that uses any of them, I do not use "from g import var", as this only results in a local variable which is initialized from g only at the time of the import. I make most references in the form g.var, and the "g." serves as a constant reminder that I am dealing with a variable that is potentially accessible to other modules.
If the value of such a global variable is to be used frequently in some function in a module, then that function can make a local copy: var = g.var. However, it is important to realize that assignments to var are local, and global g.var cannot be updated without referencing g.var explicitly in an assignment.
Note that you can also have multiple such globals modules shared by different subsets of your modules to keep things a little more tightly controlled. The reason I use short names for my globals modules is to avoid cluttering up the code too much with occurrences of them. With only a little experience, they become mnemonic enough with only 1 or 2 characters.
It is still possible to make an assignment to, say, g.x when x was not already defined in g, and a different module can then access g.x. However, even though the interpreter permits it, this approach is not so transparent, and I do avoid it. There is still the possibility of accidentally creating a new variable in g as a result of a typo in the variable name for an assignment. Sometimes an examination of dir(g) is useful to discover any surprise names that may have arisen by such accident.

Define a module ( call it "globalbaz" ) and have the variables defined inside it. All the modules using this "pseudoglobal" should import the "globalbaz" module, and refer to it using "globalbaz.var_name"
This works regardless of the place of the change, you can change the variable before or after the import. The imported module will use the latest value. (I tested this in a toy example)
For clarification, globalbaz.py looks just like this:
var_name = "my_useful_string"

You can pass the globals of one module to onother:
In Module A:
import module_b
my_var=2
module_b.do_something_with_my_globals(globals())
print my_var
In Module B:
def do_something_with_my_globals(glob): # glob is simply a dict.
glob["my_var"]=3

Global variables are usually a bad idea, but you can do this by assigning to __builtins__:
__builtins__.foo = 'something'
print foo
Also, modules themselves are variables that you can access from any module. So if you define a module called my_globals.py:
# my_globals.py
foo = 'something'
Then you can use that from anywhere as well:
import my_globals
print my_globals.foo
Using modules rather than modifying __builtins__ is generally a cleaner way to do globals of this sort.

You can already do this with module-level variables. Modules are the same no matter what module they're being imported from. So you can make the variable a module-level variable in whatever module it makes sense to put it in, and access it or assign to it from other modules. It would be better to call a function to set the variable's value, or to make it a property of some singleton object. That way if you end up needing to run some code when the variable's changed, you can do so without breaking your module's external interface.
It's not usually a great way to do things — using globals seldom is — but I think this is the cleanest way to do it.

I wanted to post an answer that there is a case where the variable won't be found.
Cyclical imports may break the module behavior.
For example:
first.py
import second
var = 1
second.py
import first
print(first.var) # will throw an error because the order of execution happens before var gets declared.
main.py
import first
On this is example it should be obvious, but in a large code-base, this can be really confusing.

I wondered if it would be possible to avoid some of the disadvantages of using global variables (see e.g. http://wiki.c2.com/?GlobalVariablesAreBad) by using a class namespace rather than a global/module namespace to pass values of variables. The following code indicates that the two methods are essentially identical. There is a slight advantage in using class namespaces as explained below.
The following code fragments also show that attributes or variables may be dynamically created and deleted in both global/module namespaces and class namespaces.
wall.py
# Note no definition of global variables
class router:
""" Empty class """
I call this module 'wall' since it is used to bounce variables off of. It will act as a space to temporarily define global variables and class-wide attributes of the empty class 'router'.
source.py
import wall
def sourcefn():
msg = 'Hello world!'
wall.msg = msg
wall.router.msg = msg
This module imports wall and defines a single function sourcefn which defines a message and emits it by two different mechanisms, one via globals and one via the router function. Note that the variables wall.msg and wall.router.message are defined here for the first time in their respective namespaces.
dest.py
import wall
def destfn():
if hasattr(wall, 'msg'):
print 'global: ' + wall.msg
del wall.msg
else:
print 'global: ' + 'no message'
if hasattr(wall.router, 'msg'):
print 'router: ' + wall.router.msg
del wall.router.msg
else:
print 'router: ' + 'no message'
This module defines a function destfn which uses the two different mechanisms to receive the messages emitted by source. It allows for the possibility that the variable 'msg' may not exist. destfn also deletes the variables once they have been displayed.
main.py
import source, dest
source.sourcefn()
dest.destfn() # variables deleted after this call
dest.destfn()
This module calls the previously defined functions in sequence. After the first call to dest.destfn the variables wall.msg and wall.router.msg no longer exist.
The output from the program is:
global: Hello world!
router: Hello world!
global: no message
router: no message
The above code fragments show that the module/global and the class/class variable mechanisms are essentially identical.
If a lot of variables are to be shared, namespace pollution can be managed either by using several wall-type modules, e.g. wall1, wall2 etc. or by defining several router-type classes in a single file. The latter is slightly tidier, so perhaps represents a marginal advantage for use of the class-variable mechanism.

This sounds like modifying the __builtin__ name space. To do it:
import __builtin__
__builtin__.foo = 'some-value'
Do not use the __builtins__ directly (notice the extra "s") - apparently this can be a dictionary or a module. Thanks to ΤΖΩΤΖΙΟΥ for pointing this out, more can be found here.
Now foo is available for use everywhere.
I don't recommend doing this generally, but the use of this is up to the programmer.
Assigning to it must be done as above, just setting foo = 'some-other-value' will only set it in the current namespace.

I use this for a couple built-in primitive functions that I felt were really missing. One example is a find function that has the same usage semantics as filter, map, reduce.
def builtin_find(f, x, d=None):
for i in x:
if f(i):
return i
return d
import __builtin__
__builtin__.find = builtin_find
Once this is run (for instance, by importing near your entry point) all your modules can use find() as though, obviously, it was built in.
find(lambda i: i < 0, [1, 3, 0, -5, -10]) # Yields -5, the first negative.
Note: You can do this, of course, with filter and another line to test for zero length, or with reduce in one sort of weird line, but I always felt it was weird.

I could achieve cross-module modifiable (or mutable) variables by using a dictionary:
# in myapp.__init__
Timeouts = {} # cross-modules global mutable variables for testing purpose
Timeouts['WAIT_APP_UP_IN_SECONDS'] = 60
# in myapp.mod1
from myapp import Timeouts
def wait_app_up(project_name, port):
# wait for app until Timeouts['WAIT_APP_UP_IN_SECONDS']
# ...
# in myapp.test.test_mod1
from myapp import Timeouts
def test_wait_app_up_fail(self):
timeout_bak = Timeouts['WAIT_APP_UP_IN_SECONDS']
Timeouts['WAIT_APP_UP_IN_SECONDS'] = 3
with self.assertRaises(hlp.TimeoutException) as cm:
wait_app_up(PROJECT_NAME, PROJECT_PORT)
self.assertEqual("Timeout while waiting for App to start", str(cm.exception))
Timeouts['WAIT_JENKINS_UP_TIMEOUT_IN_SECONDS'] = timeout_bak
When launching test_wait_app_up_fail, the actual timeout duration is 3 seconds.