Develop Reference

Python global variable in import * [duplicate]

I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.

Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()

As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.

This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'

A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.

The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()

Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())

Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12

The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.

Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.

Can you perform an inline import in Python?

Say you only wanted to call a regular expression a single time in you code. As far as I am aware, this means that you then need to do import re somewhere before your call of a function from re. Is it possible to combine this with the function call, in-line?
I thought maybe something like this would work
print(import re; re.search(r'<regex>', <string>).group())
but it just threw an error saying invalid syntax at the point of the import. This leads me to believe that the only way to do this is
import re
print(re.search(r'<regex>'), <string>).group())

Answering the question:
Can You Perform an Inline Import in Python?
You can use the built-in importlib module:
print(importlib.import_module('re').search("h", "hello").group())
Output:
'h'
Of course, it would require you to import the importlib module first:
import importlib
print(importlib.import_module('re').search("h", "hello").group())
From the documentation:
The import_module() function acts as a simplifying wrapper around importlib.__import__(). This means all semantics of the function are derived from importlib.__import__(). The most important difference between these two functions is that import_module() returns the specified package or module (e.g. pkg.mod), while __import__() returns the top-level package or module (e.g. pkg).

Is there an accepted way to import all from the global namespace that is not "from module import *"

I have a small application that I would like to split into modules so that my code is better structured and readable. The drawback to this has been that for every module that I import using:
import module
I then have to use module.object for any object that I want to access from that module.
In this case I don't want to pollute the global namespace, I want to fill it with the proper module names so that I don't have to use
from module import *
in order to call an object without the module prepend.
Is there a means to do this that isn't consider to be poor use of from import or import?

Two reasonable options are to import with a shorter name to prepend. E.g.
import module as m
m.foo()
Or explicitly import names that you plan to use:
from module import (foo,bar)
foo()

You should avoid using an asterisk in your imports always. So to answer your question, I would say no, there isn't a better way than just:
import module
module.method()
OR
import really_long_module_name as mm
mm.method()
Take a look here at the pep8 guide "Imports" section:
https://www.python.org/dev/peps/pep-0008/#imports
Wildcard imports ( from import * ) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).
Specific is safer than globbing and I try to only import what I need if I can help it. When I'm learning a new module I'll import the whole thing and then once it's in a good state I go back and refactor by specifically importing the methods I need:
from module import method
method()

I would have to say that you should use the module's name. It's a better way of usage, and makes your code free of namespace confusions and also very understandable.
To make your code more beautiful, you could use the as import:
import random as r
# OR
from random import randint as rint
One solution that I think is not very beautiful, but works, that comes to mind, in case you don't want to pollute the global namespace, you can try to use the import statements, for example, inside functions.
For example:
>>> def a():
... from random import randint
... x = randint(0,2)
... print x
...
>>> a()
1
>>> randint(0,2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'randint' is not defined
This way, the local namespace of the specific function is filled with the values from the module but the global one is clean.

Is there a reason why when importing python files, you still need to name the file.function_name?

I am currently doing a python tutorial, but they use IDLE, and I opted to use the interpreter on terminal. So I had to find out how to import a module I created. At first I tried
import my_file
then I tried calling the function inside the module by itself, and it failed. I looked around and doing
my_file.function
works. I am very confused why this needs to be done if it was imported. Also, is there a way around it so that I can just call the function? Can anyone point me in the right direction. Thanks in advance.

If you wanted to use my_file.function by just calling function, try using the from keyword.
Instead of import my_file try from my_file import *.
You can also do this to only import parts of a module like so :
from my_file import function1, function2, class1
To avoid clashes in names, you can import things with a different name:
from my_file import function as awesomePythonFunction
EDIT:
Be careful with this, if you import two modules (myfile, myfile2) that both have the same function inside, function will will point to the function in whatever module you imported last. This could make interesting things happen if you are unaware of it.

This is a central concept to python. It uses namespaces (see the last line of import this). The idea is that with thousands of people writing many different modules, the likelihood of a name collision is reasonably high. For example, I write module foo which provides function baz and Joe Smith writes module bar which provides a function baz. My baz is not the same as Joe Smiths, so in order to differentiate the two, we put them in a namespace (foo and bar) so mine can be called by foo.baz() and Joe's can be called by bar.baz().
Of course, typing foo.baz() all the time gets annoying if you just want baz() and are sure that none of your other modules imported will provide any problems... That is why python provides the from foo import * syntax, or even from foo import baz to only import the function/object/constant baz (as others have already noted).
Note that things can get even more complex:
Assume you have a module foo which provides function bar and baz, below are a few ways to import and then call the functions contained inside foo...
import foo # >>> foo.bar();foo.baz()
import foo as bar # >>> bar.bar();bar.baz()
from foo import bar,baz # >>> bar(); baz()
from foo import * # >>> bar(); baz()
from foo import bar as cow # >>> cow() # This calls bar(), baz() is not available
...

A basic import statement is an assignment of the module object (everything's an object in Python) to the specified name. I mean this literally: you can use an import anywhere in your program you can assign a value to a variable, because they're the same thing. Behind the scenes, Python is calling a built-in function called __import__() to do the import, then returning the result and assigning it to the variable name you provided.
import foo
means "import module foo and assign it the name foo in my namespace. This is the same as:
foo = __import__("foo")
Similarly, you can do:
import foo as f
which means "import module foo and assign it the name f in my namespace." This is the same as:
f = __import__("foo")
Since in this case, you have only a reference to the module object, referring to things contained by the module requires attribute access: foo.bar etc.
You can also do from foo import bar. This creates a variable named bar in your namespace that points to the bar function in the foo module. It's syntactic sugar for:
bar = __import__("foo").bar

I don't really understand your confusion. You've imported the name my_file, not anything underneath it, so that's how you reference it.
If you want to import functions or classes inside a module directly, you can use:
from my_file import function

I'm going to incorporate many of the comments already posted.
To have access to function without having to refer to the module my_file, you can do one of the following:
from my_file import function
or
from my_file import *
For a more in-depth description of how modules work, I would refer to the documentation on python modules.
The first is the preferred solution, and the second is not recommended for many reasons:
It pollutes your namespace
It is not a good practice for maintainability (it becomes more difficult to find where specific names reside.
You typically don't know exactly what is imported
You can't use tools such as pyflakes to statically detect errors in your code
Python imports work differently than the #includes/imports in a static language like C or Java, in that python executes the statements in a module. Thus if two modules need to import a specific name (or *) out of each other, you can run into circular referencing problems, such as an ImportError when importing a specific name, or simply not getting the expected names defined (in the case you from ... import *). When you don't request specific names, you don't run into the, risk of having circular references, as long as the name is defined by the time you actually want to use it.
The from ... import * also doesn't guarantee you get everything. As stated in the documentation on python modules, a module can defined the __all__ name, and cause from ... import * statements to miss importing all of the subpackages, except those listed by __all__.

Why import when you need to use the full name?

In python, if you need a module from a different package you have to import it. Coming from a Java background, that makes sense.
import foo.bar
What doesn't make sense though, is why do I need to use the full name whenever I want to use bar? If I wanted to use the full name, why do I need to import? Doesn't using the full name immediately describe which module I'm addressing?
It just seems a little redundant to have from foo import bar when that's what import foo.bar should be doing. Also a little vague why I had to import when I was going to use the full name.

The thing is, even though Python's import statement is designed to look similar to Java's, they do completely different things under the hood. As you know, in Java an import statement is really little more than a hint to the compiler. It basically sets up an alias for a fully qualified class name. For example, when you write
import java.util.Set;
it tells the compiler that throughout that file, when you write Set, you mean java.util.Set. And if you write s.add(o) where s is an object of type Set, the compiler (or rather, linker) goes out and finds the add method in Set.class and puts in a reference to it.
But in Python,
import util.set
(that is a made-up module, by the way) does something completely different. See, in Python, packages and modules are not just names, they're actual objects, and when you write util.set in your code, that instructs Python to access an object named util and look for an attribute on it named set. The job of Python's import statement is to create that object and attribute. The way it works is that the interpreter looks for a file named util/__init__.py, uses the code in it to define properties of an object, and binds that object to the name util. Similarly, the code in util/set.py is used to initialize an object which is bound to util.set. There's a function called __import__ which takes care of all of this, and in fact the statement import util.set is basically equivalent to
util = __import__('util.set')
The point is, when you import a Python module, what you get is an object corresponding to the top-level package, util. In order to get access to util.set you need to go through that, and that's why it seems like you need to use fully qualified names in Python.
There are ways to get around this, of course. Since all these things are objects, one simple approach is to just bind util.set to a simpler name, i.e. after the import statement, you can have
set = util.set
and from that point on you can just use set where you otherwise would have written util.set. (Of course this obscures the built-in set class, so I don't recommend actually using the name set.) Or, as mentioned in at least one other answer, you could write
from util import set
or
import util.set as set
This still imports the package util with the module set in it, but instead of creating a variable util in the current scope, it creates a variable set that refers to util.set. Behind the scenes, this works kind of like
_util = __import__('util', fromlist='set')
set = _util.set
del _util
in the former case, or
_util = __import__('util.set')
set = _util.set
del _util
in the latter (although both ways do essentially the same thing). This form is semantically more like what Java's import statement does: it defines an alias (set) to something that would ordinarily only be accessible by a fully qualified name (util.set).

You can shorten it, if you would like:
import foo.bar as whateveriwant
Using the full name prevents two packages with the same-named submodules from clobbering each other.

There is a module in the standard library called io:
In [84]: import io
In [85]: io
Out[85]: <module 'io' from '/usr/lib/python2.6/io.pyc'>
There is also a module in scipy called io:
In [95]: import scipy.io
In [96]: scipy.io
Out[96]: <module 'scipy.io' from '/usr/lib/python2.6/dist-packages/scipy/io/__init__.pyc'>
If you wanted to use both modules in the same script, then namespaces are a convenient way to distinguish the two.
In [97]: import this
The Zen of Python, by Tim Peters
...
Namespaces are one honking great idea -- let's do more of those!

in Python, importing doesn't just indicate you might use something. The import actually executes code at the module level. You can think of the import as being the moment where the functions are 'interpreted' and created. Any code that is in the _____init_____.py level or not inside a function or class definition happens then.
The import also makes an inexpensive copy of the whole module's namespace and puts it inside the namespace of the file / module / whatever where it is imported. An IDE then has a list of the functions you might be starting to type for command completion.

Part of the Python philosophy is explicit is better than implicit. Python could automatically import the first time you try to access something from a package, but that's not explicit.
I'm also guessing that package initialization would be much more difficult if the imports were automatic, as it wouldn't be done consistently in the code.

You're a bit confused about how Python imports work. (I was too when I first started.) In Python, you can't simply refer to something within a module by the full name, unlike in Java; you HAVE to import the module first, regardless of how you plan on referring to the imported item. Try typing math.sqrt(5) in the interpreter without importing math or math.sqrt first and see what happens.
Anyway... the reason import foo.bar has you required to use foo.bar instead of just bar is to prevent accidental namespace conflicts. For example, what if you do import foo.bar, and then import baz.bar?
You could, of course, choose to do import foo.bar as bar (i.e. aliasing), but if you're doing that you may as well just use from foo import bar. (EDIT: except when you want to import methods and variables. Then you have to use the from ... import ... syntax. This includes instances where you want to import a method or variable without aliasing, i.e. you can't simply do import foo.bar if bar is a method or variable.)

Other than in Java, in Python import foo.bar declares, that you are going to use the thing referred to by foo.bar.
This matches with Python's philosophy that explicit is better than implicit. There are more programming languages that make inter-module dependencies more explicit than Java, for example Ada.
Using the full name makes it possible to disambiguate definitions with the same name coming from different modules.

You don't have to use the full name. Try one of these
from foo import bar
import foo.bar as bar
import foo.bar
bar = foo.bar
from foo import *
A few reasons why explicit imports are good:
They help signal to humans and tools what packages your module depends on.
They avoid the overhead of dynamically determining which packages have to be loaded (and possibly compiled) at run time.
They (along with sys.path) unambiguously distinguish symbols with conflicting names from different namespaces.
They give the programmer some control of what enters the namespace within which he is working.