I want to know how to make variables from functions in imported modules available in the IPython interactive namespace.
I have some example code which I want to run from another script so I set it up as run_test.py:
def run_test():
a = 5
if __name__ == "__main__":
run_test()
I import the module as follows and call the function:
import run_test
run_test.run_test()
How do I make variable 'a' available in the interactive namespace? The only way I can do it is to make 'a' a global and run run_test.py directly rather than importing it and calling the function.
Any pointers appreciated.
I believe this can be accomplished by returning locals and then updating locals.
def main():
# do things
return locals()
if __name__ == '__main__':
new_locals = main()
locals().update(new_locals)
This seems to work, though it feels like a hack, so perhaps it won't always.
A reasonable example of where you want this: If you want access to all the variables in a function's namespace, but don't want that function's variables accessible to other parts of a script (i.e., other function definitions).
Related
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
I am using a test framework that loads a module and then searches the scope of that module for #test() decorated function. What I am trying to do is to dynamically create those tests by dynamically creating those tagged variables.
It looks like this:
for i in range(100):
globals()[f"Test_{i}"] = test()(Test)
For convenience I wrapped this up into reusable function and stuck it in another module:
def add_test(t, n=1):
for i in range(n):
globals()[f"{t.__name__}_{i}"] = test()(t)
However, when I import the add_test() function in a test module, and call it, no new variables are added to the test modules's global scope. If I move this function into the module where it is used, it works. It seems like globals() binds to the module scope rather than a true global scope. Is that analysis correct? How do I work around this?
I would like to use a variable that I define in my application inside of a module.
The folder structure:
myapp.py
modules/checkargs.py
modules/init.py (an empty file)
Main app (myapp.py):
_PARAMETERS = {
'stuff': 'here'
}
from modules.checkargs import checkargs
if __name__ == "__main__":
checkargs(sys.argv[1:])
checkargs.py:
def checkargs(argv):
global _PARAMETERS;
#more Python insanity here
The error:
NameError: global name '_PARAMETERS' is not defined
In general, you should avoid this style of programming. Modules shouldn't rely on global variables defined in other modules. A better solution would be to pass _PARAMETERS in to checkargs, or move _PARAMETERS to a file that can be shared by multiple modules.
Passing the data to checkargs
Generally speaking, relying on global variables is a bad idea. Perhaps the best solution is to pass PARAMETERS directly into your checkargs function.
# checkargs.py
def checkargs(argv, parameters):
...
# myapp.py
if __name__ == "__main__":
checkargs(sys.argv[1:], _PARAMETERS)
Creating a shared data module
If you have data that you want to share between modules, you can place that data in a third module that every other module imports:
# modules/shared.py
PARAMETERS = {...}
# myapp.py
from modules.shared import PARAMETERS
# checkargs.py
from modules.shared import PARAMETERS
Other solutions
There are other solutions, though I think the above two solutions are best. For example, your main program can copy the parameters to the checkargs module like this:
# myapp.py
import checkargs
checkargs._PARAMETERS = _PARAMETERS
...
You could also have checkargs directly reference the value in your main module, but that requires a circular import. Circular imports should be avoided.
Why would it be defined? It's from a different module. Inside checkargs.py, you'd need to do:
from myapp import _PARAMETERS
However:
You shouldn't name it with a _ then, since that implies private/protected variables.
You should probably pass the dictionary from myapp into the checkargs functions instead of importing it there. If you don't, you're creating a circular import, which is logically terrible and doesn't actually work.
$ cat declare_funcs.py
#!/usr/bin/python3
def declared_after():
print("good declared after")
declared_after()
$ python3 declare_funcs.py
good declared after
Change call place:
$ cat declare_funcs.py
#!/usr/bin/python3
declared_after()
def declared_after():
print("good declared after")
$ python3 declare_funcs.py
Traceback (most recent call last):
File "declare_funcs.py", line 4, in <module>
declared_after()
NameError: name 'declared_after' is not defined
Is there way to declare only header of function like it was in C/C++?
For example:
#!/usr/bin/python3
def declared_after() # declaration about defined function
declared_after()
def declared_after():
print("good declared after")
I found this Declare function at end of file in Python
Any way there appear another function in the beginning like wrapper, and this wrapper must be called after declaration of wrapped function, this is not an exit. Is there more elegant true-python way?
You can't forward-declare functions in Python. It doesn't make a lot of sense to do so, because Python is dynamically typed. You could do something silly like this, and what would expect it to do?
foo = 3
foo()
def foo():
print "bar"
Obviously, you are trying to __call__ the int object for 3. It's absolutely silly.
You ask if you can forward-declare like in C/C++. Well, you typically don't run C through an interpreter. However, although Python is compiled to bytecode, the python3 program is an interpreter.
Forward declaration in a compiled language makes sense because you are simply establishing a symbol and its type, and the compiler can run through the code several times to make sense of it. When you use an interpreter, however, you typically can't have that luxury, because you would have to run through the rest of the code to find the meaning of that forward declaration, and run through it again after having done that.
You can, of course, do something like this:
foo = lambda: None
foo()
def foo():
print "bar"
But you instantiated foo nonetheless. Everything has to point to an actual, existing object in Python.
This doesn't apply to def or class statements, though. These create a function or class object, but they don't execute the code inside yet. So, you have time to instantiate things inside them before their code runs.
def foo():
print bar()
# calling foo() won't work yet because you haven't defined bar()
def bar():
return "bar"
# now it will work
The difference was that you simply created function objects with the variable names foo and bar representing them respectively. You can now refer to these objects by those variable names.
With regard to the way that Python is typically interpreted (in CPython) you should make sure that you execute no code in your modules unless they are being run as the main program or unless you want them to do something when being imported (a rare, but valid case). You should do the following:
Put code meant to be executed into function and class definitions.
Unless the code only makes sense to be executed in the main program, put it in another module.
Use if __name__ == "__main__": to create a block of code which will only execute if the program is the main program.
In fact, you should do the third in all of your modules. You can simply write this at the bottom of every file which you don't want to be run as a main program:
if __name__ = "__main__":
pass
This prevents anything from happening if the module is imported.
Python doesn't work that way. The def is executed in sequence, top-to-bottom, with the remainder of the file's contents. You cannot call something before it is defined as a callable (e.g. a function), and even if you had a stand-in callable, it would not contain the code you are looking for.
This, of course, doesn't mean the code isn't compiled before execution begins—in fact, it is. But it is when the def is executed that declared_after is actually assigned the code within the def block, and not before.
Any tricks you pull to sort-of achieve your desired effect must have the effect of delaying the call to declared_after() until after it is defined, for example, by enclosing it in another def block that is itself called later.
One thing you can do is enclose everything in a main function:
def main():
declared_after()
def declared_after():
print("good declared after")
main()
However, the point still stands that the function must be defined prior to calling. This only works because main is called AFTER declared_after is defined.
As zigg wrote, Python files are executed in order they are written from top to bottom, so even if you could “declare” the variable before, the actual function body would only get there after the function was called.
The usual way to solve this is to just have a main function where all your standard execution stuff happens:
def main ():
# do stuff
declared_after();
def declared_after():
pass
main()
You can then also combine this with the __name__ == '__main__' idiom to make the function only execute when you are executing the module directly:
def main ():
# do stuff
declared_after();
def declared_after():
pass
if __name__ == '__main__':
main()
first.py
myGlobal = "hello"
def changeGlobal():
myGlobal="bye"
second.py
from first import *
changeGlobal()
print myGlobal
The output I get is
hello
although I thought it should be
bye
Why doesn't the global variable myGlobal changes after the call to the changeGlobal() function?
Try:
def changeGlobal():
global myGlobal
myGlobal = "bye"
Actually, that doesn't work either. When you import *, you create a new local module global myGlobal that is immune to the change you intend (as long as you're not mutating the variable, see below). You can use this instead:
import nice
nice.changeGlobal()
print nice.myGlobal
Or:
myGlobal = "hello"
def changeGlobal():
global myGlobal
myGlobal="bye"
changeGlobal()
However, if your global is a mutable container, you're now holding a reference to a mutable and are able to see changes done to it:
myGlobal = ["hello"]
def changeGlobal():
myGlobal[0] = "bye"
I had once the same concern as yours and reading the following section from Norman Matloff's Quick and Painless Python Tutorial was really a good help. Here is what you need to understand (copied from Matloff's book):
Python does not truly allow global variables in the sense that C/C++ do. An imported Python module will not have direct access to the globals in the module which imports it, nor vice versa.
For instance, consider these two files, x.py,
# x.py
import y
def f():
global x
x = 6
def main():
global x
x = 3
f()
y.g()
if __name__ == ’__main__’:
main()
and y.py:
# y.py
def g():
global x
x += 1
The variable x in x.py is visible throughout the module x.py, but not in y.py. In fact, execution of the line
x += 1
in the latter will cause an error message to appear, “global name ’x’ is not defined.”
Indeed, a global variable in a module is merely an attribute (i.e. a member entity) of that module, similar to a class variable’s role within a class. When module B is imported by module A, B’s namespace is copied to A’s. If module B has a global variable X, then module A will create a variable of that name, whose initial value is whatever module B had for its variable of that name at the time of importing. But changes to X in one of the modules will NOT be reflected in the other.
Say X does change in B, but we want code in A to be able to get the latest value of X in B. We can do that by including a function, say named GetX() in B. Assuming that A imported everything from B, then A will get a function GetX() which is a copy of B’s function of that name, and whose sole purpose is to return the value of X. Unless B changes that function (which is possible, e.g. functions may be assigned), the functions in the two modules will always be the same, and thus A can use its function to get the value of X in B.
Python global variables are not global
As wassimans points out above they are essentially attributes within the scope of the module they are defined in (or the module that contains the function that defined them).
The first confusion(bug) people run into is not realizing that functions have a local name space and that setting a variable in a function makes it a local to the function even when they intended for it to change a (global) variable of the same name in the enclosing module. (declaring the name
in a 'global' statement in the function, or accessing the (global) variable before setting it.)
The second confusion(bug) people run into is that each module (ie imported file) contains its own so called 'global' name space. I guess python things the world(globe) is the module -- perhaps we are looking for 'universal' variables that span more than one globe.
The third confusion (that I'm starting to understand now) is where are the 'globals' in the __main__ module? Ie if you start python from the command line in interactive mode, or if you invoke python script (type the name of the foo.py from the command shell) -- there is no import of a module whose name you can use.
The contents of 'globals()' or globals().keys() -- which gives you a list of the globals -- seems to be accessible as: dir(sys.modules['__main__'])
It seems that the module for the loaded python script (or the interactive session with no loaded script), the one named in: __name__, has no global name, but is accessible as the module whose name is '__main__' in the system's list of all active modules, sys.modules