Develop Reference

variable for class defined in __main__ with access in different module

I know I'm being dense and I know what the problem is but I can't seem to figure out how to fix it.
testcode.py
import t2
class t1:
saved = 0
#classmethod
def set_class(cls, number):
t1.saved = number
not_in_main = t1
if __name__ == "__main__":
not_in_main.set_class(6)
used_everywhere = t1
used_everywhere.set_class(4)
t2.f1()
and then in t2.py
import testcode
def f1():
print(f'not_in_main: {testcode.not_in_main.saved}')
print(f'got here: {testcode.used_everywhere.saved}')
Now the output after running python testcode.py will be
not_in_main: 0
AttributeError: module 'testcode' has no attribute 'used_everywhere'
So not_in_main has two different instances one in the main scope and one module scope. Used_everywhere is also in the main scope but another module can't see in instance.
I've looked at many of the scoping issues but I don't see how to fix this. What am I doing wrong?

The fix for this circular issue was to move the class def and creation of the module variable to a file other than where main is. The project now just has main in the a module and most all other code in a separate file.
Breakpoints on the not_in_main show it only being instantiated once as opposed to twice as in the listed code.
When necessary, as in generated code for the UI, that is in a separate module, the import of the module where not_in_main is declared does not cause a circular reference and the fields are properly filled out.
Thanks to #junpa.arrivillaga for the pointers and hints to put me on the right path.

Python global variable in import * [duplicate]

I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.

Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()

As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.

This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'

A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.

The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()

Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())

Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12

The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.

Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.

Misunderstanding differences between inside-class and outside-class imports in Python [duplicate]

This question already has answers here:
Short description of the scoping rules?
(9 answers)
Closed 1 year ago.
Context: I'm writing a translator from one Python API to another, both in Python 3.5+. I load the file to be translated with a class named FileLoader, described by Fileloader.py. This file loader allows me to transfer the file's content to other classes doing the translation job.
All of the .py files describing each class are in the same folder
I tried two different ways to import my FileLoader module inside the other modules containing the classes doing the translation job. One seems to work, but the other didn't and I don't understand why.
Here are two code examples illustrating both ways:
The working way
import FileLoader
class Parser:
#
def __init__(self, fileLoader):
if isinstance(fileLoader, FileLoader.FileLoader)
self._fileLoader = fileLoader
else:
# raise a nice exception
The crashing way
class Parser:
import FileLoader
#
def __init__(self, fileLoader):
if isinstance(fileLoader, FileLoader.FileLoader)
self._fileLoader = fileLoader
else:
# raise a nice exception
I thought doing the import inside the class's scope (where it's the only scope FileLoader is used) would be enough, since it would know how to relate to the FileLoader module and its content. I'm obviously wrong since it's the first way which worked.
What am I missing about scopes in Python? Or is it about something different?

2 things : this won't work. And there is no benefit to doing it this way.
First, why not?
class Parser:
#this assigns to the Parser namespace, to refer to it
#within a method you need to use `self.FileLoader` or
#Parser.FileLoader
import FileLoader
#`FileLoader` works fine here, under the Parser indentation
#(in its namespace, but outside of method)
copy_of_FileLoader = FileLoader
#
def __init__(self, fileLoader):
# you need to refer to modules under in Parser namespace
# with that `self`, just like you would with any other
# class or instance variable 👇
if isinstance(fileLoader, self.FileLoader.FileLoader)
self._fileLoader = fileLoader
else:
# raise a nice exception
#works here again, since we are outside of method,
#in `Parser` scope/indent.
copy2_of_FileLoader = FileLoader
Second it's not Pythonic and it doesn't help
Customary for the Python community would be to put import FileLoader at the top of the program. Since it seems to be one of your own modules, it would go after std library imports and after third party module imports. You would not put it under a class declaration.
Unless... you had a good (probably bad actually reason to).
My own code, and this doesn't reflect all that well on me, sometimes has stuff like.
class MainManager(batchhelper.BatchManager):
....
def _load(self, *args, **kwargs):
👉 from pssystem.models import NotificationConfig
So, after stating this wasn't a good thing, why am I doing this?
Well, there are some specific circumstances to my code going here. This is a batch, command-line, script, usable within a Django context and it uses some Django ORM models. In order for those to be used, Django needs to be imported first and then setup. But that often happens too early in the context of these types of batch programs and I get circular import errors, with Django complaining that it hasn't initialized yet.
The solution? Defer execution until the method is called, when all the other modules have been imported and Django has been setup elsewhere.
NotificationConfig is now available, but only within that method as it is a local variable in it. It works, but... it's really not great practice.
Remember: anything in the global scope gets executed at module load time, anything under classes at module load time, anything withing method/function bodies when the method/function is called.
#happens at module load time, you could have circular import errors
import X1
class DoImportsLater:
.
#happens at module load time, you could have circular import errors
import X2
def _load(self, *args, **kwargs):
#only happens when this method is called, if ever
#so you shouldn't be seeing circular imports
import X3
import X1 is std practice, Pythonic.
import X2, what are doing, is not and doesn't help
import X3, what I did, is a hack and is covering up circular import references. But it "fixes" the issue.

Python would not let me use methods inside the class while the class definition has not been finished

I am using Python 2.6 as a batch script replacement. It will be started by double-clicking, so all output to stdout will be lost/ignored by the user. So, I decided to add logging, and to make matters simple, I wrote a class for that. The idea is that I can use Logging.Logger anywhere in my code, and the logger will be ready to go.
I wish to have no more than 10 old log files in a directory, so I clear out the old ones manually. I have not found a functionality like that through API, plus I am paranoid, and want to log everything, event the fact that there were files with unexpected names inside the log directory.
So, below is my attempt at such class, but I get an error when I try to test (run) it:
>>> ================================ RESTART ================================
>>>
Traceback (most recent call last):
File "C:/AutomationScripts/build scripts/Deleteme.py", line 6, in <module>
class Logging:
File "C:/AutomationScripts/build scripts/Deleteme.py", line 42, in Logging
__clearOldLogs(dummySetting)
File "C:/AutomationScripts/build scripts/Deleteme.py", line 38, in __clearOldLogs
_assert(Logger, 'Logger does not exist yet! Why?')
NameError: global name '_assert' is not defined
>>>
Yes, I come from Java/C# background. I am probably not doing things "the Pythonic" way. Please help me do the right thing, and please give a complete answer that would work, instead of simply pointing out holes in my knowledge. I believe I have provided enough of a code sample. Sorry, it would not run without a Settings class, but hopefully you get the idea.
# This file has been tested on Python 2.6.*. For Windows only.
import logging # For actual logging
import tkMessageBox # To display an error (if logging is unavailable)
class Logging:
"""
Logging class provides simplified interface to logging
as well as provides some handy functions.
"""
# To be set when the logger is properly configured.
Logger = None
#staticmethod
def _assert(condition, message):
""" Like a regular assert, except that it should be visible to the user. """
if condition: return
# Else log and fail
if Logger:
Logger.debug(traceback.format_stack())
Logger.error('Assert failed: ' + message)
else:
tkMessageBox.showinfo('Assert failed: ' + message, traceback.format_stack())
assert(condition)
#staticmethod
def _removeFromEnd(string, endString):
_assert(string.endswith(endString),
"String '{0}' does not end in '{1}'.".format(string, endString))
return string[:-len(endString)]
def __clearOldLogs(logSettings):
"""
We want to clear old (by date) files if we get too many.
We should call this method only after variable 'Logger' has been created.
"""
# The following check should not be necessary from outside of
# Logging class, when it is fully defined
_assert(Logger, 'Logger does not exist yet! Why?')
# Do more work here
def __createLogger(logSettings):
logFileName = logSettings.LogFileNameFunc()
#_assert(False, 'Test')
logName = _removeFromEnd(logFileName, logSettings.FileExtension)
logFileName = os.path.join(logSettings.BaseDir, logFileName)
# If someone tried to log something before basicConfig is called,
# Python creates a default handler that goes to the console and will
# ignore further basicConfig calls. Remove the handler if there is one.
root = logging.getLogger()
if root.handlers:
for handler in root.handlers:
root.removeHandler(handler)
logging.basicConfig(filename = logFileName, name = logName, level = logging.DEBUG, format = "%(asctime)s - %(levelname)s - %(message)s")
logger = logging.getLogger(logName)
return logger
# Settings is a separate class (not dependent on this one).
Logger = __createLogger(Settings.LOGGING)
__clearOldLogs(Settings.LOGGING)
if __name__ == '__main__':
# This code section will not run when the class is imported.
# If it is run directly, then we will print debugging info.
logger = Logging.Logger
logger.debug('Test debug message.')
logger.info('Test info message.')
logger.warning('Test warning message.')
logger.error('Test error message.')
logger.critical('Test critical message.')
Relevant questions, style suggestions and complete answers are welcome. Thanks!

You're getting that exception because you're calling _assert() rather than Logging._assert(). The error message tells you it's looking for _assert() in the module's global namespace rather than in the class namespace; to get it to look in the latter, you have to explicitly specify it.
Of course, in this case, you're trying to do it while the class is still being defined and the name is not available until the class is complete, so it's going to be tough to make that work.
A solution would be to un-indent the following two lines (which I've edited to use the fully qualified name) so that they are outside the class definition; they will be executed right after it.
Logger = Logging.__createLogger(Settings.LOGGING)
Logging.__clearOldLogs(Settings.LOGGING)
A style suggestion that will help: rather than making a class with a bunch of static methods, consider making them top-level functions in the module. Users of your module (including yourself) will find it easier to get to the functionality they want. There's no reason to use a class just to be a container; the module itself is already just such a container.
A module is basically a single *.py file (though you can create modules that have multiple files, this will do for now). When you do an import, what you're importing is a module. In your example, tkMessageBox and logging are both modules. So just make a separate file (making sure its name does not conflict with existing Python module names), save it in the same directory as your main script, and import it in your main script. If you named it mylogging.py then you would import mylogging and access functions in it as mylogging.clearOldLogs() or whatever (similar to how you'd address them now as a class).
"Global" names in Python are not truly global, they're only global to the module they're defined in. So a module is a good way to compartmentalize your functionality, especially parts (like logging) that you anticipate reusing in a lot of your future scripts.

Replace the line
_assert(Logger, 'Logger does not exist yet! Why?')
with
Logging._assert(Logger, 'Logger does not exist yet! Why?')
This is because you define _assert as a static method of the class, and static method has to be referred to as ClassName.methodName even if you are calling it from a method for an instance of this class.

Some comments.
Do not use double underscores unless you are absolutely sure that you have to and why.
To access the _assert method, you call it with self. Like so: self._assert(Logger, 'Logger does not exist yet! Why?') In a static method, as in your example, you use the class name: Logger._assert(). Python is very explicit.
Classes are only created at the END of the class definition. That's just how it is with Python. But your error is not related to that.
I'm not sure what this code is supposed to do:
# Settings is a separate class (not dependent on this one).
Logger = __createLogger(Settings.LOGGING)
__clearOldLogs(Settings.LOGGING)
But I suspect you should put it in the __init__ method. I do not immediately see any code that needs access to the class during class construction.
__createLogger is a strange function. Isn't the class called Logger? Shouldn't that just be the __init__ of the Logger class?

I think it just needs to be Logging._assert. Python doesn't do namespace resolution the way java does: an unqualified name is either method-local or global. Enclosing scopes like classes won't be searched automatically.
The chapter and verse on how Python deals with names is in section 4.1. Naming and binding in the language reference manual. The bit that's relevant here is probably:
The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods
Contrast this with names defined in functions, which are inherited downwards (into method-local functions and classes):
If the definition occurs in a function block, the scope extends to any blocks contained within the defining one, unless a contained block introduces a different binding for the name

How can I inject an object into another namespace in python?

I'm writing some unittests for code written by someone else here at the office. Python is not my strongest language. While I've been successful with basic unit tests, mocking in python is throwing me for a loop.
What I need to do is override a call to ConfigObj and inject my own mock config/fixture into any ConfigObj call.
settings.py
from configobj import ConfigObj
config = ConfigObj('/etc/myapp/config')
utils.py
from settings import config
"""lots of stuff methods using various config values."""
What I would like to do is, in my unittests for utils.py, inject myself either for ANY call to ConfigObj or settings.py itself.
Many of the mocking libraries expect me to Mock my own classes but in the case of this app, it doesn't have any explicit classes.
Can it be done or are the python namespace restrictions too strict that I can't intervene in what a module that I'm importing imports itself?
Side note: running 2.7 so I can't do any of the tricks I've read about from 2.5.

If the tests are in a separate file from from settings.py and utils.py you can create a file mock.py
import configobj
class MockConfigObj(object):
#mock whatever you wan
configobj.ConfigObj = MockConfigObj
and then import mock before importing (from) any module that itself imports settings. This will ensure that settings.config is created with MockConfigObj. If you want a uniform global mocking, import it before any file that imports configobj.
This works because python will store configobj in sys.modules and check that before actually reading from a file on subsequent imports. in mock.py, the identifier ConfigObj is just a reference to the entry in sys.modules so that any changes that you make will be globally visible.
This strikes me as a little hacky though but it's the best that I can think of.

Python namespaces are not strict at all within the same scope. Just override the variable name containing your object (or the class itself and provided it) within the same scope you'd be expecting the original and that is good enough.
Now, whether or not what you're replacing it with behaves the same is up to you...

Couldn't you just overwrite the original function with another one?
There are no constants in Python, you can change everything, you could even do True = False.

I faced a similar situation before. Here is how I would go about addressing your problem.
Consider a test case for a function from utils.py.
import utils, unittest
class FooFunctionTests(unittest.TestCase):
def setUp(self):
utils._old_config = utils.config
utils.config = MockClass()
def tearDown(self):
utils.config = utils._old_config
del utils._old_config
def test_foo_function_returns_correct_value(self):
self.assertEqual("success!", utils.foo())

The following page is a good one on mocking and import
http://www.relaxdiego.com/2014/04/mocking-objects-in-python.html
Say you have a file named my_package1.py with the following code:
class A(object):
def init(self):
and you then import that in my_package2.py with the code
from my_package1 import A
class A(object):
def init(self):
The first line of my_package2.py creates a variable under the my_package2 namespace called A. Now you have two variables my_package1.A and my_package2.A that both point to the same class in memory. If you want the code in my_package2.py to use a mocked up class A, then you will need to mock my_package2.A not my_package1.A