Develop Reference

Using global to create a singleton object resulting in multiple initialization

I am facing a problem where the global variable within my module keeps on getting initialized multiple times when produce() is called. The module is imported just once but I am still unable to understand why the global variable within the module will become None.
Am I taking the right approach?
Looking to make it work in python 2.7 and upwards.
producer_instance = None
def produce(message):
producer = get_producer()
producer.produce(message)
def get_producer():
global producer_instance
if producer_instance is None:
producer_instance = Producer()
return producer_instance

Change same file-scope variable from several Python files

It seems that if I import the same Python file via the import statements from several files and change some file-scope variable in it, the reference change will not be visible to another modules while container changes will.
For example,
First example
first.py
import reader
import helper
reader.change_buf('modified')
helper.foo()
second.py
import reader
def foo():
print(reader.BUF)
reader.py
buf = 'original'
def change_buf(buf):
buf = buf
Output
> python first.py
original
Second example
first.py
import reader
import helper
reader.change_first_element('1')
helper.foo()
second.py
import reader
def foo():
print(reader.BUF)
reader.py
buf = ['0', '1', '2']
def change_first_element(new_elem):
buf[0] = new_elem
Output
> python first.py
['1', '1', '2']
Why?

Simply because in first example you are hiding the global variable buf behind a local one (the function parameter buf), while in second example the function parameter(new_elem) does not collide. But that's not all. In first example, you define the variable buf in a function, so it is still local by default - in second example, you use it since you set buf[0]. You must declare it global
You should write reader.py that way:
buf = 'original'
def change_buf(new_val):
global buf
buf = new_val

tl;dr Because you're doing different things with different behaviours.
In your first example, you create a local variable buf in the function change_buf; in the second example, there is no such variable to access locally, so python descends the stack upwards looking for variables that fit your request, and finds one at the module level scope.
Your first example uses an assignment statement, your second statement is fancy setitem notation. The behaviour is different for the two.
Explaining the behaviour in (1), This question is similar. If you want to access a global variable explicitly using an assignment statement, use the global or nonlocal keyword, as is done here
From the docs:
It would be impossible to assign to a global variable without global, although free variables may refer to globals without being declared global.
Note that your second example isn't an assignment statement; it is a method call to __setitem__ on the variable buf, which happens to be type list. It might make more sense looking like this
foo = [3]
def set_foo():
foo.__setitem__(0,3)
Now you see that foo is a "free variable" being referenced, and can only be referencing the global scope element by that name. If there was any ambiguity as to which scope foo belonged to, python would not be able to handle your case.
Python's rules about scoping can be a little complex, but are unambiguous

Not able to access object in python while creating object of other class defined in some other file

I have a directory structure like this:
home/
main.py
lib/
mylib/
Textfile_Class.py
Excelfile_Class.py
globals.py (has all the global variables declared here)
functions.py
I created an object of Textfile_class in main.py using
txt = TEXT(file).
Now, I want to use this variable txt during creating object of Excelfile_Class for some operations (eg, if the value of a variable in txt object is 5, then do certain action in Excelfile_Class )
In Excelfile_Class, I am also importing all the global variables. String variables are accessible there but I dont know why this object txt is not accessible there. Wherever I am refering to txt in Excelfile_Class(self.line_count = txt.count), I am getting below error: AttributeError: 'NoneType' object has no attribute 'count'
Please help me to know why this is happening even though I have defined all the variables in a seperate file and importing those in all the files.
Eg:
main.py
path = os.path.abspath('./lib')
sys.path.insert(0, path)
from mylib.Textfile_class import *
from mylib.Excelfile_Class import *
from mylib.globals import *
from mylib.functions import *
if __name__ == "__main__":
txt = TEXT(file)
xcel = EXCEL(file)
Eg globals.py
global txt, xcel
txt=None
xcel=None
Eg Textfile_class.py
from globals import *
class TEXT:
def __init__(self, filename):
self.count = 0
with open(filename) as fp:
for line in fp:
self.count = self.count + 1`
Eg Excelfile_Class.py
from globals import *
class EXCEL:
def __init__(self, filename):
self.line_count = 0
self.operation(filename)
def operation(self, file):
self.line_count = txt.count
if self.line_count:
self.some_operation()
else:
self.other_operation()

When you assign a value to a variable name inside a function, you're not working with the global version of the variable any more, instead you have a completely new variable.
You have to use global keyword inside the function to indicate you're working with a global variable.
From Python Doc FAQ
What are the rules for local and global variables in Python?
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a new value anywhere within the function’s body, it’s assumed to be a local. If a variable is ever assigned a new value inside the function, the variable is implicitly local, and you need to explicitly declare it as ‘global’.
Read more...
Example:
x = None # x is momdule-level global variable
def local():
x = 5 # assign 5 to x, here x is a new variable, global x is not affected.
def glob():
global x # indicate we'll working with the global x.
x = 5 # this affect the global variable.
def print_global():
print(x) # Just print the global variable.
local()
print_global() # Prints None
glob()
print_global() # Prints 5
So, every time you refer to txt inside a function, you have to tell the context you'll be working with the global version of txt.
Other thing can be happening!
Python is interpreted, that means it execute the code line by line, if in the others modules (not in the main module) you have code trying to access txt before some value be assigned
if __name__ == "__main__":
txt = TEXT(file)
, then you'll get the same error.
A recommendation:
Try to avoid the use of global variables, you already know that isn't a good practice and it leads to unestable code.
If your problem is that you want to txt and xcel to be available at any time anywhere, you could use the pattern Singleton (warning, Singleton is considered an anti-pattern ref)
I will post an example for you, but before I will encorage you to redesign your program, I ensure you it will be a good exercise!
Singleton example: (again this is an anti-pattern, but I preffer it to nude global variables).
class Globals(object):
__instance = None
def __init__(self):
class wrapped_class:
def __init__(self):
self.txt = None
self.excel = None
if (Globals.__instance is None):
Globals.__instance = wrapped_class()
def __getattr__(self, attrname):
return getattr(self.__instance, attrname)
def __setattr__(self, attrname, value):
setattr(self.__instance, attrname, value)
glob = Globals() # Since __instance is None this will instantiate wrapped_class and save the reference in __instance
glob.txt = "Txt example" # Modify the txt attribute of __instance.
glob_1 = Globals() # Since __instance is not None it stays as it is.
print(glob.txt) # Prints "Txt example"
print(glob_1.txt) # Prints "Txt example"

It is quite hard to tell where your problem is without having the code or the stack trace but I would strongly advice that you read some documentation about Python best practices, the use of global variables and naming conventions. This by itself might solve your issues.
Naming conventions might sound silly but those things and other syntax-related choices do matter in python.
Also, you seem to be missing __init__.py file in your module which may or may not be important depending of the python version you are using.
Here are a few links to get you started:
https://www.python.org/dev/peps/pep-0008/#naming-conventions
https://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references
http://www.python-course.eu/global_vs_local_variables.php
http://gettingstartedwithpython.blogspot.be/2012/05/variable-scope.html
http://c2.com/cgi/wiki?GlobalVariablesAreBad

I dont know why it happened like that. If anyone knows they can tell.
The problem got resolved when I used the method given in Using global variables between files in Python.
So, finally I put all the global variables in a function in globals.py and instantiated that once in main.py. Then, I used from mylib import globals and referenced the global variables inside the classes as globals.txt and it worked fine.

Multiprocessing and a global manager

I have
import concurrent.futures
from multiprocessing import Process,Manager,freeze_support
from functools import partial
def setup():
freeze_support()
global manager
manager=Manager()
global dict1
dict1=manager.dict()
global dict2
dict2=manager.dict()
load_stuff()
and later:
def foo(file):
#do some stuff
foobar()
def foobar():
#look up some stuff in dict1,dict2
def load_stuff():
f=partial(foo,dict1,dict2)
with concurrent.futures.ProcessPoolExecutor() as executor:
for users, tweets in executor.map(f, list_of_files):
But I keep getting
NameError: global name 'dict1' is not defined
Basically: I'm spawning multiple processes, who while doing some work call a function that looks up values in two dicts. The dicts are global because I need them when I may just call foo() and foobar() without being in a new process. The processes are unable to access the dicts. How can I fix this?
Edit: I ended up making dict1 a normal global dictionary and then passing it as a variable to f, then redeclaring it as global inside f and it works. Not memory efficient, but this process only runs once and the dicts use only around 45mb.

Since default parameters values are bound at compile time, this should work:
def foobar(dict1=dict1, dict2=dict2):
pass
Wait, so load_stuff needs to have been defined before setup and setup needs to have been defined before foobar, but it seems foobar and load_stuff are next to each other.
It's not very clear to me how your functions are laid out. Perhaps you need to have an additional declaration of dict1 and dict2 as module-level variables somewhere (while keeping the global statements unchanged).
That said, I believe that your method of accessing shared variables in your concurrent program is not idiomatic.

Global variable causing an error

So I am trying to declare the variable "checks" as a global variable because I get the following issue:
File "C:\Python27\Projects\Automatic Installer\autoinstall.py", line 11, in installFunc
if checks[0] == 1:
NameError: global name 'checks' is not defined
Here's my code, I've tried to add global checks to both the main body of the program as well as the installFunc function. Is there another location I should be adding it/some other way to indicate that checks should contain the information in the program?
import urllib
import subprocess
from Tkinter import *
global checks
def installFunc():
global checks
subprocess.call("md c:\MGInstall", shell=True)
subprocess.call (u"net use w: \\it01\files")
if checks[0] == 1:
subprocess.call(u"w:\\software\\snagitup.exe")
if checks[1] == 1:
subprocess.call(u"w:\\software\\camtasia.exe")
if checks[2] == 1:
urllib.urlretrieve(u"SUPERLONGURLLOLOLOL", u"c:\\MGinstall\\gotomeeting.exe")
subprocess.call (u"c:\\MGinstall\\gotomeeting.exe")
urllib.urlretrieve(u"http://ninite.com/.net-7zip-air-chrome-cutepdf-dropbox-essentials-firefox-flash-flashie-java-klitecodecs-quicktime-reader-safari-shockwave-silverlight-vlc/ninite.exe", u"c:\\MGinstall\\MGinstall.exe")
subprocess.call (u"c:\\MGinstall\\MGinstall.exe")
subprocess.call (u"w:\\printers\\installer\\printer.exe")
app = Tk()
w = Label(app, text="CompanyName IT Automatic Installer")
w.pack()
text = ["Snagit", "Camtasia", "GotoMeeting"]
variables = []
for name in text:
variables.append(IntVar())
Checkbutton(text=name, variable=variables[-1]).pack()
b = Button(text="OK", command=installFunc)
b.pack()
app.mainloop()
checks = [variable.get() for variable in variables]

I think this is because checks gets set after the mainloop (the last line of the posted code). the function installFunc gets called from the mainloop via a button press, but checks hasn't been defined yet.
Using the global data in this case isn't a good idea anyway. You should probably do something like:
def installFunc(checks):
...
checks = [variable.get() for variable in variables]
b = Button(text="OK", command=lambda : installFunc(checks))
Or, even better, wrap all this up in a class... that way you can do:
self.b=Button(..., command=self.installFunc)

Replace the first 'global checks' (the one at the global level) with 'global = ...', initializing it appropriately. Using 'global' is really relevant only within a function/method. Per the Python docs: The global statement is a declaration which holds for the entire current code block. It means that the listed identifiers are to be interpreted as globals. It would be impossible to assign to a global variable without global, although free variables may refer to globals without being declared global. You may want to read this as well - has lots of relevant info: Using global variables in a function other than the one that created them

The problem is not the first 'global checks'. What causes the error is that checks is accessed before it is initialized.
You must initialize checks before calling the main loop of the application.

First of all, Python is not PHP.
You need to use keyword global if an only if, you're going to assign to global variable within scope of a function.
global checks at top level makes no sense at all, and more importantly does not define that variable.
global checks in your installFunc() does not make sense, as you do not assign anything to that variable, in fact you don't even modify it.
In Python variables from outer scope are visible in local scope, unless of course you try to assign something, which would be interpreted as redefining that variable within local scope.
Problem with your code is that you only define checks after exiting main loop. Correct code would look like this
import urllib
import subprocess
from Tkinter import *
#no global definition here...
def installFunc():
#no global definition here...
subprocess.call("md c:\MGInstall", shell=True)
...
...
#define checks before starting main loop
checks = [variable.get() for variable in variables]
app.mainloop()