I just started learning Python and I'm confused about this example:
def append_to(element, to=None):
if to is None:
to = []
to.append(element)
return to
If to was initialized once, wouldn't to not be None the 2nd time it's called? I know the code above works but can't wrap my head around this "initialized once" description.
def append_to(element, to=None):
to = ...
to here becomes a local variable and is assigned to a list, and is deallocated if you don't assign return value to another variable.
If you want to staying alive for subsequent calls to append_to you should do:
def append_to(element, to=[]):
to.append(element)
return to
Demo:
>>> lst = append_to(5)
>>> append_to(6)
>>> append_to(7)
>>> print lst
[5, 6, 7]
If "to" was initialized once, wouldn't "to" won't be "None" the 2nd time it's called?
to would become None if you don't pass a value for it: append_to(1) and only when to is None will your code rebind the local name to to a newly created list inside the body of your function: to = [].
The default values of functions are assigned only once, that's whatever you assign as a default value, that object will be used for every call you make to the function and will not change, typically the same reference of the default value will be used for every call you make to the function. This matters when you assign mutables as defaults:
l = []
def defArgs(d=l) # default arguments, same default list for every call
d.append(1)
return d
defArgs() is l # Object identity test: True
Run the above function multiple times and you will observe the list growing with more elements because you get only one single copy of argument defaults for every function shared by every function call. But notice here:
def localVars(d=None):
if d is None:
d = [] # Different list for every call when d is None
d = [] is executed every time you call localVars; when the function finishes its job, every local variable is garbage-collected when reference count drops to 0, but not the default values of arguments, they live after the execution of the function and are not typically garbage-collected after the execution of the function.
In Python there is no declaration and initialization stage when you use a variable. Instead, all assignment is a definition where you bind an instance to the variable name.
The interpreter bind the instance to the default value when instantiate the function. When the default value is a mutable object and you only change its state by its methods than the value will be "shared" between calls.
Related
I'm trying to build a function that returns the value of its previous call using closure. The first time function is called, it will return None. I'm not sure how to update last_in from one call to another.
def last_in(x):
last_in = [None]
def get():
temp = last_in[0]
last_in[0] = x
# print(last_in)
return temp
return get()
For example, print(last_in(1),last_in(2),last_in(3)) should print: None 1 2
The problem with your approach is that whenever you call last_in, i.e. the "outer" function, the previous value stored in last_in (the array, not the function) is reset to None. Instead, the outer function should be called only once so that the value is not reset each time you call it.
Not sure what you need this for, but I think it would make sense to create a decorator function for this, i.e. a function modifying an existing function. This way, all the storing-and-retrieving-the-last-result can be done in the decorator without cluttering the actual function. The outer function (the decorator) is called only once, and the original function is replaced with the decorated version that will correctly retrieve the stored value.
def return_previous(f):
f.last_result = None
def _f(*args, **kwargs):
res = f.last_result
f.last_result = f(*args, **kwargs)
return res
return _f
#return_previous
def some_function(x):
return x**2
print(some_function(1), some_function(2), some_function(3))
# None 1 4
I like the solution that #tobias_k provides, but here is another alternative which conforms to the current organization/structure of your code.
def last_in(x):
def get():
temp = last_in.__dict__.get('prev', None)
last_in.__dict__['prev'] = x
return temp
return get()
print(last_in(1),last_in(2),last_in(3))
None 1 2
This is a slight deviation from the request since it requires a second keyword argument but you could take advantage of the fact that default arguments are only set once (see here) to do something like this:
def last_in(x, __last=[None]):
last = __last[0]
__last[0] = x
return last
__last is set once when the function is declared and since it is mutable, you can update it at each function call and the updated value will persist between calls.
My program has a lambda as a command for a tkinter object inside a loop. I want the lambda to pass an argument for a function where the function will make some changes according to the argument.
for cat in self.t_m_scope:
print(cat)
self.t_m_scopeObj[cat] = [tk.Checkbutton(self, text = cat, command = lambda: self.t_m_checkUpdate(cat)), []]
if self.t_m_scopeVars[cat][0]: self.t_m_scopeObj[cat][0].toggle()
self.t_m_scopeObj[cat][0].grid(row = 5, column = colNum, padx = (10,10), pady = (2,5), sticky = tk.W)
Whenever I try to run this, it always passes in the last iteration of cat because of some weird nature in tkinter commands. Instead, I want it so that the argument matches exactly whatever iteration it was at when the lambda was defined. I don't want the argument to change just because I clicked the check button after its iteration.
Here is the function that the lambda calls.
def t_m_checkUpdate(self, cat):
self.t_m_scopeVars[cat][0] = not self.t_m_scopeVars[cat][0]
for subcat in range(len(self.t_m_scopeVars[cat][1])):
if self.t_m_scopeVars[cat][0] != self.t_m_scopeVars[cat][1][subcat]:
self.t_m_scopeVars[cat][1][subcat] = not self.t_m_scopeVars[cat][1][subcat]
self.t_m_scopeObj[cat][1][subcat].toggle()
As some context to what this program is, I am trying to have a bunch of checkbuttons toggle on and off in response to a click on a main checkbutton. The only way I know which checkbuttons to toggle (since I have many) is to pass an argument to a function using lambda.
I do understand that I could just find the value of the current iteration using if/elif/else and pass a string instead of a variable but I seriously don't want to do that because it's just not scalable.
If it helps, the text for my checkbutton and the argument I want to pass is one and the same.
Is there any workaround so I can preserve the current iteration without Tkinter going all wonky?
You need to bind the data in cat to the function when it's created. For an explanation, see this question. There are a few ways to do this, including partials and closures.
This is how you would apply a partial to your example:
from functools import partial
for cat in ...:
check_update_cat = partial(self.t_m_checkUpdate, cat)
self.t_m_scopeObj[cat] = [tk.Checkbutton(self, text=cat, command=check_update_cat), []]
Explanation
In your code, the lambda refers to an iterator variable (cat) declared in the outer scope of the loop. One thing to note is that the scope of a loop "leaks" the iterator variable, i.e. you can still access it after the loop. The lambda functions refer to the iterator variable by reference, so they access the most recent value of the iterator.
A runnable example:
a = []
for i in range(5):
a.append(lambda: i)
a[0]() # returns 4
a[1]() # also returns 4
del i # if we delete the variable leaked by the for loop
a[0]() # raises NameError, i is not defined because the lambdas refer to i by reference
Instead, we want to bind the value of the iterator to the functions at each step. We need to make a copy of the iterator's value here.
import functools
a = []
for i in range(5):
a.append(functools.partial(lambda x: x, i)) # i is passed by value here
a[0]() # returns 0
a[1]() # returns 1
del i
a[0]() # still returns 0
Edit: Why are people downvoting this post? Are Python developers really this inept? It's a legitimate question, not one that's been answered in other places. I searched for a solution. I'm not an idiot. One parameter has a value and the other one is undefined, but if you actually read the post, you will see that both of them appear to be equally scoped.
First of all, I assure you that this question is unlike other questions involving the error message:
UnboundLocalError: local variable referenced before assignment closure method
As I'm looking at this code, it appears that the parameter, uuidString, of the top-level method, getStockDataSaverFactory, should actually be in-scope when the method returns its inner method, saveData, as a first-class function object... because to my amazement, the parameter tickerName IS in-scope and does have the value 'GOOG' when the saveData() method is called (e.g. by the test method testDataProcessing_getSaverMethodFactory), so we can actually see that it has an actual value when the method, getDataMethodFactory(..)
is called, unlike uuidString.
To make the matter more obvious, I added the lines:
localUuidString = uuidString
and
experimentUuidString = localUuidString
to show that the parameter uuidString has an available value when the method is inspected by a breakpoint.
def getStockDataSaverFactory(self, tickerName, uuidString, methodToGetData, columnList):
# This method expects that methodToGetData returns a pandas dataframe, such as the method returned by: self.getDataFactory(..)
localUuidString = uuidString
def saveData():
(data, meta_data) = methodToGetData()
experimentUuidString = localUuidString
methodToNameFile = self.getDataMethodFactory(tickerName, uuidString)
(full_filepathname, full_filename, uuidString) = methodToNameFile()
methodToSaveData = self.getDataFrameSaverFactory(methodToGetData, columnList, full_filepathname)
# We might want try/catch here:
methodToSaveData()
# A parameterless method that has immutable state (from a closure) is often easier to design around than one that expects parameters when we want to pass it with a list of similar methods
return (full_filepathname, full_filename, uuidString)
return saveData
def testDataProcessing_getSaverMethodFactory(self):
dataProcessing = DataProcessing()
getSymbols = dataProcessing.getSymbolFactory(
dataProcessing.getNasdaqSymbols(dataProcessing.getListOfNASDAQStockTickers))
tickers = getSymbols()
uuidString = 'FAKEUUID'
columnList = ['low', 'high']
tickerSubset = tickers[0:2]
methodsToPullData = map(lambda ticker: dataProcessing.getStockDataSaverFactory(ticker,
uuidString,
dataProcessing.getDataFactory(
ticker),
columnList), tickerSubset)
savedPathTuples = [f() for f in methodsToPullData]
savedFileNames = [pathTuple[0] for pathTuple in savedPathTuples]
for fileName in savedFileNames:
self.assertTrue(os.path.isfile(fileName))
os.remove(fileName)
Just to make it clear that uuidString has no value but ticker does have a value, I'm including this screenshot:
Notice that in the variable watch window, uuidString is undefined, but ticker has the string value of "A".
Is there something unique about Python (or Python 3) that is resulting in this behavior?
The problem is that you reference uuidString in the call to self.getMethodThatProvidesFullFilePathNameForPricesCsvFromUUIDAndTickerName before you assign to it. The assignment makes it local to the scope of the innermost function and therefore, it is unassigned when you reference it.
A full description of the scoping rules is provided by: https://stackoverflow.com/a/292502/7517724
This simpler example reproduces your error to make the problem more clear:
class aclass():
def outer(self, uuidString):
def inner():
print(uuidString)
uuidString = 'new value'
return uuidString
return inner
a = aclass()
func = a.outer('a uuid')
val = func()
print(val)
The assignment in inner() causes the uuidString to be local to inner() and therefore it is unassigned when the print(uuidString) is call, which causes Python to raise the UnboundLocalError.
You can fix the error by passing the variable in to your function with a default argument. Changing the definition of saveData to pass uuidString as a default argument, as:
def saveData(uuidString=uuidString):
will make it work as you expect.
Say we have a function in a file, something like:
..... other fucntions
def add_to_list(l = []):
l.append("foo")
..... other functions
When will def be called? What exactly does def do?
[] creates a list and, for efficiency reasons, when a list is passed, it isn't duplicated or passed by value; it is sent by reference. A similar thing is happening with this. So, firstly all def's are initialized by Python. Then your code is run. However, the initializer has already created the list.
>>> id(add_to_list())
12516768
>>> id(add_to_list())
12516768
>>> id(add_to_list())
12516768
Excerpt from http://effbot.org/zone/default-values.htm
Notice how they have the same id? That's because it's returning the same instance of list every time. Thus, l = [] isn't creating a new instance at run time. So, we keep on appending to the same instance.
That is why we don't experience this behavior with ints, floats, and etc.
Consider your code:
def add_to_list(l = []):
l.append("foo")
print(l)
add_to_list()
add_to_list()
Then it's output:
['foo']
['foo', 'foo']
Now consider this similar function:
def add_to_list():
l = []
l.append("foo")
print(l)
add_to_list()
add_to_list()
The output will be:
['foo']
['foo']
That's because the l = [] is run after the constructors are initialized (in live time). When you consider the first code, you will see that the constructor is ran first, then the code executes live time.
def is executed whenever you hit it in parsing the source file. It defines the function. This means that the body of the function is assigned to the name, the parameters are included in the "signature" (calling format), etc. IN other words, the function is now ready to call from anywhere below that point, within the scope of the containing program.
Since l is a list, it's bound to whatever you pass into the function call. If you pass in a list, then it's bound by reference to that list, a mutable object.
As to your specific case, there's really no use case that will make sense: since you have to pass an object with an append method, anything you supply will have a value: it will never use the default value. The only way to get that default value into play is to call with empty parentheses, in which case there's no way to get the value back to the calling program (since you return nothing).
Thus, when you call the routine a second time, it appears that you're using the same list you passed the first time. That list already has the "foo" element you added that first time. Here's some test code and the output to illustrate the effect:
def add_to_list(l = []):
l.append("foo")
print "l =", l
empty = []
add_to_list(empty)
print "empty #1", empty
add_to_list(empty)
print "empty #2", empty
add_to_list()
Output:
l = ['foo']
empty #1 ['foo']
l = ['foo', 'foo']
empty #2 ['foo', 'foo']
l = ['foo']
Does that clarify things?
This is a simplified section of code from a game I'm trying to make. There is a condition in primary_function that if satisfied will run the action, in this case secondary_function fed into it, action itself being a function which alters a particular variable. Problem is that it doesn't alter the variable. Print(variable) returns 1.
variable = 1
def secondary_function(var):
var += 1
def function(parameter_1, parameter_2, action = None, var = None):
if condition == 'satisfied':
action(var)
function(1, 2, secondary_function, variable)
print(variable)
Variables in Python are not passed by reference, and there is no way to do so. So a function that takes an integer argument and just increments it can never impact the original variable. If you want to modify the original variable, you will have to assign it a new value instead.
def secondary_function(var):
return var + 1
variable = 1
variable = secondary_function(variable) # variable is now 2
Similary, you will have to modify your function function too to return the value instead:
def function (parameter_1, parameter_2, action = None, var = None):
if parameter_1 == 1:
return action(var)
variable = function(1, 2, secondary_function, variable)
Python has no notion of pass-by-reference or pass-by-value. Instead, every variable is passed by assignment. That means that whenever you call a function and pass a variable, the behavior is the same as if you just did function_argument = variable; you are assigning the value of the source variable to the function argument. Since Python is object oriented, this effectively means that you copy the reference to the object that variable refers to. As such, when the object is mutable, you can modify it and impact other variables referencing the same object. But for immutable objects (ints, strings, tuples, etc.), you cannot modify the existing object, so when you “modify” it, you always assign a new object to the variable, without affecting all the other variables that still point to the old object.
Function arguments in Python as passed by value. Thus, function(var) sees a copy of variable in var and secondary_function(var) sees yet another copy.