Develop Reference

C-like Static Variable inside a Python class method

After 20 years of C++ experience I am struggling to learn something of Python.
Now I'd like to have a method (a function inside a class) that has a "static" variable of its own, and not a static class variable.
Probably a pseudo code example can illustrate better what I want.
class dummy:
#staticmethod
def foo():
foo.counter += 1
print "You have called me {} times.".format(foo.counter)
foo.counter = 0
NOTE 1: I used #staticmethod just for simplicity, but this is irrelevant.
NOTE 2: This crashes with AttributeError: 'staticmethod' object has no attribute 'counter' but as I said above, this is a pseudo code to clarify my objective.
I have already learned that this works outside a class:
def foo():
foo.counter += 1
print "You have called me {} times.".format(foo.counter)
foo.counter = 0
But the same trick doesn't seem to work for member-functions.
Last minute information, I am restricted to using Python 2.7 (not my choice).
Is there any legal and reliable way to have a persistent variable (or constant) with scope restricted to the member-function scope?
Some relevant links
"What is the Python equivalent of static variables inside a function?":
https://stackoverflow.com/a/279586/466339
"There are no function-level static variables in Python":
https://code-maven.com/slides/python-programming/static-variable
Thanks in advance.

One way to achieve this is to tuck your variable away in a closure, so it will effectively be static for your purposes. Unfortunately, Python 2 does not support the nonlocal keyword, so we have to wrap our variable's value in an object (unless you only mean to reference and not mutate the variable (i.e. assign to the variable) in the method:
In [7]: class _Nonlocal:
...: def __init__(self, value):
...: self.counter = value
...:
...: def foo_maker():
...: nonlocal = _Nonlocal(0)
...: def foo(self):
...: nonlocal.counter += 1
...: print "You have called me {} times.".format(nonlocal.counter)
...: return foo
...:
In [8]: class Dummy(object): #you should always inherit from object explicitely in python 2
...: foo = foo_maker()
...:
In [9]: dummy = Dummy()
In [10]: dummy.foo()
You have called me 1 times.
In [11]: dummy.foo()
You have called me 2 times.
Of course, this is a lot of rigamarole simply to avoid using an instance variable. Perhaps the best solution is to make your method a custom object, and you can implement the descriptor protocol to make it callable as a method, and it will be usable as an instance method if you'd like:
In [35]: import types
...:
...: class Foo(object):
...: def __init__(this):
...: this.counter = 0
...: def __call__(this, self):
...: this.counter += 1
...: print "You have called me {} times.".format(this.counter)
...: print "here is some instance state, self.bar: {}".format(self.bar)
...: def __get__(this, obj, objtype=None):
...: "Simulate func_descr_get() in Objects/funcobject.c"
...: if obj is None:
...: return this
...: return types.MethodType(this, obj)
...:
In [36]: class Dummy(object): #you should always inherit from object explicitely in python 2
...: foo = Foo()
...: def __init__(self):
...: self.bar = 42
...:
In [37]: dummy = Dummy()
In [38]: dummy.foo()
You have called me 1 times.
here is some instance state, self.bar: 42
In [39]: dummy.bar = 99
In [40]: dummy.foo()
You have called me 2 times.
here is some instance state, self.bar: 99
All of this would be highly irregular and confusing to someone else who is used to python conventions, although I hope you see, the Python data-model offers a lot of power to customize things.
note, i've used this as the name of the first argument to avoid confusion with self that will actually come from the object that Foo get's bound to as a method.
Again, I should reiterate, I would never do this. I would just use an instance variable, or perhaps a generator if your function needs to maintain state, and could be used as an iterator.

No, there is not. You've already found the Python version: a class variable that you, the supreme overlord of class dummy development, will access only within function foo.
If it would help to know the rationale for this, you can start that path here. I expect that you've already been through much of this; however, this answer gives Python specifics for more Pythonic ways to implement what you need.

As #Prune already mentioned there is no real way of doing so.
However, if you want the static variable inside a method to be available only to the object it belongs to (as it is in C++ as far as I remember), you should define it in the constructor or as a class variable with a non-static method:
from __future__ import print_function
class dummy:
def __init__(self, counter=0):
self._foo_counter = 0
def foo(self):
self._foo_counter += 1
print("You have called me {} times.".format(self._foo_counter))
or:
class dummy:
def foo(self):
self._foo_counter += 1
print("You have called me {} times.".format(self._foo_counter))
_foo_counter = 0
This way, running:
x = dummy()
for _ in range(4):
x.foo()
y = dummy()
for _ in range(4):
y.foo()
Results in:
You have called me 1 times.
You have called me 2 times.
You have called me 3 times.
You have called me 4 times.
You have called me 1 times.
You have called me 2 times.
You have called me 3 times.
You have called me 4 times.
Note that the two versions do not behave in exactly the same way.
When you define _foo_counter in the class directly, you will have access to the _foo_counter variable both for the object (self._foo_counter) and for the class itself (dummy._foo_counter).
The dummy._foo_counter will be static for every use of the class and will persist across multiple instances of the class, so across multiple objects.
This is also the only variable that you can access if you use the #staticmethod decorator on dummy.foo():
class dummy:
#staticmethod
def foo():
dummy._foo_counter += 1
print("You have called me {} times.".format(dummy._foo_counter))
_foo_counter = 0
Here, self or _foo_counter will not be accessible, and your only option is to use the class-wide variable dummy._foo_counter (which, as already mentioned, you could use with methods not decorated with #staticmethod as well).
So that running again:
x = dummy()
for _ in range(4):
x.foo()
y = dummy()
for _ in range(4):
y.foo()
results in:
You have called me 1 times.
You have called me 2 times.
You have called me 3 times.
You have called me 4 times.
You have called me 5 times.
You have called me 6 times.
You have called me 7 times.
You have called me 8 times.

Using a mutable type as the default value for a keyword argument for your function is maybe the simplest approach:
class Dummy:
#staticmethod
def foo(_counter=[0]): # here using a list, but you could use a dictionary, or a deque
_counter[0] += 1
print "You have called me {} times.".format(_counter[0])
The rationale is that this variable is initialized only once; its latest value remains in the closure formed.

I already posted this in an old post, but nobody noticed it
As I have a different idiomatic objective with static variables, I would like to expose the following:
In a function, I want to initialize a variable only once with a calculated value which may be a bit costly.
As I love nice-writing, and being an old C-style programmer. I tried to define a macro-like writing:
def Foo () :
StaticVar( Foo, ‘Var’, CalculateStatic())
StaticVar( Foo, ‘Step’, CalculateStep())
Foo.Var += Foo.Step
print(‘Value of Var : ‘, Foo.Var)
Then, I wrote ‘StaticVar’ like this:
def StaticVar(Cls, Var, StaticVal) :
if not hasattr(Cls, Var) :
setattr(Cls, Var, StaticVal)
I can even write nicer code in Python:
def StaticVars(Cls, **Vars) :
for Var, StaticVal in Vars.items() :
if not hasattr(Cls, Var) :
setattr(Cls, Var, StaticVal)
def Foo () :
StaticVars( Foo, Var = CalculateStatic(),Step= CalculateStep()))
Foo.Var += Foo. Step
print(‘Value of Var : ‘, Foo.Var)
Sure, this is a nice way to write the code, but my objective (only one call of initialization functions) is not met (just add a print in the initialization function to see that the it is called often) ! The fact is that, in a function call, the parameter value is evaluated even before the function is called.
def CalculateStatic() :
print("Costly Initialization")
return 0
def CalculateStep() :
return 2
def Test() :
Foo()
Foo()
Foo()
>>> Test()
Costly Initialization
Value of Var : 2
Costly Initialization
Value of Var : 4
Costly Initialization
Value of Var : 6
To meet my objective, I’d rather write something like this:
def Foo () :
if not hasattr(Foo, ‘Var’) :
setattr ( Foo, ‘Var’, CalculateStatic())
setattr ( Foo, ‘Step’, CalculateStep())
Foo.Var += Foo. Step
print(‘Value of Var : ‘, Foo.Var)
>>> Test()
Costly Initialization
Value of Var : 2
Value of Var : 4
Value of Var : 6
And it could be “nicely written” like this (I used the underscore notation refering to “private == static”):
def StaticVars(Cls, **Vars) :
for Var, StaticVal in Vars.items() :
setattr(Cls, Var, StaticVal)
def Foo () :
_ = Foo
try :
__ = _.Var
except AttributeError : # The above code could only generate AttributeError Exception
# the following code is executed only once
StaticDefVars(_, Var= CalculateStatic(), Step = CalculateStep())
_.Var += _. Step
print(‘Value of Var : ‘, Foo.Var)
Attention must be paid to not put 'calculation code' in the 'try' clause which could generate extra 'AttributeError' exception.
Sure, if Python had had 'Marcro preprocessing', it would be even nicer
"'

Calling function with unknown number of parameters

I am trying to create a set of functions in python that will all do a similar operation on a set of inputs. All of the functions have one input parameter fixed and half of them also need a second parameter. For the sake of simplicity, below is a toy example with only two functions.
Now, I want, in my script, to run the appropriate function, depending on what the user input as a number. Here, the user is the random function (so the minimum example works). What I want to do is something like this:
def function_1(*args):
return args[0]
def function_2(*args):
return args[0] * args[1]
x = 10
y = 20
i = random.randint(1,2)
f = function_1 if i==1 else function_2
return_value = f(x,y)
And it works, but it seems messy to me. I would rather have function_1 defined as
def function_1(x):
return x
Another way would be to define
def function_1(x,y):
return x
But that leaves me with a dangling y parameter.
but that will not work as easily. Is my way the "proper" way of solving my problem or does there exist a better way?

There are couple of approaches here, all of them adding more boiler-plate code.
There is also this PEP which may be interesting to you.
But 'pythonic' way of doing it is not as elegant as usual function overloading due to the fact that functions are just class attributes.
So you can either go with function like that:
def foo(*args):
and then count how many args you've got which will be very broad but very flexible as well.
another approach is the default arguments:
def foo(first, second=None, third=None)
less flexible but easier to predict, and then lastly you can also use:
def foo(anything)
and detect the type of anything in your function acting accordingly.
Your monkey-patching example can work too, but it becomes more complex if you use it with class methods, and does make introspection tricky.
EDIT: Also, for your case you may want to keep the functions separate and write single 'dispatcher' function that will call appropriate function for you depending on the arguments, which is probably best solution considering above.
EDIT2: base on your comments I believe that following approach may work for you
def weigh_dispatcher(*args, **kwargs):
#decide which function to call base on args
if 'somethingspecial' in kwargs:
return weight2(*args, **kwargs)
def weight_prep(arg):
#common part here
def weight1(arg1, arg2):
weitht_prep(arg1)
#rest of the func
def weight2(arg1, arg2, arg3):
weitht_prep(arg1)
#rest of the func
alternatively you can move the common part into the dispatcher

You may also have a function with optional second argument:
def function_1(x, y = None):
if y != None:
return x + y
else:
return x
Here's the sample run:
>>> function_1(3)
3
>>> function_1(3, 4)
7
Or even optional multiple arguments! Check this out:
def function_2(x, *args):
return x + sum(args)
And the sample run:
>>> function_2(3)
3
>>> function_2(3, 4)
7
>>> function_2(3, 4, 5, 6, 7)
25
You may here refer to args as to list:
def function_3(x, *args):
if len(args) < 1:
return x
else:
return x + sum(args)
And the sample run:
>>> function_3(1,2,3,4,5)
15

Python: Using one argument to handle choice between a number and a string?

Basically I am writing a function that depends on a numerical input x, a number between 0 and 1. I want the default value of x to be, say, x=0.5. However, I also want to provide an option to the user that allows them to let the program select x for them using some algorithm. Is there an elegant way to handle that choice with one function argument?
I'm thinking something like this:
def foo(x=0.5):
if x == "pick for me":
return complicated_algorithm_that_picks_x()
else:
return x
def complicated_algorithm_that_picks_x():
print "Thinking hard..."
return 0.1234567
which would return:
>>> foo()
0.5
>>> foo(0.3)
0.3
>>> foo("pick for me")
Thinking hard...
0.1234567
But this looks really inelegant, since the user has to know what magic string to pass to invoke the selection algorithm. Any ideas how I can handle this more cleanly?
I was thinking having an additional Boolean argument called pick (that defaults to False), which when True will invoke the x picking function. But then users might pass both, say, x=0.3 and pass=True, in which case I have to arbitrarily ignore one of the choices. Looks clumsy again.

There are three things you might consider:
Split the one function into two.
Use a class.
Multiple default arguments.
Here there are, in no particular order:
Split one function into two
If you want to do two different things in one function and you're having trouble designing a natural interface, it might be a sign that the one function should become two:
def foo_picked_for_me():
x = pick_x()
return foo(x)
def foo(x):
# do foo
pass
I don't know how this strikes you, but it's simple, clear, and that means its often preferable.
Use a class
Default arguments are nice, but a function's interface can only get so complicated before it starts making more sense to handle option setting with a class:
class Foo:
def __init__(self):
self.x = 0.5
def pick_x_for_me(self):
self.x = pick_x()
def foo(self):
# do foo with self.x
As EOL suggests below, it's perfectly pythonic to leave x "exposed", and to allow the user to change it. You say, though, that x must be between 0 and 1, so it might make sense to do some bounds checking with the setter for x:
class Foo(object):
def __init__(self):
self._x = 0.5
#property
def x(self):
return self._x
#x.setter
def x(self, value):
if 0 <= value <= 1:
self._x = value
else:
raise ValueError("x must be between 0 and 1")
def pick_x_for_me(self):
self._x = pick_x()
def foo(self):
pass
# do foo with self._x
Multiple default arguments
The last option is analogous to what other posters have given: use two arguments, and throw an exception if the user does something contradictory. I'd consider allowing three forms of call:
# x gets its default value of 0.5
foo()
# x gets the specified value
foo(x=.42)
# x is picked for me
foo(pick_for_me=True)
Additionally, if I write:
foo(x=.42, pick_for_me=True)
I'll throw an exception. Some code that implements this follows:
def foo(x=None, pick_for_me=None):
if x is None and pick_for_me is None:
x = 0.5
elif pick_for_me and x:
raise RuntimeError("You can't set both!")
elif pick_for_me:
x = picking_algorithm()
# else x was set, so leave it be
This is kind of complicated, and I'm not so sure I like the API. Just make sure you document the behavior well enough so that the user knows how to use the thing.

Try this:
def foo(x=0.5, use_complex_algo=False):
if use_complex_algo == False:
return x
else:
return complicated_algorithm_that_picks_x()
#ouput
print foo() # returns: 0.5
print foo(0.3) # returns: 0.3
print foo(use_complex_algo=True) # returns: 0.1234567

I would probably define a constant in the module that indicates to generate a random value. For example:
# foo.py
import random
RANDOM = 'random'
def foo(x=0.5):
if x == RANDOM:
x = random.random()
if x < 0 or x > 1:
raise ValueError('x must be between 0 and 1')
return x
Then, to use it is pretty straightforward:
>>> import foo
>>> foo.foo(0.1)
0.1
>>> foo.foo()
0.5
>>> foo.foo(foo.RANDOM)
0.4388309758578337
>>> foo.foo(foo.RANDOM)
0.5351558099071574
>>> foo.foo(7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foo.py", line 9, in foo
raise ValueError('x must be between 0 and 1')
ValueError: x must be between 0 and 1

I think this code would do what all you want!
import random
def foo(x=0.5):
if x=="pick for me":
return random.random()
else:
return x
print foo()
print foo("pick for me")
print foo(0.3)

Assuming the parameter type passed is float (can also be checked), you could define that any input < 0 or > 1 means "pick for me"...
It also kind of validates that the passed parameter is within range, and if not generates a proper value.
But this does not really answer the question about choosing between a number & a string ;-) (type() could help for this)
def foo(x=0.5):
if 0 <= x <= 1:
return x
else:
return complicated_algorithm_that_picks_x()

Returning intermediate results from function in Python

Imagine I've got a Python module with some function in it:
def sumvars(x, y, z):
s = x
s += y
s += z
return s
But sometimes I want to get results of some intermediate calculations (for example, I could have a function which reverses a matrix and would like to know the determinant which has been calculated as an intermediate step as well). Obviously, I wouldn't want to redo those calculations again if they were already done within that function.
My first idea is to return a dict:
def sumvars(x, y, z):
d = {}
s = x
d['first_step'] = s
s += y
d['second_step'] = s
s += z
d['final'] = s
return d
But I don't recall any functions in numpy or scipy which return dicts and so it seems like this might be not a good idea. (Why?) Also routinely I'll always have to type sumvars(x,y,z)['final'] for a default return value...
Another option I see is creating global variables but seems wrong having a bunch of them in my module, I would need to remember their names and in addition not being attached to the function itself looks like a bad design choice.
What would be the proper function design for such situation?

Generally when you have two different ways you want to return data, go ahead and make two different functions. "Flat is better than nested", after all. Just have one call the other so that you Don't Repeat Yourself.
For example, in the standard library, urllib.parse has parse_qs (which returns a dict) and parse_qsl (which returns a list). parse_qs just then calls the other:
def parse_qs(...):
parsed_result = {}
pairs = parse_qsl(qs, keep_blank_values, strict_parsing,
encoding=encoding, errors=errors)
for name, value in pairs:
if name in parsed_result:
parsed_result[name].append(value)
else:
parsed_result[name] = [value]
return parsed_result
Pretty straightforward. So in your example it seems fine to have
def sumvars(x, y, z):
return sumvars_with_intermediates(x, y, z).final
def sumvars_with_intermediates(x, y, z):
...
return my_namedtuple(final, first_step, second_step)
(I favor returning namedtuples instead of dicts from my APIs, it's just prettier)
Another obvious example is in re: re.findall is its own function, not some configuration flag to search.
Now, the standard library is a sprawling thing made by many authors, so you'll find counterexamples to every example. You'll far more often see the above pattern rather than one omnibus function that accepts some configuration flags, though, and I find it far more readable.

Put the common calculation into its own function as Jayanth Koushik recommended if that calculation can be named appropriately. If you want to return many values (an intermediate result and a final result) from a single function then a dict may be an overkill depending on what is your goal but in python it is much more natural to simply return a tuple if your function has many values to return:
def myfunc():
intermediate = 5
result = 6
return intermediate, result
# using the function:
intermediate, result = myfunc()

Not sure if function attributes is a good idea:
In [569]: def sumvars(x, y, z):
...: s = x
...: sumvars.first_step = s
...: s += y
...: sumvars.second_step = s
...: s += z
...: return s
In [570]: res=sumvars(1,2,3)
...: print res, sumvars.first_step, sumvars.second_step
...:
6 1 3
Note: as #BrenBarn mentioned, this idea is just like global variables, your previously calculated "intermediate results" could not be stored when you want to reuse them.

Just came up with this idea which could be a better solution:
def sumvars(x, y, z, mode = 'default'):
d = {}
s = x
d['first_step'] = s
s += y
d['second_step'] = s
s += z
d['final'] = s
if mode == 'default':
return s
else:
return d

I belive the proper solution is to use a class, to have a better grasp of what you are modeling. For example in the case of the Matrix, you could simply store the determinant in the "determinant" attribute.
Here is an example using your matrix example.
class Matrix:
determinant = 0
def calculate_determinant(self):
#calculations
return determinant
def some_method(self, args):
# some calculations here
self.determinant = self.calculate_determinant()
# other calculations
matrix = Matrix()
matrix.some_method(x, y, z)
print matrix.determinant
This also allows you to separate your method into simpler methods, like one for calculating the determinant of your matrix.

Another variation:
def sumvars(x, y, z, d=None):
s = x
if not d is None:
d['first_step'] = s
s += y
if not d is None:
d['second_step'] = s
s += z
return s
The function always returns the desired value without packing it into a tuple or dictionary. The intermediate results are still available, but only if requested. The call
sumvars(1, 2, 3)
just returns 6 without storing intermediate values. But the call
d = {}
sumvars(1, 2, 3, d)
returns the same answer 6 and inserts the intermediate calculations into the supplied dictionary.

Option 1. Make two separate functions.
Option 2. Use a generator:
>>> def my_func():
... yield 1
... yield 2
...
>>> result_gen = my_func()
>>> result_gen
<generator object my_func at 0x7f62a8449370>
>>> next(result_gen)
1
>>> next(result_gen)
2
>>> next(result_gen)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>>

Inspired by #zhangxaochen solution, here's my take on your problem using class attributes:
class MyClass():
def __init__(self):
self.i = 4
def f(self):
s = self.i
MyClass.first_step = s
print(MyClass.first_step)
s += self.i
MyClass.second_step = s
print(MyClass.second_step)
s += self.i
return s
def main():
x = MyClass()
print(x.f()) # print final s
print(x.first_step)
print(x.second_step)
print(MyClass.second_step)
Note: I included several prints to make it more explicit how attribute values can be retrieved.
Result:
4
8
12
4
8
8

Taking Input from a Function Name in Python

I am attempting to write a program to perform arithmetic mod n, given n. I was wondering if there is any way within Python (preferably 2.7) to dynamically define a function such that its behavior depends on the name used to call it. More concretely, I would like to define a function named "*mod", where * is an integer, that then does arithmetic mod *. Perhaps more clearly, I would like to write one function definition for *mod that defines the functions 2mod, 3mod, 4mod, and so on. Is this possible? I apologize if a similar question has already been asked or if my answer is readily available in documentation; I tried to search for it, but I didn't know exactly how to describe the functionality that I'm looking for, so I may have missed it.
Thanks!

You don't want to do that. Just make a simple function and pass both numbers as arguments:
def mod(x, n):
return x % n
print mod(5, 2)
# 1

Well, if you really, really want to, look at this quick hack. It uses a wrapper class to wrap the module in a class, so you can use __getattr__:
import sys
import functools
def add(a, b):
return a + b
def sub(a, b):
return a - b
class Wrapper(object):
def __init__(self, wrapped):
self.wrapped = wrapped
def __getattr__(self, name):
try:
# quick hack. Don't try this at home :-)
f = ''.join(x for x in name if not x.isdigit())
n = ''.join(x for x in name if x.isdigit())
return functools.partial(getattr(self.wrapped, f), int(n))
except:
return getattr(self.wrapped, name)
sys.modules[__name__] = Wrapper(sys.modules[__name__])
Now, when you call e.g. add10(12) on this module, the result is 22. Note that method names must not start with a number, but you could use names like _add and call the methods like _55add(45) and so on.
But I would follow Haidro advice: You don't want to do that. Just calling the method with two arguments is a lot simpler.

Using globals, lambda:
for i in range(2, 5):
globals()['mod{}'.format(i)] = lambda x, n=i: x % n
assert mod2(4) == 0
assert mod2(3) == 1
assert mod3(2) == 2
assert mod3(1) == 1
assert mod4(1) == 1
assert mod4(2) == 2
assert mod4(3) == 3
assert mod4(9) == 1

You could achieve this by generating the functions as a string, and then exec this string to get the function in the current namespace. Something like:
n = 2
s = 'def mod%i(x):' % n
s += ' return x %% %i' % n
exec s
This would define the function mod2(x)