There is a questions asking how to simulate static variables in python.
Also, on the web one can find many different solutions to create static variables. (Though I haven't seen one that I like yet.)
Why doesn't Python support static variables in methods? Is this considered unpythonic or has it something to do with Python's syntax?
Edit:
I asked specifically about the why of the design decision and I haven't provided any code example because I wanted to avoid explanation to simulate static variables.
The idea behind this omission is that static variables are only useful in two situations: when you really should be using a class and when you really should be using a generator.
If you want to attach stateful information to a function, what you need is a class. A trivially simple class, perhaps, but a class nonetheless:
def foo(bar):
static my_bar # doesn't work
if not my_bar:
my_bar = bar
do_stuff(my_bar)
foo(bar)
foo()
# -- becomes ->
class Foo(object):
def __init__(self, bar):
self.bar = bar
def __call__(self):
do_stuff(self.bar)
foo = Foo(bar)
foo()
foo()
If you want your function's behavior to change each time it's called, what you need is a generator:
def foo(bar):
static my_bar # doesn't work
if not my_bar:
my_bar = bar
my_bar = my_bar * 3 % 5
return my_bar
foo(bar)
foo()
# -- becomes ->
def foogen(bar):
my_bar = bar
while True:
my_bar = my_bar * 3 % 5
yield my_bar
foo = foogen(bar)
foo.next()
foo.next()
Of course, static variables are useful for quick-and-dirty scripts where you don't want to deal with the hassle of big structures for little tasks. But there, you don't really need anything more than global — it may seem a but kludgy, but that's okay for small, one-off scripts:
def foo():
global bar
do_stuff(bar)
foo()
foo()
One alternative to a class is a function attribute:
def foo(arg):
if not hasattr(foo, 'cache'):
foo.cache = get_data_dict()
return foo.cache[arg]
While a class is probably cleaner, this technique can be useful and is nicer, in my opinion, then a global.
In Python 3, I would use a closure:
def makefoo():
x = 0
def foo():
nonlocal x
x += 1
return x
return foo
foo = makefoo()
print(foo())
print(foo())
I think most uses of local static variables is to simulate generators, that is, having some function which performs some iteration of a process, returns the result, but mantains the state for the subsequent invocation. Python handles this very elegantly using the yield command, so it seems there is not so much need for static variables.
It's a design choice.
I'm assuming Guido thinks you don't need them very often, and you never really need them: you can always just use a global variable and tell everyone to keep their greasy paws offa' your variable ;-)
For caching or memoization purposes, decorators can be used as an elegant and general solution.
The answer's pretty much the same as why nobody uses static methods (even though they exist). You have a module-level namespace that serves about the same purpose as a class would anyway.
An ill-advised alternative:
You can also use the side-effects of the definition time evaluation of function defaults:
def func(initial=0, my_static=[])
if not my_static:
my_static.append(initial)
my_static[0] += 1
return my_static[0]
print func(0), func(0), func(0)
Its really ugly and easily subverted, but works. Using global would be cleaner than this, imo.
From one of your comments: "I'd like to use them to cache things loaded from disk. I think it clutters the instance less, if I could assign them to the function"
Use a caching class then, as a class or instance attribute to your other class. That way, you can use the full feature set of classes without cluttering other things. Also, you get a reusable tool.
This shows that on SO it always pays off to state one's problem instead of asking for a specific, low level solution (e.g. for a missing language feature). That way, instead of endless debates about simulating "static" (a deprecated feature from an ancient language, in my view) someone could have given a good answer to you problem sooner.
Related
Question
In Python, if I have a function with local variables and nested functions, I can assign these variables in the nested functions with the help of nonlocal:
def f():
x = 0
def _g():
nonlocal x
x = 1
def _h():
x = 2
_g()
_h()
print(x)
f() # 1
My issue with this solution is that if I have many local variables in f(), the use of nonlocal is verbose, and more disturbingly, it is very easy to forget nonlocal for a variable, and to create local variables in the nested functions without noticing it (what did I really mean in the h() function above, for instance?).
I have seen and used an alternative:
def f():
class state:
x = 0
def _g():
state.x = 1
def _h():
x = 2
_g()
_h()
print(state.x)
f() # 1
If uses the fact that a class in Python is actually an object too. Using 'class' that way is actually the least verbose way of creating a mutable container for local values (I believe).
I think there are two questionable aspects in this pattern:
That particular use of the 'class'-keyword might be considered as a hack.
Since the container is somewhat artificial, it is sometimes difficult to find a good name for it (I went so far as testing the use of the name 'self' for it, but that seemed even more like a hack).
Is this a good pattern? What alternatives do you usually use?
If StackOverflow is not the right forum for this question, please advise on a different forum that you think is better suited (I did read the FAQs, but what the correct forum is for this question not obvious to me).
Thanks in advance.
P.S.
For the sake of completeness, there is at least one more alternative, which feels even more like a hack:
class f:
def __init__(self):
self.x = 0
self._g()
self._h()
print(self.x)
def _g(self):
self.x = 1
def _h(self):
x = 2
f() # 1
This works because in Python, class instantiation has the same syntax as a function call.
Answer
See the accepted answer below. For a discussion about a solution when the function needs to return a value see there.
Your "last resort" alernative is of course cleaner - if you can do it in flattened methods that share an state, that is more readable code than nested functions to do the same job. "Flat is better than nested".
Other than that, you are creating a class to have a namespace. You could simply create an object to work as namespace, and it would work as a namespace - that is more usual. The only thing there is that if you simply create an instance of object itself, it can't work as namespace, because it have no __dict__, so you can't freely attribute objects for this.
That is why buried in the types module in the stdlib there is a class named SimpleNamespace exactly for this use case.
Just do:
from types import SimpleNamespace
def f():
state = SimpleNamespace()
state.x = 0
...
(but only if you don't change your mind and go for the class-based solution which is cleaner anyway).
Why not just use a dict instead of the class. This removes some of the "hackyness" and is Python 2 compatible:
def f():
state=dict(x=0)
def _g():
state["x"] = 1
def _h():
x = 2
_g()
_h()
print(state["x"])
f() # 1
I'd like to make a copy of class, while updating all of its methods to refer a new set of __globals__
I was thinking something like below, however unlike types.FunctionType, the constructor for types.UnboundMethodType does not accept __globals__, any suggestions how to work around this?
def copy_class(old_class, new_module):
"""Copies a class, updating __globals__ of all methods to point to new_module"""
new_dict = {}
for name, entry in old_class.__dict__.items():
if isinstance(entry, types.UnboundMethodType):
entry = types.UnboundMethodType(name, None, old_class.__class__, globals=new_module.__dict__)
new_dict[name] = entry
return type(old_class.name, old_class.__bases__, new_dict)
The __dict__ values are functions, not unbound methods. The unbound method objects only get created on attribute access. If you are seeing unbound method objects in the __dict__, something weird happened with your class object before this function got to it.
I don't know about you, but I generally don't like to use types for anything other than type checking (which I don't do very often ;-). I'd much rather inspect...
I have to preface this code by saying that I hope you have a really good reason for wanting to do this ;-) -- to me, it seems like just subclassing and overriding class properties should get the job done much more elegantly ... However, If you really want to copy a class -- Why not just execute it's source again in the new namespace?
I've put together the following simple modules:
# test.py
# Just some test data
FOO = 1
class Bar(object):
def subclass_method(self):
print('Hello World!')
class Foo(Bar):
def method(self):
return FOO
And then something to do the heavy lifting:
import sys
import inspect
def copy_class(cls, new_globals):
source = inspect.getsource(cls)
globs = {}
globs.update(sys.modules[cls.__module__].__dict__)
globs.update(new_globals)
exec source in globs
return globs[cls.__name__]
# Check that it works...
import test
NewFoo = copy_class(test.Foo, {'FOO': 2})
print NewFoo().method()
NewFoo().subclass_method()
print test.Foo().method()
test.Foo().subclass_method()
This has some possibly desirable properties and undesirable... First, it only works on classes that are inspectable. That's pretty much anything user-defined so probably not too restrictive... It also might be a bit slower than other solutions that don't involve re-parsing the source string -- But again, it doesn't seem like this should be executed too frequently, so that's probably Ok.
Now the "advantages"...
If a global is requested by a function but not supplied, this will use the global from the old namespace. If this behavior isn't desireable (i.e. you'd rather have the NameError), you can modify the function easily to remove it.
The "copy" doesn't inherit from the original. For most purposes, that probably doesn't matter, but it's a bit weird to have the copy of something inherit from the original ...
Some people might see the exec in here and immediately think "Oh no! exec!?!?! The world is about to end!!!". Franky, that's a good default response. However, I argue that if you're copying a function that you plan to use later in the code, it is no more safe than using exec (after all, the function's code has already been executed).
I am learning Python and am trying to figure out the best way to structure my code.
Lets say I have a long function, and want to break it up into smaller functions. In C, I would make it a 'static' function at the top level (since that is the only level of functions). I would also probably forward declare it and place it after the now-shortened function that uses it.
Now for Python. In Python, I have the option to create a nested function. Since this new "inner" function is really only a piece of the larger function broken off for readability purposes, and only used by it, it sounds like it should be a nested function, but having this function inside the parent function causes the whole function to still be very long, since no code was actually moved out of it! And especially since the functions have to be fully coded before they are called, it means the actual short function is all the way down at the end of this pseudo-long function, making readability terrible!
What is considered good practice for situations like this?
How about placing the smaller functions in an own file and import that in your main function? You'd have something like:
def main_func():
from impl import a, b, c
a()
b()
c()
I think this approach leads to high readability: You see where the smaller functions come from in case you want to look into them, importing them is a one-liner, and the implementation of the main function is directly visible. By choosing an appropriate file name / location, you can also tell the user that these functions are not intended for use outside of main_func (you don't have real information hiding in Python anyway).
By the way: This question doesn't have one correct answer.
As far as I know, the main advantage of inner functions in Python is that they inherit the scope of the enclosing function. So if you need access to variables in the main function's scope (eg. argument or local variable), an inner function is the way to go. Otherwise, do whatever you like and/or find most readable.
EDIT:
See this answer too.
So what I could understand is that you have a long function like:
def long_func(blah, foo, *args):
...
...
my_val = long_func(foo, blah, a, b, c)
What you have done is:
def long_func(blah, foo, *args):
def short_func1():
...
def short_func2():
...
...
short_func1()
short_func2()
...
...
my_val = long_func(foo, blah, a, b, c)
You have lots more options, I'll list two:
Make it into a class
class SomeName(object):
def __init__(self, blah, foo, *args):
self.blah = blah
self.foo = foo
self.args = args
self.result = None # Might keep this for returning values or see (2)
def short_func1(self):
...
def short_func2(self):
...
def run(self): # name it as you like!
self.short_func1()
self.short_func2()
return self.result # (2) or return the last call, on you
...
my_val = SomeName(foo, blah, a, b, c).run()
Make another module and put the short_funcs into it. Just like flyx has suggested.
def long_func(foo, blah, *args):
from my_module import short_func1, short_func2
short_func1(foo)
short_func2(blah)
The good practice is to keep cycomatic complexity low. This practically means breaking your long function into many smaller functions.
The complexity is measured by the number of if, while, do, for, ?:,
catch, switch, case statements, and operators && and || (plus one) in
the body of a constructor, method, static initializer, or instance
initializer. It is a measure of the minimum number of possible paths
through the source and therefore the number of required tests.
Generally 1-4 is considered good, 5-7 ok, 8-10 consider re-factoring,
and 11+ re-factor now !
I suggest to take this advice, coming from Sonar, a code quality analysis tool. A good way to refactor such code is using TDD. First write unit tests to cover all the execution paths of your current function. After that you can refactor with the peace of mind that the unit tests will guarantee you didn't break anything.
If on the other hand your long function is just long, but otherwise already has a low cyclomatic complexity, then I think it doesn't matter much whether the function is nested or not.
I have a function in my Python script where multiple variables are created, and I want to use them in other functions. I thought of using global for this but I thought it would be the incorrect way to do so.
So can anyone tell me what would be the best way to create variables in a function for other functions?
Organize the variables into a class. Instantiate the class in one function and then pass the instance to wherever you need it.
Rule of thumb: If making something global seems like a good solution at some point, don't do it. There is always a better way.
You could create a "namespace" object -- an object which functions as a namespace for the purpose of keeping your global namespace clear:
class namespace():
pass
global_data=namespace()
def add_attributes(obj):
obj.foo="bar"
add_attributes(global_data)
print (global_data.foo) #prints "bar"
However, this is only marginally better than using the global keyword. You really do want a class here as mentioned by Paul.
Declare the variables as attributes of the function.
def f1():
f1.a=100
f2.b=200
def f2():
print(f1.a,f2.b)
f1()
f2()
output:
100 200
This might be a good place to implement a class. This has lots of advantages. See Classes in the tutorials.
Say I have a simple function:
def myfunc():
a = 4.2
b = 5.5
... many similar variables ...
I use this function one time only and I am wondering what is the easiest way to make all the variables inside the function accessible to my main name-space.
Do I have to declare global for each item? or any other suggested methods?
Thanks.
Best way, in my biased opinion, is to wrap the dictionary into a nice object where the "variables" are accessed as attributes -- the pattern I named Bunch when I introduced it many years ago, and a great example of the last item in the Zen of Python (if you don't know what that is, import this at an interpreter interactive prompt). To wit...:
class Bunch(object):
def __init__(self, d=None):
if d is not None: self.__dict__.update(d)
def myfunc():
a = 4.2
b = 5.5
...
return Bunch(locals())
x = myfunc()
print x.a, x.b
Using qualified names like x.a and x.b, rather than barenames such as a and b, is the crucial idea here: qualified names let you separate namespaces and treat them right, while barenames would make one big soup of everything and violate the Zen of Python;-). And of course there's no need to use an unwrapped dict and unsightly x['a'], x['b'] accesses!-)
If what you want is separation of your constants then put them in another module:
consts.py:
foo = 42
bar = 'quux'
main.py:
import consts
print consts.foo
print consts.bar[::-1]
I can't think of any good reason for doing this, but if you REALLY want to for some strange reason:
def myfunc():
x = 5.6
y = 7.3
# many other variables
globals().update( locals() )
This works because the globals function returns a dictionary of the global variables in your module's namespace, which you can dynamically update with the dictionary of all local variables returned by the locals function.
This is very dangerous because it will clobber everything in your global namespace, so if you already had an x or y variable, then they'll get overwritten. And as I said, I really can't think of a good reason to ever do this, so definitely think twice before resorting to this kind of trickery!
It sounds like what you want to do isn't natural because it isn't something you should generally do much.
Perhaps what you really want is to make a class and set all these things on an instance of it.
class MyState(object):
def __init__(self):
self.a = 4.2
self.b = 5.5
... many similar variables ...
Storing state on an instance of a class you create is usually very strongly preferred to mutating global state. This method also does not require any magic.