Develop Reference

What happens when we edit(append, remove...) a list and can we execute actions each time a list is edited

I would like to know if there is a way to create a list that will execute some actions each time I use the method append(or an other similar method).
I know that I could create a class that inherits from list and overwrite append, remove and all other methods that change content of list but I would like to know if there is an other way.
By comparison, if I want to print 'edited' each time I edit an attribute of an object I will not execute print("edited") in all methods of the class of that object. Instead, I will only overwrite __setattribute__.
I tried to create my own type which inherits of list and overwrite __setattribute__ but that doesn't work. When I use myList.append __setattribute__ isn't called. I would like to know what's realy occured when I use myList.append ? Is there some magic methods called that I could overwrite ?
I know that the question have already been asked there : What happens when you call `append` on a list?. The answer given is just, there is no answer... I hope it's a mistake.
I don't know if there is an answer to my request so I will also explain you why I'm confronted to that problem. Maybe I can search in an other direction to do what I want. I have got a class with several attributes. When an attribute is edited, I want to execute some actions. Like I explain before, to do this I am use to overwrite __setattribute__. That works fine for most of attributes. The problem is lists. If the attribute is used like this : myClass.myListAttr.append(something), __setattribute__ isn't called while the value of the attribute have changed.
The problem would be the same with dictionaries. Methods like pop doesn't call __setattribute__.

If I understand correctly, you would want something like Notify_list that would call some method (argument to the constructor in my implementation) every time a mutating method is called, so you could do something like this:
class Test:
def __init__(self):
self.list = Notify_list(self.list_changed)
def list_changed(self,method):
print("self.list.{} was called!".format(method))
>>> x = Test()
>>> x.list.append(5)
self.list.append was called!
>>> x.list.extend([1,2,3,4])
self.list.extend was called!
>>> x.list[1] = 6
self.list.__setitem__ was called!
>>> x.list
[5, 6, 2, 3, 4]
The most simple implementation of this would be to create a subclass and override every mutating method:
class Notifying_list(list):
__slots__ = ("notify",)
def __init__(self,notifying_method, *args,**kw):
self.notify = notifying_method
list.__init__(self,*args,**kw)
def append(self,*args,**kw):
self.notify("append")
return list.append(self,*args,**kw)
#etc.
This is obviously not very practical, writing the entire definition would be very tedious and very repetitive, so we can create the new subclass dynamically for any given class with functions like the following:
import functools
import types
def notify_wrapper(name,method):
"""wraps a method to call self.notify(name) when called
used by notifying_type"""
#functools.wraps(method)
def wrapper(*args,**kw):
self = args[0]
# use object.__getattribute__ instead of self.notify in
# case __getattribute__ is one of the notifying methods
# in which case self.notify will raise a RecursionError
notify = object.__getattribute__(self, "_Notify__notify")
# I'd think knowing which method was called would be useful
# you may want to change the arguments to the notify method
notify(name)
return method(*args,**kw)
return wrapper
def notifying_type(cls, notifying_methods="all"):
"""creates a subclass of cls that adds an extra function call when calling certain methods
The constructor of the subclass will take a callable as the first argument
and arguments for the original class constructor after that.
The callable will be called every time any of the methods specified in notifying_methods
is called on the object, it is passed the name of the method as the only argument
if notifying_methods is left to the special value 'all' then this uses the function
get_all_possible_method_names to create wrappers for nearly all methods."""
if notifying_methods == "all":
notifying_methods = get_all_possible_method_names(cls)
def init_for_new_cls(self,notify_method,*args,**kw):
self._Notify__notify = notify_method
namespace = {"__init__":init_for_new_cls,
"__slots__":("_Notify__notify",)}
for name in notifying_methods:
method = getattr(cls,name) #if this raises an error then you are trying to wrap a method that doesn't exist
namespace[name] = notify_wrapper(name, method)
# I figured using the type() constructor was easier then using a meta class.
return type("Notify_"+cls.__name__, (cls,), namespace)
unbound_method_or_descriptor = ( types.FunctionType,
type(list.append), #method_descriptor, not in types
type(list.__add__),#method_wrapper, also not in types
)
def get_all_possible_method_names(cls):
"""generates the names of nearly all methods the given class defines
three methods are blacklisted: __init__, __new__, and __getattribute__ for these reasons:
__init__ conflicts with the one defined in notifying_type
__new__ will not be called with a initialized instance, so there will not be a notify method to use
__getattribute__ is fine to override, just really annoying in most cases.
Note that this function may not work correctly in all cases
it was only tested with very simple classes and the builtin list."""
blacklist = ("__init__","__new__","__getattribute__")
for name,attr in vars(cls).items():
if (name not in blacklist and
isinstance(attr, unbound_method_or_descriptor)):
yield name
Once we can use notifying_type creating Notify_list or Notify_dict would be as simple as:
import collections
mutating_list_methods = set(dir(collections.MutableSequence)) - set(dir(collections.Sequence))
Notify_list = notifying_type(list, mutating_list_methods)
mutating_dict_methods = set(dir(collections.MutableMapping)) - set(dir(collections.Mapping))
Notify_dict = notifying_type(dict, mutating_dict_methods)
I have not tested this extensively and it quite possibly contains bugs / unhandled corner cases but I do know it worked correctly with list!

Check whether class variable has been defined

I am dealing with a scenario where I have a python class Foo. Foo, among other things does many big calculations, which I would not do unless required. So, when I define the getter method for one of those big calculations, how do I make sure that the method corresponding to the calculation (here bigcalculation()) has already run?
class Foo:
def __init__(self):
#initialize some stuff
def bigcalculation(self):
#perform calculation
self.big_calc_result=[big list of numbers];
def get_big_calc_result(self):
if hasattr(self,'big_calc_result')==False:
self.bigcalculations();
return sef.big_calc_result;
If its run once, I don't want it to run again. and I don't want caller to have to keep track of whether it has run once or not.
Now, I do it using hasattr() function as above, but I think this is a really ugly way to do it. Is there a more elegant pythonic way to do it?
An alternative I can think of, is to define, in my init() function, all the variables that I would ever use in the class, as empty list. Then check whether big_calc_result is an empty list or not to determine if self.bigcalculation() has already run. Is this a better approach?
related question:Python lets me define variables on the fly in a class. But is that bad programming practice?
Edit: In retrospect, I also found that using exceptions can also be another way of handling this situation. That might be a more pythonic way of doing things.
def get_big_calc_result(self):
try:
return self.big_calc_result;
except AttributeError:
self.bigcalculations();
return self.big_calc_result;
The answers to this question are useful:
Checking for member existence in Python

You can memoize the result and store it as a property:
class Foo(object):
def __init__(self):
self._big_calc_result = None
#property
def big_calc_result(self):
if self._big_calc_result is not None:
return self._big_calc_result
else:
self._big_calc_result = self.big_calc_run()
return self._big_calc_result
def big_calc_run(self):
time.sleep(10) # Takes a long time ...
Now, you just initialize the class and get the result:
f = Foo()
x = f.big_calc_result
y = f.bic_calc_result # Look mom, this happened really quick
Of course, you don't have to use a property if that is less intuitive and you can change things around here to suit the API you're trying to provide. The real meat is in caching the result in the variable prefixed with an underscore which is to say "This is an implementation detail -- if you mess with it, you deserve to have your code break at some point in the future".

How to know which next attribute is requested in python

I have class with custom getter, so I have situations when I need to use my custom getter, and situations when I need to use default.
So consider following.
If I call method of object c in this way:
c.somePyClassProp
In that case I need to call custom getter, and getter will return int value, not Python object.
But if I call method on this way:
c.somePyClassProp.getAttributes()
In this case I need to use default setter, and first return need to be Python object, and then we need to call getAttributes method of returned python object (from c.somePyClassProp).
Note that somePyClassProp is actually property of class which is another Python class instance.
So, is there any way in Python on which we can know whether some other methods will be called after first method call?

No. c.someMethod is a self-contained expression; its evaluation cannot be influenced by the context in which the result will be used. If it were possible to achieve what you want, this would be the result:
x = c.someMethod
c.someMethod.getAttributes() # Works!
x.getAttributes() # AttributeError!
This would be confusing as hell.
Don't try to make c.someMethod behave differently depending on what will be done with it, and if possible, don't make c.someMethod a method call at all. People will expect c.someMethod to return a bound method object that can then be called to execute the method; just define the method the usual way and call it with c.someMethod().

You don't want to return different values based on which attribute is accessed next, you want to return an int-like object that also has the required attribute on it. To do this, we create a subclass of int that has a getAttributes() method. An instance of this class, of course, needs to know what object it is "bound" to, that is, what object its getAttributes() method should refer to, so we'll add this to the constructor.
class bound_int(int):
def __new__(cls, value, obj):
val = int.__new__(cls, value)
val.obj = obj
return val
def getAttributes(self):
return self.obj.somePyClassProp
Now in your getter for c.somePyClassProp, instead of returning an integer, you return a bound_int and pass it a reference to the object its getAttributes() method needs to know about (here I'll just have it refer to self, the object it's being returned from):
#property
def somePyClassProp(self):
return bound_int(42, self)
This way, if you use c.somePyPclassProp as an int, it acts just like any other int, because it is one, but if you want to further call getAttributes() on it, you can do that, too. It's the same value in both cases; it just has been built to fulfill both purposes. This approach can be adapted to pretty much any problem of this type.

It looks like you want two ways to get the property depending on what you want to do with it. I don't think there's any inherent Pythonic way to implement this, and you therefore need to store a variable or property name for each case. Maybe:
c.somePyClassProp
can be used in the __get__ and
c.somePyClassProp__getAttributes()
can be implemented in a more custom way inside the __getattribute__ function.
One way I've used (which is probably not the best) is to check for that exact variable name:
def __getattribute__(self, var_name):
if ('__' in var_name):
var_name, method = var_name.split('__')
return object.__getattribute__(self, var_name).__getattribute__(method)
Using object.__get__(self, var_name) uses the object class's method of getting a property directly.

You can store the contained python object as a variable and the create getters via the #property dectorator for whatever values you want. When you want to read the int, reference the property. When you want the contained object, use its variable name instead.
class SomePyClass(object):
def getInt(self):
return 1
def getAttributes(self):
return 'a b c'
class MyClass(object):
def __init__(self, py_class):
self._py_class = py_class
#property
def some_property(self):
return self._py_class.getInt()
x = MyClass(SomePyClass())
y = self.some_property
x._py_class.getAttributes()

What happens when you call `append` on a list?

Firstly, I don't know what the most appropriate title for this question would be. Contender: "how to implement list.append in custom class".
I have a class called Individual. Here's the relevant part of the class:
from itertools import count
class Individual:
ID = count()
def __init__(self, chromosomes):
self.chromosomes = list(chromosomes)
self.id = self.ID.next()
Here's what I want to do with this class:
Suppose I instantiate a new individual with no chromosomes: indiv = Individual([]) and I want to add a chromosome to this individual later on. Currently, I'd have to do:
indiv.chromosomes.append(makeChromosome(params))
Instead, what I would ideally like to do is:
indiv.append(makeChromosome(params))
with the same effect.
So my question is this: when I call append on a list, what really happens under the hood? Is there an __append__ (or __foo__) that gets called? Would implementing that function in my Individual class get me the desired behavior?
I know for instance, that I can implement __contains__ in Individual to enable if foo in indiv functionality. How would I go about enable indiv.append(…) functionality?

.append() is simply a method that takes one argument, and you can easily define one yourself:
def append(self, newitem):
self.chromosomes.append(newitem)
No magic methods required.

Given a class type how do I create an instance in Python?

Let's say I have this :
class whatever(object):
def __init__(self):
pass
and this function:
def create_object(type_name):
# create an object of type_name
I'd like to be able to call the create_object like this:
inst = create_object(whatever)
and get back an instance of whatever. I think this should be doable without using eval, I'd like to know how to do this. Please notice that I'm NOT using a string as a parameter for create_object.

The most obvious way:
def create_object(type_name):
return type_name()

def create_object(typeobject):
return typeobject()
As you so explicitly say that the arg to create_object is NOT meant to be a string, I assume it's meant to be the type object itself, just like in the create_object(whatever) example you give, in which whatever is indeed the type itself.

If I understand correctly, what you want is:
def create_object(type_name, *args):
# create an object of type_name
return type_name(*args)
inst = create_object(whatever)
I don't really know why you want to do this, but would be interesting to hear from you what are your reasons to need such a construct.

def create_object(type_name):
return type_name()
you can of course skip the function altogether and create the instance of whatever like this:
inst = whatever()