Develop Reference

Return a custom value when a class method is accessed as an attribute, but still allow for it to perform a computation when called?

Specifically, I would want MyClass.my_method to be used for lookup of a value in the class dictionary, but MyClass.my_method() to be a method that accepts arguments and performs a computation to update an attribute in MyClass and then returns MyClass with all its attributes (including the updated one).
I am thinking that this might be doable with Python's descriptors (maybe overriding __get__ or __call__), but I can't figure out how this would look. I understand that the behavior might be confusing, but I am interested if it is possible (and if there are any other major caveats).
I have seen that you can do something similar for classes and functions by overriding __repr__, but I can't find a similar way for a method within a class. My returned value will also not always be a string, which seems to prohibit the __repr__-based approaches mentioned in these two questions:
Possible to change a function's repr in python?
How to create a custom string representation for a class object?

Thank you Joel for the minimal implementation. I found that the remaining problem is the lack of initialization of the parent, since I did not find a generic way of initializing it, I need to check for attributes in the case of list/dict, and add the initialization values to the parent accordingly.
This addition to the code should make it work for lists/dicts:
def classFactory(parent, init_val, target):
class modifierClass(parent):
def __init__(self, init_val):
super().__init__()
dict_attr = getattr(parent, "update", None)
list_attr = getattr(parent, "extend", None)
if callable(dict_attr): # parent is dict
self.update(init_val)
elif callable(list_attr): # parent is list
self.extend(init_val)
self.target = target
def __call__(self, *args):
self.target.__init__(*args)
return modifierClass(init_val)
class myClass:
def __init__(self, init_val=''):
self.method = classFactory(init_val.__class__, init_val, self)
Unfortunately, we need to add case by case, but this works as intended.
A slightly less verbose way to write the above is the following:
def classFactory(parent, init_val, target):
class modifierClass(parent):
def __init__(self, init_val):
if isinstance(init_val, list):
self.extend(init_val)
elif isinstance(init_val, dict):
self.update(init_val)
self.target = target
def __call__(self, *args):
self.target.__init__(*args)
return modifierClass(init_val)
class myClass:
def __init__(self, init_val=''):
self.method = classFactory(init_val.__class__, init_val, self)

As jasonharper commented,
MyClass.my_method() works by looking up MyClass.my_method, and then attempting to call that object. So the result of MyClass.my_method cannot be a plain string, int, or other common data type [...]
The trouble comes specifically from reusing the same name for this two properties, which is very confusing just as you said. So, don't do it.
But for the sole interest of it you could try to proxy the value of the property with an object that would return the original MyClass instance when called, use an actual setter to perform any computation you wanted, and also forward arbitrary attributes to the proxied value.
class MyClass:
_my_method = whatever
#property
def my_method(self):
my_class = self
class Proxy:
def __init__(self, value):
self.__proxied = value
def __call__(self, value):
my_class.my_method = value
return my_class
def __getattr__(self, name):
return getattr(self.__proxied, name)
def __str__(self):
return str(self.__proxied)
def __repr__(self):
return repr(self.__proxied)
return Proxy(self._my_method)
#my_method.setter
def my_method(self, value):
# your computations
self._my_method = value
a = MyClass()
b = a.my_method('do not do this at home')
a is b
# True
a.my_method.split(' ')
# ['do', 'not', 'do', 'this', 'at', 'home']
And today, duck typing will abuse you, forcing you to delegate all kinds of magic methods to the proxied value in the proxy class, until the poor codebase where you want to inject this is satisfied with how those values quack.

This is a minimal implementation of Guillherme's answer that updates the method instead of a separate modifiable parameter:
def classFactory(parent, init_val, target):
class modifierClass(parent):
def __init__(self, init_val):
self.target = target
def __call__(self, *args):
self.target.__init__(*args)
return modifierClass(init_val)
class myClass:
def __init__(self, init_val=''):
self.method = classFactory(init_val.__class__, init_val, self)
This and the original answer both work well for single values, but it seems like lists and dictionaries are returned as empty instead of with the expected values and I am not sure why so help is appreciated here:

Dynamically creating properties of a class

Take a look at this code snippet:
class Face():
pass
class Cube():
def __init__(self):
self.faces = {
'front': Face(1),
...
}
#property
def front(self):
return self.faces['front']
#front.setter
def front(self, f):
pass
I've created getters and setters for all the faces. Is there any way to make this code more compact, maybe by dynamically creating the getters and setters?

The following code assumes that you
have a reason to have the self.faces dict instead of setting attributes like front directly on the instance
and/or want to implement some meaningful getter and setter logic for the keys in self.faces.
Otherwise, this exercise is pretty pointless because as Corentin Limier noted you can simply set self.front = Face(1), and so on.
You can use descriptors, a class variable holding the face names and a class decorator. Think of descriptors as reusable properties.
In the following sample code I added a num instance variable to Face and the face 'side' just for demonstration purposes.
class FaceDescriptor:
def __get__(self, instance, owner):
# your custom getter logic
# dummy implementation
if instance is not None:
return instance.faces[self.face]
def __set__(self, instance, value):
# your custom setter logic
# dummy implementation
instance.faces[self.face] = value
def set_faces(cls):
for face in cls._faces:
desc = FaceDescriptor()
desc.face = face
setattr(cls, face, desc)
return cls
class Face():
def __init__(self, num):
self.num = num
#set_faces
class Cube():
_faces = ['front', 'side']
def __init__(self):
self.faces = {face:Face(i) for i, face in enumerate(self._faces, 1)}
In action:
>>> c = Cube()
>>> c.front.num
1
>>> c.side.num
2
>>> c.front = 'stuff'
>>> c.front
'stuff'
>>> c.faces
{'front': 'stuff', 'side': <__main__.Face at 0x7fd0978f37f0>}

Assuming that's all your class does, you could do something like
class Cube:
...
def __getattr__(self, name):
return self.faces[name]
def __setattr__(self, name, value):
self.faces[name] = value

if you really want to do that you could use __getattr__ and __setattr__:
class Cube:
...
def __getattr__(self, item):
return self.faces[item]
def __setattr__(self, item, value):
self.faces[item] = value
but as you set front in the __init__ methoud you could just as well make it a regular member...

Your code is redundant, since instance attributes are already stored in a dictionary which is the __dict__ property. I recognize that you are focused on writing your code in fewer lines. It is a good challenge to keep yourself growing, but in the long term you should be focused on the clarity of your code instead.
Here is a simpler way to write your code without using properties:
class Face():
pass
class Cube():
def __init__(self):
self.front = Face()
self.rear = Face()
It is a tenet of encapsulation that you should hide your "attributes" behind "properties". Even though this isn't strongly enforced in python, it's not a bad idea to do that. Here's the proper way to do that:
class Face():
pass
class Cube():
def __init__(self):
self._front = Face()
#property
def front(self):
return self._front
#front.setter
def front(self, value):
self._front = value
To answer your question at the end, yes you can dynamically create properties.
https://stackoverflow.com/a/1355444/3368572
But keep in mind that writing dynamic code should be reserved for special cases since it will make it more difficult for your IDE to follow the flow of your program. If you use the conventions as they are intended then your code becomes self-explanatory to people and to your IDE.

Ruby like DSL in Python

I'm currently writing my first bigger project in Python, and I'm now wondering how to define a class method so that you can execute it in the class body of a subclass of the class.
First to give some more context, a slacked down (I removed everything non essential for this question) example of how I'd do the thing I'm trying to do in Ruby:
If I define a class Item like this:
class Item
def initialize(data={})
#data = data
end
def self.define_field(name)
define_method("#{name}"){ instance_variable_get("#data")[name.to_s] }
define_method("#{name}=") do |value|
instance_variable_get("#data")[name.to_s] = value
end
end
end
I can use it like this:
class MyItem < Item
define_field("name")
end
item = MyItem.new
item.name = "World"
puts "Hello #{item.name}!"
Now so far I tried achieving something similar in Python, but I'm not happy with the result I've got so far:
class ItemField(object):
def __init__(self, name):
self.name = name
def __get__(self, item, owner=None):
return item.values[self.name]
def __set__(self, item, value):
item.values[self.name] = value
def __delete__(self, item):
del item.values[self.name]
class Item(object):
def __init__(self, data=None):
if data == None: data = {}
self.values = data
for field in type(self).fields:
self.values[field.name] = None
setattr(self, field.name, field)
#classmethod
def define_field(cls, name):
if not hasattr(cls, "fields"): cls.fields = []
cls.fields.append(ItemField(name, default))
Now I don't know how I can call define_field from withing a subclass's body. This is what I wished that it was possible:
class MyItem(Item):
define_field("name")
item = MyItem({"name": "World"})
puts "Hello {}!".format(item.name)
item.name = "reader"
puts "Hello {}!".format(item.name)
There's this similar question but none of the answers are really satisfying, somebody recommends caling the function with __func__() but I guess I can't do that, because I can't get a reference to the class from within its anonymous body (please correct me if I'm wrong about this.)
Somebody else pointed out that it's better to use a module level function for doing this which I also think would be the easiest way, however the main intention of me doing this is to make the implementation of subclasses clean and having to load that module function wouldn't be to nice either. (Also I'd have to do the function call outside the class body and I don't know but I think this is messy.)
So basically I think my approach is wrong, because Python wasn't designed to allow this kind of thing to be done. What would be the best way to achieve something as in the Ruby example with Python?
(If there's no better way I've already thought about just having a method in the subclass which returns an array of the parameters for the define_field method.)

Perhaps calling a class method isn't the right route here. I'm not quite up to speed on exactly how and when Python creates classes, but my guess is that the class object doesn't yet exist when you'd call the class method to create an attribute.
It looks like you want to create something like a record. First, note that Python allows you to add attributes to your user-created classes after creation:
class Foo(object):
pass
>>> foo = Foo()
>>> foo.x = 42
>>> foo.x
42
Maybe you want to constrain which attributes the user can set. Here's one way.
class Item(object):
def __init__(self):
if type(self) is Item:
raise NotImplementedError("Item must be subclassed.")
def __setattr__(self, name, value):
if name not in self.fields:
raise AttributeError("Invalid attribute name.")
else:
self.__dict__[name] = value
class MyItem(Item):
fields = ("foo", "bar", "baz")
So that:
>>> m = MyItem()
>>> m.foo = 42 # works
>>> m.bar = "hello" # works
>>> m.test = 12 # raises AttributeError
Lastly, the above allows you the user subclass Item without defining fields, like such:
class MyItem(Item):
pass
This will result in a cryptic attribute error saying that the attribute fields could not be found. You can require that the fields attribute be defined at the time of class creation by using metaclasses. Furthermore, you can abstract away the need for the user to specify the metaclass by inheriting from a superclass that you've written to use the metaclass:
class ItemMetaclass(type):
def __new__(cls, clsname, bases, dct):
if "fields" not in dct:
raise TypeError("Subclass must define 'fields'.")
return type.__new__(cls, clsname, bases, dct)
class Item(object):
__metaclass__ = ItemMetaclass
fields = None
def __init__(self):
if type(self) == Item:
raise NotImplementedError("Must subclass Type.")
def __setattr__(self, name, value):
if name in self.fields:
self.__dict__[name] = value
else:
raise AttributeError("The item has no such attribute.")
class MyItem(Item):
fields = ("one", "two", "three")

You're almost there! If I understand you correctly:
class Item(object):
def __init__(self, data=None):
fields = data or {}
for field, value in data.items():
if hasattr(self, field):
setattr(self, field, value)
#classmethod
def define_field(cls, name):
setattr(cls, name, None)
EDIT: As far as I know, it's not possible to access the class being defined while defining it. You can however call the method on the __init__ method:
class Something(Item):
def __init__(self):
type(self).define_field("name")
But then you're just reinventing the wheel.

When defining a class, you cannot reference the class itself inside its own definition block. So you have to call define_field(...) on MyItem after its definition. E.g.,
class MyItem(Item):
pass
MyItem.define_field("name")
item = MyItem({"name": "World"})
print("Hello {}!".format(item.name))
item.name = "reader"
print("Hello {}!".format(item.name))

Python: showing attributes assigned to a class object in the class code

One of my classes does a lot of aggregate calculating on a collection of objects, then assigns an attribute and value appropriate to the specific object: I.e.
class Team(object):
def __init__(self, name): # updated for typo in code, added self
self.name = name
class LeagueDetails(object):
def __init__(self): # added for clarity, corrected another typo
self.team_list = [Team('name'), ...]
self.calculate_league_standings() # added for clarity
def calculate_league_standings(self):
# calculate standings as a team_place_dict
for team in self.team_list:
team.place = team_place_dict[team.name] # a new team attribute
I know, as long as the calculate_league_standings has been run, every team has team.place. What I would like to be able to do is to scan the code for class Team(object) and read all the attributes, both created by class methods and also created by external methods which operate on class objects. I am getting a little sick of typing for p in dir(team): print p just to see what the attribute names are. I could define a bunch of blank attributes in the Team __init__. E.g.
class Team(object):
def __init__(self, name): # updated for typo in code, added self
self.name = name
self.place = None # dummy attribute, but recognizable when the code is scanned
It seems redundant to have calculate_league_standings return team._place and then add
#property
def place(self): return self._place
I know I could comment a list of attributes at the top class Team, which is the obvious solution, but I feel like there has to be a best practice here, something pythonic and elegant here.

If I half understand your question, you want to keep track of which attributes of an instance have been added after initialization. If this is the case, you could use something like this:
#! /usr/bin/python3.2
def trackable (cls):
cls._tracked = {}
oSetter = cls.__setattr__
def setter (self, k, v):
try: self.initialized
except: return oSetter (self, k, v)
try: self.k
except:
if not self in self.__class__._tracked:
self.__class__._tracked [self] = []
self.__class__._tracked [self].append (k)
return oSetter (self, k, v)
cls.__setattr__ = setter
oInit = cls.__init__
def init (self, *args, **kwargs):
o = oInit (self, *args, **kwargs)
self.initialized = 42
return o
cls.__init__ = init
oGetter = cls.__getattribute__
def getter (self, k):
if k == 'tracked': return self.__class__._tracked [self]
return oGetter (self, k)
cls.__getattribute__ = getter
return cls
#trackable
class Team:
def __init__ (self, name, region):
self.name = name
self.region = region
#set name and region during initialization
t = Team ('A', 'EU')
#set rank and ELO outside (hence trackable)
#in your "aggregate" functions
t.rank = 4 # a new team attribute
t.ELO = 14 # a new team attribute
#see witch attributes have been created after initialization
print (t.tracked)
If I did not understand the question, please do specify which part I got wrong.

Due to Python's dynamic nature, I don't believe there is a general answer to your question. An attribute of an instance can be set in many ways, including pure assignment, setattr(), and writes to __dict__ . Writing a tool to statically analyze Python code and correctly determine all possible attributes of an class by analyzing all these methods would be very difficult.
In your specific case, as the programmer you know that class Team will have a place attribute in many instances, so you can decide to be explicit and write its constructor like so:
class Team(object):
def __init__(name ,place=None):
self.name = name
self.place = place
I would say there is no need to define a property of a simple attribute, unless you wanted side effects or derivations to happen at read or write time.

OO design: an object that can be exported to a "row", while accessing header names, without repeating myself

Sorry, badly worded title. I hope a simple example will make it clear. Here's the easiest way to do what I want to do:
class Lemon(object):
headers = ['ripeness', 'colour', 'juiciness', 'seeds?']
def to_row(self):
return [self.ripeness, self.colour, self.juiciness, self.seeds > 0]
def save_lemons(lemonset):
f = open('lemons.csv', 'w')
out = csv.writer(f)
out.write(Lemon.headers)
for lemon in lemonset:
out.writerow(lemon.to_row())
This works alright for this small example, but I feel like I'm "repeating myself" in the Lemon class. And in the actual code I'm trying to write (where the number of variables I'm exporting is ~50 rather than 4, and where to_row calls a number of private methods that do a bunch of weird calculations), it becomes awkward.
As I write the code to generate a row, I need to constantly refer to the "headers" variable to make sure I'm building my list in the correct order. If I want to change the variables being outputted, I need to make sure to_row and headers are being changed in parallel (exactly the kind of thing that DRY is meant to prevent, right?).
Is there a better way I could design this code? I've been playing with function decorators, but nothing has stuck. Ideally I should still be able to get at the headers without having a particular lemon instance (i.e. it should be a class variable or class method), and I don't want to have a separate method for each variable.

In this case, getattr() is your friend: it allows you to get a variable based on a string name. For example:
def to_row(self):
return [getattr(self, head) for head in self.headers]
EDIT: to properly use the header seeds?, you would need to set the attribute seeds? for the objects. setattr(self, 'seeds?', self.seeds > 0) right above the return statement.

We could use some metaclass shenanegans to do this...
In python 2, attributes are passed to the metaclass in a dict, without
preserving order, we'll also want a base class to work with so we can
distinguish class attributes that should be mapped into the row. In python3, we could dispense with just about all of this base descriptor class.
import itertools
import functools
#functools.total_ordering
class DryDescriptor(object):
_order_gen = itertools.count()
def __init__(self, alias=None):
self.alias = alias
self.order = next(self._order_gen)
def __lt__(self, other):
return self.order < other.order
We will want a python descriptor for every attribute we wish to map into the
row. slots are a nice way to get data descriptors without much work. One
caveat, though, we'll have to manually remove the helper instance to make the
real slot descriptor visible.
class slot(DryDescriptor):
def annotate(self, attr, attrs):
del attrs[attr]
self.attr = attr
slots = attrs.setdefault('__slots__', []).append(attr)
def annotate_class(self, cls):
if self.alias is not None:
setattr(cls, self.alias, getattr(self.attr))
For computed fields, we can memoize results. Memoizing off of the annotated
instance is tricky without a memory leak, we need weakref. alternatively, we
could have arranged for another slot just to store the cached value. This also isn't quite thread safe, but pretty close.
import weakref
class memo(DryDescriptor):
_memo = None
def __call__(self, method):
self.getter = method
return self
def annotate(self, attr, attrs):
if self.alias is not None:
attrs[self.alias] = self
def annotate_class(self, cls): pass
def __get__(self, instance, owner):
if instance is None:
return self
if self._memo is None:
self._memo = weakref.WeakKeyDictionary()
try:
return self._memo[instance]
except KeyError:
return self._memo.setdefault(instance, self.getter(instance))
On the metaclass, all of the descriptors we created above are found, sorted by
creation order, and instructed to annotate the new, created class. This does
not correctly treat derived classes and could use some other conveniences like
an __init__ for all the slots.
class DryMeta(type):
def __new__(mcls, name, bases, attrs):
descriptors = sorted((value, key)
for key, value
in attrs.iteritems()
if isinstance(value, DryDescriptor))
for descriptor, attr in descriptors:
descriptor.annotate(attr, attrs)
cls = type.__new__(mcls, name, bases, attrs)
for descriptor, attr in descriptors:
descriptor.annotate_class(cls)
cls._header_descriptors = [getattr(cls, attr) for descriptor, attr in descriptors]
return cls
Finally, we want a base class to inherit from so that we can have a to_row
method. this just invokes all of the __get__s for all of the respective
descriptors, in order.
class DryBase(object):
__metaclass__ = DryMeta
def to_row(self):
cls = type(self)
return [desc.__get__(self, cls) for desc in cls._header_descriptors]
Assuming all of that is tucked away, out of sight, the definition of a class
that uses this feature is mostly free of repitition. The only short coming is
that to be practical, every field needs a python friendly name, thus we had the
alias key to associate 'seeds?' to has_seeds
class ADryRow(DryBase):
__slots__ = ['seeds']
ripeness = slot()
colour = slot()
juiciness = slot()
#memo(alias='seeds?')
def has_seeds(self):
print "Expensive!!!"
return self.seeds > 0
>>> my_row = ADryRow()
>>> my_row.ripeness = "tart"
>>> my_row.colour = "#8C2"
>>> my_row.juiciness = 0.3479
>>> my_row.seeds = 19
>>>
>>> print my_row.to_row()
Expensive!!!
['tart', '#8C2', 0.3479, True]
>>> print my_row.to_row()
['tart', '#8C2', 0.3479, True]