Develop Reference

Dynamically add method to class from property function?

I think a code sample will better speak for itself:
class SomeClass:
example = create_get_method()
Yes, that's all – ideally.
In that case, create_get_method would add a get_example() to SomeClass in a way that it can be accessed via an instance of SomeClass:
obj = SomeClass()
obj.get_example() <- returns the value of self.example
(Of course, the idea is to implement a complex version of get_contact, that's why I want to do that in a non-repetitive way, and this is a simplified version that represents well the issue.)
I don't know if that's possible, because it require to have access to the property name (example) and the class (SomeClass) since these can not be guessed in advance (that function will be used on many and various classes).
I know it's something possible, because that's kind of what SQLAlchemy does with their relationship() function on a class:
class Model(BaseModel):
id = ...
contact_id = db.Integer(db.ForeignKey..)
contact = relationship('contact') <-- This !
How can this be done?

Objects bound to class-level variables can have a __set_name__ method that will be called immediately after the class object has been created. It will be called with two arguments, the class object, and the name of the variable the object is saved as in the class.
You could use this to create your extra getter method, though I'm not sure why exactly you want to (you could make the object a descriptor instead, which would probably be better than adding a separate getter function to the parent class).
class create_get_method:
def __set_name__(self, owner, name):
def getter(self):
return getattr(self, name)
getter_name = f"get_{name}"
getter.__name__ = getter_name
setattr(owner, getter_name, getter)
# you might also want a __get__ method here to give a default value (like None)
Here's how that would work:
>>> class Test:
... example = create_get_method()
...
>>> t = Test()
>>> print(t.get_example())
<__main__.create_get_method at 0x000001E0B4D41400>
>>> t.example = "foo"
>>> print(t.get_example())
foo
You could change the value returned by default (in the first print call), so that the create_get_method object isn't as exposed. Just add a __get__ method to the create_get_method class.

You can do this with a custom non-data descriptor, like a property, except that you don't need a __set__ method:
class ComplicatedDescriptor:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
Now you can have any number of classes that use this descriptor:
class SomeClass:
example = ComplicatedDescriptor('example')
Now you can do something like:
>>> inst0 = SomeClass()
>>> inst1 = SomeClass()
>>> inst0.example
1
>>> inst1.example
1
>>> inst1.example
2
>>> inst0.example
2
The line name = f'_{self.name} is necessary because the descriptor here is a non-data descriptor: it has no __set__ method, so if you create inst0.__dict__['example'], the lookup will no longer happen: inst0.example will return inst0.__dict__['example'] instead of calling SomeClass.example.__get__(inst0, type(inst0)). One workaround is to store the value under the attribute name _example. The other is to make your descriptor into a data descriptor:
class ComplicatedDescriptor_v2:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
# These two lines for demo only
value = owner.__dict__.get(self.name, 0)
value += 1
owner.__dict__[self.name] = value
return value
def __set__(self, *args):
raise AttributeError(f'{self.name} is a read-only attribute')
The usage is generally identical:
class SomeClass:
example = ComplicatedDescriptor_v2('example')
Except that now you can't accidentally override your attribute:
>>> inst = SomeClass()
>>> inst.example
1
>>> inst.example
2
>>> inst.example = 0
AttributeError: example is a read-only attribute
Descriptors are a fairly idiomatic way to get and set values in python. They are preferred to getters and setters in almost all cases. The simplest cases are handled by the built-in property. That being said, if you wanted to explicitly have a getter method, I would recommend doing something very similar, but just returning a method instead of calling __get__ directly.
For example:
def __get__(self, owner, type):
def enclosed():
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
return enclosed
There is really no point to doing something like this unless you plan on really just want to be able to call inst.example().

When and why to use self.__dict__ instead of self.variable

I'm trying to understand some code which is using this class below:
class Base(object):
def __init__(self, **kwargs):
self.client = kwargs.get('client')
self.request = kwargs.get('request')
...
def to_dict(self):
data = dict()
for key in iter(self.__dict__): # <------------------------ this
if key in ('client', 'request'):
continue
value = self.__dict__[key]
if value is not None:
if hasattr(value, 'to_dict'):
data[key] = value.to_dict()
else:
data[key] = value
return data
I understand that it gets keyword arguments passed to the Base class like for example, Base(client="foo", request="bar").
My confusion is, why is it using self.__dict__ which turns variables inside __init__ to a dict (e.g {"client": "foo", "request": "bar"}) instead of just calling them by self.client & self.request inside other methods? When and why I should use self.__dict__ instead?

Almost all of the time, you shouldn't use self.__dict__.
If you're accessing an attribute like self.client, i.e. the attribute name is known and fixed, then the only difference between that and self.__dict__['client'] is that the latter won't look up the attribute on the class if it's missing on the instance. There is very rarely any reason to do this, but the difference is demonstrated below:
>>> class A:
... b = 3 # class attribute, not an instance attribute
...
>>> A.b # the class has this attribute
3
>>> a = A()
>>> a.b # the instance doesn't have this attribute, fallback to the class
3
>>> a.__dict__['b'] # the instance doesn't have this attribute, but no fallback
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'b'
The main use-case for self.__dict__ is when you don't want to access a fixed, known attribute name. In almost all code, you always know which attribute you want to access; and if you do need to look something up dynamically using an unknown string, you should create a dictionary yourself, and write self.that_dict[key] instead of self.__dict__[key].
So the only times you should really use __dict__ is when you are writing code which needs to work regardless of which attributes the instance might have; i.e. you specifically want code which will work even if you change the class's structure or its attribute names, or code which will work across multiple classes with different structures. I'll show one example below.
The __repr__ method
The __repr__ method is meant to return a string representing the instance, for the programmer's convenience when using a REPL. For debugging/testing purposes this string usually contains information about the object's state. Here's a common way to implement it:
class Foo:
def __init__(self, foo, bar, baz):
self.foo = foo
self.bar = bar
self.baz = baz
def __repr__(self):
return 'Foo({!r}, {!r}, {!r})'.format(self.foo, self.bar, self.baz)
This means if you write obj = Foo(1, 'y', True) to create an instance, then repr(obj) will be the string "Foo(1, 'y', True)", which is convenient because it shows the instance's entire state, and also the string itself is Python code which creates an instance with the same state.
But there are a few issues with the above implementation: we have to change it if the class's attributes change, it won't give useful results for instances of subclasses, and we have to write lots of similar code for different classes with different attributes. If we use __dict__ instead, we can solve all of those problems:
def __repr__(self):
return '{}({})'.format(
self.__class__.__name__,
', '.join('{}={!r}'.format(k, v) for k, v in self.__dict__.items())
)
Now repr(obj) will be Foo(foo=1, bar='y', baz=True), which also shows the instance's entire state, and is also executable Python code. This generalised __repr__ method will still work if the structure of Foo changes, it can be shared between multiple classes via inheritance, and it returns executable Python code for any class whose attributes are accepted as keyword arguments by __init__.

__dict__ holds all of the variables in the class. Take the following class:
class A():
def __init__(self, foo):
self.foo = foo
def new_var(self, bar):
self.bar = bar
Then in this case, notice:
a = A('var1')
print(a.__dict__) # {'foo': 'var1'}
b = A('var1')
b.new_var('var2')
b.foobar = 'var3'
print(b.__dict__) # {'foo': 'var1', 'bar': 'var2', 'foobar': 'var3'}
In your case you could do either or. __dict__ is a great way to grab all of the variables that are part of that class at the current instance in which it is called. You can check out the documentation on __dict__ here.

__dict__ is used when checking what instance variables(data attributes) an object has.
So, if there is Person class below:
class Person:
x1 = "Hello"
x2 = "World"
def __init__(self, name, age):
self.name = name
self.age = age
def test1(self):
print(self.__dict__) # Here
#classmethod
def test2(cls):
pass
#staticmethod
def test3():
pass
obj = Person("John", 27)
obj.test1() # Here
__dict__ gets name and age with their values in a dictionary as shown below:
{'name': 'John', 'age': 27} # Here
And, if the new instance variable gender is added after instanciation as shown below:
# ...
obj= Person("John", 27)
obj.test1()
obj.gender = "Male" # Here
obj.test1()
__dict__ gets name, age and gender with their values in a dictionary as shown below:
{'name': 'John', 'age': 27}
{'name': 'John', 'age': 27, 'gender': 'Male'} # Here

Initialize an object built from a class with a specific dictionary

The goal is to build a class with an __init__ method that will allow to create objects which will be initialized with a specific dictionary.
Details:
A file mydicts.py is a collection of various dictionaries.
The object built from the class that I intent to create will be initialized to one of the dictionaries from mydicts.py.
mydicts.py example:
dict1 = {k1:v1, k2:v2,...,kn:vn}
dict2 = {k1:v1, k2:v2,...,km:vm}
etc.
My attempts:
class Example(dict):
def __init__(self, dictx):
self.dictx = getattr(__import__("mydicts", fromlist=['m']), dictx)
Results:
e = Example('dict1')
print e
{} # <- got empty dictionary. Want {k1:v1, k2:v2,...,kn:vn}
The goal is to create objects such that:
a = Example(dict1)
print a
# {k1:v1, k2:v2,...,kn:vn}
b = Example(dict2)
print b
# {k1:v1, k2:v2,...,km:vm}

Since you did not define a custom __str__ method for your class, print is calling the __str__ method of the parent class dict. Moreover, this is causing Python to print the empty dictionary created by the parent class in dict.__new__ instead of the dictionary that you have stored in self.dictx.
Because your class inherits from dict, you should be passing the dictionary returned by getattr to the __init__ method of the parent class. You can use super for this:
class Example(dict):
def __init__(self, dictx):
dct = getattr(__import__("__main__", fromlist=['m']), dictx)
super(Example, self).__init__(dct)
This will initialize the parent class with data taken from dct. In other words, the empty dictionary has been replaced with the dictionary returned by getattr. So, when print calls the parent's __str__ method, the correct output will be given:
>>> dict1 = {'k1':'v1', 'k2':'v2', 'kn':'vn'}
>>> class Example(dict):
... def __init__(self, dictx):
... dct = getattr(__import__("__main__", fromlist=['m']), dictx)
... super(Example, self).__init__(dct)
...
>>> e = Example('dict1')
>>> print e
{'k2': 'v2', 'k1': 'v1', 'kn': 'vn'}
>>>
An alternate solution would be to define a custom __str__ method for your class that returns self.dictx as a string:
class Example(dict):
def __init__(self, dictx):
self.dictx = getattr(__import__("mydicts", fromlist=['m']), dictx)
def __str__(self):
return str(self.dictx)
Note however that if you use this approach, there isn't really a reason to inherit from dict since your class is not a new type of dictionary; it just has a dictionary stored as an attribute.

I think you are making this far more complicated than it need be. Noting the manual says:
Direct use of __import__() is rare, except in cases where you want to import a module whose name is only known at runtime.
But you do know the module name at loading time, your data definition just has too many variable names. Much clearer would be my_dicts.py:
my_dicts = [
{k1:v1, k2:v2,...,kn:vn},
{k1:v1, k2:v2,...,km:vm},
…
]
and example.py:
import my_dicts
class Example(dict):
def __init__(self, n):
"""Returns an Example instance loaded with the nth element
of my_dict."""
super(Example, self).__init__(my_dicts.my_dicts[n])

How can I dynamically create derived classes from a base class

For example I have a base class as follows:
class BaseClass(object):
def __init__(self, classtype):
self._type = classtype
From this class I derive several other classes, e.g.
class TestClass(BaseClass):
def __init__(self):
super(TestClass, self).__init__('Test')
class SpecialClass(BaseClass):
def __init__(self):
super(TestClass, self).__init__('Special')
Is there a nice, pythonic way to create those classes dynamically by a function call that puts the new class into my current scope, like:
foo(BaseClass, "My")
a = MyClass()
...
As there will be comments and questions why I need this: The derived classes all have the exact same internal structure with the difference, that the constructor takes a number of previously undefined arguments. So, for example, MyClass takes the keywords a while the constructor of class TestClass takes b and c.
inst1 = MyClass(a=4)
inst2 = MyClass(a=5)
inst3 = TestClass(b=False, c = "test")
But they should NEVER use the type of the class as input argument like
inst1 = BaseClass(classtype = "My", a=4)
I got this to work but would prefer the other way, i.e. dynamically created class objects.

This bit of code allows you to create new classes with dynamic
names and parameter names.
The parameter verification in __init__ just does not allow
unknown parameters, if you need other verifications, like
type, or that they are mandatory, just add the logic
there:
class BaseClass(object):
def __init__(self, classtype):
self._type = classtype
def ClassFactory(name, argnames, BaseClass=BaseClass):
def __init__(self, **kwargs):
for key, value in kwargs.items():
# here, the argnames variable is the one passed to the
# ClassFactory call
if key not in argnames:
raise TypeError("Argument %s not valid for %s"
% (key, self.__class__.__name__))
setattr(self, key, value)
BaseClass.__init__(self, name[:-len("Class")])
newclass = type(name, (BaseClass,),{"__init__": __init__})
return newclass
And this works like this, for example:
>>> SpecialClass = ClassFactory("SpecialClass", "a b c".split())
>>> s = SpecialClass(a=2)
>>> s.a
2
>>> s2 = SpecialClass(d=3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in __init__
TypeError: Argument d not valid for SpecialClass
I see you are asking for inserting the dynamic names in the naming scope --
now, that is not considered a good practice in Python - you either have
variable names, known at coding time, or data - and names learned in runtime
are more "data" than "variables" -
So, you could just add your classes to a dictionary and use them from there:
name = "SpecialClass"
classes = {}
classes[name] = ClassFactory(name, params)
instance = classes[name](...)
And if your design absolutely needs the names to come in scope,
just do the same, but use the dictionary returned by the globals()
call instead of an arbitrary dictionary:
name = "SpecialClass"
globals()[name] = ClassFactory(name, params)
instance = SpecialClass(...)
(It indeed would be possible for the class factory function to insert the name dynamically on the global scope of the caller - but that is even worse practice, and is not compatible across Python implementations. The way to do that would be to get the caller's execution frame, through sys._getframe(1) and setting the class name in the frame's global dictionary in its f_globals attribute).
update, tl;dr: This answer had become popular, still its very specific to the question body. The general answer on how to
"dynamically create derived classes from a base class"
in Python is a simple call to type passing the new class name, a tuple with the baseclass(es) and the __dict__ body for the new class -like this:
>>> new_class = type("NewClassName", (BaseClass,), {"new_method": lambda self: ...})
update
Anyone needing this should also check the dill project - it claims to be able to pickle and unpickle classes just like pickle does to ordinary objects, and had lived to it in some of my tests.

type() is the function that creates classes and in particular sub-classes, like in the question:
def set_x(self, value):
self.x = value
# type() takes as argument the new class name, its base
# classes, and its attributes:
SubClass = type('SubClass', (BaseClass,), {'set_x': set_x})
# (More methods can be put in SubClass, including __init__().)
obj = SubClass()
obj.set_x(42)
print obj.x # Prints 42
print isinstance(obj, BaseClass) # True

In my case :
inst3 = globals()["SpecialClass"](b=False, c = "test")

To create a class with a dynamic attribute value, checkout the code below.
NB. This are code snippets in python programming language
def create_class(attribute_data, **more_data): # define a function with required attributes
class ClassCreated(optional extensions): # define class with optional inheritance
attribute1 = adattribute_data # set class attributes with function parameter
attribute2 = more_data.get("attribute2")
return ClassCreated # return the created class
# use class
myclass1 = create_class("hello") # *generates a class*

OO design: an object that can be exported to a "row", while accessing header names, without repeating myself

Sorry, badly worded title. I hope a simple example will make it clear. Here's the easiest way to do what I want to do:
class Lemon(object):
headers = ['ripeness', 'colour', 'juiciness', 'seeds?']
def to_row(self):
return [self.ripeness, self.colour, self.juiciness, self.seeds > 0]
def save_lemons(lemonset):
f = open('lemons.csv', 'w')
out = csv.writer(f)
out.write(Lemon.headers)
for lemon in lemonset:
out.writerow(lemon.to_row())
This works alright for this small example, but I feel like I'm "repeating myself" in the Lemon class. And in the actual code I'm trying to write (where the number of variables I'm exporting is ~50 rather than 4, and where to_row calls a number of private methods that do a bunch of weird calculations), it becomes awkward.
As I write the code to generate a row, I need to constantly refer to the "headers" variable to make sure I'm building my list in the correct order. If I want to change the variables being outputted, I need to make sure to_row and headers are being changed in parallel (exactly the kind of thing that DRY is meant to prevent, right?).
Is there a better way I could design this code? I've been playing with function decorators, but nothing has stuck. Ideally I should still be able to get at the headers without having a particular lemon instance (i.e. it should be a class variable or class method), and I don't want to have a separate method for each variable.

In this case, getattr() is your friend: it allows you to get a variable based on a string name. For example:
def to_row(self):
return [getattr(self, head) for head in self.headers]
EDIT: to properly use the header seeds?, you would need to set the attribute seeds? for the objects. setattr(self, 'seeds?', self.seeds > 0) right above the return statement.

We could use some metaclass shenanegans to do this...
In python 2, attributes are passed to the metaclass in a dict, without
preserving order, we'll also want a base class to work with so we can
distinguish class attributes that should be mapped into the row. In python3, we could dispense with just about all of this base descriptor class.
import itertools
import functools
#functools.total_ordering
class DryDescriptor(object):
_order_gen = itertools.count()
def __init__(self, alias=None):
self.alias = alias
self.order = next(self._order_gen)
def __lt__(self, other):
return self.order < other.order
We will want a python descriptor for every attribute we wish to map into the
row. slots are a nice way to get data descriptors without much work. One
caveat, though, we'll have to manually remove the helper instance to make the
real slot descriptor visible.
class slot(DryDescriptor):
def annotate(self, attr, attrs):
del attrs[attr]
self.attr = attr
slots = attrs.setdefault('__slots__', []).append(attr)
def annotate_class(self, cls):
if self.alias is not None:
setattr(cls, self.alias, getattr(self.attr))
For computed fields, we can memoize results. Memoizing off of the annotated
instance is tricky without a memory leak, we need weakref. alternatively, we
could have arranged for another slot just to store the cached value. This also isn't quite thread safe, but pretty close.
import weakref
class memo(DryDescriptor):
_memo = None
def __call__(self, method):
self.getter = method
return self
def annotate(self, attr, attrs):
if self.alias is not None:
attrs[self.alias] = self
def annotate_class(self, cls): pass
def __get__(self, instance, owner):
if instance is None:
return self
if self._memo is None:
self._memo = weakref.WeakKeyDictionary()
try:
return self._memo[instance]
except KeyError:
return self._memo.setdefault(instance, self.getter(instance))
On the metaclass, all of the descriptors we created above are found, sorted by
creation order, and instructed to annotate the new, created class. This does
not correctly treat derived classes and could use some other conveniences like
an __init__ for all the slots.
class DryMeta(type):
def __new__(mcls, name, bases, attrs):
descriptors = sorted((value, key)
for key, value
in attrs.iteritems()
if isinstance(value, DryDescriptor))
for descriptor, attr in descriptors:
descriptor.annotate(attr, attrs)
cls = type.__new__(mcls, name, bases, attrs)
for descriptor, attr in descriptors:
descriptor.annotate_class(cls)
cls._header_descriptors = [getattr(cls, attr) for descriptor, attr in descriptors]
return cls
Finally, we want a base class to inherit from so that we can have a to_row
method. this just invokes all of the __get__s for all of the respective
descriptors, in order.
class DryBase(object):
__metaclass__ = DryMeta
def to_row(self):
cls = type(self)
return [desc.__get__(self, cls) for desc in cls._header_descriptors]
Assuming all of that is tucked away, out of sight, the definition of a class
that uses this feature is mostly free of repitition. The only short coming is
that to be practical, every field needs a python friendly name, thus we had the
alias key to associate 'seeds?' to has_seeds
class ADryRow(DryBase):
__slots__ = ['seeds']
ripeness = slot()
colour = slot()
juiciness = slot()
#memo(alias='seeds?')
def has_seeds(self):
print "Expensive!!!"
return self.seeds > 0
>>> my_row = ADryRow()
>>> my_row.ripeness = "tart"
>>> my_row.colour = "#8C2"
>>> my_row.juiciness = 0.3479
>>> my_row.seeds = 19
>>>
>>> print my_row.to_row()
Expensive!!!
['tart', '#8C2', 0.3479, True]
>>> print my_row.to_row()
['tart', '#8C2', 0.3479, True]