I want to create a custom interface on top of SQLAlchemy so that some pre-defined hybrid properties are supported transparently.
Specifically, I want to create a class SpecialColumn and a metaclass so that when a user adds SpecialColumn as an attribute of a class, my custom metaclass replaces that attribute with two SQLAlchemy Columns and adds a hybrid property that gets and sets those two columns as a tuple.
Here's my approach so far:
First, I defined my special column type:
class SpecialColumn(object):
pass
Then, I defined a metaclass inheriting from DeclarativeMeta which scans the class for instances of SpecialColumn and replaces them with two Columns and a hybrid property (defined as a closure):
class MyDeclarativeMeta(DeclarativeMeta):
def __new__(cls, name, bases, attrs):
for name, col in attrs.items():
if isinstance(col, SpecialColumn):
# Replacing the column
del attrs[name]
col1_name = '_{0}_1'.format(name)
col2_name = '_{0}_2'.format(name)
attrs[col1_name] = Column(...)
attrs[col2_name] = Column(...)
# Adding the hybrid property
def getter(self):
return (getattr(self, col1_name), getattr(self, col2_name))
attrs[name] = hybrid_property(getter)
And finally I constructed an instance of declarative_base with it, and let the user define classes with the new base:
MyBase = declarative_base(metaclass=MyDeclarativeMeta)
class MyClass(MyBase):
col1 = SpecialColumn()
col2 = Column(...)
Now for my questions:
Firstly, is my approach correct?
Secondly, how can I use the metaclass to add the setter? Would it be correct to do:
def setter(self, (v1, v2)):
setattr(self, col1_name, v1)
setattr(self, col2_name, v2)
And then simply do attrs[name].setter(setter) ?
There's no need to use metaclasses for a SQLAlchemy mapped class as we supply plenty of events to add features to classes as they are created and/or mapped. mapper_configured might be good here, which if you're on 0.8 you can apply to MyBase directly:
#event.listens_for(MyBase, 'mapper_configured')
def get_special_columns(mapper, cls):
for attrname in dir(cls):
val = getattr(cls, attrname)
if isinstance(val, SpecialColumn):
name1, name2 = "_%s_1" % attrname, "_%s_2" % attrname
setattr(cls, name1, Column(...))
setattr(cls, name2, Column(...))
#hybrid_property
def myhybrid(self):
return getattr(self, name1), getattr(self, name2)
#myhybrid.setter
def myhybrid(self, value):
setattr(self, name1, value[0])
setattr(self, name2, value[1])
setattr(cls, attrname, myhybrid)
note that setattr() is the best way to go here, simple and to the point.
Related
In my object's init, I would like to create object properties from an iterable. For example:
class MyClass(object):
def __init__(self, parameters):
attributes = ['name',
'memory',
'regressors',
'use_const']
for attr_name in attributes():
try:
attr_val = parameters[attr_name]
except KeyError:
raise Error("parameters must contain {}".format(attr_name))
setattr(self, attr_name, attr_val)
This lets me get the attributes that I want. However, what I lose compared to defining
#property
def name(self):
"""str: This class' name"""
return self._name
is that I don't get the docstrings for the properties now.
I'd like to have the docstrings for each property (for my auto-generated documentation), but I'd also like to use an iterable instead of having to define each property separately. For example, can I turn attributes into a dict with the docstring as a value, and set the attribute's docstring dynamically?
Can I have my cake and eat it too?
You can only set property objects on the class. You can do this in a loop, but this has to be done when building the class, not instances.
Simply produce property objects:
def set_property(cls, name, attr, docstring):
def getter(self):
return getattr(self, attr)
prop = property(getter, None, None, docstring)
setattr(cls, name, prop)
for name in attributes:
attr = '_' + name
docstring = "str: This class' {}".format(name)
set_property(SomeClass, name, attr, docstring)
I'm using sqlalchemy's ORM and I have a bunch of Declarative classes that with class attributes. For instance:
class Example(Declarative):
id = Column(Integer, primary_key=True)
datum = Column(String(65))
Naturally, my classes are much longer than this example and I have about two dozen. I'd like to be able to populate their fields at instantiation time, so it would be nice to have __init__ functions for each class.
The naive way to do this is as follows:
def __init__(self, id, datum):
self.id = id
self.datum = datumi
This gets very tedious. Is there some sort of shortcut? Perhaps I can exploit Example.__dict__? These are new-style classes.
You don't need to create a __init__ method at all; SQLAlchemy provides you with one out of the box.
Even if you did have to build one, you could just accept arbitrary keyword arguments. That is just what the SQLAlchemy version does:
def _declarative_constructor(self, **kwargs):
"""A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and
values in ``kwargs``.
Only keys that are present as
attributes of the instance's class are allowed. These could be,
for example, any mapped columns or relationships.
"""
cls_ = type(self)
for k in kwargs:
if not hasattr(cls_, k):
raise TypeError(
"%r is an invalid keyword argument for %s" %
(k, cls_.__name__))
setattr(self, k, kwargs[k])
Translated into prose, this method tests if each and every keyword you pass in is an attribute on the class before using setattr() to set the value on the instance.
Note that this uses keyword arguments; you could not do this with positional arguments, because your class attributes are stored in a dictionary and thus have no set order. If you want to use positional arguments, you need to manually create an __init__ method, or define an order on the class for an (inherited) __init__ to make use of:
class Example(Declarative):
id = Column(Integer, primary_key=True)
datum = Column(String(65))
__order__ = ('id', 'datum')
def __init__(self, *args, **kwargs):
cls_ = type(self)
kwargs.update(zip(cls_.__order__, args))
for k in kwargs:
if not hasattr(cls_, k):
raise TypeError(
"%r is an invalid keyword argument for %s" %
(k, cls_.__name__))
setattr(self, k, kwargs[k])
Here, any positional args are merged into kwargs given a stored field order in a new __order__ attribute on the class.
You could define a class that takes a dictionary as an input and does something like:
for (k, v) in inital_dict.iteritems():
setattr(self, k, v)
If you inherited from that class all your new classes would have the same method and you could invoke it with super.
I need to make a bunch of class variables and I would like to do it by looping through a list like that:
vars=('tx','ty','tz') #plus plenty more
class Foo():
for v in vars:
setattr(no_idea_what_should_go_here,v,0)
is it possible? I don't want to make them for an instance (using self in the __init__) but as class variables.
You can run the insertion code immediately after a class is created:
class Foo():
...
vars=('tx', 'ty', 'tz') # plus plenty more
for v in vars:
setattr(Foo, v, 0)
Also, you can dynamically store the variable while the class is being created:
class Bar:
locals()['tx'] = 'texas'
Late to the party but use the type class constructor!
Foo = type("Foo", (), {k: 0 for k in ("tx", "ty", "tz")})
If for any reason you can't use Raymond's answer of setting them up after the class creation then perhaps you could use a metaclass:
class MetaFoo(type):
def __new__(mcs, classname, bases, dictionary):
for name in dictionary.get('_extra_vars', ()):
dictionary[name] = 0
return type.__new__(mcs, classname, bases, dictionary)
class Foo(): # For python 3.x use 'class Foo(metaclass=MetaFoo):'
__metaclass__=MetaFoo # For Python 2.x only
_extra_vars = 'tx ty tz'.split()
The locals() version did not work for me in a class.
The following can be used to dynamically create the attributes of the class:
class namePerson:
def __init__(self, value):
exec("self.{} = '{}'".format("name", value)
me = namePerson(value='my name')
me.name # returns 'my name'
setattr(object, name, value)
This is the counterpart of getattr(). The arguments are an object, a string and an arbitrary value. The string may name an existing attribute or a new attribute. The function assigns the value to the attribute, provided the object allows it.
For example, setattr(x, 'name', value) is equivalent to x.name = value.
The function you need is:
setattr(obj, name, value)
This allows you to set named attributes for a given class (this can be self).
The built in documentation for this function is pretty self-explanatory:
Signature: setattr(obj, name, value, /)
Docstring:
Sets the named attribute on the given object to the specified value.
setattr(x, 'y', v) is equivalent to ``x.y = v''
Type: builtin_function_or_method
Example use
One use of this is to use a dictionary to set multiple class attributes, in my case this was from xpath definitions. I felt this improved maintainability by keeping potentially more fragile xpath definitions all in one place:
class Person:
def _extract_fields(self):
''' Process the page using XPath definitions '''
logging.debug("_extract_fields(): {}".format(repr(self)))
# describe how to extract data from the publicly available site
# (kept together for maintainability)
fields = {
'staff_name':
'//div[#id="staff_name"]//text()',
'staff_dob':
'(//div[#id="staff_dob"]//text())[1]'
}
# populate named class attributes from the dict
for key in fields:
setattr(self, key, self._parsed_content.xpath(fields[key]))
def __init__(self):
self._extract_fields()
I would like to have a special obj that does the following:
obj.newly_created_attribute = some_value
Obviously, all objects will allow this. But I would like the previous code to automatically call a method when newly_created_attribute is not yet a attribute of obj. In my particular case, I wish to set up a custom get and set method for obj.newly_created_attribute (a property now).
Is there any way to do this? Some way to specify a callback that will be run whenever a new attribute is added to a object?
You can accomplish this by overriding __setattr__:
class SomeClass(object):
def __setattr__(self, name, value):
if not hasattr(self, name):
print "new attribute", name
# do stuff here
return object.__setattr__(self, name, value)
__setattr__ will help you there:
Called when an attribute assignment is attempted. This is called instead of the normal mechanism (i.e. store the value in the instance dictionary). name is the attribute name, value is the value to be assigned to it.
#!/usr/bin/env python
class Klass(object):
def __setattr__(self, name, value):
if not hasattr(self, name):
self.on_first_setattr()
return object.__setattr__(self, name, value)
def on_first_setattr(self):
print "I am just a callback and my story's seldom told."
obj = Klass()
obj.some_attr = 1 # will call callback
obj.some_attr = 2 # no output
Overload __setattr__. Example:
class Foo(object):
def __setattr__(self, attr, val):
print "setattr"
if attr not in self.__dict__:
print "new attr:", attr
self.__dict__[attr] = val
else:
print "extant attr:", attr
self.__dict__[attr] = val
im looking into mongoengine, and i wanted to make a class an "EmbeddedDocument" dynamically, so i do this
def custom(cls):
cls = type( cls.__name__, (EmbeddedDocument,), cls.__dict__.copy() )
cls.a = FloatField(required=True)
cls.b = FloatField(required=True)
return cls
A = custom( A )
and tried it on some classes, but its not doing some of the base class's init or sumthing
in BaseDocument
def __init__(self, **values):
self._data = {}
# Assign initial values to instance
for attr_name, attr_value in self._fields.items():
if attr_name in values:
setattr(self, attr_name, values.pop(attr_name))
else:
# Use default value if present
value = getattr(self, attr_name, None)
setattr(self, attr_name, value)
but this never gets used, thus never setting ._data, and giving me errors. how do i do this?
update:
im playing with it more, and it seems to have an issue with classes with init methods. maybe i need to make it explicit?
The class you are creating isn't a subclass of cls. You can mix-in EmbeddedDocument, but you still need to be subclassing the original to get the parent's methods (like __init__).
cls = type(cls.__name__, (cls, EmbeddedDocument), {'a': FloatField(required=True), 'b': FloatField(required=True)})
EDIT: you can put the 'a' and 'b' attributes right in the attribute dict passed to type()