This is some code I found on the internet. I'm not sure how it is meant to be used. I simply filled members with the enum keys/values and it works, but I'm curious what this metaclass is all about. I am assuming it has something to do with ctypes, but I can't find much information on subclassing ctypes. I know EnumerationType isn't doing anything the way I'm using Enumeration.
from ctypes import *
class EnumerationType(type(c_uint)):
def __new__(metacls, name, bases, dict):
if not "_members_" in dict:
_members_ = {}
for key,value in dict.items():
if not key.startswith("_"):
_members_[key] = value
dict["_members_"] = _members_
cls = type(c_uint).__new__(metacls, name, bases, dict)
for key,value in cls._members_.items():
globals()[key] = value
return cls
def __contains__(self, value):
return value in self._members_.values()
def __repr__(self):
return "<Enumeration %s>" % self.__name__
class Enumeration(c_uint):
__metaclass__ = EnumerationType
_members_ = {}
def __init__(self, value):
for k,v in self._members_.items():
if v == value:
self.name = k
break
else:
raise ValueError("No enumeration member with value %r" % value)
c_uint.__init__(self, value)
#classmethod
def from_param(cls, param):
if isinstance(param, Enumeration):
if param.__class__ != cls:
raise ValueError("Cannot mix enumeration members")
else:
return param
else:
return cls(param)
def __repr__(self):
return "<member %s=%d of %r>" % (self.name, self.value, self.__class__)
And an enumeration probably done the wrong way.
class TOKEN(Enumeration):
_members_ = {'T_UNDEF':0, 'T_NAME':1, 'T_NUMBER':2, 'T_STRING':3, 'T_OPERATOR':4, 'T_VARIABLE':5, 'T_FUNCTION':6}
A metaclass is a class used to create classes. Think of it this way: all objects have a class, a class is also an object, therefore, it makes sense that a class can have a class.
http://www.ibm.com/developerworks/linux/library/l-pymeta.html
To understand what this is doing, you can look at a few points in the code.
_members_ = {'T_UNDEF':0, 'T_NAME':1, 'T_NUMBER':2, 'T_STRING':3, 'T_OPERATOR':4, 'T_VARIABLE':5, 'T_FUNCTION':6}
globals()[key] = value
Here it takes every defined key in your dictionary: "T_UNDEF" "T_NUMBER" and makes them available in your globals dictionary.
def __init__(self, value):
for k,v in self._members_.items():
if v == value:
self.name = k
break
Whenever you make an instance of your enum, it will check to see if the "value" is in your list of allowable enum names when you initialized the class. When the value is found, it sets the string name to self.name.
c_uint.__init__(self, value)
This is the actual line which sets the "ctypes value" to an actual c unsigned integer.
That is indeed a weird class.
The way you are using it is correct, although another way would be:
class TOKEN(Enumeration):
T_UNDEF = 0
T_NAME = 1
T_NUMBER = 2
T_STRING = 3
T_OPERATOR = 4
T_VARIABLE = 5
T_FUNCTION = 6
(That's what the first 6 lines in __new__ are for)
Then you can use it like so:
>>> TOKEN
<Enumeration TOKEN>
>>> TOKEN(T_NAME)
<member T_NAME=1 of <Enumeration TOKEN>>
>>> T_NAME in TOKEN
True
>>> TOKEN(1).name
'T_NAME'
The from_param method seems to be for convenience, for writing methods that accept either an int or an Enumeration object. Not really sure if that's really its purpose.
I think this class is meant to be used when working with external APIs the use c-style enums, but it looks like a whole lot of work for very little gain.
Related
Using py3, I have an object that uses the #property decorator
class O(object):
def __init__(self):
self._a = None
#property
def a(self):
return 1
accessing the attribute a via __dict__ (with _a) doesn't seem to return the property decorated value but the initialized value None
o = O()
print(o.a, o.__dict__['_a'])
>>> 1, None
Is there a generic way to make this work? I mostly need this for
def __str__(self):
return ' '.join('{}: {}'.format(key, val) for key, val in self.__dict__.items())
Of course self.__dict__["_a"] will return self._a (well actually it's the other way round - self._a will return self.__dict__["_a"] - but anyway), not self.a. The only thing the property is doing here is to automatically invoke it's getter (your a(self) function) so you don't have to type the parens, otherwise it's just a plain method call.
If you want something that works with properties too, you'll have to get those manually from dir(self.__class__) and getattr(self.__class__, name), ie:
def __str__(self):
# py2
attribs = self.__dict__.items()
# py3
# attribs = list(self.__dict__.items())
for name in dir(self.__class__):
obj = getattr(self.__class__, name)
if isinstance(obj, property):
val = obj.__get__(self, self.__class__)
attribs.append((name, val))
return ' '.join('{}: {}'.format(key, val) for key, val in attribs)
Note that this won't prevent _a to appears in attribs - if you want to avoid this you'll also have to filter out protected names from the attribs list (all protected names, since you ask for something generic):
def __str__(self):
attribs = [(k, v) for k, v in self.__dict__.items() if not k.startswith("_")]
for name in dir(self.__class__):
# a protected property is somewhat uncommon but
# let's stay consistent with plain attribs
if name.startswith("_"):
continue
obj = getattr(self.__class__, name)
if isinstance(obj, property):
val = obj.__get__(self, self.__class__)
attribs.append((name, val))
return ' '.join('{}: {}'.format(key, val) for key, val in attribs)
Also note that this won't handle other computed attributes (property is just one generic implementation of the descriptor protocol). At this point, your best bet for something that's still as generic as possible but that can be customised if needed is to implement the above as a mixin class with a couple hooks for specialization:
class PropStrMixin(object):
# add other descriptor types you want to include in the
# attribs list
_COMPUTED_ATTRIBUTES_CLASSES = [property,]
def _get_attr_list(self):
attribs = [(k, v) for k, v in self.__dict__.items() if not k.startswith("_")]
for name in dir(self.__class__):
# a protected property is somewhat uncommon but
# let's stay consistent with plain attribs
if name.startswith("_"):
continue
obj = getattr(self.__class__, name)
if isinstance(obj, *self._COMPUTED_ATTRIBUTES_CLASSES):
val = obj.__get__(self, self.__class__)
attribs.append((name, val))
return attribs
def __str__(self):
attribs = self._get_attr_list()
return ' '.join('{}: {}'.format(key, val) for key, val in attribs)
class YouClass(SomeParent, PropStrMixin):
# here you can add to _COMPUTED_ATTRIBUTES_CLASSES
_COMPUTED_ATTRIBUTES_CLASSES = PropStrMixin + [SomeCustomDescriptor])
Property is basically a "computed attribute". In general, the property's value is not stored anywhere, it is computed on demand. That's why you cannot find it in the __dict__.
#property decorator replaces the class method by a descriptor object which then calls the original method as its getter. This happens at the class level.
The lookup for o.a starts at the instance. It does not exist there, the class is checked in the next step. O.a exists and is a descriptor (because it has special methods for the descriptor protocol), so the descriptor's getter is called and the returned value is used.
(EDITED)
There is not a general way to dump the name:value pairs for the descriptors. Classes including the bases must be inspected, this part is not difficult. However retrieving the values is equivalent to a function call and may have unexpected and undesirable side-effects. For a different perspective I'd like to quote a comment by bruno desthuilliers here: "property get should not have unwanted side effects (if it does then there's an obvious design error)".
You can also update self._a as getter since the return of the getter should always reflect what self._a is stored:
class O(object):
def __init__(self):
self._a = self.a
#property
def a(self):
self._a = 1
return self._a
A bit redundant, maybe, but setting self._a = None initially is useless in this case.
In case you need a setter
This would also be compatible given remove the first line in getter:
#a.setter
def a(self, value):
self._a = value
I am a total noob in Python and just couldn't help myself but did it again and dreamed of something I just couldn't achieve.
I wanted to have a class, which can be instantiated as such:
my_class = MyClass(**params)
and be consumed as such, in Flask:
jsonify(my_class)
The expected outcome would be a JSON:
{ "key" : "value", ... }
Now, the implementation of MyClass is,
class MyClas(NamedMutableSequence, Document):
__slots__ = (
'key_1',
'key_2',
'key_3'
)
def __init__(self, **params):
NamedMutableSequence.__init__(self, **params)
Document.__init__(self, 'myclass')
def save(self):
self._db_col.update({'key_1': self.key_1}, {'key_2': self.key_2, 'key_3': self.key_3})
By now, you are wondering what NamedMutableSequence and Document are...
class NamedMutableSequence(Sequence):
___slots__ = ()
def __init__(self, *positional_values, **keyword_values):
subclass_propeties = self.__slots__
for key in subclass_propeties:
setattr(self, key, keyword_values.get(key))
if positional_values:
for key, value in zip(subclass_propeties, positional_values):
setattr(self, key, value)
def __str__(self):
values = ', '.join('%s=%r' % (key, getattr(self, key)) for key in self.__slots__)
return '%s(%s)' % (clsname, values)
__repr__ = __str__
def __getitem__(self, item):
return getattr(self, item)
def __setitem__(self, item, value):
return setattr(self, item, value)
def __len__(self):
return len(self.__slots__)
Admittedly, I just copied someone's solution to a mutable namedtuple for this base class and fixed __getitem__ & __setitem__ to allow my_class.key_1 = 'some value'
class Document():
__slots__ = ('_db_col')
def __init__(self, collection):
self._db_col = mongo_db[collection]
This is just what I spew out in attempt for a base class which I will be using throughout my model classes for db connection.
This is, in my opinion, where it starts I got too over myself and just created a mess. Because no matter what I try, I can't stop raising TypeError: {string value of my_class} is not JSON serializable.
To make matters worse, when I try to dict(my_class), I get a shiny attributes must be string error raised on getattr().
I would still like to keep the base classes and I still need to make it JSON serializable.
How can I save myself?
I found an answer finally, and the solution was found from another stackoverflow post (How can I convert python class with slots to dictionary?)
What I did was just to add another method on the NamedMutableSequence as such:
def json(self):
return {key : getattr(self, key, None) for key in self.__slots__}
and just call it when I need a JSON parsable dictionary, as such:
my_class = MyClass(**params)
jsonify(my_class.json())
I had to write a class of some sort that overrides __getattribute__.
basically my class is a container, which saves every user-added property to self._meta which is a dictionary.
class Container(object):
def __init__(self, **kwargs):
super(Container, self).__setattr__('_meta', OrderedDict())
#self._meta = OrderedDict()
super(Container, self).__setattr__('_hasattr', lambda key : key in self._meta)
for attr, value in kwargs.iteritems():
self._meta[attr] = value
def __getattribute__(self, key):
try:
return super(Container, self).__getattribute__(key)
except:
if key in self._meta : return self._meta[key]
else:
raise AttributeError, key
def __setattr__(self, key, value):
self._meta[key] = value
#usage:
>>> a = Container()
>>> a
<__main__.Container object at 0x0000000002B2DA58>
>>> a.abc = 1 #set an attribute
>>> a._meta
OrderedDict([('abc', 1)]) #attribute is in ._meta dictionary
I have some classes which inherit Container base class and some of their methods have #property decorator.
class Response(Container):
#property
def rawtext(self):
if self._hasattr("value") and self.value is not None:
_raw = self.__repr__()
_raw += "|%s" %(self.value.encode("utf-8"))
return _raw
problem is that .rawtext isn't accessible. (I get attributeerror.) every key in ._meta is accessible, every attributes added by __setattr__ of object base class is accessible, but method-to-properties by #property decorator isn't. I think it has to do with my way of overriding __getattribute__ in Container base class. What should I do to make properties from #property accessible?
I think you should probably think about looking at __getattr__ instead of __getattribute__ here. The difference is this: __getattribute__ is called inconditionally if it exists -- __getattr__ is only called if python can't find the attribute via other means.
I completely agree with mgilson. If you want a sample code which should be equivalent to your code but work well with properties you can try:
class Container(object):
def __init__(self, **kwargs):
self._meta = OrderedDict()
#self._hasattr = lambda key: key in self._meta #???
for attr, value in kwargs.iteritems():
self._meta[attr] = value
def __getattr__(self, key):
try:
return self._meta[key]
except KeyError:
raise AttributeError(key)
def __setattr__(self, key, value):
if key in ('_meta', '_hasattr'):
super(Container, self).__setattr__(key, value)
else:
self._meta[key] = value
I really do not understand your _hasattr attribute. You put it as an attribute but it's actually a function that has access to self... shouldn't it be a method?
Actually I think you should simple use the built-in function hasattr:
class Response(Container):
#property
def rawtext(self):
if hasattr(self, 'value') and self.value is not None:
_raw = self.__repr__()
_raw += "|%s" %(self.value.encode("utf-8"))
return _raw
Note that hasattr(container, attr) will return True also for _meta.
An other thing that puzzles me is why you use an OrderedDict. I mean, you iterate over kwargs, and the iteration has random order since it's a normal dict, and add the items in the OrderedDict. Now you have _meta which contains the values in random order.
If you aren't sure whether you need to have a specific order or not, simply use dict and eventually swap to OrderedDict later.
By the way: never ever use an try: ... except: without specifying the exception to catch. In your code you actually wanted to catch only AttributeErrors so you should have done:
try:
return super(Container, self).__getattribute__(key)
except AttributeError:
#stuff
I want to understand python metaclasses. For practice I'm implementing a declarative way for writing classes (similar to sqlalchemy.ext.declarative). This looks promising as long as I only have one attribute.
But when I add another attribute, some part of the first attribute is changed and the value of the first attribute is validated against the pattern of the second attribute. This might be caused by the metaclass, by a closure, by the property or a combination of them. I try to give a minimal, complete but readable example.
#! /usr/bin/env python
"""
Something like:
class Artist:
locale = Pattern('[A-Z]{2}-[A-Z]{2}')
should be equivalent to:
class Artist:
def __init__(self):
self._locale = None
#property
def locale(self):
return self._locale
#locale.setter
def locale(self, value):
validate(value, '[A-Z]{2}-[A-Z]{2}')
self._locale = value
Problem:
The code below works if Artist has only one attribute.
When I add another one with a different pattern, only that last
pattern is used in validation.
"""
import re
import unittest
# this class (and future siblings) are used to describe attributes
class Pattern(object):
def __init__(self, pattern):
self.pattern = pattern
def validate(self, value):
if value is None:
return
if not re.match("^%s$" % self.pattern, value):
raise ValueError("invalid value: %r" % value)
def __repr__(self):
return "%s(pattern=%r)" % (self.__class__.__name__, self.pattern)
# __metaclass__ based class creation
def createClassFromDeclaration(name, bases, dct):
""" Examine dct, create initialization in __init__ and property. """
attributes = dict()
properties = dict()
for key, value in dct.iteritems():
if not isinstance(value, Pattern):
continue
pattern = value
pattern.attribute = "_%s" % key
attributes[key] = pattern
def fget(self):
return getattr(self, pattern.attribute)
def fset(self, value):
pattern.validate(value)
return setattr(self, pattern.attribute, value)
properties[key] = property(fget, fset)
def __init__(self, **kwargs):
# set all attributes found in the keyword arguments
for key, value in kwargs.iteritems():
if key in self.__attributes__:
setattr(self, key, value)
# set all attributes _NOT_ found to None
for key, declaration in attributes.iteritems():
if not hasattr(self, declaration.attribute):
setattr(self, key, None)
dct = dict(dct)
dct.update(properties)
dct['__init__'] = __init__
dct['__attributes__'] = attributes
return type(name, bases, dct)
# declarative class
class Artist(object):
__metaclass__ = createClassFromDeclaration
# FIXME: adding a second attribute changes the first pattern
locale = Pattern('[A-Z]{2}-[A-Z]{2}')
date = Pattern('[0-9]{4}-[0-9]{2}-[0-9]{2}')
# some unit tests
class TestArtist(unittest.TestCase):
def test_attributes_are_default_initialized(self):
artist = Artist()
self.assertIsNone(artist.date)
self.assertIsNone(artist.locale)
def test_attributes_are_initialized_from_keywords(self):
artist = Artist(locale="EN-US", date="2013-02-04")
self.assertEqual(artist.date, "2013-02-04")
# FIXME: the following does not work.
# it validates against the date pattern
self.assertEqual(artist.locale, "EN-US")
def test_locale_with_valid_value(self):
artist = Artist()
artist.date = "2013-02-04"
self.assertEqual(artist.locale, "2013-02-04")
# FIXME: the following does not work.
# it validates against the date pattern
artist.locale = "EN-US"
self.assertEqual(artist.locale, "EN-US")
def test_locale_with_invalid_value_throws(self):
artist = Artist()
with self.assertRaises(ValueError):
artist.locale = ""
with self.assertRaises(ValueError):
artist.locale = "EN-USA"
if __name__ == '__main__':
unittest.main()
# vim: set ft=python sw=4 et sta:
When I comment out the second attribute ('date') the tests succeed, but with the second attribute the tests that try to set the first attribute ('locale') fail. What causes the unittests to fail?
Disclaimer: This code is only for training. There are ways to create the same functionality that do not involve metaclasses, properties and closures (as you and I know). But we don't learn anything new if we only walk the streets we know. Please help me expand my Python knowledge.
The problem doesn't really have anything to do with metaclasses or properties per se. It has to do with how you're defining your get/set functions. Your fget and fset reference the variable pattern from the enclosing function. This creates a closure. The value of pattern will be looked up at the time fget/fset are called, not at the time they're defined. So when you overwrite pattern on the next loop iteration, you cause all fget/fset functions to now reference the new pattern.
Here's a simpler example that shows what's going on:
def doIt(x):
funs = []
for key, val in x.iteritems():
thingy = val + 1
def func():
return thingy
funs.append(func)
return funs
>>> dct = {'a': 1, 'b': 2, 'c': 3}
>>> funs = doIt(dct)
>>> for f in funs:
... print f()
3
3
3
Notice that, even though the three functions are defined at times when thingy has different values, when I call them later they all return the same value. This is because they are all looking up thingy when they're called, which is after the loop is done, so thingy just equals the last value it was set to.
The usual way to get around this is to pass in the variable you want to close over as the default value of an additional function argument. Try doing your getter and setter like this:
def fget(self, pattern=pattern):
return getattr(self, pattern.attribute)
def fset(self, value, pattern=pattern):
pattern.validate(value)
return setattr(self, pattern.attribute, value)
Default arguments are evaluated at function definition time, not call time, so this forces each function to "save" the value of pattern it wants to use.
I'm pretty new to Python, so if there's anything here that's flat-out bad, please point it out.
I have an object with this dictionary:
traits = {'happy': 0, 'worker': 0, 'honest': 0}
The value for each trait should be an int in the range 1-10, and new traits should not be allowed to be added. I want getter/setters so I can make sure these constraints are being kept. Here's how I made the getter and setter now:
def getTrait(self, key):
if key not in self.traits.keys():
raise KeyError
return traits[key]
def setTrait(self, key, value):
if key not in self.traits.keys():
raise KeyError
value = int(value)
if value < 1 or value > 10:
raise ValueError
traits[key] = value
I read on this website about the property() method. But I don't see an easy way to make use of it for getting/setting the values inside the dictionary. Is there a better way to do this? Ideally I would like the usage of this object to be obj.traits['happy'] = 14, which would invoke my setter method and throw a ValueError since 14 is over 10.
If you are willing to use syntax like obj['happy'] = 14 then you could use __getitem__ and __setitem__:
def __getitem__(self, key):
if key not in self.traits.keys():
raise KeyError
...
return traits[key]
def __setitem__(self, key, value):
if key not in self.traits.keys():
raise KeyError
...
traits[key] = value
If you really do want obj.traits['happy'] = 14 then you could define a subclass of dict and make obj.traits an instance of this subclass.
The subclass would then override __getitem__ and __setitem__ (see below).
PS. To subclass dict, inherit from both collections.MutableMapping, and dict. Otherwise, dict.update would not call the new __setitem__.
import collections
class TraitsDict(collections.MutableMapping,dict):
def __getitem__(self,key):
return dict.__getitem__(self,key)
def __setitem__(self, key, value):
value = int(value)
if not 1 <= value <= 10:
raise ValueError('{v} not in range [1,10]'.format(v=value))
dict.__setitem__(self,key,value)
def __delitem__(self, key):
dict.__delitem__(self,key)
def __iter__(self):
return dict.__iter__(self)
def __len__(self):
return dict.__len__(self)
def __contains__(self, x):
return dict.__contains__(self,x)
class Person(object):
def __init__(self):
self.traits=TraitsDict({'happy': 0, 'worker': 0, 'honest': 0})
p=Person()
print(p.traits['happy'])
# 0
p.traits['happy']=1
print(p.traits['happy'])
# 1
p.traits['happy']=14
# ValueError: 14 not in range [1,10]
Some obvious tips come to my mind first:
Do not use .keys() method when checking for existence of some key (instead of if key not in self.traits.keys() use if key not in self.traits).
Do not explicitly throw KeyError exception - it is thrown if you try to access inexistent key.
Your code could look like this after above changes:
def getTrait(self, key):
return traits[key]
def setTrait(self, key, value):
if key not in self.traits:
raise KeyError
value = int(value)
if value < 1 or value > 10:
raise ValueError
traits[key] = value
Ps. I did no check the correctness of your code thoroughly - there may be some other issues.
and new traits should not be allowed to be added.
The natural way to do this is to use an object instead of a dictionary, and set the class' __slots__.
The value for each trait should be an int in the range 1-10... I want getter/setters so I can make sure these constraints are being kept.
The natural way to do this is to use an object instead of a dictionary, so that you can write getter/setter logic that's part of the class, and wrap them up as properties. Since all these properties will work the same way, we can do some refactoring to write code that generates a property given an attribute name.
The following is probably over-engineered:
def one_to_ten(attr):
def get(obj): return getattr(obj, attr)
def set(obj, val):
val = int(val)
if not 1 <= val <= 10: raise ValueError
setattr(obj, attr, val)
return property(get, set)
def create_traits_class(*traits):
class Traits(object):
__slots__ = ['_' + trait for trait in traits]
for trait in traits: locals()[trait] = one_to_ten('_' + trait)
def __init__(self, **kwargs):
for k, v in kwargs.items(): setattr(self, k, v)
for trait in traits: assert hasattr(self, trait), "Missing trait in init"
def __repr__(self):
return 'Traits(%s)' % ', '.join(
'%s = %s' % (trait, getattr(self, trait)) for trait in traits
)
return Traits
example_type = create_traits_class('happy', 'worker', 'honest')
example_instance = example_type(happy=3, worker=8, honest=4)
# and you can set the .traits of some other object to example_instance.