Develop Reference

Python how to create a class that wraps any value

Let's say I have an Entity class:
class Entity(dict):
pass
def save(self):
...
I can wrap a dict object with Entity(dict_obj)
But is it possible to create a class that can wrap any type of objects, eg. int, list etc.
PS I have come up the following work around, it doesn't work on the more complex objects, but seems to work with basic ones, completely unsure if there are any gotchas, might get penalised with efficiency by creating the class every time, please let me know:
class EntityMixin(object):
def save(self):
...
def get_entity(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity(obj)
Usage:
>>> a = get_entity(1)
>>> a + 1
2
>>> b = get_entity('b')
>>> b.upper()
'B'
>>> c = get_entity([1,2])
>>> len(c)
2
>>> d = get_entity({'a':1})
>>> d['a']
1
>>> d = get_entity(map(lambda x : x, [1,2]))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jlin/projects/django-rest-framework-queryset/rest_framework_queryset/entity.py", line 11, in get_entity
return Entity(obj)
TypeError: map() must have at least two arguments.
Improve efficiency:
EntityClsCache = {}
class EntityMixin(object):
def save(self):
...
def _get_entity_cls(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity
def get_entity(obj)
cls = None
try:
cls = EntityClsCache[obj.__class__]
except AttributeError:
cls = _get_entity_cls(obj)
EntityClsCache[obj.__class__] = cls
return cls(obj)

The solution you propose looks elegant, but it lacks caching, as in, you'll construct a unique class every time get_entity() is called, even if types are all the same.
Python has metaclasses, which act as class factories. Given that metaclass' methods override these of class, not the instance, we can implement class caching:
class EntityMixin(object):
pass
class CachingFactory(type):
__registry__ = {}
# Instead of declaring an inner class,
# we can also return type("Wrapper", (type_, EntityMixin), {}) right away,
# which, however, looks more obscure
def __makeclass(cls, type_):
class Wrapper(type_, EntityMixin):
pass
return Wrapper
# This is the simplest form of caching; for more realistic and less error-prone example,
# better use a more unique/complex key, for example, tuple of `value`'s ancestors --
# you can obtain them via type(value).__mro__
def __call__(cls, value):
t = type(value)
typename = t.__name__
if typename not in cls.__registry__:
cls.__registry__[typename] = cls.__makeclass(t)
return cls.__registry__[typename](value)
class Factory(object):
__metaclass__ = CachingFactory
This way, Factory(1) performs Factory.__call__(1), which is CachingFactory.__call__(1) (without metaclass, that'd be a constructor call instead, which would result in a class instance -- but we want to make a class first and only then instantiate it).
We can ensure that the objects created by Factory are the instances of the same class, which is crafted specifically for them at the first time:
>>> type(Factory(map(lambda x: x, [1, 2]))) is type(Factory([1]))
True
>>> type(Factory("a")) is type(Factory("abc"))
True

Can you have constraints on Python3 NamedTuple attributes?

I have a simple NamedTuple that I want to enforce a constraint on. Is it possible?
Take the following example:
from typing import NamedTuple
class Person(NamedTuple):
first_name: str
last_name: str
If I had a desired maximum length for the name fields (e.g. 50 characters), how can I ensure that you cannot make a Person object with a name longer than that?
Normally, if this were just a class, not a NamedTuple, I'd handle this with a #property, #attr.setter and override the __init__ method. But NamedTuples can't have an __init__, and I can't see a way of having just a setter for one of the attributes (and if I could, I don't know if upon construction, the NamedTuple would even use it).
So, is this possible?
Note: I specifically want to use a NamedTuple (rather than trying to make a class immutable via my own methods/magic)

So I coded something that basically does what I wanted. I forgot to post it here, so it's evolved slightly from my original question, but I thought I'd best post here so that others can make use of it if they want.
import inspect
from collections import namedtuple
class TypedTuple:
_coerce_types = True
def __new__(cls, *args, **kwargs):
# Get the specified public attributes on the class definition
typed_attrs = cls._get_typed_attrs()
# For each positional argument, get the typed attribute, and check it's validity
new_args = []
for i, attr_value in enumerate(args):
typed_attr = typed_attrs[i]
new_value = cls.__parse_attribute(typed_attr, attr_value)
# Build a new args list to construct the namedtuple with
new_args.append(new_value)
# For each keyword argument, get the typed attribute, and check it's validity
new_kwargs = {}
for attr_name, attr_value in kwargs.items():
typed_attr = (attr_name, getattr(cls, attr_name))
new_value = cls.__parse_attribute(typed_attr, attr_value)
# Build a new kwargs object to construct the namedtuple with
new_kwargs[attr_name] = new_value
# Return a constructed named tuple using the named attribute, and the supplied arguments
return namedtuple(cls.__name__, [attr[0] for attr in typed_attrs])(*new_args, **new_kwargs)
#classmethod
def __parse_attribute(cls, typed_attr, attr_value):
# Try to find a function defined on the class to do checks on the supplied value
check_func = getattr(cls, f'_parse_{typed_attr[0]}', None)
if inspect.isroutine(check_func):
attr_value = check_func(attr_value)
else:
# If the supplied value is not the correct type, attempt to coerce it if _coerce_type is True
if not isinstance(attr_value, typed_attr[1]):
if cls._coerce_types:
# Coerce the value to the type, and assign back to the attr_value for further validation
attr_value = typed_attr[1](attr_value)
else:
raise TypeError(f'{typed_attr[0]} is not of type {typed_attr[1]}')
# Return the original value
return attr_value
#classmethod
def _get_typed_attrs(cls) -> tuple:
all_items = cls.__dict__.items()
public_items = filter(lambda attr: not attr[0].startswith('_') and not attr[0].endswith('_'), all_items)
public_attrs = filter(lambda attr: not inspect.isroutine(attr[1]), public_items)
return [attr for attr in public_attrs if isinstance(attr[1], type)]
This is my TypedTuple class, it basically behaves like a NamedTuple, except that you get type checking. It has the following basic usage:
>>> class Person(TypedTuple):
... """ Note, syntax is var=type, not annotation-style var: type
... """
... name=str
... age=int
...
>>> Person('Dave', 21)
Person(name='Dave', age=21)
>>>
>>> # Like NamedTuple, argument order matters
>>> Person(21, 'dave')
Traceback (most recent call last):
...
ValueError: invalid literal for int() with base 10: 'dave'
>>>
>>> # Can used named arguments
>>> Person(age=21, name='Dave')
Person(name='Dave', age=21)
So now you have a named tuple, which behaves in basically the same way, but it will type check the arguments you supply.
By default, the TypedTuple will also attempt to coerce the data you give it, into the types you say that it should be:
>>> dave = Person('Dave', '21')
>>> type(dave.age)
<class 'int'>
This behaviour can be turned off:
>>> class Person(TypedTuple):
... _coerce_types = False
... name=str
... age=int
...
>>> Person('Dave', '21')
Traceback (most recent call last):
...
TypeError: age is not of type <class 'int'>
Finally, you can also specify special parse methods, that can do any specific checking or coercing you want to do. These methods have the naming convention _parse_ATTR:
>>> class Person(TypedTuple):
... name=str
... age=int
...
... def _parse_age(value):
... if value < 0:
... raise ValueError('Age cannot be less than 0')
...
>>> Person('dave', -3)
Traceback (most recent call last):
...
ValueError: Age cannot be less than 0
I hope someone else finds this useful.
(Please note, this code will only work in Python3)

You are going to have to overload the __new__ method that constructs the subclass.
Here is an example that defines a name checking function inside of __new__ and checks each of the arguments.
from collections import namedtuple
# create the named tuple
BasePerson = namedtuple('person', 'first_name last_name')
# subclass the named tuple, overload new
class Person(BasePerson):
def __new__(cls, *args, **kwargs):
def name_check(name):
assert len(name)<50, 'Length of input name "{}" is too long'.format(name)
# check the arguments
for a in args + tuple(kwargs.values()):
name_check(a)
self = super().__new__(cls, *args, **kwargs)
return self
Now we can test a few inputs...
Person('hello','world')
# returns:
Person(first_name='hello', last_name='world')
Person('hello','world'*10)
# raises:
AssertionError Traceback (most recent call last)
<ipython-input-42-1ee8a8154e81> in <module>()
----> 1 Person('hello','world'*10)
<ipython-input-40-d0fa9033c890> in __new__(cls, *args, **kwargs)
12 # check the arguments
13 for a in args + tuple(kwargs.values()):
---> 14 name_check(a)
15
16 self = super().__new__(cls, *args, **kwargs)
<ipython-input-40-d0fa9033c890> in name_check(name)
8 def __new__(cls, *args, **kwargs):
9 def name_check(name):
---> 10 assert len(name)<50, 'Length of input name "{}" is too long'.format(name)
11
12 # check the arguments
AssertionError: Length of input name "worldworldworldworldworldworldworldworldworldworld" is too long

How to define enum values that are functions?

I have a situation where I need to enforce and give the user the option of one of a number of select functions, to be passed in as an argument to another function:
I really want to achieve something like the following:
from enum import Enum
#Trivial Function 1
def functionA():
pass
#Trivial Function 2
def functionB():
pass
#This is not allowed (as far as i can tell the values should be integers)
#But pseudocode for what I am after
class AvailableFunctions(Enum):
OptionA = functionA
OptionB = functionB
So the following can be executed:
def myUserFunction(theFunction = AvailableFunctions.OptionA):
#Type Check
assert isinstance(theFunction,AvailableFunctions)
#Execute the actual function held as value in the enum or equivalent
return theFunction.value()

Your assumption is wrong. Values can be arbitrary, they are not limited to integers. From the documentation:
The examples above use integers for enumeration values. Using integers
is short and handy (and provided by default by the Functional API),
but not strictly enforced. In the vast majority of use-cases, one
doesn’t care what the actual value of an enumeration is. But if the
value is important, enumerations can have arbitrary values.
However the issue with functions is that they are considered to be method definitions instead of attributes!
In [1]: from enum import Enum
In [2]: def f(self, *args):
...: pass
...:
In [3]: class MyEnum(Enum):
...: a = f
...: def b(self, *args):
...: print(self, args)
...:
In [4]: list(MyEnum) # it has no values
Out[4]: []
In [5]: MyEnum.a
Out[5]: <function __main__.f>
In [6]: MyEnum.b
Out[6]: <function __main__.MyEnum.b>
You can work around this by using a wrapper class or just functools.partial or (only in Python2) staticmethod:
from functools import partial
class MyEnum(Enum):
OptionA = partial(functionA)
OptionB = staticmethod(functionB)
Sample run:
In [7]: from functools import partial
In [8]: class MyEnum2(Enum):
...: a = partial(f)
...: def b(self, *args):
...: print(self, args)
...:
In [9]: list(MyEnum2)
Out[9]: [<MyEnum2.a: functools.partial(<function f at 0x7f4130f9aae8>)>]
In [10]: MyEnum2.a
Out[10]: <MyEnum2.a: functools.partial(<function f at 0x7f4130f9aae8>)>
Or using a wrapper class:
In [13]: class Wrapper:
...: def __init__(self, f):
...: self.f = f
...: def __call__(self, *args, **kwargs):
...: return self.f(*args, **kwargs)
...:
In [14]: class MyEnum3(Enum):
...: a = Wrapper(f)
...:
In [15]: list(MyEnum3)
Out[15]: [<MyEnum3.a: <__main__.Wrapper object at 0x7f413075b358>>]
Also note that if you want you can define the __call__ method in your enumeration class to make the values callable:
In [1]: from enum import Enum
In [2]: def f(*args):
...: print(args)
...:
In [3]: class MyEnum(Enum):
...: a = partial(f)
...: def __call__(self, *args):
...: self.value(*args)
...:
In [5]: MyEnum.a(1,2,3) # no need for MyEnum.a.value(1,2,3)
(1, 2, 3)

Since Python 3.11 there is much more concise and understandable way. member and nonmember functions were added to enum among other improvements, so you can now do the following:
from enum import Enum, member
def fn(x):
print(x)
class MyEnum(Enum):
meth = fn
mem = member(fn)
#classmethod
def this_is_a_method(cls):
print('No, still not a member')
def this_is_just_function():
print('No, not a member')
#member
def this_is_a_member(x):
print('Now a member!', x)
And now
>>> list(MyEnum)
[<MyEnum.mem: <function fn at ...>>, <MyEnum.this_is_a_member: <function MyEnum.this_is_a_member at ...>>]
>>> MyEnum.meth(1)
1
>>> MyEnum.mem(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'MyEnum' object is not callable
>>> MyEnum.mem.value(1)
1
>>> MyEnum.this_is_a_method()
No, still not a member
>>> MyEnum.this_is_just_function()
No, not a member
>>> MyEnum.this_is_a_member()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'MyEnum' object is not callable
>>> MyEnum.this_is_a_member.value(1)
Now a member! 1

Another less clunky solution is to put the functions in a tuple. As Bakuriu mentioned, you may want to make the enum callable.
from enum import Enum
def functionA():
pass
def functionB():
pass
class AvailableFunctions(Enum):
OptionA = (functionA,)
OptionB = (functionB,)
def __call__(self, *args, **kwargs):
self.value[0](*args, **kwargs)
Now you can use it like this:
AvailableFunctions.OptionA() # calls functionA

In addition to the answer of Bakuriu... If you use the wrapper approach like above you loose information about the original function like __name__, __repr__
and so on after wrapping it. This will cause problems for example if you want to use sphinx for generation of source code documentation. Therefore add the following to your wrapper class.
class wrapper:
def __init__(self, function):
self.function = function
functools.update_wrapper(self, function)
def __call__(self,*args, **kwargs):
return self.function(*args, **kwargs)
def __repr__(self):
return self.function.__repr__()

Building on top of #bakuriu's approach, I just want to highlight that we can also use dictionaries of multiple functions as values and have a broader polymorphism, similar to enums in Java. Here is a fictitious example to show what I mean:
from enum import Enum, unique
#unique
class MyEnum(Enum):
test = {'execute': lambda o: o.test()}
prod = {'execute': lambda o: o.prod()}
def __getattr__(self, name):
if name in self.__dict__:
return self.__dict__[name]
elif not name.startswith("_"):
value = self.__dict__['_value_']
return value[name]
raise AttributeError(name)
class Executor:
def __init__(self, mode: MyEnum):
self.mode = mode
def test(self):
print('test run')
def prod(self):
print('prod run')
def execute(self):
self.mode.execute(self)
Executor(MyEnum.test).execute()
Executor(MyEnum.prod).execute()
Obviously, the dictionary approach provides no additional benefit when there is only a single function, so use this approach when there are multiple functions. Ensure that the keys are uniform across all values as otherwise, the usage won't be polymorphic.
The __getattr__ method is optional, it is only there for syntactic sugar (i.e., without it, mode.execute() would become mode.value['execute']().
Since dictionaries can't be made readonly, using namedtuple would be better and require only minor changes to the above.
from enum import Enum, unique
from collections import namedtuple
EnumType = namedtuple("EnumType", "execute")
#unique
class MyEnum(Enum):
test = EnumType(lambda o: o.test())
prod = EnumType(lambda o: o.prod())
def __getattr__(self, name):
if name in self.__dict__:
return self.__dict__[name]
elif not name.startswith("_"):
value = self.__dict__['_value_']
return getattr(value, name)
raise AttributeError(name)

Dynamically adding a property to a class

This has been previously asked on Stack Overflow, but none of the answers seem to address exactly what I need to do. In my case, I want these dynamically-added properties to be a shortcut to store and read values from a database, so unfortunately it's not as easy as in this answer (where a lambda function was used) or this one (where values where stored in a dictionary): I must call other methods of the class.
This is my attempt:
import operator
class Foo(object):
def get_value(self, name):
# read and return value from database
return -1
def set_value(self, name, value):
# store value in database
pass
def add_metadata_property(name):
getter = operator.methodcaller('get_value', name)
setter = operator.methodcaller('set_value', name) # gets value at runtime
setattr(Foo, name, property(getter, setter))
add_metadata_property('spam')
f = Foo()
f.spam # works!
f.spam = 2
The last line, however, raises:
Traceback (most recent call last):
File "<stdin>", line 27, in <module>
TypeError: methodcaller expected 1 arguments, got 2
Any ideas on how to achieve this?

I don't know why you use operator.methodcaller here.
When you call f.spam=2, it will invoke setter.
setter = operator.methodcaller('set_value', name) means setter(r) = r.set_value(name). Make no sense in your case.
I suggest you write this way, using #classmethod:
class Foo(object):
#classmethod
def get_value(self, name):
# read and return value from database
return -1
#classmethod
def set_value(self, name, value):
# store value in database
pass
def add_metadata_property(name):
setattr(Foo, name, property(Foo.get_value, Foo.set_value))
add_metadata_property('spam')
f = Foo()
f.spam # works!
f.spam = 2
If this helped you, please confirm it as the answer. Thanks!

Modifying class template looks a bit odd for me. I would suggest to overload __getattr__() and __setattr__() methods in your case.
class Foo:
def __getattr__(self, name):
print('read and return value from database for ', name)
return 123
def __setattr__(self, name, value):
print('store value', value, 'for', name, 'in database')

Getting NameError when calling function in constructor

I ran the code below, by calling the function in the constructor
First --
>>> class PrintName:
... def __init__(self, value):
... self._value = value
... printName(self._value)
... def printName(self, value):
... for c in value:
... print c
...
>>> o = PrintName('Chaitanya')
C
h
a
i
t
a
n
y
a
Once again I run this and I get this
>>> class PrintName:
... def __init__(self, value):
... self._value = value
... printName(self._value)
... def printName(self, value):
... for c in value:
... print c
...
>>> o = PrintName('Hello')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __init__
NameError: global name 'printName' is not defined
Can I not call a function in the constructor? and whay a deviation in the execution of similar code?
Note: I forgot to call a function local to the class, by using self (ex: self.printName()). Apologize for the post.

You need to call self.printName since your function is a method belonging to the PrintName class.
Or, since your printname function doesn't need to rely on object state, you could just make it a module level function.
class PrintName:
def __init__(self, value):
self._value = value
printName(self._value)
def printName(value):
for c in value:
print c

Instead of
printName(self._value)
you wanted
self.printName(self._value)
It probably worked the first time because you had another function printName in a parent scope.

What you want is self.printName(self._value) in __init__, not just printName(self._value).

I know this is an old question, but I just wanted to add that you can also call the function using the Class name and passing self as the first argument.
Not sure why you'd want to though, as I think it might make things less clear.
class PrintName:
def __init__(self, value):
self._value = value
PrintName.printName(self, self._value)
def printName(self, value):
for c in value:
print(c)
See Chapter 9 of the python manuals for more info:
9.3.4. Method Objects
Actually, you may have guessed the answer: the special thing about methods is that the object is passed as the first argument of the function. In our example, the call x.f() is exactly equivalent to MyClass.f(x). In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s object before the first argument.