Let's say I have an Entity class:
class Entity(dict):
pass
def save(self):
...
I can wrap a dict object with Entity(dict_obj)
But is it possible to create a class that can wrap any type of objects, eg. int, list etc.
PS I have come up the following work around, it doesn't work on the more complex objects, but seems to work with basic ones, completely unsure if there are any gotchas, might get penalised with efficiency by creating the class every time, please let me know:
class EntityMixin(object):
def save(self):
...
def get_entity(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity(obj)
Usage:
>>> a = get_entity(1)
>>> a + 1
2
>>> b = get_entity('b')
>>> b.upper()
'B'
>>> c = get_entity([1,2])
>>> len(c)
2
>>> d = get_entity({'a':1})
>>> d['a']
1
>>> d = get_entity(map(lambda x : x, [1,2]))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jlin/projects/django-rest-framework-queryset/rest_framework_queryset/entity.py", line 11, in get_entity
return Entity(obj)
TypeError: map() must have at least two arguments.
Improve efficiency:
EntityClsCache = {}
class EntityMixin(object):
def save(self):
...
def _get_entity_cls(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity
def get_entity(obj)
cls = None
try:
cls = EntityClsCache[obj.__class__]
except AttributeError:
cls = _get_entity_cls(obj)
EntityClsCache[obj.__class__] = cls
return cls(obj)
The solution you propose looks elegant, but it lacks caching, as in, you'll construct a unique class every time get_entity() is called, even if types are all the same.
Python has metaclasses, which act as class factories. Given that metaclass' methods override these of class, not the instance, we can implement class caching:
class EntityMixin(object):
pass
class CachingFactory(type):
__registry__ = {}
# Instead of declaring an inner class,
# we can also return type("Wrapper", (type_, EntityMixin), {}) right away,
# which, however, looks more obscure
def __makeclass(cls, type_):
class Wrapper(type_, EntityMixin):
pass
return Wrapper
# This is the simplest form of caching; for more realistic and less error-prone example,
# better use a more unique/complex key, for example, tuple of `value`'s ancestors --
# you can obtain them via type(value).__mro__
def __call__(cls, value):
t = type(value)
typename = t.__name__
if typename not in cls.__registry__:
cls.__registry__[typename] = cls.__makeclass(t)
return cls.__registry__[typename](value)
class Factory(object):
__metaclass__ = CachingFactory
This way, Factory(1) performs Factory.__call__(1), which is CachingFactory.__call__(1) (without metaclass, that'd be a constructor call instead, which would result in a class instance -- but we want to make a class first and only then instantiate it).
We can ensure that the objects created by Factory are the instances of the same class, which is crafted specifically for them at the first time:
>>> type(Factory(map(lambda x: x, [1, 2]))) is type(Factory([1]))
True
>>> type(Factory("a")) is type(Factory("abc"))
True
Related
I have inherited a project with many large classes constituent of nothing but class objects (integers, strings, etc). I'd like to be able to check if an attribute is present without needed to define a list of attributes manually.
Is it possible to make a python class iterable itself using the standard syntax? That is, I'd like to be able to iterate over all of a class's attributes using for attr in Foo: (or even if attr in Foo) without needing to create an instance of the class first. I think I can do this by defining __iter__, but so far I haven't quite managed what I'm looking for.
I've achieved some of what I want by adding an __iter__ method like so:
class Foo:
bar = "bar"
baz = 1
#staticmethod
def __iter__():
return iter([attr for attr in dir(Foo) if attr[:2] != "__"])
However, this does not quite accomplish what I'm looking for:
>>> for x in Foo:
... print(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'classobj' object is not iterable
Even so, this works:
>>> for x in Foo.__iter__():
... print(x)
bar
baz
Add the __iter__ to the metaclass instead of the class itself (assuming Python 2.x):
class Foo(object):
bar = "bar"
baz = 1
class __metaclass__(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
For Python 3.x, use
class MetaFoo(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
class Foo(metaclass=MetaFoo):
bar = "bar"
baz = 1
this is how we make a class object iterable. provide the class with a iter and a next() method, then you can iterate over class attributes or their values.you can leave the next() method if you want to, or you can define next() and raise StopIteration on some condition.
e.g:
class Book(object):
def __init__(self,title,author):
self.title = title
self.author = author
def __iter__(self):
for each in self.__dict__.values():
yield each
>>> book = Book('The Mill on the Floss','George Eliot')
>>> for each in book: each
...
'George Eliot'
'The Mill on the Floss'
this class iterates over attribute value of class Book.
A class object can be made iterable by providing it with a getitem method too.
e.g:
class BenTen(object):
def __init__(self, bentenlist):
self.bentenlist = bentenlist
def __getitem__(self,index):
if index <5:
return self.bentenlist[index]
else:
raise IndexError('this is high enough')
>>> bt_obj = BenTen([x for x in range(15)])
>>>for each in bt_obj:each
...
0
1
2
3
4
now when the object of BenTen class is used in a for-in loop, getitem is called with succesively higher index value, till it raises IndexError.
You can iterate over the class's unhidden attributes with for attr in (elem for elem in dir(Foo) if elem[:2] != '__').
A less horrible way to spell that is:
def class_iter(Class):
return (elem for elem in dir(Class) if elem[:2] != '__')
then
for attr in class_iter(Foo):
pass
class MetaItetaror(type):
def __iter__(cls):
return iter(
filter(
lambda k: not k[0].startswith('__'),
cls.__dict__.iteritems()
)
)
class Klass:
__metaclass__ = MetaItetaror
iterable_attr_names = {'x', 'y', 'z'}
x = 5
y = 6
z = 7
for v in Klass:
print v
An instance of enum.Enum happens to be iterable, and while it is not a general solution, it is a reasonable option for some use cases:
from enum import Enum
class Foo(Enum):
bar = "qux"
baz = 123
>>> print(*Foo)
Foo.bar Foo.baz
names = [m.name for m in Foo]
>>> print(*names)
bar baz
values = [m.value for m in Foo]
print(*values)
>>> qux 123
As with .__dict__, the order of iteration using this Enum based approach is the same as the order of definition.
You can make class members iterable within just a single line.
Despite the easy and compact code there are two mayor features included, additionally:
Type checking allows using additional class members not to be iterated.
The technique is also working if (public) class methods are defined. The proposals above using the "__" string checking filtering method propably fail in such cases.
# How to make class members iterable in a single line within Python (O. Simon, 14.4.2022)
# Includes type checking to allow additional class members not to be iterated
class SampleVector():
def __init__(self, x, y, name):
self.x = x
self.y = y
self.name = name
def __iter__(self):
return [value for value in self.__dict__.values() if isinstance(value, int) or isinstance(value, float)].__iter__()
if __name__ == '__main__':
v = SampleVector(4, 5, "myVector")
print (f"The content of sample vector '{v.name}' is:\n")
for m in v:
print(m)
This solution is fairly close and inspired by answer 12 from Hans Ginzel and Vijay Shanker.
I think a code sample will better speak for itself:
class SomeClass:
example = create_get_method()
Yes, that's all – ideally.
In that case, create_get_method would add a get_example() to SomeClass in a way that it can be accessed via an instance of SomeClass:
obj = SomeClass()
obj.get_example() <- returns the value of self.example
(Of course, the idea is to implement a complex version of get_contact, that's why I want to do that in a non-repetitive way, and this is a simplified version that represents well the issue.)
I don't know if that's possible, because it require to have access to the property name (example) and the class (SomeClass) since these can not be guessed in advance (that function will be used on many and various classes).
I know it's something possible, because that's kind of what SQLAlchemy does with their relationship() function on a class:
class Model(BaseModel):
id = ...
contact_id = db.Integer(db.ForeignKey..)
contact = relationship('contact') <-- This !
How can this be done?
Objects bound to class-level variables can have a __set_name__ method that will be called immediately after the class object has been created. It will be called with two arguments, the class object, and the name of the variable the object is saved as in the class.
You could use this to create your extra getter method, though I'm not sure why exactly you want to (you could make the object a descriptor instead, which would probably be better than adding a separate getter function to the parent class).
class create_get_method:
def __set_name__(self, owner, name):
def getter(self):
return getattr(self, name)
getter_name = f"get_{name}"
getter.__name__ = getter_name
setattr(owner, getter_name, getter)
# you might also want a __get__ method here to give a default value (like None)
Here's how that would work:
>>> class Test:
... example = create_get_method()
...
>>> t = Test()
>>> print(t.get_example())
<__main__.create_get_method at 0x000001E0B4D41400>
>>> t.example = "foo"
>>> print(t.get_example())
foo
You could change the value returned by default (in the first print call), so that the create_get_method object isn't as exposed. Just add a __get__ method to the create_get_method class.
You can do this with a custom non-data descriptor, like a property, except that you don't need a __set__ method:
class ComplicatedDescriptor:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
Now you can have any number of classes that use this descriptor:
class SomeClass:
example = ComplicatedDescriptor('example')
Now you can do something like:
>>> inst0 = SomeClass()
>>> inst1 = SomeClass()
>>> inst0.example
1
>>> inst1.example
1
>>> inst1.example
2
>>> inst0.example
2
The line name = f'_{self.name} is necessary because the descriptor here is a non-data descriptor: it has no __set__ method, so if you create inst0.__dict__['example'], the lookup will no longer happen: inst0.example will return inst0.__dict__['example'] instead of calling SomeClass.example.__get__(inst0, type(inst0)). One workaround is to store the value under the attribute name _example. The other is to make your descriptor into a data descriptor:
class ComplicatedDescriptor_v2:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
# These two lines for demo only
value = owner.__dict__.get(self.name, 0)
value += 1
owner.__dict__[self.name] = value
return value
def __set__(self, *args):
raise AttributeError(f'{self.name} is a read-only attribute')
The usage is generally identical:
class SomeClass:
example = ComplicatedDescriptor_v2('example')
Except that now you can't accidentally override your attribute:
>>> inst = SomeClass()
>>> inst.example
1
>>> inst.example
2
>>> inst.example = 0
AttributeError: example is a read-only attribute
Descriptors are a fairly idiomatic way to get and set values in python. They are preferred to getters and setters in almost all cases. The simplest cases are handled by the built-in property. That being said, if you wanted to explicitly have a getter method, I would recommend doing something very similar, but just returning a method instead of calling __get__ directly.
For example:
def __get__(self, owner, type):
def enclosed():
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
return enclosed
There is really no point to doing something like this unless you plan on really just want to be able to call inst.example().
For example I have a base class as follows:
class BaseClass(object):
def __init__(self, classtype):
self._type = classtype
From this class I derive several other classes, e.g.
class TestClass(BaseClass):
def __init__(self):
super(TestClass, self).__init__('Test')
class SpecialClass(BaseClass):
def __init__(self):
super(TestClass, self).__init__('Special')
Is there a nice, pythonic way to create those classes dynamically by a function call that puts the new class into my current scope, like:
foo(BaseClass, "My")
a = MyClass()
...
As there will be comments and questions why I need this: The derived classes all have the exact same internal structure with the difference, that the constructor takes a number of previously undefined arguments. So, for example, MyClass takes the keywords a while the constructor of class TestClass takes b and c.
inst1 = MyClass(a=4)
inst2 = MyClass(a=5)
inst3 = TestClass(b=False, c = "test")
But they should NEVER use the type of the class as input argument like
inst1 = BaseClass(classtype = "My", a=4)
I got this to work but would prefer the other way, i.e. dynamically created class objects.
This bit of code allows you to create new classes with dynamic
names and parameter names.
The parameter verification in __init__ just does not allow
unknown parameters, if you need other verifications, like
type, or that they are mandatory, just add the logic
there:
class BaseClass(object):
def __init__(self, classtype):
self._type = classtype
def ClassFactory(name, argnames, BaseClass=BaseClass):
def __init__(self, **kwargs):
for key, value in kwargs.items():
# here, the argnames variable is the one passed to the
# ClassFactory call
if key not in argnames:
raise TypeError("Argument %s not valid for %s"
% (key, self.__class__.__name__))
setattr(self, key, value)
BaseClass.__init__(self, name[:-len("Class")])
newclass = type(name, (BaseClass,),{"__init__": __init__})
return newclass
And this works like this, for example:
>>> SpecialClass = ClassFactory("SpecialClass", "a b c".split())
>>> s = SpecialClass(a=2)
>>> s.a
2
>>> s2 = SpecialClass(d=3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in __init__
TypeError: Argument d not valid for SpecialClass
I see you are asking for inserting the dynamic names in the naming scope --
now, that is not considered a good practice in Python - you either have
variable names, known at coding time, or data - and names learned in runtime
are more "data" than "variables" -
So, you could just add your classes to a dictionary and use them from there:
name = "SpecialClass"
classes = {}
classes[name] = ClassFactory(name, params)
instance = classes[name](...)
And if your design absolutely needs the names to come in scope,
just do the same, but use the dictionary returned by the globals()
call instead of an arbitrary dictionary:
name = "SpecialClass"
globals()[name] = ClassFactory(name, params)
instance = SpecialClass(...)
(It indeed would be possible for the class factory function to insert the name dynamically on the global scope of the caller - but that is even worse practice, and is not compatible across Python implementations. The way to do that would be to get the caller's execution frame, through sys._getframe(1) and setting the class name in the frame's global dictionary in its f_globals attribute).
update, tl;dr: This answer had become popular, still its very specific to the question body. The general answer on how to
"dynamically create derived classes from a base class"
in Python is a simple call to type passing the new class name, a tuple with the baseclass(es) and the __dict__ body for the new class -like this:
>>> new_class = type("NewClassName", (BaseClass,), {"new_method": lambda self: ...})
update
Anyone needing this should also check the dill project - it claims to be able to pickle and unpickle classes just like pickle does to ordinary objects, and had lived to it in some of my tests.
type() is the function that creates classes and in particular sub-classes, like in the question:
def set_x(self, value):
self.x = value
# type() takes as argument the new class name, its base
# classes, and its attributes:
SubClass = type('SubClass', (BaseClass,), {'set_x': set_x})
# (More methods can be put in SubClass, including __init__().)
obj = SubClass()
obj.set_x(42)
print obj.x # Prints 42
print isinstance(obj, BaseClass) # True
In my case :
inst3 = globals()["SpecialClass"](b=False, c = "test")
To create a class with a dynamic attribute value, checkout the code below.
NB. This are code snippets in python programming language
def create_class(attribute_data, **more_data): # define a function with required attributes
class ClassCreated(optional extensions): # define class with optional inheritance
attribute1 = adattribute_data # set class attributes with function parameter
attribute2 = more_data.get("attribute2")
return ClassCreated # return the created class
# use class
myclass1 = create_class("hello") # *generates a class*
I have inherited a project with many large classes constituent of nothing but class objects (integers, strings, etc). I'd like to be able to check if an attribute is present without needed to define a list of attributes manually.
Is it possible to make a python class iterable itself using the standard syntax? That is, I'd like to be able to iterate over all of a class's attributes using for attr in Foo: (or even if attr in Foo) without needing to create an instance of the class first. I think I can do this by defining __iter__, but so far I haven't quite managed what I'm looking for.
I've achieved some of what I want by adding an __iter__ method like so:
class Foo:
bar = "bar"
baz = 1
#staticmethod
def __iter__():
return iter([attr for attr in dir(Foo) if attr[:2] != "__"])
However, this does not quite accomplish what I'm looking for:
>>> for x in Foo:
... print(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'classobj' object is not iterable
Even so, this works:
>>> for x in Foo.__iter__():
... print(x)
bar
baz
Add the __iter__ to the metaclass instead of the class itself (assuming Python 2.x):
class Foo(object):
bar = "bar"
baz = 1
class __metaclass__(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
For Python 3.x, use
class MetaFoo(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
class Foo(metaclass=MetaFoo):
bar = "bar"
baz = 1
this is how we make a class object iterable. provide the class with a iter and a next() method, then you can iterate over class attributes or their values.you can leave the next() method if you want to, or you can define next() and raise StopIteration on some condition.
e.g:
class Book(object):
def __init__(self,title,author):
self.title = title
self.author = author
def __iter__(self):
for each in self.__dict__.values():
yield each
>>> book = Book('The Mill on the Floss','George Eliot')
>>> for each in book: each
...
'George Eliot'
'The Mill on the Floss'
this class iterates over attribute value of class Book.
A class object can be made iterable by providing it with a getitem method too.
e.g:
class BenTen(object):
def __init__(self, bentenlist):
self.bentenlist = bentenlist
def __getitem__(self,index):
if index <5:
return self.bentenlist[index]
else:
raise IndexError('this is high enough')
>>> bt_obj = BenTen([x for x in range(15)])
>>>for each in bt_obj:each
...
0
1
2
3
4
now when the object of BenTen class is used in a for-in loop, getitem is called with succesively higher index value, till it raises IndexError.
You can iterate over the class's unhidden attributes with for attr in (elem for elem in dir(Foo) if elem[:2] != '__').
A less horrible way to spell that is:
def class_iter(Class):
return (elem for elem in dir(Class) if elem[:2] != '__')
then
for attr in class_iter(Foo):
pass
class MetaItetaror(type):
def __iter__(cls):
return iter(
filter(
lambda k: not k[0].startswith('__'),
cls.__dict__.iteritems()
)
)
class Klass:
__metaclass__ = MetaItetaror
iterable_attr_names = {'x', 'y', 'z'}
x = 5
y = 6
z = 7
for v in Klass:
print v
An instance of enum.Enum happens to be iterable, and while it is not a general solution, it is a reasonable option for some use cases:
from enum import Enum
class Foo(Enum):
bar = "qux"
baz = 123
>>> print(*Foo)
Foo.bar Foo.baz
names = [m.name for m in Foo]
>>> print(*names)
bar baz
values = [m.value for m in Foo]
print(*values)
>>> qux 123
As with .__dict__, the order of iteration using this Enum based approach is the same as the order of definition.
You can make class members iterable within just a single line.
Despite the easy and compact code there are two mayor features included, additionally:
Type checking allows using additional class members not to be iterated.
The technique is also working if (public) class methods are defined. The proposals above using the "__" string checking filtering method propably fail in such cases.
# How to make class members iterable in a single line within Python (O. Simon, 14.4.2022)
# Includes type checking to allow additional class members not to be iterated
class SampleVector():
def __init__(self, x, y, name):
self.x = x
self.y = y
self.name = name
def __iter__(self):
return [value for value in self.__dict__.values() if isinstance(value, int) or isinstance(value, float)].__iter__()
if __name__ == '__main__':
v = SampleVector(4, 5, "myVector")
print (f"The content of sample vector '{v.name}' is:\n")
for m in v:
print(m)
This solution is fairly close and inspired by answer 12 from Hans Ginzel and Vijay Shanker.
This question already has answers here:
Automatically initialize instance variables?
(17 answers)
Closed last month.
Say, I have the following class in Python
class Foo(object):
a = None
b = None
c = None
def __init__(self, a = None, b = None, c = None):
self.a = a
self.b = b
self.c = c
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
Please note that
class Foo(object):
a = None
sets a key-value pair in Foo's dict:
Foo.__dict__['a']=None
while
def __init__(self, a = None, b = None, c = None):
self.a = a
sets a key-value pair in the Foo instance object's dict:
foo=Foo()
foo.__dict__['a']=a
So setting the class members at the top of your definition is not directly related to the setting of the instance attributes in the lower half of your definition (inside the __init__.
Also, it is good to be aware that __init__ is Python's initializer. __new__ is the class constructor.
If you are looking for a way to automatically add some instance attributes based on __init__'s arguments, you could use this:
import inspect
import functools
def autoargs(*include,**kwargs):
def _autoargs(func):
attrs,varargs,varkw,defaults=inspect.getargspec(func)
def sieve(attr):
if kwargs and attr in kwargs['exclude']: return False
if not include or attr in include: return True
else: return False
#functools.wraps(func)
def wrapper(self,*args,**kwargs):
# handle default values
for attr,val in zip(reversed(attrs),reversed(defaults)):
if sieve(attr): setattr(self, attr, val)
# handle positional arguments
positional_attrs=attrs[1:]
for attr,val in zip(positional_attrs,args):
if sieve(attr): setattr(self, attr, val)
# handle varargs
if varargs:
remaining_args=args[len(positional_attrs):]
if sieve(varargs): setattr(self, varargs, remaining_args)
# handle varkw
if kwargs:
for attr,val in kwargs.iteritems():
if sieve(attr): setattr(self,attr,val)
return func(self,*args,**kwargs)
return wrapper
return _autoargs
So when you say
class Foo(object):
#autoargs()
def __init__(self,x,path,debug=False,*args,**kw):
pass
foo=Foo('bar','/tmp',True, 100, 101,verbose=True)
you automatically get these instance attributes:
print(foo.x)
# bar
print(foo.path)
# /tmp
print(foo.debug)
# True
print(foo.args)
# (100, 101)
print(foo.verbose)
# True
PS. Although I wrote this (for fun), I don't recommend using autoargs for serious work. Being explicit is simple, clear and infallible. I can't say the same for autoargs.
PPS. Is it just me, or are a lot of buttons broken on Stackoverflow? The editor window has lost all its icons... :( Clearing the browser cache fixed the problem.
Python 3.7 provides dataclasses which are helpful in situations like this:
from dataclasses import dataclass
#dataclass
class Foo:
a: str = None
b: str = None
c: str = None
This saves you from having to write out the __init__ method when you just want to store a few attributes.
Gives you a good __repr__ method:
>>> a = Foo()
>>> a
Foo(a=None, b=None, c=None)
If you need to do calculations on a param, you can implement __post_init__.
See also namedtuple:
from collections import namedtuple
Foo = namedtuple('Foo', ['a', 'b', 'c'])
All fields are required with namedtuple though.
>>> a = Foo(1, 2, 3)
>>> a
Foo(a=1, b=2, c=3)
There are elegant ways to do this.
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
There is also a crude way. It will work, but is NOT recommended. See and decide.
>>> class Foo(object):
def __init__(self, **attrs):
self.__dict__.update(**attrs)
def __getattr__(self, attr):
return self.__dict__.get(attr, None)
>>> f = Foo(a = 1, b = 2, c = 3)
>>> f.a, f.b
(1, 2)
>>> f = Foo(bar = 'baz')
>>> f.bar
'baz'
>>> f.a
>>>
The keyword argument constructor lets you get away without explicitly defining any arguments. Warning: this goes against the "explicit is better than implicit" principle.
You need to override __getattr__ ONLY if you want to return a default value for an attribute that is not present instead of getting an AttributeError.
http://code.activestate.com/recipes/286185-automatically-initializing-instance-variables-from/
This recipe and its comments provide some methods.
Python: Automatically initialize instance variables?
This is a previous question.