I have been using namedlist to create lightweight classes:
from namedlist import namedlist
# create a class
SomeClass = namedlist('SomeClass', 'foo bar', use_slots=False)
# create an object
my_list = SomeClass(1,2)
# set an attribute not specified in the class
my_list.baz = 3
# the attribute is there if I reference it
print(my_list.baz)
# output: 3
Sometimes I want to take an object and see if any extra attributes have been set:
# this doesn't show 'baz'
import inspect
inspect.getmembers(my_list)
# neither does this
my_list.__dict__
Is there a way I can see any attributes that have been added in this way?
Looking at the source of namedlist, we can see that the factory function namedlist(), generates the type (SomeClass in your example).
Now this is interesting.
On one hand, __getattribute__ and __setattribute__ were not overloaded, which lets you do things like my_list.baz = 3 and then access it as my_list.baz.
On the other, __dict__, was overridden with property(_asdict) (generated in _common_fields()). This causes whoever uses __dict__ to fail seeing baz - function such as dir() and the inspect module.
While I failed to find a function that will list the added attributes in this case, if you know what attribute you are looking for, you can still check if it exists using hasattr(my_list, 'baz'):
>>> from namedlist import namedlist
>>> SomeClass = namedlist('SomeClass', 'foo bar', use_slots=False)
>>> my_list = SomeClass(1,2)
>>> my_list.baz = 3
>>> hasattr(my_list, 'baz')
True
If switching types is going to be problematic (maybe there is legacy code already using namedlist), I found that the following makes viewing namedlist bearable:
def set_attr(self, attr_name, attr_val):
setattr(self, attr_name, attr_val)
self.opt_attrs.append(attr_name)
TypeA = namedlist('TypeA', 'field_a opt_attrs', use_slots=False)
TypeA.set_attr = set_attr
TypeB = namedlist('TypeB', 'field_b opt_attrs', use_slots=False)
TypeB.set_attr = set_attr
objA = TypeA(1, [])
objA.set_attr('field_x', 2)
objB = TypeB(7, [])
objA
# Out: TypeA(field_a=1, opt_attrs=['field_x'])
objA.field_x
# Out: 2
objB
# Out: TypeB(field_b=7, opt_attrs=[])
It is probably best to just to use python classes though. More up-front code, less after-the-fact confusion:
class TypeA:
def __init__(self, a):
self.a = a
def __repr__(self):
return "A(a={})".format(self.a)
class TypeB:
def __init__(self, b):
self.b = b
def __repr__(self):
return "B(b={})".format(self.b)
A = TypeA(1)
A.x = 2
B = TypeB(7)
class TypeA:
def __init__(self, a):
self.a = a
def __repr__(self):
return "A(a={})".format(self.a)
class TypeB:
def __init__(self, b):
self.b = b
def __repr__(self):
return "B(b={})".format(self.b)
objA = TypeA(1)
objA.x = 2
objB = TypeB(7)
objA
# Out: A(a=1)
objA.__dict__
# Out: {'a': 1, 'x': 2}
objB
# Out: B(b=7)
Related
Let's say we have class with data:
class Foo:
def __init__(self, x, y):
self.x = x
self.y = y
And collection class:
class Bar:
def __init__(self, foos):
self.foos = []
if foos:
self.foos = foos
def set_z(self):
for foo in self.foos:
foo.z = randint()
def print_z(self):
print([foo.z for foo in self.foos])
Basic stuff. Now the question.
How can we store additional variable z in each object of class Foo, but different for each instance of class Bar this object is in.
What I want to do:
>>> f1 = Foo(x=13, y=42)
>>> f2 = Foo(x=-3, y=21)
>>> b1 = Bar(foos=[f1, f2])
>>> b2 = Bar(foos=[f1, f2])
>>> b1.set_z()
>>> b2.set_z()
>>> b1.print_z()
[9, 11]
>>> b2.print_z()
[32, 8]
First thought is to make wrapper class like this:
class FooWrapper:
def __init__(self, foo):
self.foo = foo
self.z = None
And change Bar to automatically wrap each object:
class Bar:
def __init__(self, foos):
self.foos = []
if foos:
self.foos = [FooWrapper(foo) for foo in foos]
Is there maybe cleaner way without writting additional class? It not look bad here, but when there is lot of different properties in both base class and wrapper class, it becomes messy. Changing bar.foos into dict is not an option, since it's not guaranteed foos will all be hashable.
But now looking at Python 3.7 docs I read about contextvars. It seems like this is something that can be used in this case, but I have problem grasping this concept. Can I set every instance of class Bar as context and write z as contextvar inside class Foo? Will it be reasonable?
An instance can only have a single value for a given attribute at any given time, but as long as you're accessing this "attribute" through the container, you can store the values on the container.
One way is to construct a new list in which each item is a 2-tuple consisting of a foo and a dict. The dict can then hold the z values in a manner that supports adding other attributes and values.
from random import randint
class Foo:
def __init__(self, x, y):
self.x = x
self.y = y
class Bar:
def __init__(self, foos):
self.foos_with_benefits = [(foo, {}) for foo in foos]
def set_attribute(self, name):
for foo, attributes in self.foos_with_benefits:
attributes[name] = randint(1, 100)
def print_attribute(self, name):
print([attributes[name] for _, attributes in self.foos_with_benefits])
def set_z(self):
self.set_attribute('z')
def print_z(self):
self.print_attribute('z')
It performs along the lines of what you were requesting:
>>> f1 = Foo(x=13, y=42)
>>> f2 = Foo(x=-3, y=21)
>>> b1 = Bar(foos=[f1, f2])
>>> b2 = Bar(foos=[f1, f2])
>>> b1.set_z()
>>> b2.set_z()
>>> b1.print_z()
[81, 19]
>>> b2.print_z()
[66, 99]
As for contextvars, they don't seem well-suited to this kind of use case. For one thing, the value of a contextvar can vary across async contexts, not across instances within the same async context. If you're interested in learning more about how you might use contextvars, I've written a context manager for using contextvars on the fly: FlexContext.
class Books():
def __init__(self):
self.__dict__['referTable'] = 1
#property
def referTable(self):
return 2
book = Books()
print(book.referTable)
print(book.__dict__['referTable'])
Running:
vic#ubuntu:~/Desktop$ python3 test.py
2
1
Books.referTable being a data descriptor is not shadowed by book.__dict__['referTable']:
The property() function is implemented as a data descriptor.
Accordingly, instances cannot override the behavior of a property.
To shadow it, instead of property built-in descriptor i must use my own descriptor. Is there a built in descriptor like property but which is non-data?
To expand on my comment, why not simply something like this:
>>> class Books():
... def __init__(self):
... self.__dict__['referTable'] = 1
... #property
... def referTable(self):
... try:
... return self.__dict__['referTable']
... except KeyError:
... return 2
...
>>> a = Books()
>>> a.referTable
1
>>> del a.__dict__['referTable']
>>> a.referTable
2
Now, I'd like to note that I don't think this is good design, and you'd be much better off using a private variable rather than accessing __dict__ directly. E.g:
class Books():
def __init__(self):
self._referTable = 1
#property
def referTable(self):
return self._referTable if self._referTable else 2
In short, the answer is no, there is no alternative to property() that works in the way you want in the Python standard library.
There is something very similar to a built-in non-data descriptor -- the class attribute:
class Books():
referTable = 'default'
def __init__(self, referTable=None):
if referTable is not None:
self.referTable = referTable
book = Books()
print(book.referTable)
# default
book.referTable = 'something specific'
print(book.referTable)
# something specific
If you need something more like a property (for example, you want a function to do some heavy-lifting the first time, but then use that first value for all future references), then you will need to build it yourself:
class OneTime(object):
def __init__(self, method):
self.name = method.__name__
self.method = method
def __get__(self, inst, cls):
if inst is None:
return self
result = self.method(inst)
inst.__dict__[self.name] = result
return result
class Books(object):
#OneTime
def referTable(self):
print 'calculating'
return 1 * 2 * 3 * 4 * 5
b = Books()
print b.__dict__
print b.referTable
print b.__dict__
print b.referTable
With the following results:
{}
calculating
120
{'referTable': 120}
120
I am dynamically creating some classes and I want them to have different docstrings. I have:
def make_class(class_docstring):
class X:
pass
X.__doc__ = class_docstring
return X
That didn't work because docstrings are read-only. Then, I tried:
def make_class(class_name, class_docstring):
class X:
def __init__(self):
super().__init__()
d = {'__doc__': class_docstring}
d.update(X.__dict__)
return type(class_name, (), d)
ClassName = make_class(
'ClassName',
"""
Some docstring...
""")
which worked until it had to call super.
What is the correct way to dynamically set the docstring attribute?
You can set the docstring inside the class.
>>> def make_class(class_docstring):
... class X:
... __doc__ = class_docstring
... return X
...
>>> x = make_class('test doc')
>>> x
<class '__main__.X'>
>>> xx = x()
>>> xx.__doc__
'test doc'
I'm not sure why your 2nd attempt is failing.
I was wondering if it is somehow possible to modify Python code live, while keeping all state of instantiated objects and methods, like I think is possible in Lisp or Erlang (*) ?
Say, I have an active Python sessions, where I instantiated the foo class from a self-written module:
class foo():
#classmethod
def do_something(self):
print "this is good"
Python command line:
>>> f =foo()
>>> f.do_something()
Now, I would like to change the print statement into something else (e.g. print "this is better"). If I edit my module file to do so, and reload it, I have to re-instantiate the f object. Is there a way to be able to just call f.do_something() again without having to call f=foo() first?
So, I have to do this:
>>> reload my_module
>>> f =foo()
>>> f.do_something() # with changed print statement
But I want to do this:
>>> reload my_module
>>> f.do_something() # with changed print statement
(*) I am basing this statement on the cool Erlang movie and this fragment from Practical Common Lisp: 'When the bug manifested in the wild--100 million miles away from Earth--the team was able to diagnose and fix the running code, allowing the experiments to complete.'
Edit: I've been thinking a bit more about this and maybe what I want is inherently flawed for applying to OO (i.e., what about the state of the class and methods). I think Erlang allows this because, as far as I recall, it is more about separate communicating objects, so live updating the code of an object makes more sense. I am not sure though, so still open for answers.
Edit2: Maybe the best way to describe what I want is recapitulate what I said in a comment in a post below: "When called, the methods just have to point to the new method definitions/locations."
Yes you can, pretty simple too. You want to change only the instance f, not the class foo, right?
>>> class foo():
#classmethod
def do_something(self):
print "this is good"
>>> f = foo()
>>> f.do_something()
this is good
>>> def new_ds():
print "this is better"
>>> f.do_something = new_ds
>>> f.do_something()
this is better
>>> f2 = foo()
>>> f2.do_something() #unchanged
this is good
EDIT
This is almost certainly less than desirable due to the change in scope, but changes like this took place for me immediately upon reload
testmod.py -- initially
class foo():
#classmethod
def do_something(self):
outside_func()
def outside_func():
print "this is good"
testmod.py -- after change
class foo():
#classmethod
def do_something(self):
outside_func()
def outside_func():
print "this is better"
Interpreter
>>> import testmod
>>> f = testmod.foo()
>>> f.do_something()
this is good
>>> reload(testmod)
<module 'testmod' from 'C:\Python26\testmod.py'>
>>> f.do_something()
this is better
You can create a class decorator or a metaclass that makes sure that the class of the old objects is changed on class reload. Here's a working (at least for me) example, though I wouldn't suggest you to use it as it is, but use it as an inspiration to create something that matches your intentions and needs. (It is also not tested on classes that don't define __init__, so be wary.)
import sys
import weakref
class _ObSet(weakref.WeakValueDictionary):
def add(self, ob):
self[id(ob)] = ob
def remove(self, ob):
del self[id(ob)]
def __iter__(self):
return self.itervalues()
def reloadable(cls):
# Change the __init__ of the old class to store the instances
# in cls.__instances (you might stick this into a class as a
# static method to avoid name collisions)
if '__init__' in vars(cls):
old_init = vars(cls)['__init__']
def __init__(self, *a, **kw):
self.__class__.__instances.add(self)
old_init(self, *a, **kw)
cls.__init__ = __init__
elif '__new__' in vars(cls):
old_new = vars(cls)['__new__']
def __new__(cls, *a, **kw):
self = old_new(cls, *a, **kw)
cls.__instances.add(self)
return self
cls.__new__ = __new__
else:
def __init__(self, *a, **kw):
self.__class__.__instances.add(self)
super(cls, self).__init__(*a, **kw)
cls.__init__ = __init__
cls.__instances = _ObSet()
module = sys.modules.get(cls.__module__)
if module is None:
return cls
old_cls = getattr(module, cls.__name__, None)
if old_cls is None:
return cls
# Change the bases of all subclasses of the old class
for ob in old_cls.__instances:
if ob.__class__ is old_cls:
ob.__class__ = cls
# Change the class of all instances of the old class
for child_cls in old_cls.__subclasses__():
child_cls.__bases__ = tuple(cls if base is old_cls else base
for base in child_cls.__bases__)
return cls
Here's an example of how it is used:
from reloading import reloadable
#reloadable
class A(object):
def __init__(self, a, b):
self.a = a
self.b = b
class B1(A):
def __init__(self, c, *a):
super(B1, self).__init__(*a)
self.c = c
#reloadable
class B2(A):
def __init__(self, c, *a):
super(B2, self).__init__(*a)
self.c = c
And then how it works:
>>> import test_reload
>>> a = test_reload.A(1, 2)
>>> b1 = test_reload.B1(1, 2, 3)
>>> b2 = test_reload.B2(1, 4, 6)
>>> isinstance(a, test_reload.A)
True
>>> isinstance(b1, test_reload.A)
True
>>> isinstance(b1, test_reload.B1)
True
>>> isinstance(b2, test_reload.A)
True
>>> isinstance(b2, test_reload.B2)
True
>>> reload(test_reload)
<module 'test_reload' from 'test_reload.pyc'>
>>> isinstance(a, test_reload.A)
True
>>> isinstance(b1, test_reload.A)
True
>>> isinstance(b1, test_reload.B1) # will fail, not #reloadable
False
>>> isinstance(b2, test_reload.A)
True
>>> isinstance(b2, test_reload.B2)
True
>>> a.a, a.b
(1, 2)
>>> b1.a, b1.b, b1.c
(2, 3, 1)
>>> b2.a, b2.b, b2.c
(4, 6, 1)
This shows that you can modify an existing class and have those changes be manifest in instances of that class. The key is to modify the existing class rather than (re)create a new class with the same name as the old class.
>>> class foo():
... #classmethod
... def do_something(self):
... print "this is good"
...
>>> f = foo()
>>> f.do_something()
this is good
>>> def do_something_else(self):
... print "this is better"
...
>>> foo.do_something = do_something_else
>>> f.do_something()
this is better
This question already has answers here:
Automatically initialize instance variables?
(17 answers)
Closed last month.
Say, I have the following class in Python
class Foo(object):
a = None
b = None
c = None
def __init__(self, a = None, b = None, c = None):
self.a = a
self.b = b
self.c = c
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
Please note that
class Foo(object):
a = None
sets a key-value pair in Foo's dict:
Foo.__dict__['a']=None
while
def __init__(self, a = None, b = None, c = None):
self.a = a
sets a key-value pair in the Foo instance object's dict:
foo=Foo()
foo.__dict__['a']=a
So setting the class members at the top of your definition is not directly related to the setting of the instance attributes in the lower half of your definition (inside the __init__.
Also, it is good to be aware that __init__ is Python's initializer. __new__ is the class constructor.
If you are looking for a way to automatically add some instance attributes based on __init__'s arguments, you could use this:
import inspect
import functools
def autoargs(*include,**kwargs):
def _autoargs(func):
attrs,varargs,varkw,defaults=inspect.getargspec(func)
def sieve(attr):
if kwargs and attr in kwargs['exclude']: return False
if not include or attr in include: return True
else: return False
#functools.wraps(func)
def wrapper(self,*args,**kwargs):
# handle default values
for attr,val in zip(reversed(attrs),reversed(defaults)):
if sieve(attr): setattr(self, attr, val)
# handle positional arguments
positional_attrs=attrs[1:]
for attr,val in zip(positional_attrs,args):
if sieve(attr): setattr(self, attr, val)
# handle varargs
if varargs:
remaining_args=args[len(positional_attrs):]
if sieve(varargs): setattr(self, varargs, remaining_args)
# handle varkw
if kwargs:
for attr,val in kwargs.iteritems():
if sieve(attr): setattr(self,attr,val)
return func(self,*args,**kwargs)
return wrapper
return _autoargs
So when you say
class Foo(object):
#autoargs()
def __init__(self,x,path,debug=False,*args,**kw):
pass
foo=Foo('bar','/tmp',True, 100, 101,verbose=True)
you automatically get these instance attributes:
print(foo.x)
# bar
print(foo.path)
# /tmp
print(foo.debug)
# True
print(foo.args)
# (100, 101)
print(foo.verbose)
# True
PS. Although I wrote this (for fun), I don't recommend using autoargs for serious work. Being explicit is simple, clear and infallible. I can't say the same for autoargs.
PPS. Is it just me, or are a lot of buttons broken on Stackoverflow? The editor window has lost all its icons... :( Clearing the browser cache fixed the problem.
Python 3.7 provides dataclasses which are helpful in situations like this:
from dataclasses import dataclass
#dataclass
class Foo:
a: str = None
b: str = None
c: str = None
This saves you from having to write out the __init__ method when you just want to store a few attributes.
Gives you a good __repr__ method:
>>> a = Foo()
>>> a
Foo(a=None, b=None, c=None)
If you need to do calculations on a param, you can implement __post_init__.
See also namedtuple:
from collections import namedtuple
Foo = namedtuple('Foo', ['a', 'b', 'c'])
All fields are required with namedtuple though.
>>> a = Foo(1, 2, 3)
>>> a
Foo(a=1, b=2, c=3)
There are elegant ways to do this.
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
There is also a crude way. It will work, but is NOT recommended. See and decide.
>>> class Foo(object):
def __init__(self, **attrs):
self.__dict__.update(**attrs)
def __getattr__(self, attr):
return self.__dict__.get(attr, None)
>>> f = Foo(a = 1, b = 2, c = 3)
>>> f.a, f.b
(1, 2)
>>> f = Foo(bar = 'baz')
>>> f.bar
'baz'
>>> f.a
>>>
The keyword argument constructor lets you get away without explicitly defining any arguments. Warning: this goes against the "explicit is better than implicit" principle.
You need to override __getattr__ ONLY if you want to return a default value for an attribute that is not present instead of getting an AttributeError.
http://code.activestate.com/recipes/286185-automatically-initializing-instance-variables-from/
This recipe and its comments provide some methods.
Python: Automatically initialize instance variables?
This is a previous question.