Say I have this class:
class MyClass(object):
my_attrib = 'foo'
my_other_attrib = 'bar'
def mymethod():
pass
Now how can I get ONLY the attributes of the class MyClass, WITHOUT methods and builtins like __dict__ and so on?
I want to get a dictionary like {'my_attrib':'foo', 'my_other_attrib':'bar'}, when applied to the class above.
You can filter out everything you don't need from __dict__:
def getAttributes(clazz):
return {name: attr for name, attr in clazz.__dict__.items()
if not name.startswith("__")
and not callable(attr)
and not type(attr) is staticmethod}
Edit: An alternative that behaves slightly differently for class properties and descriptors:
def getAttributes2(clazz):
attrs = {}
for name in vars(clazz):
if name.startswith("__"):
continue
attr = getattr(clazz, name)
if callable(attr):
continue
attrs[name] = attr
return attrs
(In practice, this should be rarely different from the first version.)
This should get you close:
import inspect
class MyClass(object):
my_attrib = 'foo'
my_other_attrib = 'bar'
def mymethod():
pass
for name, value in inspect.getmembers(MyClass):
if not inspect.ismethod(value) and not name.startswith('__'):
print name
This outputs:
my_attrib
my_other_attrib
NOTE - There may be a better / more-official way to do this, but this should point you in the right direction.
__dict__ gives you all that but you could use a C extension maybe to get what you want. Not sure why you would do that though.
You can use types (doc) to distinguish between members of __dict__.
You can use the builtin dir() to get everything, then filter. You will not need the inspect module.
def get_attrs_without_methods(klass):
attrs = dir(klass)
d = {}
for x in attrs:
if x.startswith('__'): continue
value = getattr(self,x)
if not callable(value):
d[x] = value
return d
Sometimes, you may want to get ONLY class variables instead of class variables AND instance variable.
You can filter out instance variables by relying on __dict__. Or you can get the attributes using __class__ and filter out the methods. __class__ does not return instance variables.
#after collecting your attributes using the above example...
for attr, value in vars(obj).items():
d.pop(attr) #remove instance variables from the dict
#both vars(obj).items() and obj.__dict__.items() return similar iterable.
Note that if the object implementation overrides __dict__ and returns None, vars(obj) and obj.__dict__.items() will not return a dictionary.
Related
I think a code sample will better speak for itself:
class SomeClass:
example = create_get_method()
Yes, that's all – ideally.
In that case, create_get_method would add a get_example() to SomeClass in a way that it can be accessed via an instance of SomeClass:
obj = SomeClass()
obj.get_example() <- returns the value of self.example
(Of course, the idea is to implement a complex version of get_contact, that's why I want to do that in a non-repetitive way, and this is a simplified version that represents well the issue.)
I don't know if that's possible, because it require to have access to the property name (example) and the class (SomeClass) since these can not be guessed in advance (that function will be used on many and various classes).
I know it's something possible, because that's kind of what SQLAlchemy does with their relationship() function on a class:
class Model(BaseModel):
id = ...
contact_id = db.Integer(db.ForeignKey..)
contact = relationship('contact') <-- This !
How can this be done?
Objects bound to class-level variables can have a __set_name__ method that will be called immediately after the class object has been created. It will be called with two arguments, the class object, and the name of the variable the object is saved as in the class.
You could use this to create your extra getter method, though I'm not sure why exactly you want to (you could make the object a descriptor instead, which would probably be better than adding a separate getter function to the parent class).
class create_get_method:
def __set_name__(self, owner, name):
def getter(self):
return getattr(self, name)
getter_name = f"get_{name}"
getter.__name__ = getter_name
setattr(owner, getter_name, getter)
# you might also want a __get__ method here to give a default value (like None)
Here's how that would work:
>>> class Test:
... example = create_get_method()
...
>>> t = Test()
>>> print(t.get_example())
<__main__.create_get_method at 0x000001E0B4D41400>
>>> t.example = "foo"
>>> print(t.get_example())
foo
You could change the value returned by default (in the first print call), so that the create_get_method object isn't as exposed. Just add a __get__ method to the create_get_method class.
You can do this with a custom non-data descriptor, like a property, except that you don't need a __set__ method:
class ComplicatedDescriptor:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
Now you can have any number of classes that use this descriptor:
class SomeClass:
example = ComplicatedDescriptor('example')
Now you can do something like:
>>> inst0 = SomeClass()
>>> inst1 = SomeClass()
>>> inst0.example
1
>>> inst1.example
1
>>> inst1.example
2
>>> inst0.example
2
The line name = f'_{self.name} is necessary because the descriptor here is a non-data descriptor: it has no __set__ method, so if you create inst0.__dict__['example'], the lookup will no longer happen: inst0.example will return inst0.__dict__['example'] instead of calling SomeClass.example.__get__(inst0, type(inst0)). One workaround is to store the value under the attribute name _example. The other is to make your descriptor into a data descriptor:
class ComplicatedDescriptor_v2:
def __init__(self, name):
self.name = name
def __get__(self, owner, type):
# Here, `owner` is the instance of `SomeClass` that contains this descriptor
# Use `owner` to do some complicated stuff, like DB lookup or whatever
# These two lines for demo only
value = owner.__dict__.get(self.name, 0)
value += 1
owner.__dict__[self.name] = value
return value
def __set__(self, *args):
raise AttributeError(f'{self.name} is a read-only attribute')
The usage is generally identical:
class SomeClass:
example = ComplicatedDescriptor_v2('example')
Except that now you can't accidentally override your attribute:
>>> inst = SomeClass()
>>> inst.example
1
>>> inst.example
2
>>> inst.example = 0
AttributeError: example is a read-only attribute
Descriptors are a fairly idiomatic way to get and set values in python. They are preferred to getters and setters in almost all cases. The simplest cases are handled by the built-in property. That being said, if you wanted to explicitly have a getter method, I would recommend doing something very similar, but just returning a method instead of calling __get__ directly.
For example:
def __get__(self, owner, type):
def enclosed():
# Use `owner` to do some complicated stuff, like DB lookup or whatever
name = f'_{self.name}'
# These two lines for demo only
value = owner.__dict__.get(name, 0)
value += 1
setattr(owner, name, value)
return value
return enclosed
There is really no point to doing something like this unless you plan on really just want to be able to call inst.example().
Suppose we have the following class hierarchy:
class ClassA:
#property
def foo(self): return "hello"
class ClassB(ClassA):
#property
def bar(self): return "world"
If I explore __dict__ on ClassB like so, I only see the bar attribute:
for name,_ in ClassB.__dict__.items():
if name.startswith("__"):
continue
print(name)
Output is bar
I can roll my own means to get attributes on not only the specified type but its ancestors. However, my question is whether there's already a way in python for me to do this without re-inventing a wheel.
def return_attributes_including_inherited(type):
results = []
return_attributes_including_inherited_helper(type,results)
return results
def return_attributes_including_inherited_helper(type,attributes):
for name,attribute_as_object in type.__dict__.items():
if name.startswith("__"):
continue
attributes.append(name)
for base_type in type.__bases__:
return_attributes_including_inherited_helper(base_type,attributes)
Running my code as follows...
for attribute_name in return_attributes_including_inherited(ClassB):
print(attribute_name)
... gives back both bar and foo.
Note that I'm simplifying some things: name collisions, using items() when for this example I could use dict, skipping over anything that starts with __, ignoring the possibility that two ancestors themselves have a common ancestor, etc.
EDIT1 - I tried to keep the example simple. But I really want both the attribute name and the attribute reference for each class and ancestor class. One of the answers below has me on a better track, I'll post some better code when I get it to work.
EDIT2 - This does what I want and is very succinct. It's based on Eli's answer below.
def get_attributes(type):
attributes = set(type.__dict__.items())
for type in type.__mro__:
attributes.update(type.__dict__.items())
return attributes
It gives back both the attribute names and their references.
EDIT3 - One of the answers below suggested using inspect.getmembers. This appears very useful because it's like dict only it operates on ancestor classes as well.
Since a large part of what I was trying to do was find attributes marked with a particular descriptor, and include ancestors classes, here is some code that would help do that in case it helps anyone:
class MyCustomDescriptor:
# This is greatly oversimplified
def __init__(self,foo,bar):
self._foo = foo
self._bar = bar
pass
def __call__(self,decorated_function):
return self
def __get__(self,instance,type):
if not instance:
return self
return 10
class ClassA:
#property
def foo(self): return "hello"
#MyCustomDescriptor(foo="a",bar="b")
def bar(self): pass
#MyCustomDescriptor(foo="c",bar="d")
def baz(self): pass
class ClassB(ClassA):
#property
def something_we_dont_care_about(self): return "world"
#MyCustomDescriptor(foo="e",bar="f")
def blah(self): pass
# This will get attributes on the specified type (class) that are of matching_attribute_type. It just returns the attributes themselves, not their names.
def get_attributes_of_matching_type(type,matching_attribute_type):
return_value = []
for member in inspect.getmembers(type):
member_name = member[0]
member_instance = member[1]
if isinstance(member_instance,matching_attribute_type):
return_value.append(member_instance)
return return_value
# This will return a dictionary of name & instance of attributes on type that are of matching_attribute_type (useful when you're looking for attributes marked with a particular descriptor)
def get_attribute_name_and_instance_of_matching_type(type,matching_attribute_type):
return_value = {}
for member in inspect.getmembers(ClassB):
member_name = member[0]
member_instance = member[1]
if isinstance(member_instance,matching_attribute_type):
return_value[member_name] = member_instance
return return_value
You should use python's inspect module for any such introspective capabilities.
.
.
>>> class ClassC(ClassB):
... def baz(self):
... return "hiya"
...
>>> import inspect
>>> for attr in inspect.getmembers(ClassC):
... print attr
...
('__doc__', None)
('__module__', '__main__')
('bar', <property object at 0x10046bf70>)
('baz', <unbound method ClassC.baz>)
('foo', <property object at 0x10046bf18>)
Read more about the inspect module here.
You want to use dir:
for attr in dir(ClassB):
print attr
Sadly there isn't a single composite object. Every attribute access for a (normal) python object first checks obj.__dict__, then the attributes of all it's base classes; while there are some internal caches and optimizations, there isn't a single object you can access.
That said, one thing that could improve your code is to use cls.__mro__ instead of cls.__bases__... instead of the class's immediate parents, cls.__mro__ contains ALL the ancestors of the class, in the exact order Python would search, with all common ancestors occuring only once. That would also allow your type-searching method to be non-recursive. Loosely...
def get_attrs(obj):
attrs = set(obj.__dict__)
for cls in obj.__class__.__mro__:
attrs.update(cls.__dict__)
return sorted(attrs)
... does a fair approximation of the default dir(obj) implementation.
Here is a function I wrote, back in the day. The best answer is using the inspect module, as using __dict__ gives us ALL functions (ours + inherited) and (ALL?) data members AND properties. Where inspect gives us enough information to weed out what we don't want.
def _inspect(a, skipFunctionsAlways=True, skipMagic = True):
"""inspects object attributes, removing all the standard methods from 'object',
and (optionally) __magic__ cruft.
By default this routine skips __magic__ functions, but if you want these on
pass False in as the skipMagic parameter.
By default this routine skips functions, but if you want to see all the functions,
pass in False to the skipFunctionsAlways function. This works together with the
skipMagic parameter: if the latter is True, you won't see __magic__ methods.
If skipFunctionsAlways = False and skipMagic = False, you'll see all the __magic__
methods declared for the object - including __magic__ functions declared by Object
NOT meant to be a comprehensive list of every object attribute - instead, a
list of every object attribute WE (not Python) defined. For a complete list
of everything call inspect.getmembers directly"""
objType = type(object)
def weWantIt(obj):
#return type(a) != objType
output= True
if (skipFunctionsAlways):
output = not ( inspect.isbuiltin(obj) ) #not a built in
asStr = ""
if isinstance(obj, types.MethodType):
if skipFunctionsAlways: #never mind, we don't want it, get out.
return False
else:
asStr = obj.__name__
#get just the name of the function, we don't want the whole name, because we don't want to take something like:
#bound method LotsOfThings.bob of <__main__.LotsOfThings object at 0x103dc70>
#to be a special method because it's module name is special
#WD-rpw 02-23-2008
#TODO: it would be great to be able to separate out superclass methods
#maybe by getting the class out of the method then seeing if that attribute is in that class?
else:
asStr = str(obj)
if (skipMagic):
output = (asStr.find("__") == -1 ) #not a __something__
return (output)
for value in inspect.getmembers( a, weWantIt ):
yield value
{k: getattr(ClassB, k) for k in dir(ClassB)}
Proper values (instead of <property object...>) will be presented when using ClassB instance.
And of course You can filter this by adding things like if not k.startswith('__') in the end.
I was messing around with inherited classes and wondered if it was possible to set a custom object attribute using a method.
It would work something like this:
class MyClass(object):
def __init__(self):
super.__init__()
def setCustAttr(self, name, value):
#...
g=MyClass()
g.setCustAttr("var",5)
g.var+=6
g.var="text"
Is there any way to do this?
Could you use exec("self."+string+"="+value)?
This is what the setattr function does:
setattr(g, 'var', 5)
# g.var is now 5
The arguments passed to setCustAttr are exactly the arguments you would pass to setattr.
def setCustAttr(self, name, value):
setattr(self, name, value)
Why would you want a wrapper around setattr? You might try to perform some validation:
def setCustAttr(self, name, value):
if name not in ['bar', 'baz']:
raise ValueError("Custom attribute must be 'bar' or 'baz'")
if name == 'bar' and value < 0:
raise ValueError("'bar' attribute must be non-negative")
if name == 'baz' and value % 2:
raise ValueError("'baz' attribute must be even")
setattr(self, name, value)
However, this doesn't prevent the user of your class from ignoring your setCustAttr method and assigning directly to the object:
g = MyClass()
g.bar = -5 # Negative bar!
g.baz = 3 # Odd baz!
g.quux = 2 # Non-bar/baz attribute!
Python has deep magic for providing more control over how attributes are set on an object (see __slots__, __{get,set}attr__, __getattribute__, properties, etc), but generally, they aren't used merely to prevent the examples shown above. The Python way is to just document how an instance of your class should be used, and trust the user to abide by your instructions. (And if they don't, caveat emptor.)
Suppose we have the following class hierarchy:
class ClassA:
#property
def foo(self): return "hello"
class ClassB(ClassA):
#property
def bar(self): return "world"
If I explore __dict__ on ClassB like so, I only see the bar attribute:
for name,_ in ClassB.__dict__.items():
if name.startswith("__"):
continue
print(name)
Output is bar
I can roll my own means to get attributes on not only the specified type but its ancestors. However, my question is whether there's already a way in python for me to do this without re-inventing a wheel.
def return_attributes_including_inherited(type):
results = []
return_attributes_including_inherited_helper(type,results)
return results
def return_attributes_including_inherited_helper(type,attributes):
for name,attribute_as_object in type.__dict__.items():
if name.startswith("__"):
continue
attributes.append(name)
for base_type in type.__bases__:
return_attributes_including_inherited_helper(base_type,attributes)
Running my code as follows...
for attribute_name in return_attributes_including_inherited(ClassB):
print(attribute_name)
... gives back both bar and foo.
Note that I'm simplifying some things: name collisions, using items() when for this example I could use dict, skipping over anything that starts with __, ignoring the possibility that two ancestors themselves have a common ancestor, etc.
EDIT1 - I tried to keep the example simple. But I really want both the attribute name and the attribute reference for each class and ancestor class. One of the answers below has me on a better track, I'll post some better code when I get it to work.
EDIT2 - This does what I want and is very succinct. It's based on Eli's answer below.
def get_attributes(type):
attributes = set(type.__dict__.items())
for type in type.__mro__:
attributes.update(type.__dict__.items())
return attributes
It gives back both the attribute names and their references.
EDIT3 - One of the answers below suggested using inspect.getmembers. This appears very useful because it's like dict only it operates on ancestor classes as well.
Since a large part of what I was trying to do was find attributes marked with a particular descriptor, and include ancestors classes, here is some code that would help do that in case it helps anyone:
class MyCustomDescriptor:
# This is greatly oversimplified
def __init__(self,foo,bar):
self._foo = foo
self._bar = bar
pass
def __call__(self,decorated_function):
return self
def __get__(self,instance,type):
if not instance:
return self
return 10
class ClassA:
#property
def foo(self): return "hello"
#MyCustomDescriptor(foo="a",bar="b")
def bar(self): pass
#MyCustomDescriptor(foo="c",bar="d")
def baz(self): pass
class ClassB(ClassA):
#property
def something_we_dont_care_about(self): return "world"
#MyCustomDescriptor(foo="e",bar="f")
def blah(self): pass
# This will get attributes on the specified type (class) that are of matching_attribute_type. It just returns the attributes themselves, not their names.
def get_attributes_of_matching_type(type,matching_attribute_type):
return_value = []
for member in inspect.getmembers(type):
member_name = member[0]
member_instance = member[1]
if isinstance(member_instance,matching_attribute_type):
return_value.append(member_instance)
return return_value
# This will return a dictionary of name & instance of attributes on type that are of matching_attribute_type (useful when you're looking for attributes marked with a particular descriptor)
def get_attribute_name_and_instance_of_matching_type(type,matching_attribute_type):
return_value = {}
for member in inspect.getmembers(ClassB):
member_name = member[0]
member_instance = member[1]
if isinstance(member_instance,matching_attribute_type):
return_value[member_name] = member_instance
return return_value
You should use python's inspect module for any such introspective capabilities.
.
.
>>> class ClassC(ClassB):
... def baz(self):
... return "hiya"
...
>>> import inspect
>>> for attr in inspect.getmembers(ClassC):
... print attr
...
('__doc__', None)
('__module__', '__main__')
('bar', <property object at 0x10046bf70>)
('baz', <unbound method ClassC.baz>)
('foo', <property object at 0x10046bf18>)
Read more about the inspect module here.
You want to use dir:
for attr in dir(ClassB):
print attr
Sadly there isn't a single composite object. Every attribute access for a (normal) python object first checks obj.__dict__, then the attributes of all it's base classes; while there are some internal caches and optimizations, there isn't a single object you can access.
That said, one thing that could improve your code is to use cls.__mro__ instead of cls.__bases__... instead of the class's immediate parents, cls.__mro__ contains ALL the ancestors of the class, in the exact order Python would search, with all common ancestors occuring only once. That would also allow your type-searching method to be non-recursive. Loosely...
def get_attrs(obj):
attrs = set(obj.__dict__)
for cls in obj.__class__.__mro__:
attrs.update(cls.__dict__)
return sorted(attrs)
... does a fair approximation of the default dir(obj) implementation.
Here is a function I wrote, back in the day. The best answer is using the inspect module, as using __dict__ gives us ALL functions (ours + inherited) and (ALL?) data members AND properties. Where inspect gives us enough information to weed out what we don't want.
def _inspect(a, skipFunctionsAlways=True, skipMagic = True):
"""inspects object attributes, removing all the standard methods from 'object',
and (optionally) __magic__ cruft.
By default this routine skips __magic__ functions, but if you want these on
pass False in as the skipMagic parameter.
By default this routine skips functions, but if you want to see all the functions,
pass in False to the skipFunctionsAlways function. This works together with the
skipMagic parameter: if the latter is True, you won't see __magic__ methods.
If skipFunctionsAlways = False and skipMagic = False, you'll see all the __magic__
methods declared for the object - including __magic__ functions declared by Object
NOT meant to be a comprehensive list of every object attribute - instead, a
list of every object attribute WE (not Python) defined. For a complete list
of everything call inspect.getmembers directly"""
objType = type(object)
def weWantIt(obj):
#return type(a) != objType
output= True
if (skipFunctionsAlways):
output = not ( inspect.isbuiltin(obj) ) #not a built in
asStr = ""
if isinstance(obj, types.MethodType):
if skipFunctionsAlways: #never mind, we don't want it, get out.
return False
else:
asStr = obj.__name__
#get just the name of the function, we don't want the whole name, because we don't want to take something like:
#bound method LotsOfThings.bob of <__main__.LotsOfThings object at 0x103dc70>
#to be a special method because it's module name is special
#WD-rpw 02-23-2008
#TODO: it would be great to be able to separate out superclass methods
#maybe by getting the class out of the method then seeing if that attribute is in that class?
else:
asStr = str(obj)
if (skipMagic):
output = (asStr.find("__") == -1 ) #not a __something__
return (output)
for value in inspect.getmembers( a, weWantIt ):
yield value
{k: getattr(ClassB, k) for k in dir(ClassB)}
Proper values (instead of <property object...>) will be presented when using ClassB instance.
And of course You can filter this by adding things like if not k.startswith('__') in the end.
I need to make a bunch of class variables and I would like to do it by looping through a list like that:
vars=('tx','ty','tz') #plus plenty more
class Foo():
for v in vars:
setattr(no_idea_what_should_go_here,v,0)
is it possible? I don't want to make them for an instance (using self in the __init__) but as class variables.
You can run the insertion code immediately after a class is created:
class Foo():
...
vars=('tx', 'ty', 'tz') # plus plenty more
for v in vars:
setattr(Foo, v, 0)
Also, you can dynamically store the variable while the class is being created:
class Bar:
locals()['tx'] = 'texas'
Late to the party but use the type class constructor!
Foo = type("Foo", (), {k: 0 for k in ("tx", "ty", "tz")})
If for any reason you can't use Raymond's answer of setting them up after the class creation then perhaps you could use a metaclass:
class MetaFoo(type):
def __new__(mcs, classname, bases, dictionary):
for name in dictionary.get('_extra_vars', ()):
dictionary[name] = 0
return type.__new__(mcs, classname, bases, dictionary)
class Foo(): # For python 3.x use 'class Foo(metaclass=MetaFoo):'
__metaclass__=MetaFoo # For Python 2.x only
_extra_vars = 'tx ty tz'.split()
The locals() version did not work for me in a class.
The following can be used to dynamically create the attributes of the class:
class namePerson:
def __init__(self, value):
exec("self.{} = '{}'".format("name", value)
me = namePerson(value='my name')
me.name # returns 'my name'
setattr(object, name, value)
This is the counterpart of getattr(). The arguments are an object, a string and an arbitrary value. The string may name an existing attribute or a new attribute. The function assigns the value to the attribute, provided the object allows it.
For example, setattr(x, 'name', value) is equivalent to x.name = value.
The function you need is:
setattr(obj, name, value)
This allows you to set named attributes for a given class (this can be self).
The built in documentation for this function is pretty self-explanatory:
Signature: setattr(obj, name, value, /)
Docstring:
Sets the named attribute on the given object to the specified value.
setattr(x, 'y', v) is equivalent to ``x.y = v''
Type: builtin_function_or_method
Example use
One use of this is to use a dictionary to set multiple class attributes, in my case this was from xpath definitions. I felt this improved maintainability by keeping potentially more fragile xpath definitions all in one place:
class Person:
def _extract_fields(self):
''' Process the page using XPath definitions '''
logging.debug("_extract_fields(): {}".format(repr(self)))
# describe how to extract data from the publicly available site
# (kept together for maintainability)
fields = {
'staff_name':
'//div[#id="staff_name"]//text()',
'staff_dob':
'(//div[#id="staff_dob"]//text())[1]'
}
# populate named class attributes from the dict
for key in fields:
setattr(self, key, self._parsed_content.xpath(fields[key]))
def __init__(self):
self._extract_fields()