I have some classes in Python:
class Class1:
def method(self):
return 1
class Class2:
def method(self):
return 2
and a list myList whose elements are all either instances of Class1 or Class2. I'd like to create a new list whose elements are the return values of method called on each element of myList. I have tried using a "virtual" base class
class Class0:
def method(self):
return 0
class Class1(Class0):
def method(self):
return 1
class Class2(Class0):
def method(self):
return 2
But if I try map(Class0.method, myList) I just get [0, 0, 0, ...]. I'm a bit new to Python, and I hear that "duck typing" is preferred to actual inheritance, so maybe this is the wrong approach. Of course, I can do
[myList[index].method() for index in xrange(len(myList))]
but I like the brevity of map. Is there a way to still use map for this?
You can use
map(lambda e: e.method(), myList)
But I think this is better:
[e.method() for e in myList]
PS.: I don't think there is ever a need for range(len(collection)).
The operator.methodcaller tool is exactly what you're looking for:
map(methodcaller("method"), myList)
Alternatively you can use a list comprehension:
[obj.method() for obj in myList]
This is best:
[o.method() for o in myList]
Map seems to be favored by people pining for Haskell or Lisp, but Python has fine iterative structures you can use instead.
Related
When initializing an attribute in a class, is there a reason to avoid using a lambda expression or list comprehension, preferring a method or vice versa?
In example:
With
class Foo():
def __init__(self, data):
self.data = data
List Comprehension
class BarListComp():
def __init__(self, listOfFoo):
self.data = [fooInst.data for fooInst in listOfFoo if fooInst.data % 2 == 0]
Lambda
class BarLambda():
def __init__(self, listOfFoo):
self.data = list(filter(lambda x: x % 2 == 0, map(lambda y: y.data, listOfFoo)))
Method
class BarMethod():
def __init__(self, listOfFoo):
self.data = self.genData(listOfFoo)
#static_method
def genData(listOfFoo):
# this could also just be the earlier list comprehension
listOut = []
for elem in listOfFoo:
if elem.data % 2 == 0:
listOut.append(elem.data)
return listOut
(Please note these might not be the greatest examples, and the needed processing could be much more complicated)
Is there a preferred method? Does an initialization process reach a suitable complexity to necessitate a new method to be split off?
Just to add to #NPE comment (because I truly agree with his statement),
is there a reason to avoid using a lambda expression or list comprehension, preferring a method or vice versa?
The answer is Yes and No.
My ambiguous answer is the consequence of the reality being, choose wathever is more readable for you specific case.
Now, opinion asides, for your above example I think it could be hardly arguable that your Lambda solution be favored. At a glance, it would be difficult to really get what is going on. So I'd say pick the method option or the list comprehension (which would be my GOTO solution).
Suppose i have a class with several attributes. I want to modify a select few in the same way. However, since i can't put attributes in a list, this is rather tedious.
For example:
class BOB:
def __init__(self,name,color,shape):
self.lenname = len(name)
self.lencolor = len(color)
self.lenshape = len(shape)
def BOBmodify_lenname(instance):
instance.lenname *= 2
def BOBmodify_lencolor(instance):
instance.lencolor *= 2
def BOBmodify_lenshape(instance):
instance.lenshape *= 2
My goal is to have an input of sorts in the form of a list of attribues, like [lenshape, lencolor] and then have a function that iterates over the list and multiplies them by two. Since this is not possible, i have to resort to a function for each attribute
Here, i only have three attributes, and i need three functions to modify each. For classes with more attributes, this quickly becomes impractical. It would be nice if this was possible:
def BOBmodify(instance,attribute_list):
for attribute in attribute_list:
instance.attribute *= 2
and then do
BOBmodify(bobinstance, [lenname, lenshape])
Aas far as i know, you can't put attributes in a list so this isn't possible. So how should i handle this situation where i want a function to do the same thing to several different attributes? Although i've searched for this on stack overflow and google, nothing relevant has come up. Please help. Thanks!
You can define a method like this, and pass attributes as strings:
def modify_attrs(self, attrs):
for attr in attrs:
val = getattr(self, attr)
setattr(self, attr, val*2)
...
bobinstance.modify_attrs(['lenname', 'lenshape'])
Demo:
>>> bobinstance = BOB('spam', 'red', 'square')
>>> bobinstance.__dict__
{'lenshape': 6, 'lencolor': 3, 'lenname': 4}
>>> bobinstance.modify_attrs(['lencolor', 'lenname'])
>>> bobinstance.__dict__
{'lenshape': 6, 'lencolor': 6, 'lenname': 8}
I have a method that returns a set of objects, and I'm writing a unit test for this method. Is there a generic, tidy and idiomatic way of comparing these for identity (rather than equality)? Or do I need to write a suitable implementation myself?
An example (somewhat contrived to keep it simple):
class Foo(object):
def has_some_property(self):
...
class Container(object):
def __init__(self):
self.foo_set = set()
def add_foo(self, foo):
self.foo_set.add(foo)
def foo_objects_that_have_property(self):
return set([foo for foo in self.foo_set if foo.has_some_property()])
import unittest
class TestCase(unittest.TestCase):
def testFoo(self):
c = Container()
x, y, z = Foo(), Foo(), Foo()
...
self.assertContentIdentity(c.foo_objects_that_have_property(), set([x, y]))
Importantly, testing here for equality won't do, since mutating the objects returned by foo_objects_that_have_property() may lead to inconsistent results depending on how those objects are used differently in Container even if they are "equal" at the time of the test.
The best I can come up with is:
#staticmethod
def set_id(c):
return set([id(e) for e in c])
def assertContentIdentity(self, a, b):
self.assertEqual(set_id(a), set_id(b))
However, this is specialised for sets and can't deal with nested containers.
A simple, albeit not the most efficient, way to do it:
def assertContentIdentity(set1, set2):
set1 = set([id(a) for a in set1])
set2 = set([id(a) for a in set2])
assert set1 == set2
x is y won't work here since that
would tell me that the sets are
different, which I know already. I
want to know if the objects that they
contain are the same objects or
different objects.
Then you need to write your own function, like
set([id(x) for x in X]) == set([id(y) for y in Y])
I need a way to inspect a class so I can safely identify which attributes are user-defined class attributes. The problem is that functions like dir(), inspect.getmembers() and friends return all class attributes including the pre-defined ones like: __class__, __doc__, __dict__, __hash__. This is of course understandable, and one could argue that I could just make a list of named members to ignore, but unfortunately these pre-defined attributes are bound to change with different versions of Python therefore making my project volnerable to changed in the python project - and I don't like that.
example:
>>> class A:
... a=10
... b=20
... def __init__(self):
... self.c=30
>>> dir(A)
['__doc__', '__init__', '__module__', 'a', 'b']
>>> get_user_attributes(A)
['a','b']
In the example above I want a safe way to retrieve only the user-defined class attributes ['a','b'] not 'c' as it is an instance attribute. So my question is... Can anyone help me with the above fictive function get_user_attributes(cls)?
I have spent some time trying to solve the problem by parsing the class in AST level which would be very easy. But I can't find a way to convert already parsed objects to an AST node tree. I guess all AST info is discarded once a class has been compiled into bytecode.
Below is the hard way. Here's the easy way. Don't know why it didn't occur to me sooner.
import inspect
def get_user_attributes(cls):
boring = dir(type('dummy', (object,), {}))
return [item
for item in inspect.getmembers(cls)
if item[0] not in boring]
Here's a start
def get_user_attributes(cls):
boring = dir(type('dummy', (object,), {}))
attrs = {}
bases = reversed(inspect.getmro(cls))
for base in bases:
if hasattr(base, '__dict__'):
attrs.update(base.__dict__)
elif hasattr(base, '__slots__'):
if hasattr(base, base.__slots__[0]):
# We're dealing with a non-string sequence or one char string
for item in base.__slots__:
attrs[item] = getattr(base, item)
else:
# We're dealing with a single identifier as a string
attrs[base.__slots__] = getattr(base, base.__slots__)
for key in boring:
del attrs['key'] # we can be sure it will be present so no need to guard this
return attrs
This should be fairly robust. Essentially, it works by getting the attributes that are on a default subclass of object to ignore. It then gets the mro of the class that's passed to it and traverses it in reverse order so that subclass keys can overwrite superclass keys. It returns a dictionary of key-value pairs. If you want a list of key, value tuples like in inspect.getmembers then just return either attrs.items() or list(attrs.items()) in Python 3.
If you don't actually want to traverse the mro and just want attributes defined directly on the subclass then it's easier:
def get_user_attributes(cls):
boring = dir(type('dummy', (object,), {}))
if hasattr(cls, '__dict__'):
attrs = cls.__dict__.copy()
elif hasattr(cls, '__slots__'):
if hasattr(base, base.__slots__[0]):
# We're dealing with a non-string sequence or one char string
for item in base.__slots__:
attrs[item] = getattr(base, item)
else:
# We're dealing with a single identifier as a string
attrs[base.__slots__] = getattr(base, base.__slots__)
for key in boring:
del attrs['key'] # we can be sure it will be present so no need to guard this
return attrs
Double underscores on both ends of 'special attributes' have been a part of python before 2.0. It would be very unlikely that they would change that any time in the near future.
class Foo(object):
a = 1
b = 2
def get_attrs(klass):
return [k for k in klass.__dict__.keys()
if not k.startswith('__')
and not k.endswith('__')]
print get_attrs(Foo)
['a', 'b']
Thanks aaronasterling, you gave me the expression i needed :-)
My final class attribute inspector function looks like this:
def get_user_attributes(cls,exclude_methods=True):
base_attrs = dir(type('dummy', (object,), {}))
this_cls_attrs = dir(cls)
res = []
for attr in this_cls_attrs:
if base_attrs.count(attr) or (callable(getattr(cls,attr)) and exclude_methods):
continue
res += [attr]
return res
Either return class attribute variabels only (exclude_methods=True) or also retrieve the methods.
My initial tests og the above function supports both old and new-style python classes.
/ Jakob
If you use new style classes, could you simply subtract the attributes of the parent class?
class A(object):
a = 10
b = 20
#...
def get_attrs(Foo):
return [k for k in dir(Foo) if k not in dir(super(Foo))]
Edit: Not quite. __dict__,__module__ and __weakref__ appear when inheriting from object, but aren't there in object itself. You could special case these--I doubt they'd change very often.
Sorry for necro-bumping the thread. I'm surprised that there's still no simple function (or a library) to handle such common usage as of 2019.
I'd like to thank aaronasterling for the idea. Actually, set container provides a more straightforward way to express it:
class dummy: pass
def abridged_set_of_user_attributes(obj):
return set(dir(obj))-set(dir(dummy))
def abridged_list_of_user_attributes(obj):
return list(abridged_set_of_user_attributes(obj))
The original solution using list comprehension is actually two level of loops because there are two in keyword compounded, despite having only one for keyword made it look like less work than it is.
This worked for me to include user defined attributes with __ that might be be found in cls.__dict__
import inspect
class A:
__a = True
def __init__(self, _a, b, c):
self._a = _a
self.b = b
self.c = c
def test(self):
return False
cls = A(1, 2, 3)
members = inspect.getmembers(cls, predicate=lambda x: not inspect.ismethod(x))
attrs = set(dict(members).keys()).intersection(set(cls.__dict__.keys()))
__attrs = {m[0] for m in members if m[0].startswith(f'_{cls.__class__.__name__}')}
attrs.update(__attrs)
This will correctly yield: {'_A__a', '_a', 'b', 'c'}
You can update to clean the cls.__class__.__name__ if you wish
Suppose code like this:
class Base:
def start(self):
pass
def stop(self)
pass
class A(Base):
def start(self):
... do something for A
def stop(self)
.... do something for A
class B(Base):
def start(self):
def stop(self):
a1 = A(); a2 = A()
b1 = B(); b2 = B()
all = [a1, b1, b2, a2,.....]
Now I want to call methods start and stop (maybe also others) for each object in the list all. Is there any elegant way for doing this except of writing a bunch of functions like
def start_all(all):
for item in all:
item.start()
def stop_all(all):
This will work
all = [a1, b1, b2, a2,.....]
map(lambda x: x.start(),all)
simple example
all = ["MILK","BREAD","EGGS"]
map(lambda x:x.lower(),all)
>>>['milk','bread','eggs']
and in python3
all = ["MILK","BREAD","EGGS"]
list(map(lambda x:x.lower(),all))
>>>['milk','bread','eggs']
It seems like there would be a more Pythonic way of doing this, but I haven't found it yet.
I use "map" sometimes if I'm calling the same function (not a method) on a bunch of objects:
map(do_something, a_list_of_objects)
This replaces a bunch of code that looks like this:
do_something(a)
do_something(b)
do_something(c)
...
But can also be achieved with a pedestrian "for" loop:
for obj in a_list_of_objects:
do_something(obj)
The downside is that a) you're creating a list as a return value from "map" that's just being throw out and b) it might be more confusing that just the simple loop variant.
You could also use a list comprehension, but that's a bit abusive as well (once again, creating a throw-away list):
[ do_something(x) for x in a_list_of_objects ]
For methods, I suppose either of these would work (with the same reservations):
map(lambda x: x.method_call(), a_list_of_objects)
or
[ x.method_call() for x in a_list_of_objects ]
So, in reality, I think the pedestrian (yet effective) "for" loop is probably your best bet.
The approach
for item in all:
item.start()
is simple, easy, readable, and concise. This is the main approach Python provides for this operation. You can certainly encapsulate it in a function if that helps something. Defining a special function for this for general use is likely to be less clear than just writing out the for loop.
The *_all() functions are so simple that for a few methods I'd just write the functions. If you have lots of identical functions, you can write a generic function:
def apply_on_all(seq, method, *args, **kwargs):
for obj in seq:
getattr(obj, method)(*args, **kwargs)
Or create a function factory:
def create_all_applier(method, doc=None):
def on_all(seq, *args, **kwargs):
for obj in seq:
getattr(obj, method)(*args, **kwargs)
on_all.__doc__ = doc
return on_all
start_all = create_all_applier('start', "Start all instances")
stop_all = create_all_applier('stop', "Stop all instances")
...
maybe map, but since you don't want to make a list, you can write your own...
def call_for_all(f, seq):
for i in seq:
f(i)
then you can do:
call_for_all(lamda x: x.start(), all)
call_for_all(lamda x: x.stop(), all)
by the way, all is a built in function, don't overwrite it ;-)
Starting in Python 2.6 there is a operator.methodcaller function.
So you can get something more elegant (and fast):
from operator import methodcaller
map(methodcaller('method_name'), list_of_objects)
Taking #Ants Aasmas answer one step further, you can create a wrapper that takes any method call and forwards it to all elements of a given list:
class AllOf:
def __init__(self, elements):
self.elements = elements
def __getattr__(self, attr):
def on_all(*args, **kwargs):
for obj in self.elements:
getattr(obj, attr)(*args, **kwargs)
return on_all
That class can then be used like this:
class Foo:
def __init__(self, val="quux!"):
self.val = val
def foo(self):
print "foo: " + self.val
a = [ Foo("foo"), Foo("bar"), Foo()]
AllOf(a).foo()
Which produces the following output:
foo: foo
foo: bar
foo: quux!
With some work and ingenuity it could probably be enhanced to handle attributes as well (returning a list of attribute values).
If you would like to have a generic function while avoiding referring to method name using strings, you can write something like that:
def apply_on_all(seq, method, *args, **kwargs):
for obj in seq:
getattr(obj, method.__name__)(*args, **kwargs)
# to call:
apply_on_all(all, A.start)
Similar to other answers but has the advantage of only using explicit attribute lookup (i.e. A.start). This can eliminate refactoring errors, i.e. it's easy to rename the start method and forget to change the strings that refer to this method.
The best solution, in my opinion, depends on whether you need the result of the method and whether your method takes any arguments except self.
If you don't need the result, I would simply write a for loop:
for instance in lst:
instance.start()
If you need the result, but method takes no arguments, I would use map:
strs = ['A', 'B', 'C']
lower_strs = list(map(str.lower, strs)) # ['a', 'b', 'c']
And finally, if you need the result and method does take some arguments, list comprehension would work great:
strs = ['aq', 'bq', 'cq']
qx_strs = [i.replace('q', 'x') for i in strs] # ['ax', 'bx', 'cx']