I have written a series of functions, but have been toying with the idea of using them to build a class. Instead of re-writing them all, I would like to call them from the class. What is the correct way of doing this? I can think of the following:
a boring function, but it gives the idea.
def my_func( a_dict, a_tuple ):
a,b = a_tuple
a_dict[a] = b
return a_dict
the two ways I had thought of were as follows:
class MyDict(dict):
def my_method(self, a_tuple):
return my_func(self, a_tuple)
or:
import functools
class MyOtherDict(dict):
def __init__(self):
self.my_parital = functools.partial(my_func, self)
it has been pointed out that the following will work as well. which seems the simplest!
class MySimpleDict(dict):
my_method = my_function
A Slight Extension of the question:
is it common to do this (call functions from methods)?
There is even a third one:
class MyThirdDict(dict):
my_method = my_func
This behaves as if the function was defined in the class scope - every access to it leads to a call of the function's __get__, yielding a method.
Besides, in your 1st approach, I would do
class MyDict(dict):
def my_method(self, a_tuple):
return my_func(self, a_tuple)
in order to have the behaviour 100% identical.
From your 2 approaches, the 1st one is definitely more readable.
What about the simpler...
def my_func( a_dict, a_tuple ):
a,b = a_tuple
a_dict[a] = b
return a_dict
class MyDict(dict):
my_method = my_func
An instance method is just a function that is an attribute of the class, so the following will work here:
class MyDict(dict):
my_method = my_func
For example:
>>> def my_func( a_dict, a_tuple ):
... a,b = a_tuple
... a_dict[a] = b
... return a_dict
...
>>> class MyDict(dict):
... my_method = my_func
...
>>> d = MyDict()
>>> d.my_method(('a', 'b'))
{'a': 'b'}
>>> d
{'a': 'b'}
This looks a little confusing because self isn't used as the first parameter name, but it works the same way. The MyDict instance is passed in as the a_dict argument.
The former would probably be considered cleaner - creating "methods" in your __init__ is kind of nasty (and also means that those methods aren't visible on the class definition).
Related
This code:
import inspect
class Obj():
def c(self):
return 1
def b(self):
return 2
def a(self):
return 3
o = Obj()
for name, value in inspect.getmembers(o, inspect.ismethod):
print str(value())+" "+name
print:
3 a
2 b
1 c
Because of inspect.getmembers return all the members of an object in a list of (name, value) pairs sorted by name, as you can read in https://docs.python.org/2/library/inspect.html#inspect.getmembers
But I want to get that list in the same order that the members was written in the code, in other words, the output would be:
1 c
2 b
3 a
Is any way to do that?
Thanks
No. Class members are not ordered. They are gathered into a dictionary, immediately losing order. You can resort to tricks like parsing the source, but it will break easily. For starters, the source could not be available.
[edit: it seems python3 allows more flexibility in class creation, making it possible to customize the way class members are gathered, if you are on python3 only, that's probably a better approach]
If changing the code is not a problem, you can use a decorator though:
import inspect
def track_order(fn):
fn.order = track_order.idx
track_order.idx += 1
return fn
track_order.idx = 0
class Obj(object):
#track_order
def c(self):
return 1
#track_order
def b(self):
return 2
#track_order
def a(self):
return 3
o = Obj()
methods = sorted((item
for item in inspect.getmembers(o, inspect.ismethod)),
key=lambda item: item[1].order)
for name, value in methods:
print str(value())+" "+name
The decorator adds an idx attribute to all methods that pass through it.
This makes use of the fact that python has first-class functions.
$ python test.py
1 c
2 b
3 a
Note: this is the method used by Django to keep track of form and model fields order. Only, they don't need a decorator because fields' classes have the instanciation order attribute built-in (it is named creation_counter).
When creating an object, all of its attributes are contained in another specialized attribute in the object called __dict__, which as the name suggests is just a normal Python non-ordered dictionary, hence they are not guaranteed to be stored in the same fashion they were added in. When retrieving the values in __dict__ using getmembers(), Python automatically reorganizes the dictionary when printing it in order to make some logical sense.
To combat this, something must be done to turn the regular Python dictionary __dict__ into some sort of ordered one.
This can be done a number of ways, for simplicity's sake, I will assume you are using Python 3.
Using the collections package, you can obtain an OrderedDict, which is exactly the technology we require for such an issue. Prepare this ordered dictionary for use in a metaclass for the class which needs ordered members to be stored, copy over the members, and finally access this new OrderedDict when wanting to print out said members.
This can be seen in action in this Stack Overflow answer.
In cpython the code is compiled down to bytecode for the VM. And the functions have a __code__ attribute, which is a code object. The code object has a co_firstlineno attribute, which is the first line in Python source code. (Detailed in the inspect module.)
If you know your methods are all in source code, and you know you are using cpython, you could use this as a sort key. But it seems awful shaky if you don't know these things.
members = [ (name,meth) for name, meth in inspect.getmembers(o, inspect.ismethod)]
members = sorted(members, key=lambda t: t[1].__code__.co_firstlineno)
print '\n'.join(m[0] for m in members)
Hm, this is very hacky, but basically I inspect the source directly and use re to find method names. This solution is pretty brittle, though, and it doesn't deal with inheritance, but maybe it works for you. Assuming I've saved your class definition in a file named test.py:
>>> import test
>>> import re
>>> findmethods = re.compile(r" def (.+)\(")
>>> findmethods.findall(inspect.getsource(test.Obj))
['c', 'b', 'a']
>>>
def print_class_methods_by_order(class_object):
for attr in class_object.__dict__:
if callable(getattr(class_object, attr)):
print(attr)
class MyObject:
def c(self):
pass
def a(self):
pass
def b(self):
pass
output:
>>> print_class_methods_by_order(MyObject)
c
a
b
Also works with decorators:
def print_class_methods_by_order(class_object):
for attr in class_object.__dict__:
if callable(getattr(class_object, attr)):
print(attr)
def my_decorator(func):
def wrapper(*args, **kwargs):
print("my_decorator")
func(*args, **kwargs)
return wrapper
class MyObject:
#my_decorator
def c(self):
pass
#my_decorator
def a(self):
pass
#my_decorator
def b(self):
pass
output:
>>> print_class_methods_by_order(MyObject)
c
a
b
And also works with "class" decorator (not sure there is any difference compared to the previous case):
def print_class_methods_by_order(class_object):
for attr in class_object.__dict__:
if callable(getattr(class_object, attr)):
print(attr)
def decorate_all(decorator):
def decorate(cls):
for attr in cls.__dict__:
if callable(getattr(cls, attr)):
setattr(cls, attr, decorator(getattr(cls, attr), cls))
return cls
return decorate
def my_decorator(func, cls):
def wrapper(*args, **kwargs):
print("my_decorator")
fn = func(*args, **kwargs)
return fn
return wrapper
#decorate_all(my_decorator)
class MyObject:
def c(self):
pass
def a(self):
pass
def b(self):
pass
output:
>>> print_class_methods_by_order(MyObject)
c
a
b
Consider the following code:
class Foo():
pass
Foo.entries = dict()
a = Foo()
a.entries['1'] = 1
b = Foo()
b.entries['3'] = 3
print(a.entries)
This will print:
{'1': 1, '3': 3}
because the entries is added as static attribute. Is there a way monkey patch the class definition in order to add new attributes (without using inheritance).
I managed to find the following way but it looks convoluted to me:
def patch_me(target, field, value):
def func(self):
if not hasattr(self, '__' + field):
setattr(self, '__' + field, value())
return getattr(self, '__' + field)
setattr(target, field, property(func))
patch_me(Foo, 'entries', dict)
Ordinarily, attributes are added either by the __init__() function or after instantiating:
foo = Foo()
foo.bar = 'something' # note case
If you want to do this automatically, inheritance is by far the simplest way to do so:
class Baz(Foo):
def __init__(self):
super().__init__() # super() needs arguments in 2.x
self.bar = 'something'
Note that classes don't need to appear at the top level of a Python module. You can declare a class inside a function:
def make_baz(value):
class Baz(Foo):
def __init__(self):
super().__init__() # super() needs arguments in 2.x
self.bar = value()
return Baz()
This example will create a new class every time make_baz() is called. That may or may not be what you want. It would probably be simpler to just do this:
def make_foo(value):
result = Foo()
result.bar = value()
return result
If you're really set on monkey-patching the original class, the example code you provided is more or less the simplest way of doing it. You might consider using decorator syntax for property(), but that's a minor change. I should also note that it will not invoke double-underscore name mangling, which is probably a good thing because it means you cannot conflict with any names used by the class itself.
I'm writing a class that has a dict containing int to method mappings. However setting the values in this dict results in the dict being populated with unbound functions.
class A:
def meth_a: ...
def meth_b: ...
...
map = {1: meth_a, 2: meth_b, ...}
for int in ...:
map[int] = meth_x
This doesn't work for a few reasons:
The methods aren't bound when the class is initialized because they're not in the class dict?
I can't bind the methods manually using __get__ because the class name isn't bound to any namespace yet.
So:
How can I do this?
Do I have to drop out of the class and define the dict after the class has been initialized?
Is it really necessary to call __get__ on the methods to bind them?
Update0
The methods will be called like this:
def func(self, int):
return self.map[int]()
Also regarding the numeric indices/list: Not all indices will be present. I'm not aware that one can do list([1]=a, [2]=b, [1337]=leet) in Python, is there an equivalent? Should I just allocate a arbitrary length list and set specific values? The only interest I have here is in minimizing the lookup time, would it really be that different to the O(1) hash that is {}? I've ignored this for now as premature optimization.
I'm not sure exactly why you're doing what you're doing, but you certainly can do it right in the class definition; you don't need __init__.
class A:
def meth_a(self): pass
m = {1: meth_a}
def foo(self, number):
self.m[number](self)
a = A()
a.foo(1)
An "unbound" instance method simply needs you to pass it an instance of the class manually, and it works fine.
Also, please don't use int as the name of a variable, either, it's a builtin too.
A dictionary is absolutely the right type for this kind of thing.
Edit: This will also work for staticmethods and classmethods if you use new-style classes.
First of all Don't use variable "map" since build in python function map will be fetched.
You need to have init method and initialize your dictionary in the init method using self. The dictionary right now is only part of the class, and not part of instances of the class. If you want instances of the class to have the dictionary as well you need to make an init method and initialize your dictionary there. So you need to do this:
def __init__(self):
self.mymap[int] = self.meth_x
or if you want the dictionary to be a class variable, then this:
def __init__(self):
A.mymap[int] = self.meth_x
It's not totally clear just what you're trying to do. I suspect you want to write code something like
class Foo(object):
def __init__(self, name):
self.name = name
def method_1(self, bar):
print self.name, bar
# ... something here
my_foo = Foo('baz')
my_foo.methods[1]('quux')
# prints "baz quux"
so, that methods attribute needs to return a bound instance method somehow, but without being called directly. This is a good opportunity to use a descriptor. We need to do something that will return a special object when accessed through an instance, and we need that special object to return a bound method when indexed. Let's start from the inside and work our way out.
>>> import types
>>> class BindMapping(object):
... def __init__(self, instance, mapping):
... self.instance, self.mapping = instance, mapping
...
... def __getitem__(self, key):
... func = self.mapping[key]
... if isinstance(func, types.MethodType):
... return types.MethodType(func.im_func, self.instance, func.im_class)
... else:
... return types.MethodType(func, self.instance, type(self))
...
We're just implementing the barest minimum of the mapping protocol, and deferring completely to an underlying collection. here we make use of types.MethodType to get a real instance method when needed, including binding something that's already an instance method. We'll see how that's useful in a minute.
We could implement a descriptor directly, but for the purposes here, property already does everything we need out of a descriptor, so we'll just define one that returns a properly constructed BindMapping instance.
>>> class Foo(object):
... def method_1(self):
... print "1"
... def method_2(self):
... print "2"
...
... _mapping = [method_1, method_2]
...
... #property
... def mapping(self):
... return BindMapping(self, self._mapping)
...
Just for kicks, we also throw in some extra methods outside the class body. Notice how the the methods added inside the class body are functions, just like functions added outside; methods added outside the class body are actual instance methods (although unbound).
>>> def method_3(self):
... print "3"
...
>>> Foo._mapping.append(method_3)
>>> Foo._mapping.append(Foo.method_1)
>>> map(type, Foo._mapping)
[<type 'function'>, <type 'function'>, <type 'function'>, <type 'instancemethod'>]
And it works as advertised:
>>> f = Foo()
>>> for i in range(len(f._mapping)):
... f.mapping[i]()
...
1
2
3
1
>>>
This seems kind of convoluted to me. What is the ultimate goal?
If you really want do to this, you can take advantage of the fact that the methods are alreaday contained in a mapping (__dict__).
class A(object):
def meth_1(self):
print("method 1")
def meth_2(self):
print("method 2")
def func(self, i):
return getattr(self, "meth_{}".format(i))()
a = A()
a.func(2)
This pattern is found in some existing library modules.
I'm simulating a distributed system in which all nodes follow some protocol. This includes assessing some small variations in the protocol. A variation means alternative implementation of a single method. All nodes always follow the same variation, which is determined by experiment configuration (only one configuration is active at any given time). What is the clearest way to do it, without sacrificing performance?
As an experiment can be quite extensive, I clearly don't want any conditionals. Before I've just used inheritance, like:
class Node(object):
def dumb_method(self, argument):
# ...
def slow_method(self, argument):
# ...
# A lot more methods
class SmarterNode(Node):
def dumb_method(self, argument):
# A somewhat smarter variant ...
class FasterNode(SmarterNode):
def slow_method(self, argument):
# A faster variant ...
But now I need to test all possible variants and don't want an exponential number of classes cluttering the source. I also want other people peeping at the code to understand it with minimal effort. What are your suggestions?
Edit: One thing I failed to emphasize enough: for all envisioned use cases, it seems that patching the class upon configuration is good. I mean: it can be made to work by simple Node.dumb_method = smart_method. But somehow it didn't feel right. Would this kind of solution cause major headache to a random smart reader?
You can create new subtypes with the type function. You just have to give it the subclasses namespace as a dict.
# these are supposed to overwrite methods
def foo(self):
return "foo"
def bar(self):
return "bar"
def variants(base, methods):
"""
given a base class and list of dicts like [{ foo = <function foo> }]
returns types T(base) where foo was overwritten
"""
for d in methods:
yield type('NodeVariant', (base,), d)
from itertools import combinations
def subdicts(**fulldict):
""" returns all dicts that are subsets of `fulldict` """
items = fulldict.items()
for i in range(len(items)+1):
for subset in combinations(items, i):
yield dict(subset)
# a list of method variants
combos = subdicts(slow_method=foo, dumb_method=bar)
# base class
class Node(object):
def dumb_method(self):
return "dumb"
def slow_method(self):
return "slow"
# use the base and our variants to make a number of types
types = variants(Node, combos)
# instantiate each type and call boths methods on it for demonstration
print [(var.dumb_method(), var.slow_method()) for var
in (cls() for cls in types)]
# [('dumb', 'slow'), ('dumb', 'foo'), ('bar', 'slow'), ('bar', 'foo')]
You could use the __slots__ mechanism and a factory class. You would need to instantiate a NodeFactory for each experiment, but it would handle creating Node instances for you from there on. Example:
class Node(object):
__slots__ = ["slow","dumb"]
class NodeFactory(object):
def __init__(self, slow_method, dumb_method):
self.slow = slow_method
self.dumb = dumb_method
def makenode(self):
n = Node()
n.dumb = self.dumb
n.slow = self.slow
return n
an example run:
>>> def foo():
... print "foo"
...
>>> def bar():
... print "bar"
...
>>> nf = NodeFactory(foo, bar)
>>> n = nf.makenode()
>>> n.dumb()
bar
>>> n.slow()
foo
I'm not sure if you're trying to do something akin to this (allows swap-out runtime "inheritance"):
class Node(object):
__methnames = ('method','method1')
def __init__(self, type):
for i in self.__methnames:
setattr(self, i, getattr(self, i+"_"+type))
def dumb_method(self, argument):
# ...
def slow_method(self, argument):
# ...
n = Node('dumb')
n.method() # calls dumb_method
n = Node('slow')
n.method() # calls slow_method
Or if you're looking for something like this (allows running (and therefore testing) of all methods of the class):
class Node(object):
#do something
class NodeTest(Node):
def run_tests(self, ending = ''):
for i in dir(self):
if(i.endswith(ending)):
meth = getattr(self, i)
if(callable(meth)):
meth() #needs some default args.
# or yield meth if you can
You can use a metaclass for this.
If will let you create a class on the fly with method names according to every variations.
Should the method to be called be decided when the class is instantiated or after? Assuming it is when the class is instantiated, how about the following:
class Node():
def Fast(self):
print "Fast"
def Slow(self):
print "Slow"
class NodeFactory():
def __init__(self, method):
self.method = method
def SetMethod(self, method):
self.method = method
def New(self):
n = Node()
n.Run = getattr(n, self.method)
return n
nf = NodeFactory("Fast")
nf.New().Run()
# Prints "Fast"
nf.SetMethod("Slow")
nf.New().Run()
# Prints "Slow"
This question already has answers here:
Automatically initialize instance variables?
(17 answers)
Closed last month.
Say, I have the following class in Python
class Foo(object):
a = None
b = None
c = None
def __init__(self, a = None, b = None, c = None):
self.a = a
self.b = b
self.c = c
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
Please note that
class Foo(object):
a = None
sets a key-value pair in Foo's dict:
Foo.__dict__['a']=None
while
def __init__(self, a = None, b = None, c = None):
self.a = a
sets a key-value pair in the Foo instance object's dict:
foo=Foo()
foo.__dict__['a']=a
So setting the class members at the top of your definition is not directly related to the setting of the instance attributes in the lower half of your definition (inside the __init__.
Also, it is good to be aware that __init__ is Python's initializer. __new__ is the class constructor.
If you are looking for a way to automatically add some instance attributes based on __init__'s arguments, you could use this:
import inspect
import functools
def autoargs(*include,**kwargs):
def _autoargs(func):
attrs,varargs,varkw,defaults=inspect.getargspec(func)
def sieve(attr):
if kwargs and attr in kwargs['exclude']: return False
if not include or attr in include: return True
else: return False
#functools.wraps(func)
def wrapper(self,*args,**kwargs):
# handle default values
for attr,val in zip(reversed(attrs),reversed(defaults)):
if sieve(attr): setattr(self, attr, val)
# handle positional arguments
positional_attrs=attrs[1:]
for attr,val in zip(positional_attrs,args):
if sieve(attr): setattr(self, attr, val)
# handle varargs
if varargs:
remaining_args=args[len(positional_attrs):]
if sieve(varargs): setattr(self, varargs, remaining_args)
# handle varkw
if kwargs:
for attr,val in kwargs.iteritems():
if sieve(attr): setattr(self,attr,val)
return func(self,*args,**kwargs)
return wrapper
return _autoargs
So when you say
class Foo(object):
#autoargs()
def __init__(self,x,path,debug=False,*args,**kw):
pass
foo=Foo('bar','/tmp',True, 100, 101,verbose=True)
you automatically get these instance attributes:
print(foo.x)
# bar
print(foo.path)
# /tmp
print(foo.debug)
# True
print(foo.args)
# (100, 101)
print(foo.verbose)
# True
PS. Although I wrote this (for fun), I don't recommend using autoargs for serious work. Being explicit is simple, clear and infallible. I can't say the same for autoargs.
PPS. Is it just me, or are a lot of buttons broken on Stackoverflow? The editor window has lost all its icons... :( Clearing the browser cache fixed the problem.
Python 3.7 provides dataclasses which are helpful in situations like this:
from dataclasses import dataclass
#dataclass
class Foo:
a: str = None
b: str = None
c: str = None
This saves you from having to write out the __init__ method when you just want to store a few attributes.
Gives you a good __repr__ method:
>>> a = Foo()
>>> a
Foo(a=None, b=None, c=None)
If you need to do calculations on a param, you can implement __post_init__.
See also namedtuple:
from collections import namedtuple
Foo = namedtuple('Foo', ['a', 'b', 'c'])
All fields are required with namedtuple though.
>>> a = Foo(1, 2, 3)
>>> a
Foo(a=1, b=2, c=3)
There are elegant ways to do this.
Is there any way to simplify this process? Whenever I add a new member to class Foo, I'm forced to modify the constructor.
There is also a crude way. It will work, but is NOT recommended. See and decide.
>>> class Foo(object):
def __init__(self, **attrs):
self.__dict__.update(**attrs)
def __getattr__(self, attr):
return self.__dict__.get(attr, None)
>>> f = Foo(a = 1, b = 2, c = 3)
>>> f.a, f.b
(1, 2)
>>> f = Foo(bar = 'baz')
>>> f.bar
'baz'
>>> f.a
>>>
The keyword argument constructor lets you get away without explicitly defining any arguments. Warning: this goes against the "explicit is better than implicit" principle.
You need to override __getattr__ ONLY if you want to return a default value for an attribute that is not present instead of getting an AttributeError.
http://code.activestate.com/recipes/286185-automatically-initializing-instance-variables-from/
This recipe and its comments provide some methods.
Python: Automatically initialize instance variables?
This is a previous question.