I have a python fuse project based on the Xmp example in the fuse documentation. I have included a small piece of the code to show how this works. For some reason get_file does get called and the class gets created, but instead of fuse calling .read() on the class from get_file (file_class) fuse keeps calling Dstorage.read() which defeats the purpose in moving the read function out of that class.
class Dstorage(Fuse, Distributor):
def get_file(self, server, path, flags, *mode):
pass
# This does some work and passes back an instance of
# a class very similar to XmpFile
def main(self, *a, **kw):
self.file_class = self.get_file
return Fuse.main(self, *a, **kw)
I have my code hosted on launchpad, you can download it with this command.
bzr co https://code.launchpad.net/~asa-ayers/+junk/dstorage
bzr branch lp:~asa-ayers/dstorage/trunk
solution:
I used a proxy class that subclasses the one I needed and in the constructor I get the instance of the class I need and overwrite all of the proxy's methods to simply call the instance methods.
Looking at the code of the Fuse class (which is a maze of twisty little passages creating method proxies), I see this bit (which is a closure used to create a setter inside Fuse.MethodProxy._add_class_type, line 865):
def setter(self, xcls):
setattr(self, type + '_class', xcls)
for m in inits:
self.mdic[m] = xcls
for m in proxied:
if hasattr(xcls, m):
self.mdic[m] = self.proxyclass(m)
When you do self.file_class = self.get_file, this gets called with self.get_file, which is a bound method. The loop over proxied attributes is expecting to be able to get the attributes off the class you set, to put them into its mdic proxy dictionary after wrapping them, but they aren't there, because it's a bound method, rather than a class. Since it can't find them, it reverts to calling them on Dstorage.
So, long story short, you can't use a callable that returns an instance (kind of a pseudo-class) instead of a class here, because Fuse is introspecting the object that you set to find the methods it should call.
You need to assign a class to file_class - if you need to refer back to the parent instance, you can use the nested class trick they show in the docs.
Related
I have an Object of the following class which inherates from the algorithm class.
class AP(Algorithm):
def evaluate(self, u):
return self.stuff *2 +u
The Algorithm class has a method called StoppingCritiria.
At some point in the project the object objAP = AP() gets created. Later on I can then actually access it.
And at that point in time I want to override the method StoppingCriteria by some function which calls the old StoppingCriteria.
I tried simply
def new_stopping(self):
return super().StoppingCriteria() and custom(self.u)
objAP.StoppingCriteria = newStoppingCriteria
But that did not work. What did work were two rather inconviniend solutions:
New AP class (not desirable since I possibly need to do that for lots of classes)
class AP_custom(AP):
def StoppingCriteria(self):
return super().StoppingCriteria() and custom(self)
Override the Method but not using super but rather copy pasting the code into the new function and adding my code to that. Not desirable since I want to changes in the original method to be applyed to my new function as well.
See Override a method at instance level for many possible solutions. None of them will really work with super though, since you're simply not defining the replacement function in a class. You can define it slightly differently though for it to work:
class Foo:
def bar(self):
print('bar')
f = Foo()
def _bar(self):
type(self).bar(self) # or Foo.bar(self)
print('baz')
from typing import MethodType
f.bar = MethodType(_bar, f)
f.bar() # outputs bar baz
Since you're replacing the method at the instance level, you don't really need to access the method of the super class, you just want to access the method of the class, which still exists in its original form.
Assuming I have a class which requires a function (or should I say method) which is:
independent from my class instance - doesn't need self argument;
is called only inside my class object
I won't need access to it at any point (to override it for example);
should I (A) place it inside the class and mark it as a #staticmethod or should I (B) define it outside my class object (but in the same namespace)? Why?
Example:
class A:
def __init__(self, my_int):
self.my_int = my_int
def my_int_and_4(self):
print(self.adder(self.my_int,4))
#staticmethod
def adder(a,b):
return a+b
or
def adder(a,b):
return a+b
class B:
def __init__(self, my_int):
self.my_int = my_int
def my_int_and_4(self):
print(adder(self.my_int,4))
EDIT: maybe the example is a bit oversimplified. I should have added that my version of "adder" is specificly used with my class and in no other case.
This is a textbook use case for a private static method.
They key point here is that you should make it a private method of that class. That way you're certain nothing else will use it and depend on its implementation. You'll be free to change it in the future, or even delete it, without breaking anything outside that class.
And yeah, make it static, because you can.
In Python, there is no way to make a method truly private, but by convention, prefixing the method name by a _ means it should be treated as private.
#staticmethod
def _adder(a,b): ## <-- note the _
return a+b
If at some point you suddenly need to use it outside the class, then exposing it will be no trouble at all, e.g. using a public wrapper method.
The reverse, however, isn't true; once exposed, it's difficult to retract that exposure.
I would definitely use a private static method in this case, for the reasons described by Jean-Francois Corbett. There are two types of methods in Python that belong to the class itself, rather than an instance: class methods and static methods.
The first parameter of a class method (created with #classmethod) references the class in exactly the same manner that the first parameter of an instance method (self) references an instance. It is the equivalent of static methods in most other languages. If your method requires access to other class members, use a class method.
A static method (created with #staticmethod) does not contain a reference to the class, and therefore cannot reference other class members. It's generally used for private helper methods and the like.
For your adder method, I would definitely use a static method. However, in this modified (and rather useless) version, a class method is necessary:
class A:
x = 1
def __init__(self, my_int):
self.my_int = my_int
def my_int_and_4(self):
print(self._adder(self.my_int,4))
#staticmethod
def _adder(a,b):
return a+b
#classmethod
def _increment(cls, n):
return n + cls.x
Both approaches will work, so it's the matter of readability and following conventions.
Does the method need to look at the instance's private attributes? If yes, it's a good reason to keep it in the class.
Is the method only used as a helper for one of different methods? If yes, it's a good reason to put it right after the calling method so that the code can be read top-down.
Does the method seem to make sense outside of the context of your class? If yes, it's a good reason to make it a free function or even move it to a different file, like utils.
I was looking into the following code.
On many occasions the __init__ method is not really used but there is a custom initialize function like in the following example:
def __init__(self):
pass
def initialize(self, opt):
# ...
This is then called as:
data_loader = CustomDatasetDataLoader()
# other instance method is called
data_loader.initialize(opt)
I see the problem that variables, that are used in other instance methods, could still be undefined, if one forgets to call this custom initialize function. But what are the benefits of this approach?
Some APIs out in the wild (such as inside setuptools) have similar kind of thing and they use it to their advantage. The __init__ call could be used for the low level internal API while public constructors are defined as classmethods for the different ways that one might construct objects. For instance, in pkg_resources.EntryPoint, the way to create instances of this class is to make use of the parse classmethod. A similar way can be followed if a custom initialization is desired
class CustomDatasetDataLoader(object):
#classmethod
def create(cls):
"""standard creation"""
return cls()
#classmethod
def create_with_initialization(cls, opt):
"""create with special options."""
inst = cls()
# assign things from opt to cls, like
# inst.some_update_method(opt.something)
# inst.attr = opt.some_attr
return inst
This way users of the class will not need two lines of code to do what a single line could do, they can just simply call CustomDatasetDataLoader.create_with_initialization(some_obj) if that is what they want, or call the other classmethod to construct an instance of this class.
Edit: I see, you had an example linked (wish underlining links didn't go out of fashion) - that particular usage and implementation I feel is a poor way, when a classmethod (or just rely on the standard __init__) would be sufficient.
However, if that initialize function were to be an interface with some other system that receives an object of a particular type to invoke some method with it (e.g. something akin to the visitor pattern) it might make sense, but as it is it really doesn't.
I wanted to access the class on which method is to be defined. This can be used, for example, to create alias for methods with decorator. This particular case could be implemented without using decorator (alias = original_name), but I would like to use decorator, primarily so because the aliasing will be visible along side the method definition at the top, useful when the method definition is long.
def method_alias(*aliases):
def aliased(m):
class_of_m = ??? # GET class of this method
for alias in aliases:
setattr(class_of_m, alias, m)
return m
return aliased
class Test():
#method_alias('check', 'examine')
def test():
print('I am implemented with name "test"')
Later, I found here that the above could be implemented by using two decorators (first store the aliases as method attributes, later when the class is already created, add the attributes to class). Can it be done without decorating the class, i.e. only decorating the method? This requires getting access to the class name in the decorator.
The short answer is no. The contents of the class body are evaluated before the class object is created, i.e. the function test is created and passed to the decorator without class Test already existing. The decorator is therefore unable to obtain a reference to it.
To solve the problem of method aliasing, I reckon three approaches:
Using a class decorator as described by your link.
Using a metaclass, which lets you modifies the class' __dict__ before the class object is created. (Implementing a metaclass class is acutally overriding the default constructor for class objects, see here. Also the metaclass usage syntax has changed in Python 3.)
Creating the aliases in the __init__ method for each instance of Test.
The first approach is probably the most straightforward. I wrote another example. It basically does the same as your link, but is more stripped down to make it a bit clearer.
def alias(*aliases):
def decorator(f):
f.aliases = set(aliases)
return f
return decorator
def apply_aliases(cls):
for name, elem in list(cls.__dict__.items()):
if not hasattr(elem, 'aliases'):
continue
for alias in elem.aliases:
setattr(cls, alias, elem)
return cls
#apply_aliases
class Test(object):
#alias('check', 'examine')
def test(self):
print('I am implemented with name "test"')
Test().test()
Test().check()
Test().examine()
I just can't see why do we need to use #staticmethod. Let's start with an exmaple.
class test1:
def __init__(self,value):
self.value=value
#staticmethod
def static_add_one(value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
a=test1(3)
print(a.new_val) ## >>> 4
class test2:
def __init__(self,value):
self.value=value
def static_add_one(self,value):
return value+1
#property
def new_val(self):
self.value=self.static_add_one(self.value)
return self.value
b=test2(3)
print(b.new_val) ## >>> 4
In the example above, the method, static_add_one , in the two classes do not require the instance of the class(self) in calculation.
The method static_add_one in the class test1 is decorated by #staticmethod and work properly.
But at the same time, the method static_add_one in the class test2 which has no #staticmethod decoration also works properly by using a trick that provides a self in the argument but doesn't use it at all.
So what is the benefit of using #staticmethod? Does it improve the performance? Or is it just due to the zen of python which states that "Explicit is better than implicit"?
The reason to use staticmethod is if you have something that could be written as a standalone function (not part of any class), but you want to keep it within the class because it's somehow semantically related to the class. (For instance, it could be a function that doesn't require any information from the class, but whose behavior is specific to the class, so that subclasses might want to override it.) In many cases, it could make just as much sense to write something as a standalone function instead of a staticmethod.
Your example isn't really the same. A key difference is that, even though you don't use self, you still need an instance to call static_add_one --- you can't call it directly on the class with test2.static_add_one(1). So there is a genuine difference in behavior there. The most serious "rival" to a staticmethod isn't a regular method that ignores self, but a standalone function.
Today I suddenly find a benefit of using #staticmethod.
If you created a staticmethod within a class, you don't need to create an instance of the class before using the staticmethod.
For example,
class File1:
def __init__(self, path):
out=self.parse(path)
def parse(self, path):
..parsing works..
return x
class File2:
def __init__(self, path):
out=self.parse(path)
#staticmethod
def parse(path):
..parsing works..
return x
if __name__=='__main__':
path='abc.txt'
File1.parse(path) #TypeError: unbound method parse() ....
File2.parse(path) #Goal!!!!!!!!!!!!!!!!!!!!
Since the method parse is strongly related to the classes File1 and File2, it is more natural to put it inside the class. However, sometimes this parse method may also be used in other classes under some circumstances. If you want to do so using File1, you must create an instance of File1 before calling the method parse. While using staticmethod in the class File2, you may directly call the method by using the syntax File2.parse.
This makes your works more convenient and natural.
I will add something other answers didn't mention. It's not only a matter of modularity, of putting something next to other logically related parts. It's also that the method could be non-static at other point of the hierarchy (i.e. in a subclass or superclass) and thus participate in polymorphism (type based dispatching). So if you put that function outside the class you will be precluding subclasses from effectively overriding it. Now, say you realize you don't need self in function C.f of class C, you have three two options:
Put it outside the class. But we just decided against this.
Do nothing new: while unused, still keep the self parameter.
Declare you are not using the self parameter, while still letting other C methods to call f as self.f, which is required if you wish to keep open the possibility of further overrides of f that do depend on some instance state.
Option 2 demands less conceptual baggage (you already have to know about self and methods-as-bound-functions, because it's the more general case). But you still may prefer to be explicit about self not being using (and the interpreter could even reward you with some optimization, not having to partially apply a function to self). In that case, you pick option 3 and add #staticmethod on top of your function.
Use #staticmethod for methods that don't need to operate on a specific object, but that you still want located in the scope of the class (as opposed to module scope).
Your example in test2.static_add_one wastes its time passing an unused self parameter, but otherwise works the same as test1.static_add_one. Note that this extraneous parameter can't be optimized away.
One example I can think of is in a Django project I have, where a model class represents a database table, and an object of that class represents a record. There are some functions used by the class that are stand-alone and do not need an object to operate on, for example a function that converts a title into a "slug", which is a representation of the title that follows the character set limits imposed by URL syntax. The function that converts a title to a slug is declared as a staticmethod precisely to strongly associate it with the class that uses it.