Normally a descriptor is used on a class attribute like so:
class Owner(object):
attr = Attr()
When getting Owner.attr, Attr.__get__(self, instance, owner) is called where self = Owner.attr, instance = None and owner = Owner.
When Owner is instantiated instance will be the instance of Owner.
Now I would like to apply this concept to method parameters instead of class attributes.
How it would look in practice (let's assume that the functionality of Attr is to wrap a string with a given string):
class Example(object):
def funct(self, param=Attr('t')):
return param == 'test' # < param calls the descriptor here
e = Example()
e.funct('es') # < is True because 'es' wrapped with 't' becomes 'test'.
When accessing param, Attr.__get__(self, instance, owner) will be called with self = funct.param, instance = funct and owner = funct (although it doesn't make sense to have owner and instance the same, might be None?).
But since funct is not a class, this will not work. How can I get something similar to work?
A decorator on the function will be processing the parameters, so this might add to the solution I think.
The decorator must be, for example, be able to change the wrapper string.
Functions actually are first class objects in Python, but you are correct in saying that the syntax you describe would not work as you want. You could potentially do something like this with a decorator that inspects the passed attributes for characteristics that would enable this sort of functionality though. However, you'd probably be better off implementing a callable object, then attaching descriptors to that and creating instances of the callable rather than functions.
Related
I am writing a framework, and I want my base class to use different functions for renaming in the child classes. I figured the best way would be to use a class attribute, like in case of A, but I got TypeErrors when running it like in rename_columns(). However it worked with implementation like B
import pandas as pd
class A:
my_func_mask = str.lower
foo = 'bar'
def rename_columns(self, data):
return data.rename(columns=self.my_func_mask)
class B(A):
def rename_columns(self, data):
return data.rename(columns=self.__class__.my_func_mask)
So I experimented with the above a bit, and I get the following:
a = A()
a.foo # Works fine, gives back 'bar'
a.__class__.my_func_mask # Works as expected `a.__class__.my_func_mask is str.lower` is true
a.my_func_mask # throws TypeError: descriptor 'lower' for 'str' objects doesn't apply to 'A' object
My questions would be why can I use regular typed (int, str, etc.) values as class attributes and access them on the instance as well, while I cannot do that for functions?
What happens during the attribute lookup in these cases? What is the difference in the attribute resolution process?
Actually both foo and my_func_mask is in __class__.__dict__ so I am a bit puzzled. Thanks for the clarifications!
You are storing an unbound built-in method on your class, meaning it is a descriptor object. When you then try to access that on self, descriptor binding applies but the __get__ method called to complete the binding tells you that it can't be bound to your custom class instances, because the method would only work on str instances. That's a strict limitation of most methods of built-in types.
You need to store it in a different manner; putting it inside another container, such as a list or dictionary, would avoid binding. Or you could wrap it in a staticmethod descriptor to have it be bound and return the original. Another option is to not store this as a class attribute, and simply create an instance attribute in __init__.
But in this case, I'd not store str.lower as an attribute value, at all. I'd store None and fall back to str.lower when you still encounter None:
return data.rename(columns=self.my_func_mask or str.lower)
Setting my_func_mask to None is a better indicator that a default is going to be used, clearly distinguishable from explicitly setting str.lower as the mask.
You need to declare staticmethod.
class A:
my_func_mask = staticmethod(str.lower)
foo = 'bar'
>>> A().my_func_mask is str.lower
>>> True
Everything that is placed in the class definition is bound to the class, but you can't bind a built-in to your own class.
Essentially, all code that you place in a class is executed when the class is created. All items in locals() are then bound to your class at the end of the class. That's why this also works to bind a method to your class:
def abc(self):
print('{} from outside the class'.format(self))
class A:
f1 = abc
f2 = lambda self: print('{} from lambda'.format(self))
def f3(self):
print('{} from method'.format(self))
To not have the function bound to your class, you have to place it in the __init__ method of your class:
class A:
def __init__(self):
self.my_func_mask = str.lower
I have class with custom getter, so I have situations when I need to use my custom getter, and situations when I need to use default.
So consider following.
If I call method of object c in this way:
c.somePyClassProp
In that case I need to call custom getter, and getter will return int value, not Python object.
But if I call method on this way:
c.somePyClassProp.getAttributes()
In this case I need to use default setter, and first return need to be Python object, and then we need to call getAttributes method of returned python object (from c.somePyClassProp).
Note that somePyClassProp is actually property of class which is another Python class instance.
So, is there any way in Python on which we can know whether some other methods will be called after first method call?
No. c.someMethod is a self-contained expression; its evaluation cannot be influenced by the context in which the result will be used. If it were possible to achieve what you want, this would be the result:
x = c.someMethod
c.someMethod.getAttributes() # Works!
x.getAttributes() # AttributeError!
This would be confusing as hell.
Don't try to make c.someMethod behave differently depending on what will be done with it, and if possible, don't make c.someMethod a method call at all. People will expect c.someMethod to return a bound method object that can then be called to execute the method; just define the method the usual way and call it with c.someMethod().
You don't want to return different values based on which attribute is accessed next, you want to return an int-like object that also has the required attribute on it. To do this, we create a subclass of int that has a getAttributes() method. An instance of this class, of course, needs to know what object it is "bound" to, that is, what object its getAttributes() method should refer to, so we'll add this to the constructor.
class bound_int(int):
def __new__(cls, value, obj):
val = int.__new__(cls, value)
val.obj = obj
return val
def getAttributes(self):
return self.obj.somePyClassProp
Now in your getter for c.somePyClassProp, instead of returning an integer, you return a bound_int and pass it a reference to the object its getAttributes() method needs to know about (here I'll just have it refer to self, the object it's being returned from):
#property
def somePyClassProp(self):
return bound_int(42, self)
This way, if you use c.somePyPclassProp as an int, it acts just like any other int, because it is one, but if you want to further call getAttributes() on it, you can do that, too. It's the same value in both cases; it just has been built to fulfill both purposes. This approach can be adapted to pretty much any problem of this type.
It looks like you want two ways to get the property depending on what you want to do with it. I don't think there's any inherent Pythonic way to implement this, and you therefore need to store a variable or property name for each case. Maybe:
c.somePyClassProp
can be used in the __get__ and
c.somePyClassProp__getAttributes()
can be implemented in a more custom way inside the __getattribute__ function.
One way I've used (which is probably not the best) is to check for that exact variable name:
def __getattribute__(self, var_name):
if ('__' in var_name):
var_name, method = var_name.split('__')
return object.__getattribute__(self, var_name).__getattribute__(method)
Using object.__get__(self, var_name) uses the object class's method of getting a property directly.
You can store the contained python object as a variable and the create getters via the #property dectorator for whatever values you want. When you want to read the int, reference the property. When you want the contained object, use its variable name instead.
class SomePyClass(object):
def getInt(self):
return 1
def getAttributes(self):
return 'a b c'
class MyClass(object):
def __init__(self, py_class):
self._py_class = py_class
#property
def some_property(self):
return self._py_class.getInt()
x = MyClass(SomePyClass())
y = self.some_property
x._py_class.getAttributes()
Look at this code:
class MyClass():
# Why does this give me "NameError: name 'self' is not defined":
mySelf = self
# But this does not?
def myFunction(self):
mySelf2 = self
Basically I want a way for a class to refer to itself without needing to name itself specifically, hence I want self to work for the class, not just methods/functions. How can I achieve this?
EDIT: The point of this is that I'm trying to refer to the class name from inside the class itself with something like self.class._name_ so that the class name isn't hardcoded anywhere in the class's code, and thus it's easier to re-use the code.
EDIT 2: From what I've learned from the answers below, what I'm trying to do is impossible. I'll have to find a different way. Mission abandoned.
EDIT 3: Here is specifically what I'm trying to do:
class simpleObject(object):
def __init__(self, request):
self.request = request
#view_defaults(renderer='string')
class Test(simpleObject):
# this line throws an error because of self
myClassName = self.__class__.__name__
#view_config(route_name=myClassName)
def activateTheView(self):
db = self.request.db
foo = 'bar'
return foo
Note that self is not defined at the time when you want the class to refer to itself for the assignment to work. This is because (in addition to being named arbitrarily), self refers to instances and not classes. At the time that the suspect line of code attempts to run, there is as of yet no class for it to refer to. Not that it would refer to the class if there was.
In a method, you can always use type(self). That will get the subclass of MyClass that created the current instance. If you want to hard-code to MyClass, that name will be available in the global scope of the methods. This will allow you to do everything that your example would allow if it actually worked. E.g, you can just do MyClass.some_attribute inside your methods.
You probably want to modify the class attributes after class creation. This can be done with decorators or on an ad-hoc basis. Metaclasses may be a better fit. Without knowing what you actually want to do though, it's impossible to say.
UPDATE:
Here's some code to do what you want. It uses a metaclass AutoViewConfigMeta and a new decorator to mark the methods that you want view_config applied to. I spoofed the view_config decorator. It prints out the class name when it's called though to prove that it has access to it. The metaclass __new__ just loops through the class dictionary and looks for methods that were marked by the auto_view_config decorator. It cleans off the mark and applies the view_config decorator with the appropriate class name.
Here's the code.
# This just spoofs the view_config decorator.
def view_config(route=''):
def dec(f):
def wrapper(*args, **kwargs):
print "route={0}".format(route)
return f(*args, **kwargs)
return wrapper
return dec
# Apply this decorator to methods for which you want to call view_config with
# the class name. It will tag them. The metaclass will apply view_config once it
# has the class name.
def auto_view_config(f):
f.auto_view_config = True
return f
class AutoViewConfigMeta(type):
def __new__(mcls, name, bases, dict_):
#This is called during class creation. _dict is the namespace of the class and
# name is it's name. So the idea is to pull out the methods that need
# view_config applied to them and manually apply them with the class name.
# We'll recognize them because they will have the auto_view_config attribute
# set on them by the `auto_view_config` decorator. Then use type to create
# the class and return it.
for item in dict_:
if hasattr(dict_[item], 'auto_view_config'):
method = dict_[item]
del method.auto_view_config # Clean up after ourselves.
# The next line is the manual form of applying a decorator.
dict_[item] = view_config(route=name)(method)
# Call out to type to actually create the class with the modified dict.
return type.__new__(mcls, name, bases, dict_)
class simpleObject(object):
__metaclass__ = AutoViewConfigMeta
class Test(simpleObject):
#auto_view_config
def activateTheView(self):
foo = 'bar'
print foo
if __name__=='__main__':
t = Test()
t.activateTheView()
Let me know if you have any questions.
Python has an "explict is better than implicit" design philosophy.
Many languages have an implicit pointer or variable in the scope of a method that (e.g. this in C++) that refers to the object through which the method was invoked. Python does not have this. Here, all bound methods will have an extra first argument that is the object through which the method was invoked. You can call it anything you want (self is not a keyword like this in C++). The name self is convention rather than a syntactic rule.
Your method myFunction defines the variable self as a parameter so it works. There's no such variable at the class level so it's erroring out.
So much for the explanation. I'm not aware of a straightforward way for you to do what you want and I've never seen such requirement in Python. Can you detail why you want to do such a thing? Perhaps there's an assumption that you're making which can be handled in another way using Python.
self is just a name, your self in this case is a class variable and not this for the object using which it is called,
self is treated as a normal variable and it is not defined, where as the self in the function comes from the object used for calling.
you want to treat the object reference in self as a class variable which is not possible.
self isn't a keyword, it's just a convention. The methods are attributes of the class object (not the instance), but they receive the instance as their first argument. You could rename the argument to xyzzy if you wanted and it would still work the same way.
But (as should be obvious) you can't refer to a method argument outside the body of the method. Inside a class block but outside of any method, self is undefined. And the concept wouldn't even make sense -- at the time the class block is being evaluated, no instance of the class can possibly exist yet.
Because the name self is explicitly defined as part of the arguments to myFunction. The first argument to a method is the instance that the method was called on; in the class body, there isn't an "instance we're dealing with", because the class body deals with every possible instance of the class (including ones that don't necessarily exist yet) - so, there isn't a particular object that could be called self.
If you want to refer to the class itself, rather than some instance of it, this is spelled self.__class__ (or, for new-style classes in Py2 and all classes in Py3, type(self)) anywhere self exists. If you want to be able to deal with this in situations where self doesn't exist, then you may want to look at class methods which aren't associated with any particular instance, and so take the class itself in place of self. If you really need to do this in the class body (and, you probably don't), you'll just have to call it by name.
You can't refer to the class itself within the class body because the class doesn't exist at the time that the class body is executed. (If the previous sentence is confusing, reading up about metaclasses will either clear this up or make you more confused.)
Within an instance method, you can refer to the class of the instance with self.__class__, but be careful here. This will be the instance's actual class, which through the power of inheritance might not be the class in which the method was defined.
Within a class method, the class is passed in as the first argument, much like instances are the first argument to instance methods:
class MyClass(object):
#classmethod
def foo(cls):
print cls.__name__
MyClass.foo() # Should print "MyClass"
As with instance methods, the actual class might differ due to inheritance.
class OtherClass(MyClass):
pass
OtherClass.foo() # Should print "OtherClass"
If you really need to refer to MyClass within a method of MyClass, you're pretty much going to have to refer to it as MyClass unless you use magic. This sort of magic is more trouble than it is worth.
I recently developed a class named DocumentWrapper around some ORM document object in Python to transparently add some features to it without changing its interface in any way.
I just have one issue with this. Let's say I have some User object wrapped in it. Calling isinstance(some_var, User) will return False because some_var indeed is an instance of DocumentWrapper.
Is there any way to fake the type of an object in Python to have the same call return True?
You can use the __instancecheck__ magic method to override the default isinstance behaviour:
#classmethod
def __instancecheck__(cls, instance):
return isinstance(instance, User)
This is only if you want your object to be a transparent wrapper; that is, if you want a DocumentWrapper to behave like a User. Otherwise, just expose the wrapped class as an attribute.
This is a Python 3 addition; it came with abstract base classes. You can't do the same in Python 2.
Override __class__ in your wrapper class DocumentWrapper:
class DocumentWrapper(object):
#property
def __class__(self):
return User
>>> isinstance(DocumentWrapper(), User)
True
This way no modifications to the wrapped class User are needed.
Python Mock does the same (see mock.py:612 in mock-2.0.0, couldn't find sources online to link to, sorry).
Testing the type of an object is usually an antipattern in python. In some cases it makes sense to test the "duck type" of the object, something like:
hasattr(some_var, "username")
But even that's undesirable, for instance there are reasons why that expression might return false, even though a wrapper uses some magic with __getattribute__ to correctly proxy the attribute.
It's usually preferred to allow variables only take a single abstract type, and possibly None. Different behaviours based on different inputs should be achieved by passing the optionally typed data in different variables. You want to do something like this:
def dosomething(some_user=None, some_otherthing=None):
if some_user is not None:
#do the "User" type action
elif some_otherthing is not None:
#etc...
else:
raise ValueError("not enough arguments")
Of course, this all assumes you have some level of control of the code that is doing the type checking. Suppose it isn't. for "isinstance()" to return true, the class must appear in the instance's bases, or the class must have an __instancecheck__. Since you don't control either of those things for the class, you have to resort to some shenanigans on the instance. Do something like this:
def wrap_user(instance):
class wrapped_user(type(instance)):
__metaclass__ = type
def __init__(self):
pass
def __getattribute__(self, attr):
self_dict = object.__getattribute__(type(self), '__dict__')
if attr in self_dict:
return self_dict[attr]
return getattr(instance, attr)
def extra_feature(self, foo):
return instance.username + foo # or whatever
return wrapped_user()
What we're doing is creating a new class dynamically at the time we need to wrap the instance, and actually inherit from the wrapped object's __class__. We also go to the extra trouble of overriding the __metaclass__, in case the original had some extra behaviors we don't actually want to encounter (like looking for a database table with a certain class name). A nice convenience of this style is that we never have to create any instance attributes on the wrapper class, there is no self.wrapped_object, since that value is present at class creation time.
Edit: As pointed out in comments, the above only works for some simple types, if you need to proxy more elaborate attributes on the target object, (say, methods), then see the following answer: Python - Faking Type Continued
Here is a solution by using metaclass, but you need to modify the wrapped classes:
>>> class DocumentWrapper:
def __init__(self, wrapped_obj):
self.wrapped_obj = wrapped_obj
>>> class MetaWrapper(abc.ABCMeta):
def __instancecheck__(self, instance):
try:
return isinstance(instance.wrapped_obj, self)
except:
return isinstance(instance, self)
>>> class User(metaclass=MetaWrapper):
pass
>>> user=DocumentWrapper(User())
>>> isinstance(user,User)
True
>>> class User2:
pass
>>> user2=DocumentWrapper(User2())
>>> isinstance(user2,User2)
False
It sounds like you want to test the type of the object your DocumentWrapper wraps, not the type of the DocumentWrapper itself. If that's right, then the interface to DocumentWrapper needs to expose that type. You might add a method to your DocumentWrapper class that returns the type of the wrapped object, for instance. But I don't think that making the call to isinstance ambiguous, by making it return True when it's not, is the right way to solve this.
The best way is to inherit DocumentWrapper from the User itself, or mix-in pattern and doing multiple inherintance from many classes
class DocumentWrapper(User, object)
You can also fake isinstance() results by manipulating obj.__class__ but this is deep level magic and should not be done.
I know that I can dynamically add an instance method to an object by doing something like:
import types
def my_method(self):
# logic of method
# ...
# instance is some instance of some class
instance.my_method = types.MethodType(my_method, instance)
Later on I can call instance.my_method() and self will be bound correctly and everything works.
Now, my question: how to do the exact same thing to obtain the behavior that decorating the new method with #property would give?
I would guess something like:
instance.my_method = types.MethodType(my_method, instance)
instance.my_method = property(instance.my_method)
But, doing that instance.my_method returns a property object.
The property descriptor objects needs to live in the class, not in the instance, to have the effect you desire. If you don't want to alter the existing class in order to avoid altering the behavior of other instances, you'll need to make a "per-instance class", e.g.:
def addprop(inst, name, method):
cls = type(inst)
if not hasattr(cls, '__perinstance'):
cls = type(cls.__name__, (cls,), {})
cls.__perinstance = True
inst.__class__ = cls
setattr(cls, name, property(method))
I'm marking these special "per-instance" classes with an attribute to avoid needlessly making multiple ones if you're doing several addprop calls on the same instance.
Note that, like for other uses of property, you need the class in play to be new-style (typically obtained by inheriting directly or indirectly from object), not the ancient legacy style (dropped in Python 3) that's assigned by default to a class without bases.
Since this question isn't asking about only adding to a spesific instance,
the following method can be used to add a property to the class, this will expose the properties to all instances of the class YMMV.
cls = type(my_instance)
cls.my_prop = property(lambda self: "hello world")
print(my_instance.my_prop)
# >>> hello world
Note: Adding another answer because I think #Alex Martelli, while correct, is achieving the desired result by creating a new class that holds the property, this answer is intended to be more direct/straightforward without abstracting whats going on into its own method.