How can I know from where the method is called? - python

I'm trying clean implement of Objective-C's category in Python, and found this answer to similar question of mine. I copied the code below:
categories.py
class category(object):
def __init__(self, mainModule, override = True):
self.mainModule = mainModule
self.override = override
def __call__(self, function):
if self.override or function.__name__ not in dir(self.mainModule):
setattr(self.mainModule, function.__name__, function)
But I do not want to waste namespace.
By using this `categories', there remains a variable as NoneType object like below:
>>> from categories import category
>>> class Test(object):
... pass
...
>>> #category(Test)
... def foobar(self, msg):
... print msg
...
>>> test = Test()
>>> test.foobar('hello world')
hello world
>>> type(foobar)
<type 'NoneType'>
>>>
I want it to be like below
>>> from categories import category
>>> class Test(object):
... pass
...
>>> #category(Test)
... def foobar(self, msg):
... print msg
...
>>> test = Test()
>>> test.foobar('hello world')
hello world
>>> type(foobar)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'foobar' is not defined
>>>
Is there anyway to delete it automatically like below?
def __call__(self, function):
if self.override or function.__name__ not in dir(self.mainModule):
setattr(self.mainModule, function.__name__, function)
del(somewhere.function.__name__)
I found that sys._getframe give me some useful information. But I couldn't make it by myself.

No, there's no way to automatically do that. You would have to manually delete the name afterwards. category here is a decorator, which means that
#category(Test)
def f():
...
is the same as
def f():
...
f = category(Test)(f)
Even if, from inside category, you could delete the name in the outer scope, it wouldn't be enough, because that name is rebound after the decorator executes.
The code that you linked to borders on an abuse of the decorator syntax. Decorators are meant to provide a way to modify or extend the function they decorate, but that code relies on side-effects of the decorator (namely, assigning the function as a method of a class), and then discards the function. But it can only "discard" it by returning None, so None remains bound to the function's name, as you saw.
I would recommend you follow the advice of the highest-voted answer on that question, and simply assign new methods to your classes. There is no real need for an "infrastructure" like categories in Python, because you can just directly add new methods to existing classes whenever you want.

While I completely agree with what BrenBarn said, you could split the function removal into a later step. The problem is that after the decorator executed, the variable is reassigned. So you cannot perform the removal within the decorator itself.
You could however remember the functions and remove them from the module at a later point.
class category(object):
functions = []
def __init__(self, mainModule, override = True):
self.mainModule = mainModule
self.override = override
def __call__(self, function):
if self.override or function.__name__ not in dir(self.mainModule):
setattr(self.mainModule, function.__name__, function)
self.functions.append((inspect.getmodule(function), function.__name__))
return self.dummy
#staticmethod
def dummy():
pass
#classmethod
def cleanUp(cls):
for module, name in cls.functions:
if hasattr(module, name) and getattr(module, name) == cls.dummy:
delattr(module, name)
cls.functions = []
This category type will remember the functions it decorates and stores the names and modules they belong to for a later cleanup. The decorator also returns a special dummy function so that the cleanup can ensure that the variable was not reassigned later.
>>> class Test(object): pass
>>> #category(Test)
def foobar(self, msg):
print(msg)
>>> #category(Test)
def hello_world(self):
print('Hello world')
>>> test = Test()
>>> test.foobar('xy')
xy
>>> test.hello_world()
Hello world
>>> type(foobar)
<class 'function'>
>>> type(hello_world)
<class 'function'>
>>> category.cleanUp()
>>> type(foobar)
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
type(foobar)
NameError: name 'foobar' is not defined
>>> type(hello_world)
Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
type(hello_world)
NameError: name 'hello_world' is not defined

Related

Can someone explain me how "d.maximumDifference" worked in the end? Like can we even call a variable inside a function which is inside a class?

class Difference:
def __init__(self, a):
self.__elements = a
def computeDifference(self):
b = min(self.__elements)
c = max(self.__elements)
result = abs(b-c)
self.maximumDifference = result
_ = input()
a = [int(e) for e in input().split(' ')]
d = Difference(a)
d.computeDifference()
print(d.maximumDifference)
I am unable to understand how I was able to call maximumdifference, which is a variable inside the computeDifference function, Which is inside the Difference class?
how I was able to call maximumdifference, which is a variable inside the computeDifference function, Which is inside the Difference class?
I think the core of your question comes from a slight misunderstanding: self.maximumDifference is a field in the class Difference. This field was created on-the-fly when the computeDifference function was called.
As another example:
... def foo(self):
... self.bar = "hi"
...
>>> a = A()
>>> a.bar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'bar'
>>> a.foo()
>>> a.bar
'hi'
Here we can see how the bar attribute does not exist until after the foo function was run. If you're familiar with other languages, this may be surprising behavior --- many other languages require that class attributes/fields are fixed/specified in the class or constructor, but Python allows adding new fields on-the-fly.

Access static class variable in instance method

Say I have class Test defined as this:
class Test
test_var = 2
def test_func():
print(test_var)
I can find out what test_var is fine like so:
>>> Test.test_var
2
...But calling Test.test_func() does not work.
>>> Test.test_func()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in test
NameError: name 'test_var' is not defined
If I change Test.test_func() like this (note that this is pseudo-code):
redef test_func():
print(Test.test_var)
It works fine:
>>> Test.test_func()
2
...and that makes sense. But how can I make the first example work, keeping in mind that I want test_func to be an instance method?
Note that the code posted above is example code, and so typos should be ignored.
You can always access class-level attributes via the instance, ie self, as long as you have not shadowed them with an instance attribute of the same name. So:
def test_func(self):
print(self.test_var)
In your example, test_func is just a function and although its defined in the class namespace, the function itself doesn't know about the class namespace. You want either a regular instance method or a class method.
class Test:
test_var = 2
def instance_test(self):
# instance methods will look in self first and class namespace second
print(self.test_var)
#classmethod
def class_test(cls):
# class methods take the class itself as first argument
print(cls.test_var)
t = Test()
t.instance_test()
Test.class_test()
You need to either pass self (almost always what you want) to the class method or add a #classmethod or #staticmethod decorator if you don't need self. Then create an instance of the class and call the test_func method.
Examples:
# test_var is an class variable and test_func has a classmethod decorator
>>> class Test:
... test_var = 2
... #classmethod
... def test_func(cls):
... print(cls.test_var)
...
>>> t = Test()
>>> t.test_func()
2
# test_var is an class variable and test_func has a staticmethod decorator
>>> class Test:
... test_var = 2
... #staticmethod
... def test_func():
... print(Test.test_var)
...
>>> t = Test()
>>> t.test_func()
2
# test_var is an instance variable here
>>> class Test:
... self.test_var = 2
... def test_func(self):
... print(self.test_var)
...
>>> t = Test()
>>> t.test_func()
2

How do you hide from hasattr?

Let's say a function looks at an object and checks if it has a function a_method:
def func(obj):
if hasattr(obj, 'a_method'):
...
else:
...
I have an object whose class defines a_method, but I want to hide it from hasattr. I don't want to change the implementation of func to achieve this hiding, so what hack can I do to solve this problem?
If the method is defined on the class you appear to be able to remove it from the __dict__ for the class. This prevents lookups (hasattr will return false). You can still use the function if you keep a reference to it when you remove it (like the example) - just remember that you have to pass in an instance of the class for self, it's not being called with the implied self.
>>> class A:
... def meth(self):
... print "In method."
...
>>>
>>> a = A()
>>> a.meth
<bound method A.meth of <__main__.A instance at 0x0218AB48>>
>>> fn = A.__dict__.pop('meth')
>>> hasattr(a, 'meth')
False
>>> a.meth
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute 'meth'
>>> fn()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: meth() takes exactly 1 argument (0 given)
>>> fn(a)
In method.
You could redefine the hasattr function. Below is an example.
saved_hasattr = hasattr
def hasattr(obj, method):
if method == 'MY_METHOD':
return False
else:
return saved_hasattr(obj, method)
Note that you probably want to implement more detailed checks than just checking the method name. For example checking the object type might be beneficial.
Try this:
class Test(object):
def __hideme(self):
print 'hidden'
t = Test()
print hasattr(t,"__hideme") #prints False....
I believe this works b/c of the double underscore magic of hiding members (owning to name mangling) of a class to outside world...Unless someone has a strong argument against this, I'd think this is way better than popping stuff off from __dict__? Thoughts?

How to define global function in Python?

Is there a way to define a function to be global from within a class( or from within another function, as matter of fact)? Something similar to defining a global variable.
Functions are added to the current namespace like any other name would be added. That means you can use the global keyword inside a function or method:
def create_global_function():
global foo
def foo(): return 'bar'
The same applies to a class body or method:
class ClassWithGlobalFunction:
global spam
def spam(): return 'eggs'
def method(self):
global monty
def monty(): return 'python'
with the difference that spam will be defined immediately as top-level class bodies are executed on import.
Like all uses of global you probably want to rethink the problem and find another way to solve it. You could return the function so created instead, for example.
Demo:
>>> def create_global_function():
... global foo
... def foo(): return 'bar'
...
>>> foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'foo' is not defined
>>> create_global_function()
>>> foo
<function foo at 0x102a0c7d0>
>>> foo()
'bar'
>>> class ClassWithGlobalFunction:
... global spam
... def spam(): return 'eggs'
... def method(self):
... global monty
... def monty(): return 'python'
...
>>> spam
<function spam at 0x102a0cb18>
>>> spam()
'eggs'
>>> monty
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'monty' is not defined
>>> ClassWithGlobalFunction().method()
>>> monty()
'python'
You can use global to declare a global function from within a class. The problem with doing that is you can not use it with a class scope so might as well declare it outside the class.
class X:
global d
def d():
print 'I might be defined in a class, but I\'m global'
>> X.d
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'X' object has no attribute 'd'
>> d()
I might be defined in a class, but I'm global
I found a case where global does not have the desired effect: inside .pdbrc file. If you define functions in .pdbrc they will only be available from the stack frame from which pdb.set_trace() was called.
However, you can add a function globally, so that it will be available in all stack frames, by using an alternative syntax:
def cow(): print("I'm a cow")
globals()['cow']=cow
I've tested that it also works in place of the global keyword in at least the simplest case:
def fish():
def cow():
print("I'm a cow")
globals()['cow']=cow
Despite being more verbose, I thought it was worth sharing this alternative syntax. I have not tested it extensively so I can't comment on its limitations vs using the global keyword.

Python: changing methods and attributes at runtime

I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
Oh, and please don't ask why.
This example shows the differences between adding a method to a class and to an instance.
>>> class Dog():
... def __init__(self, name):
... self.name = name
...
>>> skip = Dog('Skip')
>>> spot = Dog('Spot')
>>> def talk(self):
... print 'Hi, my name is ' + self.name
...
>>> Dog.talk = talk # add method to class
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk()
Hi, my name is Spot
>>> del Dog.talk # remove method from class
>>> skip.talk() # won't work anymore
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
>>> import types
>>> f = types.MethodType(talk, skip, Dog)
>>> skip.talk = f # add method to specific instance
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk() # won't work, since we only modified skip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
I wish to create a class in Python that I can add and remove attributes and methods.
import types
class SpecialClass(object):
#classmethod
def removeVariable(cls, name):
return delattr(cls, name)
#classmethod
def addMethod(cls, func):
return setattr(cls, func.__name__, types.MethodType(func, cls))
def hello(self, n):
print n
instance = SpecialClass()
SpecialClass.addMethod(hello)
>>> SpecialClass.hello(5)
5
>>> instance.hello(6)
6
>>> SpecialClass.removeVariable("hello")
>>> instance.hello(7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'SpecialClass' object has no attribute 'hello'
>>> SpecialClass.hello(8)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'SpecialClass' has no attribute 'hello'
A possibly interesting alternative to using types.MethodType in:
>>> f = types.MethodType(talk, puppy, Dog)
>>> puppy.talk = f # add method to specific instance
would be to exploit the fact that functions are descriptors:
>>> puppy.talk = talk.__get__(puppy, Dog)
I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
You can add and remove attributes and methods to any class, and they'll be available to all instances of the class:
>>> def method1(self):
pass
>>> def method1(self):
print "method1"
>>> def method2(self):
print "method2"
>>> class C():
pass
>>> c = C()
>>> c.method()
Traceback (most recent call last):
File "<pyshell#62>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.method = method1
>>> c.method()
method1
>>> C.method = method2
>>> c.method()
method2
>>> del C.method
>>> c.method()
Traceback (most recent call last):
File "<pyshell#68>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.attribute = "foo"
>>> c.attribute
'foo'
>>> c.attribute = "bar"
>>> c.attribute
'bar'
you can just assign directly to the class (either by accessing the original class name or via __class__ ):
class a : pass
ob=a()
ob.__class__.blah=lambda self,k: (3, self,k)
ob.blah(5)
ob2=a()
ob2.blah(7)
will print
(3, <__main__.a instance at 0x7f18e3c345f0>, 5)
(3, <__main__.a instance at 0x7f18e3c344d0>, 7)
Simply:
f1 = lambda:0 #method for instances
f2 = lambda _:0 #method for class
class C: pass #class
c1,c2 = C(),C() #instances
print dir(c1),dir(c2)
#add to the Instances
c1.func = f1
c1.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
del c1.func,c1.any
#add to the Class
C.func = f2
C.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
print c2.func(),c2.any
which results in:
['__doc__', '__module__'] ['__doc__', '__module__']
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__']
0 1.23
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__', 'any', 'func']
0 1.23
0 1.23
another alternative, if you need to replace the class wholesale is to modify the class attribute:
>>> class A(object):
... def foo(self):
... print 'A'
...
>>> class B(object):
... def foo(self):
... print 'Bar'
...
>>> a = A()
>>> a.foo()
A
>>> a.__class__ = B
>>> a.foo()
Bar
Does the class itself necessarily need to be modified? Or is the goal simply to replace what object.method() does at a particular point during runtime?
I ask because I sidestep the problem of actually modifying the class to monkey patch specific method calls in my framework with getattribute and a Runtime Decorator on my Base inheritance object.
Methods retrieved by a Base object in getattribute are wrapped in a Runtime_Decorator that parses the method calls keyword arguments for decorators/monkey patches to apply.
This enables you to utilize the syntax object.method(monkey_patch="mypatch"), object.method(decorator="mydecorator"), and even object.method(decorators=my_decorator_list).
This works for any individual method call (I leave out magic methods), does so without actually modifying any class/instance attributes, can utilize arbitrary, even foreign methods to patch, and will work transparently on sublcasses that inherit from Base (provided they don't override getattribute of course).
import trace
def monkey_patched(self, *args, **kwargs):
print self, "Tried to call a method, but it was monkey patched instead"
return "and now for something completely different"
class Base(object):
def __init__(self):
super(Base, self).__init__()
def testmethod(self):
print "%s test method" % self
def __getattribute__(self, attribute):
value = super(Base, self).__getattribute__(attribute)
if "__" not in attribute and callable(value):
value = Runtime_Decorator(value)
return value
class Runtime_Decorator(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
if kwargs.has_key("monkey_patch"):
module_name, patch_name = self._resolve_string(kwargs.pop("monkey_patch"))
module = self._get_module(module_name)
monkey_patch = getattr(module, patch_name)
return monkey_patch(self.function.im_self, *args, **kwargs)
if kwargs.has_key('decorator'):
decorator_type = str(kwargs['decorator'])
module_name, decorator_name = self._resolve_string(decorator_type)
decorator = self._get_decorator(decorator_name, module_name)
wrapped_function = decorator(self.function)
del kwargs['decorator']
return wrapped_function(*args, **kwargs)
elif kwargs.has_key('decorators'):
decorators = []
for item in kwargs['decorators']:
module_name, decorator_name = self._resolve_string(item)
decorator = self._get_decorator(decorator_name, module_name)
decorators.append(decorator)
wrapped_function = self.function
for item in reversed(decorators):
wrapped_function = item(wrapped_function)
del kwargs['decorators']
return wrapped_function(*args, **kwargs)
else:
return self.function(*args, **kwargs)
def _resolve_string(self, string):
try: # attempt to split the string into a module and attribute
module_name, decorator_name = string.split(".")
except ValueError: # there was no ".", it's just a single attribute
module_name = "__main__"
decorator_name = string
finally:
return module_name, decorator_name
def _get_module(self, module_name):
try: # attempt to load the module if it exists already
module = modules[module_name]
except KeyError: # import it if it doesn't
module = __import__(module_name)
finally:
return module
def _get_decorator(self, decorator_name, module_name):
module = self._get_module(module_name)
try: # attempt to procure the decorator class
decorator_wrap = getattr(module, decorator_name)
except AttributeError: # decorator not found in module
print("failed to locate decorators %s for function %s." %\
(kwargs["decorator"], self.function))
else:
return decorator_wrap # instantiate the class with self.function
class Tracer(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
tracer = trace.Trace(trace=1)
tracer.runfunc(self.function, *args, **kwargs)
b = Base()
b.testmethod(monkey_patch="monkey_patched")
b.testmethod(decorator="Tracer")
#b.testmethod(monkey_patch="external_module.my_patch")
The downside to this approach is getattribute hooks all access to attributes, so the checking of and potential wrapping of methods occurs even for attributes that are not methods + won't be utilizing the feature for the particular call in question. And using getattribute at all is inherently somewhat complicated.
The actual impact of this overhead in my experience/for my purposes has been negligible, and my machine runs a dual core Celeron. The previous implementation I used introspected methods upon object init and bound the Runtime_Decorator to methods then. Doing things that way eliminated the need to utilize getattribute and reduced the overhead mentioned previously... however, it also breaks pickle (maybe not dill) and is less dynamic then this approach.
The only use cases I have actually come across "in the wild" with this technique were with timing and tracing decorators. However, the possibilities it opens up are extremely wide ranging.
If you have a preexisting class that cannot be made to inherit from a different base (or utilize the technique it's own class definition or in it's base class'), then the whole thing simply does not apply to your issue at all unfortunately.
I don't think setting/removing non-callable attributes on a class at runtime is necessarily so challenging? unless you want classes that inherit from the modified class to automatically reflect the changes in themselves as well... That'd be a whole 'nother can o' worms by the sound of it though.

Categories

Resources