Way to mask functions on Python Object - python

I have a class that inherit from OrderedDict, but I don't know if this is the right way to accomplish what I need.
I would like the class to have the duel method of the javascript '.' notation like obj.<property> and I would also like the users to be able to access the class properties like obj['myproperty'] but I was to hide all the key() and get() functions. The inheritance model is providing good functionality, but it cluttering up the object with additional methods that are not really needed.
Is it possible to get the dictionary behavior without all the other functions coming along?
For this discussion, let's assume my class is this:
from six.moves.urllib import request
import json
class MyClass(OrderedDict):
def __init__(self, url):
super(MyClass, self).__init__(url=url)
self._url = url
self.init()
def init(self):
# call the url and load the json
req = request.Request(self._url)
res = json.loads(request.urlopen(req).read())
for k,v in res.items():
setattr(self, k, v)
self.update(res)
self.__dict__.update(res)
if __name__ == "__main__":
url = "https://sampleserver5.arcgisonline.com/ArcGIS/rest/services?f=json"
props = MyClass(url=url)
props.currentVersion
Is there another way to approach this dilemma?
Thanks

If all you want is x['a'] to work the same way as x.a without any other functionality of dictionaries, then don't inherit from dict or OrderedDict, instead just forward key/indice operations (__getitem__, __setitem__ and __delitem__) to attribute operations:
class MyClass(object):
def __getitem__(self,key):
try: #change the error to a KeyError if the attribute doesn't exist
return getattr(self,key)
except AttributeError:
pass
raise KeyError(key)
def __setitem__(self,key,value):
setattr(self,key,value)
def __delitem__(self,key):
delattr(self,key)
As an added bonus, because these special methods don't check the instance variables for the method name it doesn't break if you use the same names:
x = MyClass()
x['__getitem__'] = 1
print(x.__getitem__) #still works
print(x["__getattr__"]) #still works
The only time it will break is when trying to use __dict__ since that is where the instance variables are actually stored:
>>> x = MyClass()
>>> x.a = 4
>>> x.__dict__ = 1 #stops you right away
Traceback (most recent call last):
File "<pyshell#36>", line 1, in <module>
x.__dict__ = 1
TypeError: __dict__ must be set to a dictionary, not a 'int'
>>> x.__dict__ = {} #this is legal but removes all the previously stored values!
>>> x.a
Traceback (most recent call last):
File "<pyshell#38>", line 1, in <module>
x.a
AttributeError: 'MyClass' object has no attribute 'a'
In addition you can still use the normal dictionary methods by using vars():
x = MyClass()
x.a = 4
x['b'] = 6
for k,v in vars(x).items():
print((k,v))
#output
('b', 6)
('a', 4)
>>> vars(x)
{'b': 6, 'a': 4}

Related

Python how to create a class that wraps any value

Let's say I have an Entity class:
class Entity(dict):
pass
def save(self):
...
I can wrap a dict object with Entity(dict_obj)
But is it possible to create a class that can wrap any type of objects, eg. int, list etc.
PS I have come up the following work around, it doesn't work on the more complex objects, but seems to work with basic ones, completely unsure if there are any gotchas, might get penalised with efficiency by creating the class every time, please let me know:
class EntityMixin(object):
def save(self):
...
def get_entity(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity(obj)
Usage:
>>> a = get_entity(1)
>>> a + 1
2
>>> b = get_entity('b')
>>> b.upper()
'B'
>>> c = get_entity([1,2])
>>> len(c)
2
>>> d = get_entity({'a':1})
>>> d['a']
1
>>> d = get_entity(map(lambda x : x, [1,2]))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jlin/projects/django-rest-framework-queryset/rest_framework_queryset/entity.py", line 11, in get_entity
return Entity(obj)
TypeError: map() must have at least two arguments.
Improve efficiency:
EntityClsCache = {}
class EntityMixin(object):
def save(self):
...
def _get_entity_cls(obj):
class Entity(obj.__class__, EntityMixin):
pass
return Entity
def get_entity(obj)
cls = None
try:
cls = EntityClsCache[obj.__class__]
except AttributeError:
cls = _get_entity_cls(obj)
EntityClsCache[obj.__class__] = cls
return cls(obj)
The solution you propose looks elegant, but it lacks caching, as in, you'll construct a unique class every time get_entity() is called, even if types are all the same.
Python has metaclasses, which act as class factories. Given that metaclass' methods override these of class, not the instance, we can implement class caching:
class EntityMixin(object):
pass
class CachingFactory(type):
__registry__ = {}
# Instead of declaring an inner class,
# we can also return type("Wrapper", (type_, EntityMixin), {}) right away,
# which, however, looks more obscure
def __makeclass(cls, type_):
class Wrapper(type_, EntityMixin):
pass
return Wrapper
# This is the simplest form of caching; for more realistic and less error-prone example,
# better use a more unique/complex key, for example, tuple of `value`'s ancestors --
# you can obtain them via type(value).__mro__
def __call__(cls, value):
t = type(value)
typename = t.__name__
if typename not in cls.__registry__:
cls.__registry__[typename] = cls.__makeclass(t)
return cls.__registry__[typename](value)
class Factory(object):
__metaclass__ = CachingFactory
This way, Factory(1) performs Factory.__call__(1), which is CachingFactory.__call__(1) (without metaclass, that'd be a constructor call instead, which would result in a class instance -- but we want to make a class first and only then instantiate it).
We can ensure that the objects created by Factory are the instances of the same class, which is crafted specifically for them at the first time:
>>> type(Factory(map(lambda x: x, [1, 2]))) is type(Factory([1]))
True
>>> type(Factory("a")) is type(Factory("abc"))
True

There is no constant in python so what to do to make variable constant [duplicate]

How do I declare a constant in Python?
In Java, we do:
public static final String CONST_NAME = "Name";
You cannot declare a variable or value as constant in Python.
To indicate to programmers that a variable is a constant, one usually writes it in upper case:
CONST_NAME = "Name"
To raise exceptions when constants are changed, see Constants in Python by Alex Martelli. Note that this is not commonly used in practice.
As of Python 3.8, there's a typing.Final variable annotation that will tell static type checkers (like mypy) that your variable shouldn't be reassigned. This is the closest equivalent to Java's final. However, it does not actually prevent reassignment:
from typing import Final
a: Final[int] = 1
# Executes fine, but mypy will report an error if you run mypy on this:
a = 2
There's no const keyword as in other languages, however it is possible to create a Property that has a "getter function" to read the data, but no "setter function" to re-write the data. This essentially protects the identifier from being changed.
Here is an alternative implementation using class property:
Note that the code is far from easy for a reader wondering about constants. See explanation below.
def constant(f):
def fset(self, value):
raise TypeError
def fget(self):
return f()
return property(fget, fset)
class _Const(object):
#constant
def FOO():
return 0xBAADFACE
#constant
def BAR():
return 0xDEADBEEF
CONST = _Const()
print(hex(CONST.FOO)) # -> '0xbaadfaceL'
CONST.FOO = 0
##Traceback (most recent call last):
## File "example1.py", line 22, in <module>
## CONST.FOO = 0
## File "example1.py", line 5, in fset
## raise TypeError
##TypeError
Code Explanation:
Define a function constant that takes an expression, and uses it to construct a "getter" - a function that solely returns the value of the expression.
The setter function raises a TypeError so it's read-only
Use the constant function we just created as a decoration to quickly define read-only properties.
And in some other more old-fashioned way:
(The code is quite tricky, more explanations below)
class _Const(object):
def FOO():
def fset(self, value):
raise TypeError
def fget(self):
return 0xBAADFACE
return property(**locals())
FOO = FOO() # Define property.
CONST = _Const()
print(hex(CONST.FOO)) # -> '0xbaadfaceL'
CONST.FOO = 0
##Traceback (most recent call last):
## File "example2.py", line 16, in <module>
## CONST.FOO = 0
## File "example2.py", line 6, in fset
## raise TypeError
##TypeError
To define the identifier FOO, firs define two functions (fset, fget - the names are at my choice).
Then use the built-in property function to construct an object that can be "set" or "get".
Note hat the property function's first two parameters are named fset and fget.
Use the fact that we chose these very names for our own getter & setter and create a keyword-dictionary using the ** (double asterisk) applied to all the local definitions of that scope to pass parameters to the property function
In Python instead of language enforcing something, people use naming conventions e.g __method for private methods and using _method for protected methods.
So in same manner you can simply declare the constant as all caps, e.g.:
MY_CONSTANT = "one"
If you want that this constant never changes, you can hook into attribute access and do tricks, but a simpler approach is to declare a function:
def MY_CONSTANT():
return "one"
Only problem is everywhere you will have to do MY_CONSTANT(), but again MY_CONSTANT = "one" is the correct way in Python (usually).
You can also use namedtuple() to create constants:
>>> from collections import namedtuple
>>> Constants = namedtuple('Constants', ['pi', 'e'])
>>> constants = Constants(3.14, 2.718)
>>> constants.pi
3.14
>>> constants.pi = 3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
I've recently found a very succinct update to this which automatically raises meaningful error messages and prevents access via __dict__:
class CONST(object):
__slots__ = ()
FOO = 1234
CONST = CONST()
# ----------
print(CONST.FOO) # 1234
CONST.FOO = 4321 # AttributeError: 'CONST' object attribute 'FOO' is read-only
CONST.__dict__['FOO'] = 4321 # AttributeError: 'CONST' object has no attribute '__dict__'
CONST.BAR = 5678 # AttributeError: 'CONST' object has no attribute 'BAR'
We define over ourselves as to make ourselves an instance and then use slots to ensure that no additional attributes can be added. This also removes the __dict__ access route. Of course, the whole object can still be redefined.
Edit - Original solution
I'm probably missing a trick here, but this seems to work for me:
class CONST(object):
FOO = 1234
def __setattr__(self, *_):
pass
CONST = CONST()
#----------
print CONST.FOO # 1234
CONST.FOO = 4321
CONST.BAR = 5678
print CONST.FOO # Still 1234!
print CONST.BAR # Oops AttributeError
Creating the instance allows the magic __setattr__ method to kick in and intercept attempts to set the FOO variable. You could throw an exception here if you wanted to. Instantiating the instance over the class name prevents access directly via the class.
It's a total pain for one value, but you could attach lots to your CONST object. Having an upper class, class name also seems a bit grotty, but I think it's quite succinct overall.
Python doesn't have constants.
Perhaps the easiest alternative is to define a function for it:
def MY_CONSTANT():
return 42
MY_CONSTANT() now has all the functionality of a constant (plus some annoying braces).
Properties are one way to create constants. You can do it by declaring a getter property, but ignoring the setter. For example:
class MyFinalProperty(object):
#property
def name(self):
return "John"
You can have a look at an article I've written to find more ways to use Python properties.
In addition to the two top answers (just use variables with UPPERCASE names, or use properties to make the values read-only), I want to mention that it's possible to use metaclasses in order to implement named constants. I provide a very simple solution using metaclasses at GitHub which may be helpful if you want the values to be more informative about their type/name:
>>> from named_constants import Constants
>>> class Colors(Constants):
... black = 0
... red = 1
... white = 15
...
>>> c = Colors.black
>>> c == 0
True
>>> c
Colors.black
>>> c.name()
'black'
>>> Colors(0) is c
True
This is slightly more advanced Python, but still very easy to use and handy. (The module has some more features, including constants being read-only, see its README.)
There are similar solutions floating around in various repositories, but to the best of my knowledge they either lack one of the fundamental features that I would expect from constants (like being constant, or being of arbitrary type), or they have esoteric features added that make them less generally applicable. But YMMV, I would be grateful for feedback. :-)
Edit: Added sample code for Python 3
Note: this other answer looks like it provides a much more complete implementation similar to the following (with more features).
First, make a metaclass:
class MetaConst(type):
def __getattr__(cls, key):
return cls[key]
def __setattr__(cls, key, value):
raise TypeError
This prevents statics properties from being changed. Then make another class that uses that metaclass:
class Const(object):
__metaclass__ = MetaConst
def __getattr__(self, name):
return self[name]
def __setattr__(self, name, value):
raise TypeError
Or, if you're using Python 3:
class Const(object, metaclass=MetaConst):
def __getattr__(self, name):
return self[name]
def __setattr__(self, name, value):
raise TypeError
This should prevent instance props from being changed. To use it, inherit:
class MyConst(Const):
A = 1
B = 2
Now the props, accessed directly or via an instance, should be constant:
MyConst.A
# 1
my_const = MyConst()
my_const.A
# 1
MyConst.A = 'changed'
# TypeError
my_const.A = 'changed'
# TypeError
Here's an example of above in action. Here's another example for Python 3.
PEP 591 has the 'final' qualifier. Enforcement is down to the type checker.
So you can do:
MY_CONSTANT: Final = 12407
Note: Final keyword is only applicable for Python 3.8 version
from enum import Enum
class StringConsts(str,Enum):
ONE='one'
TWO='two'
print(f'Truth is {StringConsts.ONE=="one"}') #Truth is True
StringConsts.ONE="one" #Error: Cannot reassign
This mixin of Enum and str gives you the power of not having to reimplement setattr (through Enum) and comparison to other str objects (through str).
This might deprecate http://code.activestate.com/recipes/65207-constants-in-python/?in=user-97991 completely.
I declare constant values using frozen data class like this:
from dataclasses import dataclass
#dataclass(frozen=True)
class _Const:
SOME_STRING = 'some_string'
SOME_INT = 5
Const = _Const()
# In another file import Const and try
print(Const.SOME_STRING) # ITS OK!
Const.SOME_INT = 6 # dataclasses.FrozenInstanceError: cannot assign to field 'SOME_INT'
You can use a namedtuple as a workaround to effectively create a constant that works the same way as a static final variable in Java (a Java "constant"). As workarounds go, it's sort of elegant. (A more elegant approach would be to simply improve the Python language --- what sort of language lets you redefine math.pi? -- but I digress.)
(As I write this, I realize another answer to this question mentioned namedtuple, but I'll continue here because I'll show a syntax that more closely parallels what you'd expect in Java, as there is no need to create a named type as namedtuple forces you to do.)
Following your example, you'll remember that in Java we must define the constant inside some class; because you didn't mention a class name, let's call it Foo. Here's the Java class:
public class Foo {
public static final String CONST_NAME = "Name";
}
Here's the equivalent Python.
from collections import namedtuple
Foo = namedtuple('_Foo', 'CONST_NAME')('Name')
The key point I want to add here is that you don't need a separate Foo type (an "anonymous named tuple" would be nice, even though that sounds like an oxymoron), so we name our namedtuple _Foo so that hopefully it won't escape to importing modules.
The second point here is that we immediately create an instance of the nametuple, calling it Foo; there's no need to do this in a separate step (unless you want to). Now you can do what you can do in Java:
>>> Foo.CONST_NAME
'Name'
But you can't assign to it:
>>> Foo.CONST_NAME = 'bar'
…
AttributeError: can't set attribute
Acknowledgement: I thought I invented the namedtuple approach, but then I see that someone else gave a similar (although less compact) answer. Then I also noticed What are "named tuples" in Python?, which points out that sys.version_info is now a namedtuple, so perhaps the Python standard library already came up with this idea much earlier.
Note that unfortunately (this still being Python), you can erase the entire Foo assignment altogether:
>>> Foo = 'bar'
(facepalm)
But at least we're preventing the Foo.CONST_NAME value from being changed, and that's better than nothing. Good luck.
Here is an implementation of a "Constants" class, which creates instances with read-only (constant) attributes. E.g. can use Nums.PI to get a value that has been initialized as 3.14159, and Nums.PI = 22 raises an exception.
# ---------- Constants.py ----------
class Constants(object):
"""
Create objects with read-only (constant) attributes.
Example:
Nums = Constants(ONE=1, PI=3.14159, DefaultWidth=100.0)
print 10 + Nums.PI
print '----- Following line is deliberate ValueError -----'
Nums.PI = 22
"""
def __init__(self, *args, **kwargs):
self._d = dict(*args, **kwargs)
def __iter__(self):
return iter(self._d)
def __len__(self):
return len(self._d)
# NOTE: This is only called if self lacks the attribute.
# So it does not interfere with get of 'self._d', etc.
def __getattr__(self, name):
return self._d[name]
# ASSUMES '_..' attribute is OK to set. Need this to initialize 'self._d', etc.
#If use as keys, they won't be constant.
def __setattr__(self, name, value):
if (name[0] == '_'):
super(Constants, self).__setattr__(name, value)
else:
raise ValueError("setattr while locked", self)
if (__name__ == "__main__"):
# Usage example.
Nums = Constants(ONE=1, PI=3.14159, DefaultWidth=100.0)
print 10 + Nums.PI
print '----- Following line is deliberate ValueError -----'
Nums.PI = 22
Thanks to #MikeGraham 's FrozenDict, which I used as a starting point. Changed, so instead of Nums['ONE'] the usage syntax is Nums.ONE.
And thanks to #Raufio's answer, for idea to override __ setattr __.
Or for an implementation with more functionality, see #Hans_meine 's
named_constants at GitHub
A tuple technically qualifies as a constant, as a tuple will raise an error if you try to change one of its values. If you want to declare a tuple with one value, then place a comma after its only value, like this:
my_tuple = (0 """Or any other value""",)
To check this variable's value, use something similar to this:
if my_tuple[0] == 0:
#Code goes here
If you attempt to change this value, an error will be raised.
Here it is a collection of idioms that I created as an attempt to improve some of the already available answers.
I know the use of constant is not pythonic, and you should not do this at home!
However, Python is such a dynamic language! This forum shows how it is possible the creation of constructs that looks and feels like constants. This answer has as the primary purpose to explore what can be expressed by the language.
Please do not be too harsh with me :-).
For more details I wrote a accompaniment blog about these idioms.
In this post, I will call a constant variable to a constant reference to values (immutable or otherwise). Moreover, I say that a variable has a frozen value when it references a mutable object that a client-code cannot update its value(s).
A space of constants (SpaceConstants)
This idiom creates what looks like a namespace of constant variables (a.k.a. SpaceConstants). It is a modification of a code snippet by Alex Martelli to avoid the use of module objects. In particular, this modification uses what I call a class factory because within SpaceConstants function, a class called SpaceConstants is defined, and an instance of it is returned.
I explored the use of class factory to implement a policy-based design look-alike in Python in stackoverflow and also in a blogpost.
def SpaceConstants():
def setattr(self, name, value):
if hasattr(self, name):
raise AttributeError(
"Cannot reassign members"
)
self.__dict__[name] = value
cls = type('SpaceConstants', (), {
'__setattr__': setattr
})
return cls()
sc = SpaceConstants()
print(sc.x) # raise "AttributeError: 'SpaceConstants' object has no attribute 'x'"
sc.x = 2 # bind attribute x
print(sc.x) # print "2"
sc.x = 3 # raise "AttributeError: Cannot reassign members"
sc.y = {'name': 'y', 'value': 2} # bind attribute y
print(sc.y) # print "{'name': 'y', 'value': 2}"
sc.y['name'] = 'yprime' # mutable object can be changed
print(sc.y) # print "{'name': 'yprime', 'value': 2}"
sc.y = {} # raise "AttributeError: Cannot reassign members"
A space of frozen values (SpaceFrozenValues)
This next idiom is a modification of the SpaceConstants in where referenced mutable objects are frozen. This implementation exploits what I call shared closure between setattr and getattr functions. The value of the mutable object is copied and referenced by variable cache define inside of the function shared closure. It forms what I call a closure protected copy of a mutable object.
You must be careful in using this idiom because getattr return the value of cache by doing a deep copy. This operation could have a significant performance impact on large objects!
from copy import deepcopy
def SpaceFrozenValues():
cache = {}
def setattr(self, name, value):
nonlocal cache
if name in cache:
raise AttributeError(
"Cannot reassign members"
)
cache[name] = deepcopy(value)
def getattr(self, name):
nonlocal cache
if name not in cache:
raise AttributeError(
"Object has no attribute '{}'".format(name)
)
return deepcopy(cache[name])
cls = type('SpaceFrozenValues', (),{
'__getattr__': getattr,
'__setattr__': setattr
})
return cls()
fv = SpaceFrozenValues()
print(fv.x) # AttributeError: Object has no attribute 'x'
fv.x = 2 # bind attribute x
print(fv.x) # print "2"
fv.x = 3 # raise "AttributeError: Cannot reassign members"
fv.y = {'name': 'y', 'value': 2} # bind attribute y
print(fv.y) # print "{'name': 'y', 'value': 2}"
fv.y['name'] = 'yprime' # you can try to change mutable objects
print(fv.y) # print "{'name': 'y', 'value': 2}"
fv.y = {} # raise "AttributeError: Cannot reassign members"
A constant space (ConstantSpace)
This idiom is an immutable namespace of constant variables or ConstantSpace. It is a combination of awesomely simple Jon Betts' answer in stackoverflow with a class factory.
def ConstantSpace(**args):
args['__slots__'] = ()
cls = type('ConstantSpace', (), args)
return cls()
cs = ConstantSpace(
x = 2,
y = {'name': 'y', 'value': 2}
)
print(cs.x) # print "2"
cs.x = 3 # raise "AttributeError: 'ConstantSpace' object attribute 'x' is read-only"
print(cs.y) # print "{'name': 'y', 'value': 2}"
cs.y['name'] = 'yprime' # mutable object can be changed
print(cs.y) # print "{'name': 'yprime', 'value': 2}"
cs.y = {} # raise "AttributeError: 'ConstantSpace' object attribute 'x' is read-only"
cs.z = 3 # raise "AttributeError: 'ConstantSpace' object has no attribute 'z'"
A frozen space (FrozenSpace)
This idiom is an immutable namespace of frozen variables or FrozenSpace. It is derived from the previous pattern by making each variable a protected property by closure of the generated FrozenSpace class.
from copy import deepcopy
def FreezeProperty(value):
cache = deepcopy(value)
return property(
lambda self: deepcopy(cache)
)
def FrozenSpace(**args):
args = {k: FreezeProperty(v) for k, v in args.items()}
args['__slots__'] = ()
cls = type('FrozenSpace', (), args)
return cls()
fs = FrozenSpace(
x = 2,
y = {'name': 'y', 'value': 2}
)
print(fs.x) # print "2"
fs.x = 3 # raise "AttributeError: 'FrozenSpace' object attribute 'x' is read-only"
print(fs.y) # print "{'name': 'y', 'value': 2}"
fs.y['name'] = 'yprime' # try to change mutable object
print(fs.y) # print "{'name': 'y', 'value': 2}"
fs.y = {} # raise "AttributeError: 'FrozenSpace' object attribute 'x' is read-only"
fs.z = 3 # raise "AttributeError: 'FrozenSpace' object has no attribute 'z'"
I would make a class that overrides the __setattr__ method of the base object class and wrap my constants with that, note that I'm using python 2.7:
class const(object):
def __init__(self, val):
super(const, self).__setattr__("value", val)
def __setattr__(self, name, val):
raise ValueError("Trying to change a constant value", self)
To wrap a string:
>>> constObj = const("Try to change me")
>>> constObj.value
'Try to change me'
>>> constObj.value = "Changed"
Traceback (most recent call last):
...
ValueError: Trying to change a constant value
>>> constObj2 = const(" or not")
>>> mutableObj = constObj.value + constObj2.value
>>> mutableObj #just a string
'Try to change me or not'
It's pretty simple, but if you want to use your constants the same as you would a non-constant object (without using constObj.value), it will be a bit more intensive. It's possible that this could cause problems, so it might be best to keep the .value to show and know that you are doing operations with constants (maybe not the most 'pythonic' way though).
Unfortunately the Python has no constants so yet and it is shame. ES6 already added support constants to JavaScript (https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/const) since it is a very useful thing in any programming language.
As answered in other answers in Python community use the convention - user uppercase variable as constants, but it does not protect against arbitrary errors in code.
If you like, you may be found useful a single-file solution as next
(see docstrings how use it).
file constants.py
import collections
__all__ = ('const', )
class Constant(object):
"""
Implementation strict constants in Python 3.
A constant can be set up, but can not be changed or deleted.
Value of constant may any immutable type, as well as list or set.
Besides if value of a constant is list or set, it will be converted in an immutable type as next:
list -> tuple
set -> frozenset
Dict as value of a constant has no support.
>>> const = Constant()
>>> del const.temp
Traceback (most recent call last):
NameError: name 'temp' is not defined
>>> const.temp = 1
>>> const.temp = 88
Traceback (most recent call last):
...
TypeError: Constanst can not be changed
>>> del const.temp
Traceback (most recent call last):
...
TypeError: Constanst can not be deleted
>>> const.I = ['a', 1, 1.2]
>>> print(const.I)
('a', 1, 1.2)
>>> const.F = {1.2}
>>> print(const.F)
frozenset([1.2])
>>> const.D = dict()
Traceback (most recent call last):
...
TypeError: dict can not be used as constant
>>> del const.UNDEFINED
Traceback (most recent call last):
...
NameError: name 'UNDEFINED' is not defined
>>> const()
{'I': ('a', 1, 1.2), 'temp': 1, 'F': frozenset([1.2])}
"""
def __setattr__(self, name, value):
"""Declaration a constant with value. If mutable - it will be converted to immutable, if possible.
If the constant already exists, then made prevent againt change it."""
if name in self.__dict__:
raise TypeError('Constanst can not be changed')
if not isinstance(value, collections.Hashable):
if isinstance(value, list):
value = tuple(value)
elif isinstance(value, set):
value = frozenset(value)
elif isinstance(value, dict):
raise TypeError('dict can not be used as constant')
else:
raise ValueError('Muttable or custom type is not supported')
self.__dict__[name] = value
def __delattr__(self, name):
"""Deny against deleting a declared constant."""
if name in self.__dict__:
raise TypeError('Constanst can not be deleted')
raise NameError("name '%s' is not defined" % name)
def __call__(self):
"""Return all constans."""
return self.__dict__
const = Constant()
if __name__ == '__main__':
import doctest
doctest.testmod()
If this is not enough, see full testcase for it.
import decimal
import uuid
import datetime
import unittest
from ..constants import Constant
class TestConstant(unittest.TestCase):
"""
Test for implementation constants in the Python
"""
def setUp(self):
self.const = Constant()
def tearDown(self):
del self.const
def test_create_constant_with_different_variants_of_name(self):
self.const.CONSTANT = 1
self.assertEqual(self.const.CONSTANT, 1)
self.const.Constant = 2
self.assertEqual(self.const.Constant, 2)
self.const.ConStAnT = 3
self.assertEqual(self.const.ConStAnT, 3)
self.const.constant = 4
self.assertEqual(self.const.constant, 4)
self.const.co_ns_ta_nt = 5
self.assertEqual(self.const.co_ns_ta_nt, 5)
self.const.constant1111 = 6
self.assertEqual(self.const.constant1111, 6)
def test_create_and_change_integer_constant(self):
self.const.INT = 1234
self.assertEqual(self.const.INT, 1234)
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.INT = .211
def test_create_and_change_float_constant(self):
self.const.FLOAT = .1234
self.assertEqual(self.const.FLOAT, .1234)
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.FLOAT = .211
def test_create_and_change_list_constant_but_saved_as_tuple(self):
self.const.LIST = [1, .2, None, True, datetime.date.today(), [], {}]
self.assertEqual(self.const.LIST, (1, .2, None, True, datetime.date.today(), [], {}))
self.assertTrue(isinstance(self.const.LIST, tuple))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.LIST = .211
def test_create_and_change_none_constant(self):
self.const.NONE = None
self.assertEqual(self.const.NONE, None)
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.NONE = .211
def test_create_and_change_boolean_constant(self):
self.const.BOOLEAN = True
self.assertEqual(self.const.BOOLEAN, True)
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.BOOLEAN = False
def test_create_and_change_string_constant(self):
self.const.STRING = "Text"
self.assertEqual(self.const.STRING, "Text")
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.STRING += '...'
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.STRING = 'TEst1'
def test_create_dict_constant(self):
with self.assertRaisesRegexp(TypeError, 'dict can not be used as constant'):
self.const.DICT = {}
def test_create_and_change_tuple_constant(self):
self.const.TUPLE = (1, .2, None, True, datetime.date.today(), [], {})
self.assertEqual(self.const.TUPLE, (1, .2, None, True, datetime.date.today(), [], {}))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.TUPLE = 'TEst1'
def test_create_and_change_set_constant(self):
self.const.SET = {1, .2, None, True, datetime.date.today()}
self.assertEqual(self.const.SET, {1, .2, None, True, datetime.date.today()})
self.assertTrue(isinstance(self.const.SET, frozenset))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.SET = 3212
def test_create_and_change_frozenset_constant(self):
self.const.FROZENSET = frozenset({1, .2, None, True, datetime.date.today()})
self.assertEqual(self.const.FROZENSET, frozenset({1, .2, None, True, datetime.date.today()}))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.FROZENSET = True
def test_create_and_change_date_constant(self):
self.const.DATE = datetime.date(1111, 11, 11)
self.assertEqual(self.const.DATE, datetime.date(1111, 11, 11))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.DATE = True
def test_create_and_change_datetime_constant(self):
self.const.DATETIME = datetime.datetime(2000, 10, 10, 10, 10)
self.assertEqual(self.const.DATETIME, datetime.datetime(2000, 10, 10, 10, 10))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.DATETIME = None
def test_create_and_change_decimal_constant(self):
self.const.DECIMAL = decimal.Decimal(13123.12312312321)
self.assertEqual(self.const.DECIMAL, decimal.Decimal(13123.12312312321))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.DECIMAL = None
def test_create_and_change_timedelta_constant(self):
self.const.TIMEDELTA = datetime.timedelta(days=45)
self.assertEqual(self.const.TIMEDELTA, datetime.timedelta(days=45))
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.TIMEDELTA = 1
def test_create_and_change_uuid_constant(self):
value = uuid.uuid4()
self.const.UUID = value
self.assertEqual(self.const.UUID, value)
with self.assertRaisesRegexp(TypeError, 'Constanst can not be changed'):
self.const.UUID = []
def test_try_delete_defined_const(self):
self.const.VERSION = '0.0.1'
with self.assertRaisesRegexp(TypeError, 'Constanst can not be deleted'):
del self.const.VERSION
def test_try_delete_undefined_const(self):
with self.assertRaisesRegexp(NameError, "name 'UNDEFINED' is not defined"):
del self.const.UNDEFINED
def test_get_all_defined_constants(self):
self.assertDictEqual(self.const(), {})
self.const.A = 1
self.assertDictEqual(self.const(), {'A': 1})
self.const.B = "Text"
self.assertDictEqual(self.const(), {'A': 1, 'B': "Text"})
Advantages:
1. Access to all constants for whole project
2. Strict control for values of constants
Lacks:
1. Not support for custom types and the type 'dict'
Notes:
Tested with Python3.4 and Python3.5 (I am use the 'tox' for it)
Testing environment:
.
$ uname -a
Linux wlysenko-Aspire 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
We can create a descriptor object.
class Constant:
def __init__(self,value=None):
self.value = value
def __get__(self,instance,owner):
return self.value
def __set__(self,instance,value):
raise ValueError("You can't change a constant")
1) If we wanted to work with constants at the instance level then:
class A:
NULL = Constant()
NUM = Constant(0xFF)
class B:
NAME = Constant('bar')
LISTA = Constant([0,1,'INFINITY'])
>>> obj=A()
>>> print(obj.NUM) #=> 255
>>> obj.NUM =100
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: You can't change a constant
2) if we wanted to create constants only at the class level, we could use a metaclass that serves as a container for our constants (our descriptor objects); all the classes that descend will inherit our constants (our descriptor objects) without any risk that can be modified.
# metaclass of my class Foo
class FooMeta(type): pass
# class Foo
class Foo(metaclass=FooMeta): pass
# I create constants in my metaclass
FooMeta.NUM = Constant(0xff)
FooMeta.NAME = Constant('FOO')
>>> Foo.NUM #=> 255
>>> Foo.NAME #=> 'FOO'
>>> Foo.NUM = 0 #=> ValueError: You can't change a constant
If I create a subclass of Foo, this class will inherit the constant without the possibility of modifying them
class Bar(Foo): pass
>>> Bar.NUM #=> 255
>>> Bar.NUM = 0 #=> ValueError: You can't change a constant
The Pythonic way of declaring "constants" is basically a module level variable:
RED = 1
GREEN = 2
BLUE = 3
And then write your classes or functions. Since constants are almost always integers, and they are also immutable in Python, you have a very little chance of altering it.
Unless, of course, if you explicitly set RED = 2.
There is a cleaner way to do this with namedtuple:
from collections import namedtuple
def make_consts(name, **kwargs):
return namedtuple(name, kwargs.keys())(**kwargs)
Usage Example
CONSTS = make_consts("baz1",
foo=1,
bar=2)
With this exactly approach you can namespace your constants.
Here's a trick if you want constants and don't care their values:
Just define empty classes.
e.g:
class RED:
pass
class BLUE:
pass
There's no perfect way to do this. As I understand it most programmers will just capitalize the identifier, so PI = 3.142 can be readily understood to be a constant.
On the otherhand, if you want something that actually acts like a constant, I'm not sure you'll find it. With anything you do there will always be some way of editing the "constant" so it won't really be a constant. Here's a very simple, dirty example:
def define(name, value):
if (name + str(id(name))) not in globals():
globals()[name + str(id(name))] = value
def constant(name):
return globals()[name + str(id(name))]
define("PI",3.142)
print(constant("PI"))
This looks like it will make a PHP-style constant.
In reality all it takes for someone to change the value is this:
globals()["PI"+str(id("PI"))] = 3.1415
This is the same for all the other solutions you'll find on here - even the clever ones that make a class and redefine the set attribute method - there will always be a way around them. That's just how Python is.
My recommendation is to just avoid all the hassle and just capitalize your identifiers. It wouldn't really be a proper constant but then again nothing would.
I am trying different ways to create a real constant in Python and perhaps I found the pretty solution.
Example:
Create container for constants
>>> DAYS = Constants(
... MON=0,
... TUE=1,
... WED=2,
... THU=3,
... FRI=4,
... SAT=5,
... SUN=6
... )
Get value from container
>>> DAYS.MON
0
>>> DAYS['MON']
0
Represent with pure python data structures
>>> list(DAYS)
['WED', 'SUN', 'FRI', 'THU', 'MON', 'TUE', 'SAT']
>>> dict(DAYS)
{'WED': 2, 'SUN': 6, 'FRI': 4, 'THU': 3, 'MON': 0, 'TUE': 1, 'SAT': 5}
All constants are immutable
>>> DAYS.MON = 7
...
AttributeError: Immutable attribute
>>> del DAYS.MON
...
AttributeError: Immutable attribute
Autocomplete only for constants
>>> dir(DAYS)
['FRI', 'MON', 'SAT', 'SUN', 'THU', 'TUE', 'WED']
Sorting like list.sort
>>> DAYS.sort(key=lambda (k, v): v, reverse=True)
>>> list(DAYS)
['SUN', 'SAT', 'FRI', 'THU', 'WED', 'TUE', 'MON']
Copability with python2 and python3
Simple container for constants
from collections import OrderedDict
from copy import deepcopy
class Constants(object):
"""Container of constant"""
__slots__ = ('__dict__')
def __init__(self, **kwargs):
if list(filter(lambda x: not x.isupper(), kwargs)):
raise AttributeError('Constant name should be uppercase.')
super(Constants, self).__setattr__(
'__dict__',
OrderedDict(map(lambda x: (x[0], deepcopy(x[1])), kwargs.items()))
)
def sort(self, key=None, reverse=False):
super(Constants, self).__setattr__(
'__dict__',
OrderedDict(sorted(self.__dict__.items(), key=key, reverse=reverse))
)
def __getitem__(self, name):
return self.__dict__[name]
def __len__(self):
return len(self.__dict__)
def __iter__(self):
for name in self.__dict__:
yield name
def keys(self):
return list(self)
def __str__(self):
return str(list(self))
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, str(self.__dict__))
def __dir__(self):
return list(self)
def __setattr__(self, name, value):
raise AttributeError("Immutable attribute")
def __delattr__(*_):
raise AttributeError("Immutable attribute")
Python dictionaries are mutable, so they don't seem like a good way to declare constants:
>>> constants = {"foo":1, "bar":2}
>>> print constants
{'foo': 1, 'bar': 2}
>>> constants["bar"] = 3
>>> print constants
{'foo': 1, 'bar': 3}
In python, a constant is simply a variable with a name in all capitals, with words separated by the underscore character,
e.g
DAYS_IN_WEEK = 7
The value is mutable, as in you can change it. But given the rules for the name tell you is a constant, why would you? I mean, it is your program after all!
This is the approach taken throughout python. There is no private keyword for the same reason. Prefix the name with an underscore and you know it is intended to be private. Code can break the rule....just as a programmer could remove the private keyword anyway.
Python could have added a const keyword... but a programmer could remove keyword and then change the constant if they want to, but why do that? If you want to break the rule, you could change the rule anyway. But why bother to break the rule if the name makes the intention clear?
Maybe there is some unit test where it makes sense to apply a change to value? To see what happens for an 8 day week even though in the real world the number of days in the week cannot be changed. If the language stopped you making an exception if there is just this one case you need to break the rule...you would then have to stop declaring it as a constant, even though it still is a constant in the application, and there is just this one test case that sees what happens if it is changed.
The all upper case name tells you it is intended to be a constant. That is what is important. Not a language forcing constraints on code you have the power to change anyway.
That is the philosophy of python.
(This paragraph was meant to be a comment on those answers here and there, which mentioned namedtuple, but it is getting too long to be fit into a comment, so, here it goes.)
The namedtuple approach mentioned above is definitely innovative. For the sake of completeness, though, at the end of the NamedTuple section of its official documentation, it reads:
enumerated constants can be implemented with named tuples, but it is simpler and more efficient to use a simple class declaration:
class Status:
open, pending, closed = range(3)
In other words, the official documentation kind of prefers to use a practical way, rather than actually implementing the read-only behavior. I guess it becomes yet another example of Zen of Python:
Simple is better than complex.
practicality beats purity.
Maybe pconst library will help you (github).
$ pip install pconst
from pconst import const
const.APPLE_PRICE = 100
const.APPLE_PRICE = 200
[Out] Constant value of "APPLE_PRICE" is not editable.
You can use StringVar or IntVar, etc, your constant is const_val
val = 'Stackoverflow'
const_val = StringVar(val)
const.trace('w', reverse)
def reverse(*args):
const_val.set(val)
You can do it with collections.namedtuple and itertools:
import collections
import itertools
def Constants(Name, *Args, **Kwargs):
t = collections.namedtuple(Name, itertools.chain(Args, Kwargs.keys()))
return t(*itertools.chain(Args, Kwargs.values()))
>>> myConstants = Constants('MyConstants', 'One', 'Two', Three = 'Four')
>>> print myConstants.One
One
>>> print myConstants.Two
Two
>>> print myConstants.Three
Four
>>> myConstants.One = 'Two'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
In Python, constants do not exist, but you can indicate that a variable is a constant and must not be changed by adding CONST_ to the start of the variable name and stating that it is a constant in a comment:
myVariable = 0
CONST_daysInWeek = 7 # This is a constant - do not change its value.
CONSTANT_daysInMonth = 30 # This is also a constant - do not change this value.
Alternatively, you may create a function that acts like a constant:
def CONST_daysInWeek():
return 7;

pickling dict inherited class is missing internal values

I've played around for a bit with the code and obviously the reason for the failure is that when setting 'wham' the value is another instance of the class TestDict which works fine as long as i don't try to pickle and unpickle it.
Because if i do self.test is missing.
Traceback:
Traceback (most recent call last):
File "test.py", line 30, in <module>
loads_a = loads(dumps_a)
File "test.py", line 15, in __setitem__
if self.test == False:
AttributeError: 'TestDict' object has no attribute 'test'
The code:
from pickle import dumps, loads
class TestDict(dict):
def __init__(self, test=False, data={}):
super().__init__(data)
self.test = test
def __getitem__(self, k):
if self.test == False:
pass
return dict.__getitem__(self, k)
def __setitem__(self, k, v):
if self.test == False:
pass
if type(v) == dict:
super().__setitem__(k, TestDict(False, v))
else:
super().__setitem__(k, v)
if __name__ == '__main__':
a = TestDict()
a['wham'] = {'bam' : 1}
b = TestDict(True)
b['wham'] = {'bam' : 2}
dumps_a = dumps(a)
dumps_b = dumps(b)
loads_a = loads(dumps_a)
loads_b = loads(dumps_b)
print(loads_a)
print(loads_b)
The code works if not replacing __setitem__ and __getitem__ but i want to add extra functionality to those two specific functions.
I've also tried:
class TestDict(dict):
__module__ = os.path.splitext(os.path.basename(__file__))[0]
Which sort of worked, as long as i don't nest TestDict within TestDict meaning i won't get to replace __setitem__ and __getitem__ in sub-parts of the dictionary.
Define __reduce__ method:
class TestDict(dict):
...
def __reduce__(self):
return type(self), (self.test, dict(self))
The problem is that pickle doesn't call the __init__ method of the object when it does the unpicling, so the self.test variable is not created at the moment when it tries to set the items of the dictionary.
Apparently setting the attributes is staged after setting items of the dictionary.
One way to solve it is to add a class level attribute that will be overridden in the instances:
class TestDict(dict):
test = False
...

Setting Property via a String

I'm trying to set a Python class property outside of the class via the setattr(self, item, value) function.
class MyClass:
def getMyProperty(self):
return self.__my_property
def setMyProperty(self, value):
if value is None:
value = ''
self.__my_property = value
my_property = property( getMyProperty, setMyProperty )
And in another script, I create an instance and want to specify the property and let the property mutator handle the simple validation.
myClass = MyClass()
new_value = None
# notice the property in quotes
setattr(myClass, 'my_property', new_value)
The problem is that it doesn't appear to be calling the setMyProperty(self, value) mutator. For a quick test to verify that it doesn't get called, I change the mutator to:
def setMyProperty(self, value):
raise ValueError('WTF! Why are you not being called?')
if value is None:
value = ''
self.__my_property = value
I'm fairly new to Python, and perhaps there's another way to do what I'm trying to do, but can someone explain why the mutator isn't being called when setattr(self, item, value) is called?
Is there another way to set a property via a string? I need the validation inside the mutator to be executed when setting the property value.
Works for me:
>>> class MyClass(object):
... def get(self): return 10
... def setprop(self, val): raise ValueError("hax%s"%str(val))
... prop = property(get, setprop)
...
>>> i = MyClass()
>>> i.prop =4
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in setprop
ValueError: hax4
>>> i.prop
10
>>> setattr(i, 'prop', 12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in setprop
ValueError: hax12
The code you pasted seems to do the same as mine, except that my class inherits from object, but that's cause I'm running Python 2.6 and I thought that in 2.7 all classes automatically inherit from object. Try that, though, and see if it helps.
To make it even clearer: try just doing myClass.my_property = 4. Does that raise an exception? If not then it's an issue with inheriting from object - properties only work for new-style classes, i.e. classes that inherit from object.

Python: changing methods and attributes at runtime

I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
Oh, and please don't ask why.
This example shows the differences between adding a method to a class and to an instance.
>>> class Dog():
... def __init__(self, name):
... self.name = name
...
>>> skip = Dog('Skip')
>>> spot = Dog('Spot')
>>> def talk(self):
... print 'Hi, my name is ' + self.name
...
>>> Dog.talk = talk # add method to class
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk()
Hi, my name is Spot
>>> del Dog.talk # remove method from class
>>> skip.talk() # won't work anymore
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
>>> import types
>>> f = types.MethodType(talk, skip, Dog)
>>> skip.talk = f # add method to specific instance
>>> skip.talk()
Hi, my name is Skip
>>> spot.talk() # won't work, since we only modified skip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Dog instance has no attribute 'talk'
I wish to create a class in Python that I can add and remove attributes and methods.
import types
class SpecialClass(object):
#classmethod
def removeVariable(cls, name):
return delattr(cls, name)
#classmethod
def addMethod(cls, func):
return setattr(cls, func.__name__, types.MethodType(func, cls))
def hello(self, n):
print n
instance = SpecialClass()
SpecialClass.addMethod(hello)
>>> SpecialClass.hello(5)
5
>>> instance.hello(6)
6
>>> SpecialClass.removeVariable("hello")
>>> instance.hello(7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'SpecialClass' object has no attribute 'hello'
>>> SpecialClass.hello(8)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'SpecialClass' has no attribute 'hello'
A possibly interesting alternative to using types.MethodType in:
>>> f = types.MethodType(talk, puppy, Dog)
>>> puppy.talk = f # add method to specific instance
would be to exploit the fact that functions are descriptors:
>>> puppy.talk = talk.__get__(puppy, Dog)
I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?
You can add and remove attributes and methods to any class, and they'll be available to all instances of the class:
>>> def method1(self):
pass
>>> def method1(self):
print "method1"
>>> def method2(self):
print "method2"
>>> class C():
pass
>>> c = C()
>>> c.method()
Traceback (most recent call last):
File "<pyshell#62>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.method = method1
>>> c.method()
method1
>>> C.method = method2
>>> c.method()
method2
>>> del C.method
>>> c.method()
Traceback (most recent call last):
File "<pyshell#68>", line 1, in <module>
c.method()
AttributeError: C instance has no attribute 'method'
>>> C.attribute = "foo"
>>> c.attribute
'foo'
>>> c.attribute = "bar"
>>> c.attribute
'bar'
you can just assign directly to the class (either by accessing the original class name or via __class__ ):
class a : pass
ob=a()
ob.__class__.blah=lambda self,k: (3, self,k)
ob.blah(5)
ob2=a()
ob2.blah(7)
will print
(3, <__main__.a instance at 0x7f18e3c345f0>, 5)
(3, <__main__.a instance at 0x7f18e3c344d0>, 7)
Simply:
f1 = lambda:0 #method for instances
f2 = lambda _:0 #method for class
class C: pass #class
c1,c2 = C(),C() #instances
print dir(c1),dir(c2)
#add to the Instances
c1.func = f1
c1.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
del c1.func,c1.any
#add to the Class
C.func = f2
C.any = 1.23
print dir(c1),dir(c2)
print c1.func(),c1.any
print c2.func(),c2.any
which results in:
['__doc__', '__module__'] ['__doc__', '__module__']
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__']
0 1.23
['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__', 'any', 'func']
0 1.23
0 1.23
another alternative, if you need to replace the class wholesale is to modify the class attribute:
>>> class A(object):
... def foo(self):
... print 'A'
...
>>> class B(object):
... def foo(self):
... print 'Bar'
...
>>> a = A()
>>> a.foo()
A
>>> a.__class__ = B
>>> a.foo()
Bar
Does the class itself necessarily need to be modified? Or is the goal simply to replace what object.method() does at a particular point during runtime?
I ask because I sidestep the problem of actually modifying the class to monkey patch specific method calls in my framework with getattribute and a Runtime Decorator on my Base inheritance object.
Methods retrieved by a Base object in getattribute are wrapped in a Runtime_Decorator that parses the method calls keyword arguments for decorators/monkey patches to apply.
This enables you to utilize the syntax object.method(monkey_patch="mypatch"), object.method(decorator="mydecorator"), and even object.method(decorators=my_decorator_list).
This works for any individual method call (I leave out magic methods), does so without actually modifying any class/instance attributes, can utilize arbitrary, even foreign methods to patch, and will work transparently on sublcasses that inherit from Base (provided they don't override getattribute of course).
import trace
def monkey_patched(self, *args, **kwargs):
print self, "Tried to call a method, but it was monkey patched instead"
return "and now for something completely different"
class Base(object):
def __init__(self):
super(Base, self).__init__()
def testmethod(self):
print "%s test method" % self
def __getattribute__(self, attribute):
value = super(Base, self).__getattribute__(attribute)
if "__" not in attribute and callable(value):
value = Runtime_Decorator(value)
return value
class Runtime_Decorator(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
if kwargs.has_key("monkey_patch"):
module_name, patch_name = self._resolve_string(kwargs.pop("monkey_patch"))
module = self._get_module(module_name)
monkey_patch = getattr(module, patch_name)
return monkey_patch(self.function.im_self, *args, **kwargs)
if kwargs.has_key('decorator'):
decorator_type = str(kwargs['decorator'])
module_name, decorator_name = self._resolve_string(decorator_type)
decorator = self._get_decorator(decorator_name, module_name)
wrapped_function = decorator(self.function)
del kwargs['decorator']
return wrapped_function(*args, **kwargs)
elif kwargs.has_key('decorators'):
decorators = []
for item in kwargs['decorators']:
module_name, decorator_name = self._resolve_string(item)
decorator = self._get_decorator(decorator_name, module_name)
decorators.append(decorator)
wrapped_function = self.function
for item in reversed(decorators):
wrapped_function = item(wrapped_function)
del kwargs['decorators']
return wrapped_function(*args, **kwargs)
else:
return self.function(*args, **kwargs)
def _resolve_string(self, string):
try: # attempt to split the string into a module and attribute
module_name, decorator_name = string.split(".")
except ValueError: # there was no ".", it's just a single attribute
module_name = "__main__"
decorator_name = string
finally:
return module_name, decorator_name
def _get_module(self, module_name):
try: # attempt to load the module if it exists already
module = modules[module_name]
except KeyError: # import it if it doesn't
module = __import__(module_name)
finally:
return module
def _get_decorator(self, decorator_name, module_name):
module = self._get_module(module_name)
try: # attempt to procure the decorator class
decorator_wrap = getattr(module, decorator_name)
except AttributeError: # decorator not found in module
print("failed to locate decorators %s for function %s." %\
(kwargs["decorator"], self.function))
else:
return decorator_wrap # instantiate the class with self.function
class Tracer(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
tracer = trace.Trace(trace=1)
tracer.runfunc(self.function, *args, **kwargs)
b = Base()
b.testmethod(monkey_patch="monkey_patched")
b.testmethod(decorator="Tracer")
#b.testmethod(monkey_patch="external_module.my_patch")
The downside to this approach is getattribute hooks all access to attributes, so the checking of and potential wrapping of methods occurs even for attributes that are not methods + won't be utilizing the feature for the particular call in question. And using getattribute at all is inherently somewhat complicated.
The actual impact of this overhead in my experience/for my purposes has been negligible, and my machine runs a dual core Celeron. The previous implementation I used introspected methods upon object init and bound the Runtime_Decorator to methods then. Doing things that way eliminated the need to utilize getattribute and reduced the overhead mentioned previously... however, it also breaks pickle (maybe not dill) and is less dynamic then this approach.
The only use cases I have actually come across "in the wild" with this technique were with timing and tracing decorators. However, the possibilities it opens up are extremely wide ranging.
If you have a preexisting class that cannot be made to inherit from a different base (or utilize the technique it's own class definition or in it's base class'), then the whole thing simply does not apply to your issue at all unfortunately.
I don't think setting/removing non-callable attributes on a class at runtime is necessarily so challenging? unless you want classes that inherit from the modified class to automatically reflect the changes in themselves as well... That'd be a whole 'nother can o' worms by the sound of it though.

Categories

Resources