Develop Reference

How do I define setter, getter for dynamically added attributes

I have a class as follows:
class A:
def __init__(self):
pass
def add_attr(self, name):
setattr(self, name, 'something')
How do I define custom setter, getter for self.name? I cannot use __setattr__, __getattribute__ because that will change the behaviour of add_attr too.
EDIT: the users of this class will add arbitrary number of attributes with arbitrary names:
a = A()
a.add_attr('attr1')
a.add_attr('attr2')
I want custom behavior for only these user added attributes.

Building off #Devesh Kumar Singh’s answer, I would implement it in some way like this:
class A:
def __init__(self):
self.attrs = {}
def __setattr__(self, key, value):
if key in self.attrs:
self.set_attr(key, value)
else:
object.__setattr__(self, key, value)
def __getattribute__(self, key):
if key in self.__dict__.get(attrs, {}):
return self.__dict__['get_attr'](self, key)
return object.__getattribute__(self, key)
def get_attr(self, key):
r = self.attrs[key]
# logic
return r
def set_attr(self, key, value):
# logic
self.attrs[key] = value
def add_attr(self, key, value=None):
self.attrs[key] = value
add_attr is only used to initialise the variable the first time. You could also edit __setattr__ to set all new attributes in the self.attrs rather than self.__dict__

Custom getter and setter logic? That's what a property is made for. Usually these are used to magically mask function calls and make them look like attribute access
class MyDoubler(object):
def __init__(self, x):
self._x = x
#property
def x(self):
return x * 2
#x.setter
def x(self, value):
self._x = value
>>> md = MyDoubler(10)
>>> md.x
20
>>> md.x = 20
>>> md.x
40
>>> md._x
20
But there's no rule saying you can't abuse that power to add custom behavior to your getters and setters.
class A(object):
def __init__(self):
pass
#staticmethod
def default_getter_factory(name):
def default_getter(self):
return self.name
return default_getter
#staticmethod
def default_setter_factory(name):
def default_setter(self, value):
setattr(self, name, value)
return default_setter
def add_attr(self, name, getterfactory=None, setterfactory=None):
private_name = f"_{name}"
if getterfactory is None:
getterfactory = self.__class__.default_getter_factory
if setterfactory is None:
setterfactory = self.__class__.default_setter_factory
getter, setter = getterfactory(private_name), setterfactory(private_name)
getter = property(getter)
setattr(self.__class__, name, getter)
setattr(self.__class__, name, getter.setter(setter))
That said this is all a bit silly, and chances are that whatever it is you're trying to do is a thing that shouldn't be done. Dynamic programming is all well and good, but if I were to review code that did this, I would think very long and hard about alternative solutions before approving it. This reeks of technical debt to me.

One possibility I could think of is to have a dictionary of dynamic attributes, and set and get the dynamic attributes using the dictionary
class A:
def __init__(self):
#Dictionary of attributes
self.attrs = {}
#Set attribute
def set_attr(self, name):
self.attrs[name] = 'something'
#Get attribute
def get_attr(self, name):
return self.attrs.get(name)
a = A()
a.set_attr('var')
print(a.get_attr('var'))
The output will be something
Or an alternate is to use property decorator to add arguments explicitly outside the class, as described here
class A:
def __init__(self):
pass
a = A()
#Add attributes via property decorator
a.attr_1 = property(lambda self: self.attr_1)
a.attr_2 = property(lambda self: self.attr_2)
#Assign them values and print them
a.attr_1 = 4
a.attr_2 = 6
print(a.attr_1, a.attr_2)
The output will be 4 6

I am gonna answer my own question just for reference. This is based on others' answers here. The idea is to use default __setattr__ and __getattribute__ on attributes not added through add_attr.
class A:
def __init__(self):
self.attrs = {}
def add_attr(self, name):
self.attrs[name] = 'something'
def __getattribute__(self, name):
try:
object.__getattribute__(self, 'attrs')[name] # valid only if added by user
# custom logic and return
except (KeyError, AttributeError):
return object.__getattribute__(self, name)
def __setattr__(self, name, val):
# similar to __getattribute__

Why can't I remove a method with delattr?

There mustn't be an error (according to the docs of Python 2.7):
class C(object):
def __init__(self):
self.d = {}
def set_key(self, key, val):
self.d[key] = val
def get_key(self, key):
return self.d[key]
c = C()
delattr(c, 'set_key')
However:
AttributeError: set_key
I can do delattr on a class object. Can I remove a bound function from an instance?

set_key is an attribute of the class, not the instance. delattr(C, 'set_key') works as expected.
If you want the function not to be available from only the one instance, you can try c.set_key = None.

I am not quite sure why Python (2.7 at least) isn't allowing bound methods to be deleted in this manner.
That said, here is one way to simulate your desired behaviour:
def delete_method(obj, method_name):
def stub(*args, **kwargs):
raise AttributeError("'%s' object has no attribute '%s'" % (
obj.__class__.__name__, method_name))
setattr(obj, method_name, stub)
c = C()
delete_method(c, 'set_key')
c.set_key('the_answer', 42)
When run, this gives
AttributeError: 'C' object has no attribute 'set_key'
Note that this technique isn't universal: for example, you won't be able to use it to remove a method from an instance of the dict class.

There is also a method to hide member methods (not deleting them) to pass restricted interface to another place:
def safe_interface(obj, enabled):
"Returns an interface object blocking functions which are not in enabled."
def assert_name(name):
if name not in enabled:
raise KeyError('No attribute %s.' % name)
class Caller(object):
def __getattribute__(self, name):
assert_name(name)
return getattr(obj, name)
def __setattr__(self, name, val):
assert_name(name)
setattr(obj, name, val)
c = Caller()
return c
if __name__ == '__main__':
class C(object):
def __init__(self):
self.d = {}
def set_key(self, key, val):
self.d[key] = val
def get_key(self, key):
return self.d[key]
c = C()
d = safe_interface(c, ['set_key'])
key = 'c'
val = 'm'
d.set_key(key, val)
print c.get_key(key)
print d.get_key(key) # error, this method is not allowed

How to instantiate objects during a unit test's setup phase in Python

I've been diving into unit testing with Python, but can't figure out how I'm supposed to instantiate the object I want to test during the setup phase, and end up with a new object for each test.
For example, I have the following class I want to test:
class Cfg():
data = {}
def set(self, key, value):
self.data[key] = value
def get(self, key):
return self.data.get(key, None)
For each unit test, I want a newly instantiated Cfg object. My tests look like this:
from cfg import Cfg
class TestCfg():
def setup(self):
self.cfg = Cfg()
def teardown(self):
self.cfg = None
def test_a(self):
self.cfg.set('foo', 'bar')
assert self.cfg.get('foo') == 'bar'
def test_b(self):
assert self.cfg.get('foo') == 'bar'
I don't understand why test_b passes. I expected setup and tearDown to 'reset' my cfg instance, but it seems that cfg is persisting between tests. What am I doing wrong here and how can I achieve the expected behavior?

This is related to how you've written Cfg class. Move data initialization into __init__ method:
class Cfg():
def __init__(self):
self.data = {}
def set(self, key, value):
self.data[key] = value
def get(self, key):
return self.data.get(key, None)
And, you'll see failing test_b.

How to clear instance data without setattr?

I wanted, to make traversable (by DB, single file or just as string) class in python. I Write this (shorted):
from json import JSONDecoder, JSONEncoder
def json_decode(object): return JSONDecoder().decode(object)
def json_encode(object): return JSONEncoder().encode(object)
class Storage:
__separator__ = 'ANY OF ANYS'
__keys__ = []
__vals__ = []
__slots__ = ('__keys__', '__vals__', '__separator__')
def __getattr__(self, key):
try:
return self.__vals__[self.__keys__.index(key)]
except IndexError:
raise AttributeError
def __setattr__(self, key, val):
self.__keys__.append(key)
self.__vals__.append(val)
def store(self):
return (json_encode(self.__keys__) + self.__separator__ +
json_encode(self.__vals__))
def restore(self, stored):
stored = stored.split(self.__separator__)
for (key, val) in zip(json_decode(stored[0]), json_decode(stored[1])):
setattr(self, key, val)
And yea - that work, but... When i'm making more instances, all of them are like singleton.
So - how to set attribute to instance without _setattr_?
PS. I got idea - make in set/getattr an pass for keys/vals, but it'll make mess.

your __separator__, __keys__, __vals__ and __slots__ are attributes of the object "Storage"(class object). I don't know if it's exactly the same, but I'd call it static variables of the class.
If you want to have different values for each instance of Storage, define each of these variables in your __init__ function:
class Storage(object):
__slots__ = ('__keys__', '__vals__', '__separator__')
def __init__(self):
super(Storage, self).__setattr__('__separator__', "ANY OF ANYS")
super(Storage, self).__setattr__('__keys__', [])
super(Storage, self).__setattr__('__vals__', [])
def __getattr__(self, key):
try:
vals = getattr(self, '__vals__')
keys = getattr(self, '__keys__')
return vals[keys.index(key)]
except IndexError:
raise AttributeError
def __setattr__(self, key, val):
vals = getattr(self, '__vals__')
keys = getattr(self, '__keys__')
vals.append(val)
keys.append(key)
edited so getattr and setattr works
I got that problem 2 days ago. Don't know if that's exactly your problem, but you said that about "its like I have a singleton"

You could make your Storage class a subclass of a special base class like this:
class Singleton(object):
def __new__(cls, *args, **kwargs):
if '_inst_' not in vars(cls):
cls._inst = type.__new__(cls, *args, *kwargs)
return cls._inst
class Storage(Singleton):
....
As long as you don't override __new__() in your subclass, all subsequent calls to create new instances after the first will return the one first created.

Case insensitive dictionary

I'd like my dictionary to be case insensitive.
I have this example code:
text = "practice changing the color"
words = {'color': 'colour',
'practice': 'practise'}
def replace(words,text):
keys = words.keys()
for i in keys:
text= text.replace(i ,words[i])
return text
text = replace(words,text)
print text
Output = practise changing the colour
I'd like another string, "practice changing the Color", (where Color starts with a capital) to also give the same output.
I believe there is a general way to convert to lowercase using
mydictionary[key.lower()] but I'm not sure how to best integrate this into my existing code. (If this would be a reasonable, simple approach anyway).

The currently accepted answer wouldn't work for lots of cases, so it cannot be used as a drop-in dict replacement. Some tricky points in getting a proper dict replacement:
overloading all of the methods that involve keys
properly handling non-string keys
properly handling the constructor of the class
The following should work much better:
class CaseInsensitiveDict(dict):
#classmethod
def _k(cls, key):
return key.lower() if isinstance(key, basestring) else key
def __init__(self, *args, **kwargs):
super(CaseInsensitiveDict, self).__init__(*args, **kwargs)
self._convert_keys()
def __getitem__(self, key):
return super(CaseInsensitiveDict, self).__getitem__(self.__class__._k(key))
def __setitem__(self, key, value):
super(CaseInsensitiveDict, self).__setitem__(self.__class__._k(key), value)
def __delitem__(self, key):
return super(CaseInsensitiveDict, self).__delitem__(self.__class__._k(key))
def __contains__(self, key):
return super(CaseInsensitiveDict, self).__contains__(self.__class__._k(key))
def has_key(self, key):
return super(CaseInsensitiveDict, self).has_key(self.__class__._k(key))
def pop(self, key, *args, **kwargs):
return super(CaseInsensitiveDict, self).pop(self.__class__._k(key), *args, **kwargs)
def get(self, key, *args, **kwargs):
return super(CaseInsensitiveDict, self).get(self.__class__._k(key), *args, **kwargs)
def setdefault(self, key, *args, **kwargs):
return super(CaseInsensitiveDict, self).setdefault(self.__class__._k(key), *args, **kwargs)
def update(self, E={}, **F):
super(CaseInsensitiveDict, self).update(self.__class__(E))
super(CaseInsensitiveDict, self).update(self.__class__(**F))
def _convert_keys(self):
for k in list(self.keys()):
v = super(CaseInsensitiveDict, self).pop(k)
self.__setitem__(k, v)

Just for the record. I found an awesome impementation on Requests:
https://github.com/kennethreitz/requests/blob/v1.2.3/requests/structures.py#L37

If I understand you correctly and you want a way to key dictionaries in a non case-sensitive fashion, one way would be to subclass dict and overload the setter / getter:
class CaseInsensitiveDict(dict):
def __setitem__(self, key, value):
super(CaseInsensitiveDict, self).__setitem__(key.lower(), value)
def __getitem__(self, key):
return super(CaseInsensitiveDict, self).__getitem__(key.lower())

In my particular instance, I needed a case insensitive lookup, however, I did not want to modify the original case of the key. For example:
>>> d = {}
>>> d['MyConfig'] = 'value'
>>> d['myconfig'] = 'new_value'
>>> d
{'MyConfig': 'new_value'}
You can see that the dictionary still has the original key, however it is accessible case-insensitively. Here's a simple solution:
class CaseInsensitiveKey(object):
def __init__(self, key):
self.key = key
def __hash__(self):
return hash(self.key.lower())
def __eq__(self, other):
return self.key.lower() == other.key.lower()
def __str__(self):
return self.key
The __hash__ and __eq__ overrides are required for both getting and setting entries in the dictionary. This is creating keys that hash to the same position in the dictionary if they are case-insensitively equal.
Now either create a custom dictionary that initializes a CaseInsensitiveKey using the provided key:
class CaseInsensitiveDict(dict):
def __setitem__(self, key, value):
key = CaseInsensitiveKey(key)
super(CaseInsensitiveDict, self).__setitem__(key, value)
def __getitem__(self, key):
key = CaseInsensitiveKey(key)
return super(CaseInsensitiveDict, self).__getitem__(key)
or simply make sure to always pass an instance of CaseInsensitiveKey as the key when using the dictionary.

Would you consider using string.lower() on your inputs and using a fully lowercase dictionary? It's a bit of a hacky solution, but it works

I've modified the simple yet good solution by pleasemorebacon (thanks!) making it slightly more compact, self-contained and with minor updates to allow construction from {'a':1, 'B':2} and support __contains__ protocol.
Finally, since the CaseInsensitiveDict.Key is expected to be string (what else can be case-sensitive or not), it is a good idea to derive Key class from the str, then it is possible, for instance, to dump CaseInsensitiveDict with json.dumps out of the box.
# caseinsensitivedict.py
class CaseInsensitiveDict(dict):
class Key(str):
def __init__(self, key):
str.__init__(key)
def __hash__(self):
return hash(self.lower())
def __eq__(self, other):
return self.lower() == other.lower()
def __init__(self, data=None):
super(CaseInsensitiveDict, self).__init__()
if data is None:
data = {}
for key, val in data.items():
self[key] = val
def __contains__(self, key):
key = self.Key(key)
return super(CaseInsensitiveDict, self).__contains__(key)
def __setitem__(self, key, value):
key = self.Key(key)
super(CaseInsensitiveDict, self).__setitem__(key, value)
def __getitem__(self, key):
key = self.Key(key)
return super(CaseInsensitiveDict, self).__getitem__(key)
Here is a basic test script for those who like to check things in action:
# test_CaseInsensitiveDict.py
import json
import unittest
from caseinsensitivedict import *
class Key(unittest.TestCase):
def setUp(self):
self.Key = CaseInsensitiveDict.Key
self.lower = self.Key('a')
self.upper = self.Key('A')
def test_eq(self):
self.assertEqual(self.lower, self.upper)
def test_hash(self):
self.assertEqual(hash(self.lower), hash(self.upper))
def test_str(self):
self.assertEqual(str(self.lower), 'a')
self.assertEqual(str(self.upper), 'A')
class Dict(unittest.TestCase):
def setUp(self):
self.Dict = CaseInsensitiveDict
self.d1 = self.Dict()
self.d2 = self.Dict()
self.d1['a'] = 1
self.d1['B'] = 2
self.d2['A'] = 1
self.d2['b'] = 2
def test_contains(self):
self.assertIn('B', self.d1)
d = self.Dict({'a':1, 'B':2})
self.assertIn('b', d)
def test_init(self):
d = self.Dict()
self.assertFalse(d)
d = self.Dict({'a':1, 'B':2})
self.assertTrue(d)
def test_items(self):
self.assertDictEqual(self.d1, self.d2)
self.assertEqual(
[v for v in self.d1.items()],
[v for v in self.d2.items()])
def test_json_dumps(self):
s = json.dumps(self.d1)
self.assertIn('a', s)
self.assertIn('B', s)
def test_keys(self):
self.assertEqual(self.d1.keys(), self.d2.keys())
def test_values(self):
self.assertEqual(
[v for v in self.d1.values()],
[v for v in self.d2.values()])

While a case insensitive dictionary is a solution, and there are answers to how to achieve that, there is a possibly easier way in this case. A case insensitive search is sufficient:
import re
text = "Practice changing the Color"
words = {'color': 'colour', 'practice': 'practise'}
def replace(words,text):
keys = words.keys()
for i in keys:
exp = re.compile(i, re.I)
text = re.sub(exp, words[i], text)
return text
text = replace(words,text)
print text

You can do a dict key case insensitive search with a one liner:
>>> input_dict = {'aBc':1, 'xyZ':2}
>>> search_string = 'ABC'
>>> next((value for key, value in input_dict.items() if key.lower()==search_string.lower()), None)
1
>>> search_string = 'EFG'
>>> next((value for key, value in input_dict.items() if key.lower()==search_string.lower()), None)
>>>
You can place that into a function:
def get_case_insensitive_key_value(input_dict, key):
return next((value for dict_key, value in input_dict.items() if dict_key.lower() == key.lower()), None)
Note that only the first match is returned.

If you only need to do this once in your code (hence, no point to a function), the most straightforward way to deal with the problem is this:
lowercase_dict = {key.lower(): value for (key, value) in original_dict}
I'm assuming here that the dict in question isn't all that large--it might be inelegant to duplicate it, but if it's not large, it isn't going to hurt anything.
The advantage of this over #Fred's answer (though that also works) is that it produces the same result as a dict when the key isn't present: a KeyError.

There are multiple approaches to this problem, each has its set of pros and cons. Just to add to the list (looks like this option wasn't mentioned), it's possible to extend str class and use it as a key:
class CaseInsensitiveStr(str):
def __hash__(self) -> 'int':
return hash(self.lower())
def __eq__(self, other:'str') -> 'bool':
return self.lower() == other.lower()
It can work well if dictionary in question is private and some kind of interface is used to access it.
class MyThing:
def __init__(self):
self._d: 'dict[CaseInsensitiveStr, int]' = dict()
def set(self, key:'str', value:'int'):
self._d[CaseInsensitiveStr(key)] = value
def get(self, key:'str') -> 'int':
return self._d[CaseInsensitiveStr(key)]

I just set up a function to handle this:
def setLCdict(d, k, v):
k = k.lower()
d[k] = v
return d
myDict = {}
So instead of
myDict['A'] = 1
myDict['B'] = 2
You can:
myDict = setLCdict(myDict, 'A', 1)
myDict = setLCdict(myDict, 'B', 2)
You can then either lower case the value before looking it up or write a function to do so.
def lookupLCdict(d, k):
k = k.lower()
return d[k]
myVal = lookupLCdict(myDict, 'a')
Probably not ideal if you want to do this globally but works well if its just a subset you wish to use it for.