I would like to know how to convert a python object from a dictionary (using python3 btw). I realize that this question has been asked (and answered) already (here). However, in my case the object is given entirely in terms of #property values, for example:
class Test:
#property
def value(self):
return 1.0
Regarding conversion to a dictionary: The __dict__ dictionary of the Test class is empty, and consequently, the vars
function does not work as expected:
>>> vars(Test())
{}
Still, I can use gettattr(Test(), 'value'), to obtain 1.0, so
the value is present.
Note: The reason I am coming up with this apparently contrived example is that I am trying to convert a cython cdef class (containing parameters) to a dictionary. The recommended way to wrap c structures with properties using cython is indeed based on properties.
I think you could use dir:
a = Test()
dir(a)
Output:
['__doc__', '__module__', 'value']
So you could maybe do something like:
d = {}
for attr in dir(a):
if not attr.startswith("__"):
d[attr] = getattr(a, attr)
Output:
d = {'value': 1.0}
Maybe you could abuse that:
In [10]: type(Test().__class__.__dict__['value']) is property
Out[10]: True
So you check the class of the object and if it has attribute of type property.
Here is how I would do it:
t = Test()
dictionary = {attr_name: getattr(t, attr_name)
for attr_name, method in t.__class__.__dict__.items()
if isinstance(method, property)}
It is even worse that that. You could imagine to build an instance __dict__ at init time, but that would not solve anything, except for read_only constant properties. Because the value in the dict will be a copy of the property at the time it was taken, and will not reflect future changes.
Why do the following lines give me the same result?
str.upper('hello')
and
'hello'.upper()
I tried to do the same with list.append but got a TypeError.
list.append([1])
Is the str type in Python overloaded? How can this be achieved by writing a class/function? I would appreciate an example.
list.append takes two arguments - the list to modify and the element to append. So you need to do it like this:
ls = [1]
list.append(ls, 2)
which is equivalent to the much more popular:
ls.append(2)
str.upper and list.append are both functions.
str.upper takes one argument.
>>> str.upper('test')
'TEST'
list.append takes two arguments.
>>> my_list = []
>>> list.append(my_list, 1)
>>> my_list
[1]
str.upper and list.append (like other functions) are also non-data-descriptors with a __get__ method which in this context has two implications:
When you access the function through the class via the dot notation (str.upper, list.append) the function's __get__ method (i.e. string.upper.__get__ and list.append.__get__) is called but it returns just the function itself.
When you access the function through an instance (my_string.upper, my_list.append) the function's __get__ method is called and it will return a new callable acting like the original function, but with whatever was "in front of the dot" automatically passed as the first argument. .
That's why you need to pass 1 - 1 = 0 arguments when calling my_string.upper() and 2 - 1 = 1 argument when calling my_list.append(1).
>>> 'my_string'.upper()
'MY_STRING'
>>>
>>> my_list = []
>>> my_list.append(1)
>>> my_list
[1]
You could even get these modified callables (methods) by explicitly calling __get__ and passing the argument to be bound (what has been before the dot) as its argument.
>>> my_string = 'my_string'
>>> upper_maker = str.upper.__get__(my_string)
>>> upper_maker()
'MY_STRING'
>>>
>>> my_list = []
>>> appender = list.append.__get__(my_list)
>>> appender(1)
>>> my_list
[1]
Finally, here's a short example demonstrating how descriptor instances can detect whether they are being accessed via their owner-class or via an instance.
class Descriptor:
def __get__(self, instance, owner_class):
if instance is None:
print('accessed through class')
# list.append.__get__ would return list.append here
else:
print('accessed through instance')
# list.append.__get__ would build a new callable here
# that takes one argument x and that internally calls
# list.append(instance, x)
class Class:
attribute = Descriptor()
Class.attribute # prints 'accessed through class'
instance = Class()
instance.attribute # prints 'accessed through instance'
Quoting Dave Kirbys answer from Relationship between string module and str:
There is some overlap between the string module and the str type,
mainly for historical reasons. In early versions of Python str objects
did not have methods, so all string manipulation was done with
functions from the string module. When methods were added to the str
type (in Python 1.5?) the functions were left in the string module for
compatibility, but now just forward to the equivalent str method.
However the string module also contains constants and functions that
are not methods on str, such as formatting, character translation etc.
There is nothing at all magical going on with str (except that we have a nice syntactic shortcut to creating one using ""). You can write a class that behaves like str and list to see more clearly what is happening here.
class MyClass():
def __init__(self, arg):
self.val=str(arg)
def do_thing(self):
self.val = "asdf"
def do_thing_with_arg(self, arg):
self.val = "asdf " + str(arg)
def __repr__(self):
return self.val
my_thing = MyClass("qwerty")
# this is like 'hello'.upper()
my_thing.do_thing()
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like str.upper('hello')
MyClass.do_thing(my_thing)
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like my_list.append('qwerty')
my_thing.do_thing_with_arg('zxcv')
print(my_thing)
# it prints 'asdf zxcv'
my_thing = MyClass("qwerty")
# this is like list.append(my_list, 'qwerty')
MyClass.do_thing_with_arg(my_thing, 'zxcv')
print(my_thing)
# it prints 'asdf zxcv'
The short version is, you're invoking what looks like an "instance method" on a class, but you are supplying the instance ('self') yourself as the first argument to the function call.
I'm very new to python and I wish I could do . notation to access values of a dict.
Lets say I have test like this:
>>> test = dict()
>>> test['name'] = 'value'
>>> print(test['name'])
value
But I wish I could do test.name to get value. Infact I did it by overriding the __getattr__ method in my class like this:
class JuspayObject:
def __init__(self,response):
self.__dict__['_response'] = response
def __getattr__(self,key):
try:
return self._response[key]
except KeyError,err:
sys.stderr.write('Sorry no key matches')
and this works! when I do:
test.name // I get value.
But the problem is when I just print test alone I get the error as:
'Sorry no key matches'
Why is this happening?
This functionality already exists in the standard libraries, so I recommend you just use their class.
>>> from types import SimpleNamespace
>>> d = {'key1': 'value1', 'key2': 'value2'}
>>> n = SimpleNamespace(**d)
>>> print(n)
namespace(key1='value1', key2='value2')
>>> n.key2
'value2'
Adding, modifying and removing values is achieved with regular attribute access, i.e. you can use statements like n.key = val and del n.key.
To go back to a dict again:
>>> vars(n)
{'key1': 'value1', 'key2': 'value2'}
The keys in your dict should be string identifiers for attribute access to work properly.
Simple namespace was added in Python 3.3. For older versions of the language, argparse.Namespace has similar behaviour.
I assume that you are comfortable in Javascript and want to borrow that kind of syntax... I can tell you by personal experience that this is not a great idea.
It sure does look less verbose and neat; but in the long run it is just obscure. Dicts are dicts, and trying to make them behave like objects with attributes will probably lead to (bad) surprises.
If you need to manipulate the fields of an object as if they were a dictionary, you can always resort to use the internal __dict__ attribute when you need it, and then it is explicitly clear what you are doing. Or use getattr(obj, 'key') to have into account the inheritance structure and class attributes too.
But by reading your example it seems that you are trying something different... As the dot operator will already look in the __dict__ attribute without any extra code.
In addition to this answer, one can add support for nested dicts as well:
from types import SimpleNamespace
class NestedNamespace(SimpleNamespace):
def __init__(self, dictionary, **kwargs):
super().__init__(**kwargs)
for key, value in dictionary.items():
if isinstance(value, dict):
self.__setattr__(key, NestedNamespace(value))
else:
self.__setattr__(key, value)
nested_namespace = NestedNamespace({
'parent': {
'child': {
'grandchild': 'value'
}
},
'normal_key': 'normal value',
})
print(nested_namespace.parent.child.grandchild) # value
print(nested_namespace.normal_key) # normal value
Note that this does not support dot notation for dicts that are somewhere inside e.g. lists.
Could you use a named tuple?
from collections import namedtuple
Test = namedtuple('Test', 'name foo bar')
my_test = Test('value', 'foo_val', 'bar_val')
print(my_test)
print(my_test.name)
__getattr__ is used as a fallback when all other attribute lookup rules have failed. When you try to "print" your object, Python look for a __repr__ method, and since you don't implement it in your class it ends up calling __getattr__ (yes, in Python methods are attributes too). You shouldn't assume which key getattr will be called with, and, most important, __getattr__ must raise an AttributeError if it cannot resolve key.
As a side note: don't use self.__dict__ for ordinary attribute access, just use the plain attribute notation:
class JuspayObject:
def __init__(self,response):
# don't use self.__dict__ here
self._response = response
def __getattr__(self,key):
try:
return self._response[key]
except KeyError,err:
raise AttributeError(key)
Now if your class has no other responsability (and your Python version is >= 2.6 and you don't need to support older versions), you may just use a namedtuple : http://docs.python.org/2/library/collections.html#collections.namedtuple
You can use the built-in method argparse.Namespace():
import argparse
args = argparse.Namespace()
args.name = 'value'
print(args.name)
# 'value'
You can also get the original dict via vars(args).
class convert_to_dot_notation(dict):
"""
Access dictionary attributes via dot notation
"""
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
test = {"name": "value"}
data = convert_to_dot_notation(test)
print(data.name)
You have to be careful when using __getattr__, because it's used for a lot of builtin Python functionality.
Try something like this...
class JuspayObject:
def __init__(self,response):
self.__dict__['_response'] = response
def __getattr__(self, key):
# First, try to return from _response
try:
return self.__dict__['_response'][key]
except KeyError:
pass
# If that fails, return default behavior so we don't break Python
try:
return self.__dict__[key]
except KeyError:
raise AttributeError, key
>>> j = JuspayObject({'foo': 'bar'})
>>> j.foo
'bar'
>>> j
<__main__.JuspayObject instance at 0x7fbdd55965f0>
Here is a simple, handy dot notation helper example that is working with nested items:
def dict_get(data:dict, path:str, default = None):
pathList = re.split(r'\.', path, flags=re.IGNORECASE)
result = data
for key in pathList:
try:
key = int(key) if key.isnumeric() else key
result = result[key]
except:
result = default
break
return result
Usage example:
my_dict = {"test1": "str1", "nested_dict": {"test2": "str2"}, "nested_list": ["str3", {"test4": "str4"}]}
print(dict_get(my_dict, "test1"))
# str1
print(dict_get(my_dict, "nested_dict.test2"))
# str2
print(dict_get(my_dict, "nested_list.1.test4"))
# str4
With a small addition to this answer you can support lists as well:
class NestedNamespace(SimpleNamespace):
def __init__(self, dictionary, **kwargs):
super().__init__(**kwargs)
for key, value in dictionary.items():
if isinstance(value, dict):
self.__setattr__(key, NestedNamespace(value))
elif isinstance(value, list):
self.__setattr__(key, map(NestedNamespace, value))
else:
self.__setattr__(key, value)
2022 answer: I've created the dotwiz package -- this is a fast, tiny library that seems to perform really well in most cases.
>>> from dotwiz import DotWiz
>>> test = DotWiz(hello='world')
>>> test.works = True
>>> test
✫(hello='world', works=True)
>>> test.hello
'world'
>>> assert test.works
This feature is baked into OmegaConf:
from omegaconf import OmegaConf
your_dict = {"k" : "v", "list" : [1, {"a": "1", "b": "2", 3: "c"}]}
adot_dict = OmegaConf.create(your_dict)
print(adot_dict.k)
print(adot_dict.list)
Installation is:
pip install omegaconf
This lib comes in handy for configurations, which it is actually made for:
from omegaconf import OmegaConf
cfg = OmegaConf.load('config.yml')
print(cfg.data_path)
I use the dotted_dict package:
>>> from dotted_dict import DottedDict
>>> test = DottedDict()
>>> test.name = 'value'
>>> print(test.name)
value
Advantages over SimpleNamespace
(See #win's answer.) DottedDict is an actual dict:
>>> isinstance(test, dict)
True
This allows, for example, checking for membership:
>>> 'name' in test
True
whereas for SimpleNamespace you need something much less readable like hasattr(test, 'name').
Don't use DotMap
I found this out the hard way. If you reference a non-member it adds it rather than throwing an error. This can lead to hard to find bugs in code:
>>> from dotmap import DotMap
>>> dm = DotMap()
>>> 'a' in dm
False
>>> x = dm.a
>>> 'a' in dm
True
#!/usr/bin/env python3
import json
from sklearn.utils import Bunch
from collections.abc import MutableMapping
def dotted(inpt: MutableMapping,
*args,
**kwargs
) -> Bunch:
"""
Enables recursive dot notation for ``dict``.
"""
return json.loads(json.dumps(inpt),
object_hook=lambda x:
Bunch(**{**Bunch(), **x}))
You can make hacks adding dot notation to Dicts mostly work, but there are always namespace problems. As in, what does this do?
x = DotDict()
x["values"] = 1989
print(x. values)
I use pydash, which is a Python port of JS's lodash, to do these things a different way when the nesting gets too ugly.
Add a __repr__() method to the class so that you can customize the text to be shown on
print text
Learn more here: https://web.archive.org/web/20121022015531/http://diveintopython.net/object_oriented_framework/special_class_methods2.html
I have a class:
class A:
s = 'some string'
b = <SOME OTHER INSTANCE>
now I want this class to have the functionality of a string whenever it can. That is:
a = A()
print a.b
will print b's value. But I want functions that expect a string (for example replace) to work. For example:
'aaaa'.replace('a', a)
to actually do:
'aaa'.replace('a', a.s)
I tried overidding __get__ but this isn't correct.
I see that you can do this by subclassing str, but is there a way without it?
If you want your class to have the functionality of a string, just extend the built in string class.
>>> class A(str):
... b = 'some other value'
...
>>> a = A('x')
>>> a
'x'
>>> a.b
'some other value'
>>> 'aaa'.replace('a',a)
'xxx'
I found an answer in Subclassing Python tuple with multiple __init__ arguments .
I used Dave's solution and extended str, and then added a new function:
def __new__(self,a,b):
s=a
return str.__new__(A,s)
Override __str__ or __unicode__ to set the string representation of an object (Python documentation).
What I want to do in my code:
myobj = <SomeBuiltinClass>()
myobj.randomattr = 1
print myobj.randomattr
...
I can implement a custom SomeClass that implements __setattr__ __getattr__.
But I wonder if there is already a built-in Python class or simple way to do this?
You can just use an empty class:
class A(object): pass
a = A()
a.randomattr = 1
I like using the Bunch idiom for this. There are list of variations and some discussion here.
One solution is to use mock's:
from mock import Mock
myobj = Mock()
myobj.randomattr = 1
print myobj.randomattr
Second solution is to use namedtuple:
from collections import namedtuple
myobj = namedtuple('MyObject', '')
myobj.randomattr = 1
print myobj.randomattr