Related
I'm very new to python and I wish I could do . notation to access values of a dict.
Lets say I have test like this:
>>> test = dict()
>>> test['name'] = 'value'
>>> print(test['name'])
value
But I wish I could do test.name to get value. Infact I did it by overriding the __getattr__ method in my class like this:
class JuspayObject:
def __init__(self,response):
self.__dict__['_response'] = response
def __getattr__(self,key):
try:
return self._response[key]
except KeyError,err:
sys.stderr.write('Sorry no key matches')
and this works! when I do:
test.name // I get value.
But the problem is when I just print test alone I get the error as:
'Sorry no key matches'
Why is this happening?
This functionality already exists in the standard libraries, so I recommend you just use their class.
>>> from types import SimpleNamespace
>>> d = {'key1': 'value1', 'key2': 'value2'}
>>> n = SimpleNamespace(**d)
>>> print(n)
namespace(key1='value1', key2='value2')
>>> n.key2
'value2'
Adding, modifying and removing values is achieved with regular attribute access, i.e. you can use statements like n.key = val and del n.key.
To go back to a dict again:
>>> vars(n)
{'key1': 'value1', 'key2': 'value2'}
The keys in your dict should be string identifiers for attribute access to work properly.
Simple namespace was added in Python 3.3. For older versions of the language, argparse.Namespace has similar behaviour.
I assume that you are comfortable in Javascript and want to borrow that kind of syntax... I can tell you by personal experience that this is not a great idea.
It sure does look less verbose and neat; but in the long run it is just obscure. Dicts are dicts, and trying to make them behave like objects with attributes will probably lead to (bad) surprises.
If you need to manipulate the fields of an object as if they were a dictionary, you can always resort to use the internal __dict__ attribute when you need it, and then it is explicitly clear what you are doing. Or use getattr(obj, 'key') to have into account the inheritance structure and class attributes too.
But by reading your example it seems that you are trying something different... As the dot operator will already look in the __dict__ attribute without any extra code.
In addition to this answer, one can add support for nested dicts as well:
from types import SimpleNamespace
class NestedNamespace(SimpleNamespace):
def __init__(self, dictionary, **kwargs):
super().__init__(**kwargs)
for key, value in dictionary.items():
if isinstance(value, dict):
self.__setattr__(key, NestedNamespace(value))
else:
self.__setattr__(key, value)
nested_namespace = NestedNamespace({
'parent': {
'child': {
'grandchild': 'value'
}
},
'normal_key': 'normal value',
})
print(nested_namespace.parent.child.grandchild) # value
print(nested_namespace.normal_key) # normal value
Note that this does not support dot notation for dicts that are somewhere inside e.g. lists.
Could you use a named tuple?
from collections import namedtuple
Test = namedtuple('Test', 'name foo bar')
my_test = Test('value', 'foo_val', 'bar_val')
print(my_test)
print(my_test.name)
__getattr__ is used as a fallback when all other attribute lookup rules have failed. When you try to "print" your object, Python look for a __repr__ method, and since you don't implement it in your class it ends up calling __getattr__ (yes, in Python methods are attributes too). You shouldn't assume which key getattr will be called with, and, most important, __getattr__ must raise an AttributeError if it cannot resolve key.
As a side note: don't use self.__dict__ for ordinary attribute access, just use the plain attribute notation:
class JuspayObject:
def __init__(self,response):
# don't use self.__dict__ here
self._response = response
def __getattr__(self,key):
try:
return self._response[key]
except KeyError,err:
raise AttributeError(key)
Now if your class has no other responsability (and your Python version is >= 2.6 and you don't need to support older versions), you may just use a namedtuple : http://docs.python.org/2/library/collections.html#collections.namedtuple
You can use the built-in method argparse.Namespace():
import argparse
args = argparse.Namespace()
args.name = 'value'
print(args.name)
# 'value'
You can also get the original dict via vars(args).
class convert_to_dot_notation(dict):
"""
Access dictionary attributes via dot notation
"""
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
test = {"name": "value"}
data = convert_to_dot_notation(test)
print(data.name)
You have to be careful when using __getattr__, because it's used for a lot of builtin Python functionality.
Try something like this...
class JuspayObject:
def __init__(self,response):
self.__dict__['_response'] = response
def __getattr__(self, key):
# First, try to return from _response
try:
return self.__dict__['_response'][key]
except KeyError:
pass
# If that fails, return default behavior so we don't break Python
try:
return self.__dict__[key]
except KeyError:
raise AttributeError, key
>>> j = JuspayObject({'foo': 'bar'})
>>> j.foo
'bar'
>>> j
<__main__.JuspayObject instance at 0x7fbdd55965f0>
Here is a simple, handy dot notation helper example that is working with nested items:
def dict_get(data:dict, path:str, default = None):
pathList = re.split(r'\.', path, flags=re.IGNORECASE)
result = data
for key in pathList:
try:
key = int(key) if key.isnumeric() else key
result = result[key]
except:
result = default
break
return result
Usage example:
my_dict = {"test1": "str1", "nested_dict": {"test2": "str2"}, "nested_list": ["str3", {"test4": "str4"}]}
print(dict_get(my_dict, "test1"))
# str1
print(dict_get(my_dict, "nested_dict.test2"))
# str2
print(dict_get(my_dict, "nested_list.1.test4"))
# str4
With a small addition to this answer you can support lists as well:
class NestedNamespace(SimpleNamespace):
def __init__(self, dictionary, **kwargs):
super().__init__(**kwargs)
for key, value in dictionary.items():
if isinstance(value, dict):
self.__setattr__(key, NestedNamespace(value))
elif isinstance(value, list):
self.__setattr__(key, map(NestedNamespace, value))
else:
self.__setattr__(key, value)
2022 answer: I've created the dotwiz package -- this is a fast, tiny library that seems to perform really well in most cases.
>>> from dotwiz import DotWiz
>>> test = DotWiz(hello='world')
>>> test.works = True
>>> test
✫(hello='world', works=True)
>>> test.hello
'world'
>>> assert test.works
This feature is baked into OmegaConf:
from omegaconf import OmegaConf
your_dict = {"k" : "v", "list" : [1, {"a": "1", "b": "2", 3: "c"}]}
adot_dict = OmegaConf.create(your_dict)
print(adot_dict.k)
print(adot_dict.list)
Installation is:
pip install omegaconf
This lib comes in handy for configurations, which it is actually made for:
from omegaconf import OmegaConf
cfg = OmegaConf.load('config.yml')
print(cfg.data_path)
I use the dotted_dict package:
>>> from dotted_dict import DottedDict
>>> test = DottedDict()
>>> test.name = 'value'
>>> print(test.name)
value
Advantages over SimpleNamespace
(See #win's answer.) DottedDict is an actual dict:
>>> isinstance(test, dict)
True
This allows, for example, checking for membership:
>>> 'name' in test
True
whereas for SimpleNamespace you need something much less readable like hasattr(test, 'name').
Don't use DotMap
I found this out the hard way. If you reference a non-member it adds it rather than throwing an error. This can lead to hard to find bugs in code:
>>> from dotmap import DotMap
>>> dm = DotMap()
>>> 'a' in dm
False
>>> x = dm.a
>>> 'a' in dm
True
#!/usr/bin/env python3
import json
from sklearn.utils import Bunch
from collections.abc import MutableMapping
def dotted(inpt: MutableMapping,
*args,
**kwargs
) -> Bunch:
"""
Enables recursive dot notation for ``dict``.
"""
return json.loads(json.dumps(inpt),
object_hook=lambda x:
Bunch(**{**Bunch(), **x}))
You can make hacks adding dot notation to Dicts mostly work, but there are always namespace problems. As in, what does this do?
x = DotDict()
x["values"] = 1989
print(x. values)
I use pydash, which is a Python port of JS's lodash, to do these things a different way when the nesting gets too ugly.
Add a __repr__() method to the class so that you can customize the text to be shown on
print text
Learn more here: https://web.archive.org/web/20121022015531/http://diveintopython.net/object_oriented_framework/special_class_methods2.html
I have a Python script that reads a file containing a command line invocation of some other tool. I'd like to modify the options of this invocation before calling the tool. For example, I might transform:
my_util --input file1.txt --option1 red --option2 blue
...to this:
my_util --input file1_001.txt --option1 red --option3 green
(More accurately, I'd be working on the arguments as lists.)
I figured that using the argparse module would be the easiest way to do this: I could parse the args, change, add or remove the options as I need to, and then reconstruct the command line.
But how do I do the last step? Given the Namespace object returned by parse_args(), can I easily reconstruct a list of command line options, such as could be passed to subprocess.Popen()?
A Namespace object is just a simple object subclass, so you can get the values out as a dict with vars:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> args = parser.parse_args(['--foo', 'BAR'])
>>> vars(args)
{'foo': 'BAR'}
Or you can assign to a class directly and get the arguments out as class variables:
>>> class C(object):
... pass
...
>>> c = C()
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo')
>>> parser.parse_args(args=['--foo', 'BAR'], namespace=c)
>>> c.foo
'BAR'
It would be fairly easy to use either of these structures to test/replace arguments and pass the results to Popen.
I know this is an old question, but I've just encountered the same problem. I realized that all I need is a way to iterate over the Action objects. Unfortunately, the internal list is not exposed by ArgParser itself. However, these objects are returned by add_argument(), so I can construct my own list. Well, putting actions.append() around each call looked like too much typing to me, so I store all options in a tuple:
def add_argument(*args, **kwargs):
return (args, kwargs)
parser = argparse.ArgumentParser()
options = (
add_argument('--verbose', action='store_true'),
add_argument('--author'),
add_argument('--subject', required=True),
add_argument('--cache', nargs='?'),
add_argument('files', nargs=''),
)
actions = []
for (args, kwargs) in options:
actions.append(parser.add_argument(*args, **kwargs))
args = parser.parse_args()
At this point, the options are parsed in args, and all argparse.Action objects are stored in the actions list. I can then iterate over this list and reconstruct the options like this:
cmdline = []
for action in actions:
value = getattr(args, action.dest)
if action.required or value != action.default:
if action.option_strings:
cmdline.append(action.option_strings[0])
if action.nargs is None:
cmdline.append(value)
elif action.nargs == '?':
if value != action.const:
cmdline.append(value)
elif action.nargs != 0:
cmdline += value
In my specific case, I also wanted to remove some options from the command line. To do that I simply added them separately with a call to parser.add_argument() and not through the options tuple.
What I want to accomplish:
dct = {'foo':0, 'bar':1, 'baz':2}
def func(**dct):
pass
#function signature is now func(foo=0, bar=1, baz=2)
However, the ** syntax is obviously clashing here between expanding a dict (what I want to do) and declaring a parameter that holds the keyword arguments (what I don't want to do).
Is this possible?
Based on my interpretation of your requirements -- you want to dynamically define a function with a signature that matches the content of adict provided at runtime -- there are two issues here which makes it impractical.
If the arguments are defined at run-time, how can your function reference the variables? Are you planning to build the function body at run-time as well?
dicts are unordered, so you cannot reliably use them to define positional arguments
I suspect this is an XY problem. If you can explain what you're trying to achieve then perhaps we can be of better help.
However, assuming you're trying to assign default keyword arguments using a dict then one way to achieve this would be to use decorators. For example:
def defaultArgs(default_kw):
"decorator to assign default kwargs"
def wrap(f):
def wrapped_f(**kwargs):
kw = {}
kw.update(default_kw) # apply defaults
kw.update(kwargs) # apply from input args
f(**kw) # run actual function with updated kwargs
return wrapped_f
return wrap
You can then use this decorator to assign default keyword arguments to a function that expects only keyword arguments:
defaults = {'foo':0, 'bar':1, 'baz':2}
#defaultArgs(defaults)
def func(**kwargs):
print kwargs # args accessible via the kwargs dict
Results:
func() # prints {'baz': 2, 'foo': 0, 'bar': 1}
func(foo=2) # prints {'baz': 2, 'foo': 2, 'bar': 1}
params = {'bar':1000, 'hello':'world'}
func(**params) # prints {'baz': 2, 'foo': 0, 'bar': 1000, 'hello': 'world'}
Note that you will not be able to use positional arguments:
func(1, 2, 3) # raises TypeError
what you want i believe is eval() link
an answer i gave on a similar question: https://stackoverflow.com/a/11865190/1561176
I'm really not sure what you plan on accomplishing here. The following works (sort of):
def func(**dct):
pass
dct = {'foo':0, 'bar':1, 'baz':2}
func(**dct)
How do you plan on using foo, bar or baz in the function if they're created dynamically? If you give some more details on what you're actually trying to accomplish, we might be able to be a little more helpful.
I'm using ConfigObj in python with Template-style interpolation. Unwrapping my config dictionary via ** doesn't seem to do interpolation. Is this a feature or a bug? Any nice workarounds?
$ cat my.conf
foo = /test
bar = $foo/directory
>>> import configobj
>>> config = configobj.ConfigObj('my.conf', interpolation='Template')
>>> config['bar']
'/test/directory'
>>> '{bar}'.format(**config)
'$foo/directory'
I'd expect the second line to be /test/directory. Why doesn't interpolation work with **kwargs?
When unpacking the keyword argument, then a new object is created: of type dict. This dictionary contains the the raw-values of the configuration (no interpolation)
Demonstration:
>>> id(config)
31143152
>>> def showKeywordArgs(**kwargs):
... print(kwargs, type(kwargs), id(kwargs))
...
>>> showKeywordArgs(**config)
({'foo': '/test', 'bar': '$foo/directory'}, <type 'dict'>, 35738944)
To resolve your problem you could create an expanded version of your configuration like this:
>>> expandedConfig = {k: config[k] for k in config}
>>> '{bar}'.format(**expandedConfig)
'/test/directory'
Another more elegant way is to simply avoid unpacking: This can be achieved by using the function string.Formatter.vformat:
import string
fmt = string.Formatter()
fmt.vformat("{bar}", None, config)
I have had a similar problem.
A workaround is to use configobj's function ".dict()". This works because configobj returns a real dictionary, which Python knows how to unpack.
Your example becomes:
>>> import configobj
>>> config = configobj.ConfigObj('my.conf', interpolation='Template')
>>> config['bar']
'/test/directory'
>>> '{bar}'.format(**config.dict())
'/test/directory'
Say you have,
foo = 'bar'
d = {'a-key':'a-value'}
And you want
d = {'a-key':'a-value','foo':'bar'}
e = {'foo':foo}
I know you can do,
d['foo'] = foo
#Either of the following for e
e = {'foo':foo}
e = dict(foo=foo)
But, in all these way to add the variable foo to dict, I have had to use the word foo twice; once to indicate the key and once for its value.
It seems wasteful to me to use foo twice. Is there a simpler way, in which you can tell python "Add this variable to the dictionary with its name as the key and its value as the value"?
you can do something like this
def add_dry_foo(d, namespace, fooName):
d[fooName] = namespace[fooName]
foo = 'oh-foo'
d = {}
add_dry_foo(d, locals(), 'foo')
print d
Actutally using foo twice is remarkably common in python programs. It is used extensively for passing on arguments eg
def f(foo, bar):
g(foo=foo)
Which is a specialised case of the dictionary manipulations in your question.
I don't think there is a way of avoiding it without resorting to magic, so I think you'll have to live with it.
You can use:
name = 'foo'
d[name] = vars[name]
I don't see the difference between your d and e cases: both set 'foo' to the value of foo.
It gets trickier if you want to bury this in a function:
def add_variable(d, name):
# blah
because then it has to use inspect to start poking around in frames.
This sounds like a larger problem that might have a nicer solution if you wanted to describe it to us. For example, if the problem is that you don't care just about foo, but in fact, a whole slew of local variables, then maybe you want something like:
d.update(locals())
which will copy the names and value of all the local variables into d.
If you don't want to pass all of locals() (which may be a security risk if you don't fully trust the function you're sending the data too), a one-line answer could be this:
dict([ (var, locals()[var]) for var in ['foo', 'bar'] ])
or in Python 3.0 this would become possible:
{ var: locals()[var] for var in ['foo', 'bar'] }
To add all the local variables to a dict you can do:
d.update(locals())
The same works for function calls:
func(**locals())
Note that depending on where you are locals() might of course contain stuff that should not end up in the dict. So you could implement a filter function:
def filtered_update(d, namespace):
for key, value in namespace.items():
if not key.startswith('__'):
d[key] = value
filtered_update(d, locals())
Of course the Python philosophy is "explicit is better than implicit", so generally I would walk the extra mile and do this kind of stuff by hand (otherwise you have to be careful about what goes on in your local namespace).
You could use eval, although I'm not sure that I'd recommend it.
>>> d = dict()
>>> foo = 'wibble'
>>> def add(d, name):
d[name] = eval(name)
>>> add(d, 'foo')
>>> d
{'foo': 'wibble'}
Edit:
I should point out why I don't recommend "eval". What happens if you do something like this? (from: http://mail.python.org/pipermail/tutor/2005-November/042854.html)
>>> s = "(lambda loop: loop(loop)) (lambda self: self(self))"
>>> add(d, s)
Traceback (most recent call last):
File "<pyshell#54>", line 1, in <module>
add(d, s)
File "<pyshell#43>", line 2, in add
d[name] = eval(name)
File "<string>", line 1, in <module>
File "<string>", line 1, in <lambda>
File "<string>", line 1, in <lambda>
...
File "<string>", line 1, in <lambda>
RuntimeError: maximum recursion depth exceeded
It seems to me what you are talking about is an
enhancement to parameter passing functionality:
def func(*vars):
provides a tuple of ordered values without keys
def func(**vars):
provides a dict of key value pairs, that
MUST be passed as key=value pairs.
def func(***vars):
WOULD PROVIDE a dict of key value pairs,
passed either explicitly as key=value,
or implicitly as key (a variable,
literals would cause error without key=)
SO:
(x1,x2,x3) = (1,2,3)
def myfunc(***vars):
retrun vars
myfunc(x1,x2,x3)
>>> {'x1':1,'x2':2,'x3':3}
But of course, this is just wishful thinking...