This question already has answers here:
Destructuring-bind dictionary contents
(17 answers)
Closed 1 year ago.
I am studying Python and currently going through some more learning with dictionaries.
I was wondering;
If I have a dictionary like: d = {'key_1': 'value_a', 'key_2': 'value_b'} and I want separate/divide this dictionary into variables where each variable is a key from the dictionary and each variables value is the value of that key in the dictionary.
What would be the best pythonic way to achieve this?
d = {'key_1': 'value_a', 'key_2': 'value_b'}
#perform the command and get
key_1 = 'value_a'
key_2 = 'value_b'
I tried: key_1, key_2 = d but it did not work.
Basically I am seeking expert's wisdom to find out if there is a better way to reduce 2 lines of code into one.
Note: This is not a dynamic variable creation.
The existing answers will work, but they're all essentially re-implementing a function that already exists in the Python standard library: operator.itemgetter()
From the docs:
Return a callable object that fetches item from its operand using the operand’s __getitem__() method. If multiple items are specified, returns a tuple of lookup values. For example:
After f = itemgetter(2), the call f(r) returns r[2].
After g = itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3]).
In other words, your destructured dict assignment becomes something like:
from operator import itemgetter
d = {'key_1': 'value_a', 'key_2': 'value_b'}
key_1, key_2 = itemgetter('key_1', 'key_2')(d)
# prints "Key 1: value_a, Key 2: value_b"
print("Key 1: {}, Key 2: {}".format(key_1, key_2))
Problem is that dicts are unordered, so you can't use simple unpacking of d.values(). You could of course first sort the dict by key, then unpack the values:
# Note: in python 3, items() functions as iteritems() did
# in older versions of Python; use it instead
ds = sorted(d.iteritems())
name0, name1, name2..., namen = [v[1] for v in ds]
You could also, at least within an object, do something like:
for k, v in dict.iteritems():
setattr(self, k, v)
Additionally, as I mentioned in the comment above, if you can get all your logic that needs your unpacked dictionary as variables in to a function, you could do:
def func(**kwargs):
# Do stuff with labeled args
func(**d)
A solution which has not been mentionned before would be
dictget = lambda d, *k: [d[i] for i in k]
and then use it:
key_1, key_2 = dictget(d, 'key_1', 'key_2')
whose advantage is that it is quite readable even with more variables to be retrieved.
Even more readable, however, would be a "real" function such as
def dictget(d, *k):
"""Get the values corresponding to the given keys in the provided dict."""
return [d[i] for i in k]
# or maybe
return (d[i] for i in k) # if we suppose that we have bigger sets of result
# or, equivalent to this
for i in k:
yield d[i]
which as well supports commenting with a docstring and is to be preferred.
var1, var2 = (lambda key1, key2: (key1, key2))(**d)
If you want to give anyone reading your code a headache you can use anonymous function to unpack values like this.
You can do this, if you're brave:
for k, v in d.items():
locals()[k] = v
But being brave might not be enough - you might also have to be reckless etc.
If you want to be a reckless hero like #ecatmur, you can do this:
locals().update(d)
But now that OP has updated his question and answered comments, it seems, this isn't what he really wants to do. Just for the record: There are good reasons for dynamically creating variables - even if everyone here agrees that it's not pythonic. A lot of interpreter style problems can be solved neetly with dynamic altering of your scope. Just do this in a controlled fashion. And... uh, don't deploy this to some production site ;)
I actually have a usecase, where i pull all the arguments of an __init__ method into the self namespace on object construction:
vars(self).update(somedict)
The vars function gets you a dict of the “namespace” associated with the object passed. However, this will not work with locals in a function, due to CPython implementation details. So it's not supposed to work on all interpreters.
For global namespace you would substitute vars(self) with globals(), but this is really a sign that something is wrong with your logic. As said, for local variables this won't work anyways (It'll NameError even if you assigned a value to the dict).
i think this should solve your problem
d = {'key_1': 'value_a', 'key_2': 'value_b'}
for k,v in d.items():
exec '%s=%s'%(k,v)
#glglgl's answer is not most voted, that answer worked for me,
solution1={'variable': np.array([75, 74]), 'function': 0}
def dict_get(d, *k):
for i in k:
yield d[i]
variables, obj_func = dict_get(solution1, 'variable', 'function')
a, b=variables
print(a)
reference: #glglgl
It's not recommended to ever declare variables dynamically, because it becomes very difficult to find the source of a variable name. That said, it's possible to hack together a solution Dynamic variable name in python but I wouldn't recommend it. Probably better off using a plain old dictionary.
Here is a solution that uses Python's inspect module on the calling stack frame to determine how to extract the right values from a supplied dictionary. Right now the only check is to make sure that there is a value available for each of the output variables, but feel free to add additional checks if you need them.
import inspect
def update(src):
result = []
cprev = inspect.currentframe().f_back
#print(cprev.f_code.co_names)
for v in cprev.f_code.co_names[1:]:
if src is cprev.f_locals[v]: continue
if v in src:
result.append(src[v])
else:
raise NameError(v + " has no source for update")
return result
Usage looks like this:
src={'a': 1, 'b': 2, 'c': 3}
a,b,c = update(src)
Related
What is the best way to replace the first character of all keys in a dictionary?
old_cols= {"~desc1":"adjustment1","~desc23":"adjustment3"}
I am trying to get
new_cols= {"desc1":"adjustment1","desc23":"adjustment3"}
I have tried:
for k,v in old_cols.items():
new_cols[k]=old_cols.pop(k[1:])
old_cols = {"~desc1":"adjustment1", "~desc23":"adjustment3"}
new_cols = {}
for k, v in old_cols.items():
new_key = k[1:]
new_cols[new_key] = v
Here it is with a dictionary comprehension:
old_cols= {"~desc1":"adjustment1","~desc23":"adjustment3"}
new_cols = {k.replace('~', ''):old_cols[k] for k in old_cols}
print(new_cols)
#{'desc1': 'adjustment1', 'desc23': 'adjustment3'}
There are many ways to do this with list-comprehension or for-loops. What is important is to understand is that dictionaries are mutable. This basically means that you can either modify the existing dictionary or create a new one.
If you want to create a new one (and I would recommend it! - see 'A word of warning ...' below), both solutions provided in the answers by #Ethem_Turgut and by #pakpe do the trick. I would probably wirte:
old_dict = {"~desc1":"adjustment1","~desc23":"adjustment3"}
# get the id of this dict, just for comparison later.
old_id = id(old_dict)
new_dict = {k[1:]: v for k, v in old_dict.items()}
print(f'Is the result still the same dictionary? Answer: {old_id == id(new_dict)}')
Now, if you want to modify the dictionary in place you might loop over the keys and adapt the key/value to your liking:
old_dict = {"~desc1":"adjustment1","~desc23":"adjustment3"}
# get the id of this dict, just for comparison later.
old_id = id(old_dict)
for k in [*old_dict.keys()]: # note the weird syntax here
old_dict[k[1:]] = old_dict.pop(k)
print(f'Is the result still the same dictionary? Answer: {old_id == id(old_dict)}')
A word of warning for the latter approach:
You should be aware that you are modifying the keys while looping over them. This is in most of the cases problematic and can even lead to a RuntimeError if you loop directly over old_dict. I avoided this by explicitly unpacking the keys into a list and the looping over that list with [*old_dict.keys()].
Why can modifying keys while looping over them be problematic? Imagine for example that you have the keys '~key1' and 'key1' in your old_dict. Now when your loop handles '~key1' it will modify it to 'key1' which already exists in old_dict and thus it will overwrite the value of 'key1' with the value from '~key1'.
So, only use the latter approach if you are certain to not produce issues like the example mentioned here before. If you are uncertain, simply create a new dictionary!
Let's say we have a Python dictionary d, and we're iterating over it like so:
for k, v in d.iteritems():
del d[f(k)] # remove some item
d[g(k)] = v # add a new item
(f and g are just some black-box transformations.)
In other words, we try to add/remove items to d while iterating over it using iteritems.
Is this well defined? Could you provide some references to support your answer?
See also How to avoid "RuntimeError: dictionary changed size during iteration" error? for the separate question of how to avoid the problem.
Alex Martelli weighs in on this here.
It may not be safe to change the container (e.g. dict) while looping over the container.
So del d[f(k)] may not be safe. As you know, the workaround is to use d.copy().items() (to loop over an independent copy of the container) instead of d.iteritems() or d.items() (which use the same underlying container).
It is okay to modify the value at an existing index of the dict, but inserting values at new indices (e.g. d[g(k)] = v) may not work.
It is explicitly mentioned on the Python doc page (for Python 2.7) that
Using iteritems() while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries.
Similarly for Python 3.
The same holds for iter(d), d.iterkeys() and d.itervalues(), and I'll go as far as saying that it does for for k, v in d.items(): (I can't remember exactly what for does, but I would not be surprised if the implementation called iter(d)).
You cannot do that, at least with d.iteritems(). I tried it, and Python fails with
RuntimeError: dictionary changed size during iteration
If you instead use d.items(), then it works.
In Python 3, d.items() is a view into the dictionary, like d.iteritems() in Python 2. To do this in Python 3, instead use d.copy().items(). This will similarly allow us to iterate over a copy of the dictionary in order to avoid modifying the data structure we are iterating over.
I have a large dictionary containing Numpy arrays, so the dict.copy().keys() thing suggested by #murgatroid99 was not feasible (though it worked). Instead, I just converted the keys_view to a list and it worked fine (in Python 3.4):
for item in list(dict_d.keys()):
temp = dict_d.pop(item)
dict_d['some_key'] = 1 # Some value
I realize this doesn't dive into the philosophical realm of Python's inner workings like the answers above, but it does provide a practical solution to the stated problem.
The following code shows that this is not well defined:
def f(x):
return x
def g(x):
return x+1
def h(x):
return x+10
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[g(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[h(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
The first example calls g(k), and throws an exception (dictionary changed size during iteration).
The second example calls h(k) and throws no exception, but outputs:
{21: 'axx', 22: 'bxx', 23: 'cxx'}
Which, looking at the code, seems wrong - I would have expected something like:
{11: 'ax', 12: 'bx', 13: 'cx'}
Python 3 you should just:
prefix = 'item_'
t = {'f1': 'ffw', 'f2': 'fca'}
t2 = dict()
for k,v in t.items():
t2[k] = prefix + v
or use:
t2 = t1.copy()
You should never modify original dictionary, it leads to confusion as well as potential bugs or RunTimeErrors. Unless you just append to the dictionary with new key names.
This question asks about using an iterator (and funny enough, that Python 2 .iteritems iterator is no longer supported in Python 3) to delete or add items, and it must have a No as its only right answer as you can find it in the accepted answer. Yet: most of the searchers try to find a solution, they will not care how this is done technically, be it an iterator or a recursion, and there is a solution for the problem:
You cannot loop-change a dict without using an additional (recursive) function.
This question should therefore be linked to a question that has a working solution:
How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete")
Also helpful as it shows how to change the items of a dict on the run: How can I replace a key:value pair by its value wherever the chosen key occurs in a deeply nested dictionary? (= "replace").
By the same recursive methods, you will also able to add items as the question asks for as well.
Since my request to link this question was declined, here is a copy of the solution that can delete items from a dict. See How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete") for examples / credits / notes.
import copy
def find_remove(this_dict, target_key, bln_overwrite_dict=False):
if not bln_overwrite_dict:
this_dict = copy.deepcopy(this_dict)
for key in this_dict:
# if the current value is a dict, dive into it
if isinstance(this_dict[key], dict):
if target_key in this_dict[key]:
this_dict[key].pop(target_key)
this_dict[key] = find_remove(this_dict[key], target_key)
return this_dict
dict_nested_new = find_remove(nested_dict, "sub_key2a")
The trick
The trick is to find out in advance whether a target_key is among the next children (= this_dict[key] = the values of the current dict iteration) before you reach the child level recursively. Only then you can still delete a key:value pair of the child level while iterating over a dictionary. Once you have reached the same level as the key to be deleted and then try to delete it from there, you would get the error:
RuntimeError: dictionary changed size during iteration
The recursive solution makes any change only on the next values' sub-level and therefore avoids the error.
I got the same problem and I used following procedure to solve this issue.
Python List can be iterate even if you modify during iterating over it.
so for following code it will print 1's infinitely.
for i in list:
list.append(1)
print 1
So using list and dict collaboratively you can solve this problem.
d_list=[]
d_dict = {}
for k in d_list:
if d_dict[k] is not -1:
d_dict[f(k)] = -1 # rather than deleting it mark it with -1 or other value to specify that it will be not considered further(deleted)
d_dict[g(k)] = v # add a new item
d_list.append(g(k))
Today I had a similar use-case, but instead of simply materializing the keys on the dictionary at the beginning of the loop, I wanted changes to the dict to affect the iteration of the dict, which was an ordered dict.
I ended up building the following routine, which can also be found in jaraco.itertools:
def _mutable_iter(dict):
"""
Iterate over items in the dict, yielding the first one, but allowing
it to be mutated during the process.
>>> d = dict(a=1)
>>> it = _mutable_iter(d)
>>> next(it)
('a', 1)
>>> d
{}
>>> d.update(b=2)
>>> list(it)
[('b', 2)]
"""
while dict:
prev_key = next(iter(dict))
yield prev_key, dict.pop(prev_key)
The docstring illustrates the usage. This function could be used in place of d.iteritems() above to have the desired effect.
Let's say we have a Python dictionary d, and we're iterating over it like so:
for k, v in d.iteritems():
del d[f(k)] # remove some item
d[g(k)] = v # add a new item
(f and g are just some black-box transformations.)
In other words, we try to add/remove items to d while iterating over it using iteritems.
Is this well defined? Could you provide some references to support your answer?
See also How to avoid "RuntimeError: dictionary changed size during iteration" error? for the separate question of how to avoid the problem.
Alex Martelli weighs in on this here.
It may not be safe to change the container (e.g. dict) while looping over the container.
So del d[f(k)] may not be safe. As you know, the workaround is to use d.copy().items() (to loop over an independent copy of the container) instead of d.iteritems() or d.items() (which use the same underlying container).
It is okay to modify the value at an existing index of the dict, but inserting values at new indices (e.g. d[g(k)] = v) may not work.
It is explicitly mentioned on the Python doc page (for Python 2.7) that
Using iteritems() while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries.
Similarly for Python 3.
The same holds for iter(d), d.iterkeys() and d.itervalues(), and I'll go as far as saying that it does for for k, v in d.items(): (I can't remember exactly what for does, but I would not be surprised if the implementation called iter(d)).
You cannot do that, at least with d.iteritems(). I tried it, and Python fails with
RuntimeError: dictionary changed size during iteration
If you instead use d.items(), then it works.
In Python 3, d.items() is a view into the dictionary, like d.iteritems() in Python 2. To do this in Python 3, instead use d.copy().items(). This will similarly allow us to iterate over a copy of the dictionary in order to avoid modifying the data structure we are iterating over.
I have a large dictionary containing Numpy arrays, so the dict.copy().keys() thing suggested by #murgatroid99 was not feasible (though it worked). Instead, I just converted the keys_view to a list and it worked fine (in Python 3.4):
for item in list(dict_d.keys()):
temp = dict_d.pop(item)
dict_d['some_key'] = 1 # Some value
I realize this doesn't dive into the philosophical realm of Python's inner workings like the answers above, but it does provide a practical solution to the stated problem.
The following code shows that this is not well defined:
def f(x):
return x
def g(x):
return x+1
def h(x):
return x+10
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[g(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
try:
d = {1:"a", 2:"b", 3:"c"}
for k, v in d.iteritems():
del d[f(k)]
d[h(k)] = v+"x"
print d
except Exception as e:
print "Exception:", e
The first example calls g(k), and throws an exception (dictionary changed size during iteration).
The second example calls h(k) and throws no exception, but outputs:
{21: 'axx', 22: 'bxx', 23: 'cxx'}
Which, looking at the code, seems wrong - I would have expected something like:
{11: 'ax', 12: 'bx', 13: 'cx'}
Python 3 you should just:
prefix = 'item_'
t = {'f1': 'ffw', 'f2': 'fca'}
t2 = dict()
for k,v in t.items():
t2[k] = prefix + v
or use:
t2 = t1.copy()
You should never modify original dictionary, it leads to confusion as well as potential bugs or RunTimeErrors. Unless you just append to the dictionary with new key names.
This question asks about using an iterator (and funny enough, that Python 2 .iteritems iterator is no longer supported in Python 3) to delete or add items, and it must have a No as its only right answer as you can find it in the accepted answer. Yet: most of the searchers try to find a solution, they will not care how this is done technically, be it an iterator or a recursion, and there is a solution for the problem:
You cannot loop-change a dict without using an additional (recursive) function.
This question should therefore be linked to a question that has a working solution:
How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete")
Also helpful as it shows how to change the items of a dict on the run: How can I replace a key:value pair by its value wherever the chosen key occurs in a deeply nested dictionary? (= "replace").
By the same recursive methods, you will also able to add items as the question asks for as well.
Since my request to link this question was declined, here is a copy of the solution that can delete items from a dict. See How can I remove a key:value pair wherever the chosen key occurs in a deeply nested dictionary? (= "delete") for examples / credits / notes.
import copy
def find_remove(this_dict, target_key, bln_overwrite_dict=False):
if not bln_overwrite_dict:
this_dict = copy.deepcopy(this_dict)
for key in this_dict:
# if the current value is a dict, dive into it
if isinstance(this_dict[key], dict):
if target_key in this_dict[key]:
this_dict[key].pop(target_key)
this_dict[key] = find_remove(this_dict[key], target_key)
return this_dict
dict_nested_new = find_remove(nested_dict, "sub_key2a")
The trick
The trick is to find out in advance whether a target_key is among the next children (= this_dict[key] = the values of the current dict iteration) before you reach the child level recursively. Only then you can still delete a key:value pair of the child level while iterating over a dictionary. Once you have reached the same level as the key to be deleted and then try to delete it from there, you would get the error:
RuntimeError: dictionary changed size during iteration
The recursive solution makes any change only on the next values' sub-level and therefore avoids the error.
I got the same problem and I used following procedure to solve this issue.
Python List can be iterate even if you modify during iterating over it.
so for following code it will print 1's infinitely.
for i in list:
list.append(1)
print 1
So using list and dict collaboratively you can solve this problem.
d_list=[]
d_dict = {}
for k in d_list:
if d_dict[k] is not -1:
d_dict[f(k)] = -1 # rather than deleting it mark it with -1 or other value to specify that it will be not considered further(deleted)
d_dict[g(k)] = v # add a new item
d_list.append(g(k))
Today I had a similar use-case, but instead of simply materializing the keys on the dictionary at the beginning of the loop, I wanted changes to the dict to affect the iteration of the dict, which was an ordered dict.
I ended up building the following routine, which can also be found in jaraco.itertools:
def _mutable_iter(dict):
"""
Iterate over items in the dict, yielding the first one, but allowing
it to be mutated during the process.
>>> d = dict(a=1)
>>> it = _mutable_iter(d)
>>> next(it)
('a', 1)
>>> d
{}
>>> d.update(b=2)
>>> list(it)
[('b', 2)]
"""
while dict:
prev_key = next(iter(dict))
yield prev_key, dict.pop(prev_key)
The docstring illustrates the usage. This function could be used in place of d.iteritems() above to have the desired effect.
Hi want to understand how to make this code shorter using dictionary comprehension:
for e in list_of_tuples:
tmp = mydict.copy()
tmp[e[0]] = tmp[e[1]]
if someFunction(tmp):
mydict = tmp
I would like to pass a dictionary comprehension to someFunction instead of relying on a temporary dictionary whose values are changed in the loop. Is it possible?
This answer assumes that someFunction does not alter the dictionary
The dictionary passed to someFunction is still going to be a basic copy of mydict, but this is the only way I can think of answering the question with comprehension.
for e in list_of_tuples:
if someFunction({key: val if key != e[0] else mydict[e[1]] for key,val in mydict }):
mydict[e[0]] = mydict[e[1]]
However the faster/ easier way would be to just make a temp variable for mydict[e[0]], and change it back after if someFunction fails. Also having extra lines isn't always a bad thing. It can usually help readability, solving bugs and maintenance.. especially for newer programmers.
for e in list_of_tuples:
temp = mydict[e[0]]
mydict[e[0]] = mydict[e[1]]
if not someFunction(mydict):
mydict[e[0]] = temp
What is the most pythonic way to set a value in a dict if the value is not already set?
At the moment my code uses if statements:
if "timeout" not in connection_settings:
connection_settings["timeout"] = compute_default_timeout(connection_settings)
dict.get(key,default) is appropriate for code consuming a dict, not for code that is preparing a dict to be passed to another function. You can use it to set something but its no prettier imo:
connection_settings["timeout"] = connection_settings.get("timeout", \
compute_default_timeout(connection_settings))
would evaluate the compute function even if the dict contained the key; bug.
Defaultdict is when default values are the same.
Of course there are many times you set primative values that don't need computing as defaults, and they can of course use dict.setdefault. But how about the more complex cases?
dict.setdefault will precisely "set a value in a dict only if the value is not already set".
You still need to compute the value to pass it in as the parameter:
connection_settings.setdefault("timeout", compute_default_timeout(connection_settings))
This is a bit of a non-answer, but I would say the most pythonic is the if statement as you have it. You resisted the urge to one-liner it with __setitem__ or other methods. You've avoided possible bugs in the logic due to existing-but-falsey values which might happen when trying to be clever with short-circuiting and/or hacks. It's immediately obvious that the compute function isn't used when it wasn't necessary.
It's clear, concise, and readable - pythonic.
One way to do this is:
if key not in dict:
dict[key] = value
Since Python 3.9 you can use the merge operator | to merge two dictionaries. The dict on the right takes precedence:
d = { key: value } | d
Note: this creates a new dictionary with the updated values.
You probably need dict.setdefault:
Create a new dictionary and set a value:
>>> d = {}
>>> d.setdefault('timeout', 120)
120
>>> d
{'timeout': 120}
If a value already set, dict.setdefault won't override it:
>>> d['port']=8080
>>> d.setdefault('port', 8888)
8080
>>> d
{'port': 8080, 'timeout': 120}
I'm using the following to modify kwargs to non-default values and pass to another function:
def f( **non_default_kwargs ):
kwargs = {
'a':1,
'b':2,
}
kwargs.update( non_default_kwargs )
f2( **kwargs )
This has the merits that
you don't have to type the keys twice
all is done in a single function
The answer by #Rotareti makes me wonder if for older version of Python then 3.9, we can do:
>>> dict_a = {'a': 1 }
>>> dict_a = {'a': 3, 'b': 2, **dict_a}
>>> dict_a
{'a': 1, 'b': 2}
(Well, it works for sure on Python3.7, but is this Pythonesque enough?)
I found it convenient and obvious to exploit the return of the dict .get() method being None (Falsy), along with or to put off evaluation of an expensive network request if the key was not present.
d = dict()
def fetch_and_set(d, key):
d[key] = ("expensive operation to fetch key")
if not d[key]:
raise Exception("could not get value")
return d[key]
...
value = d.get(key) or fetch_and_set(d, key)
In my case specifically, I was building a new dictionary from a cache then later updating the cache after expediting the fn() call.
Here's a simplified view of my use
j = load(database) # dict
d = dict()
# see if desired keys are in the cache, else fetch
for key in keys:
d[key] = j.get(key) or fetch(key, network_token)
fn(d) # use d for something useful
j.update(d) # update database with new values (if any)