Using dictionary comprehension is it possible to convert all values recursively to string?
I have this dictionary
d = {
"root": {
"a": "1",
"b": 2,
"c": 3,
"d": 4
}
}
I tried
{k: str(v) for k, v in d.items()}
But the code above turns the entire root value into string and I want this:
d = {"root": {"a": "1", "b": "2", "c": "3", "d": "4"}}
This is not a dictionary comprehension, but it works, it's just one line, and it's recursive!
(f := lambda d: {k: f(v) for k, v in d.items()} if type(d) == dict else str(d))(d)
It only works with Python 3.8+ though (because of the use of an assignment expression).
You could do a recursive solution for arbitrarily nested dicts, but if you only have 2 levels the following is sufficient:
{k: {k2: str(v2) for k2, v2 in v.items()} for k, v in d.items()}
Assuming that your given input was wrong and root's value was a dictionary, your code would somewhat work. You just need to add d['root'].items()
newDict = {k:{k: str(v) for k, v in d[k].items()} for k,v in d.items()}
output
{'root': {'a': '1', 'b': '2', 'c': '3', 'd': '4'}}
The following solution might not be using dictionary comprehension, but it is recursive and can transform dictionaries of any depth, I don't think that's possible using comprehension alone:
def convert_to_string(d):
for key, value in d.items():
if isinstance(value, dict):
convert_to_string(value)
else:
d[key] = str(value)
Found a simpler way to to achieve this using json module. Just made the following
import json
string_json = json.dumps(d) # Convert to json string
d = json.loads(string_json, parse_int=str) # This convert the `int` to `str` recursively.
Using a function
def dictionary_string(dictionary: dict) -> dict:
return json.loads(json.dumps(dictionary), parse_int=str, parse_float=str)
Regards
Related
Given a dictionary { k1: v1, k2: v2 ... } I want to get { k1: f(v1), k2: f(v2) ... } provided I pass a function f.
Is there any such built in function? Or do I have to do
dict([(k, f(v)) for (k, v) in my_dictionary.iteritems()])
Ideally I would just write
my_dictionary.map_values(f)
or
my_dictionary.mutate_values_with(f)
That is, it doesn't matter to me if the original dictionary is mutated or a copy is created.
There is no such function; the easiest way to do this is to use a dict comprehension:
my_dictionary = {k: f(v) for k, v in my_dictionary.items()}
In python 2.7, use the .iteritems() method instead of .items() to save memory. The dict comprehension syntax wasn't introduced until python 2.7.
Note that there is no such method on lists either; you'd have to use a list comprehension or the map() function.
As such, you could use the map() function for processing your dict as well:
my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1])), my_dictionary.iteritems()))
but that's not that readable, really.
These toolz are great for this kind of simple yet repetitive logic.
http://toolz.readthedocs.org/en/latest/api.html#toolz.dicttoolz.valmap
Gets you right where you want to be.
import toolz
def f(x):
return x+1
toolz.valmap(f, my_list)
Due to PEP-0469 which renamed iteritems() to items() and PEP-3113 which removed Tuple parameter unpacking, in Python 3.x you should write Martijn Pieters♦ answer like this:
my_dictionary = dict(map(lambda item: (item[0], f(item[1])), my_dictionary.items()))
You can do this in-place, rather than create a new dict, which may be preferable for large dictionaries (if you do not need a copy).
def mutate_dict(f,d):
for k, v in d.iteritems():
d[k] = f(v)
my_dictionary = {'a':1, 'b':2}
mutate_dict(lambda x: x+1, my_dictionary)
results in my_dictionary containing:
{'a': 2, 'b': 3}
While my original answer missed the point (by trying to solve this problem with the solution to Accessing key in factory of defaultdict), I have reworked it to propose an actual solution to the present question.
Here it is:
class walkableDict(dict):
def walk(self, callback):
try:
for key in self:
self[key] = callback(self[key])
except TypeError:
return False
return True
Usage:
>>> d = walkableDict({ k1: v1, k2: v2 ... })
>>> d.walk(f)
The idea is to subclass the original dict to give it the desired functionality: "mapping" a function over all the values.
The plus point is that this dictionary can be used to store the original data as if it was a dict, while transforming any data on request with a callback.
Of course, feel free to name the class and the function the way you want (the name chosen in this answer is inspired by PHP's array_walk() function).
Note: Neither the try-except block nor the return statements are mandatory for the functionality, they are there to further mimic the behavior of the PHP's array_walk.
To avoid doing indexing from inside lambda, like:
rval = dict(map(lambda kv : (kv[0], ' '.join(kv[1])), rval.iteritems()))
You can also do:
rval = dict(map(lambda(k,v) : (k, ' '.join(v)), rval.iteritems()))
Just came accross this use case. I implemented gens's answer, adding a recursive approach for handling values that are also dicts:
def mutate_dict_in_place(f, d):
for k, v in d.iteritems():
if isinstance(v, dict):
mutate_dict_in_place(f, v)
else:
d[k] = f(v)
# Exemple handy usage
def utf8_everywhere(d):
mutate_dict_in_place((
lambda value:
value.decode('utf-8')
if isinstance(value, bytes)
else value
),
d
)
my_dict = {'a': b'byte1', 'b': {'c': b'byte2', 'd': b'byte3'}}
utf8_everywhere(my_dict)
print(my_dict)
This can be useful when dealing with json or yaml files that encode strings as bytes in Python 2
My way to map over dictionary
def f(x): return x+2
bill = {"Alice": 20, "Bob": 10}
d = {map(lambda x: f(x),bill.values())}
print('d: ',dict(d))
Results
: d: {22: 12}
Map over iterable in values within dictionary
bills = {"Alice": [20, 15, 30], "Bob": [10, 35]}
d= {map(lambda v: sum(v),bills.values())}
g= dict(map(lambda v: (v[0],sum(v[1])),bills.items()))
# prints
print('d: ',dict(d))
print('g: ',g)
Results
d: {65: 45}
g: {'Alice': 65, 'Bob': 45}
Is there a convenient way to map a function to specified keys in a dictionary?
Ie, given
d = {"a": 1, "b": 2, "c": 3}
would like to map a function, say f, to keys "a" and "c":
{"a": f(1), "b": 2, "c": f(3)}
EDIT
Looking for methods that will not update the input dictionary.
You can use a dictionary comprehension:
output_dict = {k: f(v) for k, v in d.items()}
Note that f(v) will be evaluated (called) immediately and its return values will be stored as the dictionary's values.
If you want to store the function and call it later (with the arguments already stored) you can use functools.partial:
from functools import partial
def f(n):
print(n * 2)
d = {"a": 1, "b": 2, "c": 3}
output_dict = {k: partial(f, v) for k, v in d.items()}
output_dict['b']()
# 4
If you only want specific keys mapped you can of course not use .items and just override those keys:
d['a'] = partial(f, d['a'])
or more generalized
keys = ('a', 'c')
for key in keys:
d[key] = partial(f, d[key])
I have a string that could be parsed as a JSON or dict object. My string variable looks like this :
my_string_variable = """{
"a":1,
"b":{
"b1":1,
"b2":2
},
"b": {
"b1":3,
"b2":2,
"b4":8
}
}"""
When I do json.loads(my_string_variable), I have a dict but only the second value of the key "b" is kept, which is normal because a dict can't contain duplicate keys.
What would be the best way to have some sort of defaultdict like this :
result = {
"a": 1,
"b": [{"b1": 1, "b2": 2}, {"b1": 3, "b2": 2, "b4": 8}],
}
I have already looked for similar questions but they all deal with dicts or lists as an input and then create defaultdicts to handle the duplicate keys.
In my case I have a string variable and I would want to know if there is a simple way to achieve this.
something like the following can be done.
import json
def join_duplicate_keys(ordered_pairs):
d = {}
for k, v in ordered_pairs:
if k in d:
if type(d[k]) == list:
d[k].append(v)
else:
newlist = []
newlist.append(d[k])
newlist.append(v)
d[k] = newlist
else:
d[k] = v
return d
raw_post_data = '{"a":1, "b":{"b1":1,"b2":2}, "b": { "b1":3, "b2":2,"b4":8} }'
newdict = json.loads(raw_post_data, object_pairs_hook=join_duplicate_keys)
print (newdict)
Please note that above code depends on value type, if type(d[k]) == list. So if original string itself gives a list then there could be some error handling required to make the code robust.
Accepted answer is perfectly fine. I just wanted to show another approach.
So at first, you dedicate a list for values in order to easily accumulate next values. At the end, you call pop on the lists which have only one item. This means that the list doesn't have duplicate values:
import json
from collections import defaultdict
my_string_variable = '{"a":1, "b":{"b1":1,"b2":2}, "b": { "b1":3, "b2":2,"b4":8} }'
def join_duplicate_keys(ordered_pairs):
d = defaultdict(list)
for k, v in ordered_pairs:
d[k].append(v)
return {k: v.pop() if len(v) == 1 else v for k, v in d.items()}
d = json.loads(my_string_variable, object_pairs_hook=join_duplicate_keys)
print(d)
output:
{'a': 1, 'b': [{'b1': 1, 'b2': 2}, {'b1': 3, 'b2': 2, 'b4': 8}]}
I have a dictionary where I have the data already inside, i.e. keys have values and some of them have more than one value.
For example:
i = {"a": "111", "b": "222", "c": ["333", "444"]}
How can I change the type of the multiple values? I want them to be sets, not lists, such as:
i = {"a": {"111"}, "b": {"222"}, "c": {"333", "444"}}
One similar post is this one:
How to add multiple values to a dictionary key in python? [closed]
There it is explained how to add multiple elements to a dictionary, but they always seem to be lists.
How to change the type of the multiple values?
OR how to add them to the dictionary as sets, not lists?
Using a dict-comprehension makes converting an existing dict very easy:
i = {"a": "111", "b": "222", 'c': ["333", "444"]}
{k: set(v) if isinstance(v, list) else v for k, v in i.items()}
this converts all values that are lists to sets.
In a single line of code:
>>> i = {"a": "111", "b": "222", "c": ["333", "444"]}
>>> {k: set(v) for k, v in i.items()}
{'b': {'2'}, 'a': {'1'}, 'c': {'444', '333'}}
Or with a few more steps:
>>> i = {"a": "111", "b": "222", "c": ["333", "444"]}
>>> for k, v in i.items():
... i[k] = set(v)
>>> i
{'b': {'2'}, 'a': {'1'}, 'c': {'444', '333'}}
Instead of doing
my_dict['key'] = ['333', '444']
use a set literal:
my_dict['key'] = {'333', '444'}
That looks like a dict literal, but the lack of key: value like things makes it a set.
Given a dictionary { k1: v1, k2: v2 ... } I want to get { k1: f(v1), k2: f(v2) ... } provided I pass a function f.
Is there any such built in function? Or do I have to do
dict([(k, f(v)) for (k, v) in my_dictionary.iteritems()])
Ideally I would just write
my_dictionary.map_values(f)
or
my_dictionary.mutate_values_with(f)
That is, it doesn't matter to me if the original dictionary is mutated or a copy is created.
There is no such function; the easiest way to do this is to use a dict comprehension:
my_dictionary = {k: f(v) for k, v in my_dictionary.items()}
In python 2.7, use the .iteritems() method instead of .items() to save memory. The dict comprehension syntax wasn't introduced until python 2.7.
Note that there is no such method on lists either; you'd have to use a list comprehension or the map() function.
As such, you could use the map() function for processing your dict as well:
my_dictionary = dict(map(lambda kv: (kv[0], f(kv[1])), my_dictionary.iteritems()))
but that's not that readable, really.
These toolz are great for this kind of simple yet repetitive logic.
http://toolz.readthedocs.org/en/latest/api.html#toolz.dicttoolz.valmap
Gets you right where you want to be.
import toolz
def f(x):
return x+1
toolz.valmap(f, my_list)
Due to PEP-0469 which renamed iteritems() to items() and PEP-3113 which removed Tuple parameter unpacking, in Python 3.x you should write Martijn Pieters♦ answer like this:
my_dictionary = dict(map(lambda item: (item[0], f(item[1])), my_dictionary.items()))
You can do this in-place, rather than create a new dict, which may be preferable for large dictionaries (if you do not need a copy).
def mutate_dict(f,d):
for k, v in d.iteritems():
d[k] = f(v)
my_dictionary = {'a':1, 'b':2}
mutate_dict(lambda x: x+1, my_dictionary)
results in my_dictionary containing:
{'a': 2, 'b': 3}
While my original answer missed the point (by trying to solve this problem with the solution to Accessing key in factory of defaultdict), I have reworked it to propose an actual solution to the present question.
Here it is:
class walkableDict(dict):
def walk(self, callback):
try:
for key in self:
self[key] = callback(self[key])
except TypeError:
return False
return True
Usage:
>>> d = walkableDict({ k1: v1, k2: v2 ... })
>>> d.walk(f)
The idea is to subclass the original dict to give it the desired functionality: "mapping" a function over all the values.
The plus point is that this dictionary can be used to store the original data as if it was a dict, while transforming any data on request with a callback.
Of course, feel free to name the class and the function the way you want (the name chosen in this answer is inspired by PHP's array_walk() function).
Note: Neither the try-except block nor the return statements are mandatory for the functionality, they are there to further mimic the behavior of the PHP's array_walk.
To avoid doing indexing from inside lambda, like:
rval = dict(map(lambda kv : (kv[0], ' '.join(kv[1])), rval.iteritems()))
You can also do:
rval = dict(map(lambda(k,v) : (k, ' '.join(v)), rval.iteritems()))
Just came accross this use case. I implemented gens's answer, adding a recursive approach for handling values that are also dicts:
def mutate_dict_in_place(f, d):
for k, v in d.iteritems():
if isinstance(v, dict):
mutate_dict_in_place(f, v)
else:
d[k] = f(v)
# Exemple handy usage
def utf8_everywhere(d):
mutate_dict_in_place((
lambda value:
value.decode('utf-8')
if isinstance(value, bytes)
else value
),
d
)
my_dict = {'a': b'byte1', 'b': {'c': b'byte2', 'd': b'byte3'}}
utf8_everywhere(my_dict)
print(my_dict)
This can be useful when dealing with json or yaml files that encode strings as bytes in Python 2
My way to map over dictionary
def f(x): return x+2
bill = {"Alice": 20, "Bob": 10}
d = {map(lambda x: f(x),bill.values())}
print('d: ',dict(d))
Results
: d: {22: 12}
Map over iterable in values within dictionary
bills = {"Alice": [20, 15, 30], "Bob": [10, 35]}
d= {map(lambda v: sum(v),bills.values())}
g= dict(map(lambda v: (v[0],sum(v[1])),bills.items()))
# prints
print('d: ',dict(d))
print('g: ',g)
Results
d: {65: 45}
g: {'Alice': 65, 'Bob': 45}