Develop Reference

How can I merge nested lists to output the key once and all values with that key "below"?

thing = [{item, abc}, {item, def}, {item, ghi}, {item2, jkl}, {item2, mno}, {item2, pqr}, ...]
to output: item abc def ghi ... item2 jkl mno ...
I've seen similar questions but none that would help me. I appreciate any help I can get!

A solution like this could work:
thing = [{"item", "abc"}, {"item", "def"}, {"item", "ghi"}, {"item2", "jkl"}, {"item2", "mno"}, {"item2", "pqr"}]
newDict = {}
count = 0
key = ""
for s in thing:
count = 0
for e in s:
if count == 0:
if not(e in newDict):
newDict[e] = []
key = e
else:
newDict[key].append(e)
count += 1
print(newDict)
Output:
{'item': ['abc', 'def', 'ghi'], 'item2': ['jkl', 'mno', 'pqr']}
Since you are using a list of sets, the solution is a bit more difficult than if you used a list of lists, since sets are unindexed.
However, we can still access the elements we order them in by iterating through them.
First, we create our newDict, a dictionary will store our output. For every set, s, in thing, we will iterate through every element, e, in s.
If we are currently iterating the first element, where count = 0, if e (which can be only either "item" or "item2"), is not a key in newDict, we will create a new key with an empty list as its value. Then, we will set our variable key equal to e, so that in the next iteration, we can append the value to that dictionary.
Solution if thing is a list of lists:
thing = [["item", "abc"], ["item", "def"], ["item", "ghi"],["item2", "jkl"], ["item2", "mno"], ["item2", "pqr"]]
newDict = {}
for s in thing:
if not(s[0] in newDict):
newDict[s[0]] = []
newDict[s[0]].append(s[1])
print(newDict)
I hope this made sense! Please let me know if you have any further questions or clarifications :)

I want to create a function that takes a text string and returns a dictionary containing certain characters with how many tmies they occur

This is what I have so far as a function
example = "Sample String"
def func(text, let):
count= {}
for let in text.lower():
let = count.keys()
if let in text:
count[let] += 1
else:
count[let] = 1
return count
I want to return something like this
print(func(example, "sao"))
{'s': 2, 'a' : 1}
I am not very sure what I could improve on

I would use Counter from the collections built-in module:
from collections import Counter
def func(text, let):
c = Counter(text.lower())
return {l: c[l] for l in let if l in c.keys()}
Breaking it down:
Counter will return the count of letters in your string:
In [5]: Counter(example.lower())
Out[5]:
Counter({'s': 2,
'a': 1,
'm': 1,
'p': 1,
'l': 1,
'e': 1,
' ': 1,
't': 1,
'r': 1,
'i': 1,
'n': 1,
'g': 1})
So then all you need to do is return a dictionary of the appropriate letters, which can be done in a dictionary comprehension:
# iterate over every letter in `let`, and get the Counter value for that letter,
# if that letter is in the Counter keys
{l: c[l] for l in let if l in c.keys()}
Fixing your code
If you prefer to use your approach, you could make your code work properly with this:
def func(text, let):
count = {}
for l in text.lower():
if l in let:
if l in count.keys():
count[l] += 1
else:
count[l] = 1
return count

from functools import reduce
def count(text, letters):
return reduce(
lambda d, letr: d.update({letr: d.get(letr, 0) + 1}) or d,
filter(lambda l: l in letters, text), {}
)
Read it backwards.
Creates an empty dictionary.
{}
Filters letters from text.
lambda l: l in letters
This lambda function returns true if l is in letters
filter(lambda l: l in letters, text)
reduce will iterate over the object returned by filter, which will
only produce letters in text, if they are in letters.
lambda d, letr: d.update({letr: d.get(letr, 0) + 1}) or d
Updates the dictionary with the count of the letters it encounters.
Each time reduce iterates over an item generated by the filter object,
it will call this lambda function. Since dict.update() -> None, returns None, which evaluates to false, we say or d to actually return the dict back to reduce, which will pass the dict back into the lambda the next time it gets called, thus building up the counts. We also use dict.get() in the lambda instead of d[i], this allows us to pass the default of 0 if the letter is not yet in the dictionary.
At the end reduce returns the dict, and we return that from count.
This is similar to how "map reduce" works.
You can read about functional style and lambda expressions in the python docs.

>>> def func(text: str, let: str):
... text, count = text.lower(), {}
... for i in let:
... if text.count(i) != 0:
... count[i] = text.count(i)
... return count
...
>>> print(func("Sample String", "sao"))
{'s': 2, 'a': 1}

Unflatten nested Python dictionary

What would be the cleanest way to convert this
{"a.b.c[0].key1": 1, "a.b.c[1].key2": 2, "a.b.c[3].key3": 3}
Into this
{"a": {"b": {"c": [{"key1": 1}, {"key2": 2}, None, {"key3": 3}]}}}
the dictionary keys may be anything.
the length of the list may vary.
the depth of the dictionary may vary.
if there are missing values in the list the value must be None.
if values are repeated the last one declared is the one that counts.
I came up with the following working example.
Was wondering if we could find a better solution for our community.
def unflatten(data):
if type(data) != dict:
return None
regex = r'\.?([^.\[\]]+)|\[(\d+)\]'
result_holder = {}
for key,value in data.items():
cur = result_holder
prop = ""
results = re.findall(regex, key)
for result in results:
prop = int(prop) if type(cur) == list else prop
if (type(cur) == dict and cur.get(prop)) or (type(cur) == list and len(cur) > prop):
cur = cur[prop]
else:
if type(cur) == list:
if type(prop) is int:
while len(cur) <= prop:
cur.append(None)
cur[prop] = list() if result[1] else dict()
cur = cur[prop]
prop = result[1] or result[0]
prop = int(prop) if type(cur) == list else prop
if type(cur) == list:
if type(prop) is int:
while len(cur) <= prop:
cur.append(None)
print(len(cur), prop)
cur[prop] = data[key]
return result_holder[""] or result_holder

You can use recursion:
d = {"a.b.c[0].key1": 1, "a.b.c[1].key2": 2, "a.b.c[3].key3": 3}
from itertools import groupby
import re
def group_data(data):
new_results = [[a, [i[1:] for i in b]] for a, b in groupby(sorted(data, key=lambda x:x[0]), key=lambda x:x[0])]
arrays = [[a, list(b)] for a, b in groupby(sorted(new_results, key=lambda x:x[0].endswith(']')), key=lambda x:x[0].endswith(']'))]
final_result = {}
for a, b in arrays:
if a:
_chars = [[c, list(d)] for c, d in groupby(sorted(b, key=lambda x:re.findall('^\w+', x[0])[0]), key=lambda x:re.findall('^\w+', x[0])[0])]
_key = _chars[0][0]
final_result[_key] = [[int(re.findall('\d+', c)[0]), d[0]] for c, d in _chars[0][-1]]
_d = dict(final_result[_key])
final_result[_key] = [group_data([_d[i]]) if i in _d else None for i in range(min(_d), max(_d)+1)]
else:
for c, d in b:
final_result[c] = group_data(d) if all(len(i) >1 for i in d) else d[0][0]
return final_result
print(group_data([[*a.split('.'), b] for a, b in d.items()]))
Output:
{'a': {'b': {'c': [{'key1': 1}, {'key2': 2}, None, {'key3': 3}]}}}

A recursive function would probably be much easier to work with and more elegant.
This is partly pseudocode, but it may help you get thinking.
I haven't tested it, but I'm pretty sure it should work so long as you don't have any lists that are directly elements of other lists. So you can have dicts of dicts, dicts of lists, and lists of dicts, but not lists of lists.
def unflatten(data):
resultDict = {}
for e in data:
insertElement(e.split("."), data[e], resultDict)
return resultDict
def insertElement(path, value, subDict):
if (path[0] is of the form "foo[n]"):
key, index = parseListNotation(path[0])
if (key not in subDict):
subDict[key] = []
if (index >= subDict[key].len()):
subDict[key].expandUntilThisSize(index)
if (subDict[key][index] == None):
subDict[key][index] = {}
subDict[key][index] = insertElement(path.pop(0), value, subDict[key][index])
else:
key = path[0]
if (path.length == 1):
subDict[key] = value
else:
if (key not in subDict):
subDict[key] = {}
subDict[key] = insertElement(path.pop(0), value, subDict[key])
return subDict;
The idea is to build the dictionary from the inside, out. E.g.:
For the first element, first create the dictionary `
{key1: 1},
Then assign that to an element of a new dictionary
{c : [None]}, c[0] = {key1: 1}
Then assign that dictionary to the next element b in a new dict, like
- {b: {c : [{key1: 1}]}
Assign that result to a in a new dict
- {a: {b: {c : [{key1: 1}]}}
And lastly return that full dictionary, to use to add the next value.
If you're not familiar with recursive functions, I'd recommend practicing with some simpler ones, and then writing one that does what you want but for input that's only dictionaries.
General path of a dictionary-only recursive function:
Given a path that's a list of attributes of nested dictionaries ( [a, b, c, key1] in your example, if c weren't a list):
Start (path, value):
If there's only item in your path, build a dictionary setting
that key to your value, and you're done.
If there's more than one, build a dictionary using the first
element as a key, and set the value as the output of Start(path.remove(0), value)

Here is another variation on how to achieve the desired results. Not as pretty as I would like though, so I expect there is a much more elegant way. Probably more regex than is really necessary if you spent a bit more time on this, and also seems like the break approach to handling the final key is probably just an indicator that the loop logic could be improved to eliminate that sort of manual intervention. That said, hopefully this is helpful in the process of refining your approach here.
import re
def unflatten(data):
results = {}
list_rgx = re.compile(r'[^\[\]]+\[\d+\]')
idx_rgx = re.compile(r'\d+(?=\])')
key_rgx = re.compile(r'[^\[]+')
for text, value in data.items():
cur = results
keys = text.split('.')
idx = None
for i, key in enumerate(keys):
stop = (i == len(keys) - 1)
if idx is not None:
val = value if stop else {}
if len(cur) > idx:
cur[idx] = {key: val}
else:
for x in range(len(cur), idx + 1):
cur.append({key: val}) if x == idx else cur.append(None)
if stop:
break
else:
cur[idx].get(key)
idx = None
if stop:
cur[key] = value
break
elif re.match(list_rgx, key):
idx = int(re.search(idx_rgx, key).group())
key = re.search(key_rgx, key).group()
cur.setdefault(key, [])
else:
cur.setdefault(key, {})
cur = cur.get(key)
print(results)
Output:
d = {"a.b.c[0].key1": 1, "a.b.c[1].key2": 2, "a.b.c[3].key3": 3}
unflatten(d)
# {'a': {'b': {'c': [{'key1': 1}, {'key2': 2}, None, {'key3': 3}]}}}

Is it possible to turn a list into a nested dict of keys *without* recursion?

Supposing I had a list as follows:
mylist = ['a','b','c','d']
Is it possible to create, from this list, the following dict without using recursion/a recursive function?
{
'a': {
'b': {
'c': {
'd': { }
}
}
}
}

For the simple case, simply iterate and build, either from the end or the start:
result = {}
for name in reversed(mylist):
result = {name: result}
or
result = current = {}
for name in mylist:
current[name] = {}
current = current[name]
The first solution can also be expressed as a one-liner using reduce():
reduce(lambda res, name: {name: res}, reversed(mylist), {})

For this simple case at least, yes:
my_list = ['a', 'b', 'c', 'd']
cursor = built_dict = {}
for value in my_list:
cursor[value] = {}
cursor = cursor[value]

Or for fancyness and reduced readability:
dict = reduce(lambda x, y: {y: x}, reversed(myList), {})

It's worth mentioning that every recursion can be converted into iteration, although sometimes that might not be so easy. For the particular example in the question, it is simple enough, it's just a matter of accumulating the expected result in a variable and traversing the input list in the appropriate order. This is what I mean:
def convert(lst):
acc = {}
for e in reversed(lst):
acc = {e: acc}
return acc
Or even shorter, the above algorithm can be expressed as a one-liner (assuming Python 2.x, in Python 3.x reduce was moved to the functools module). Notice how the variable names in the previous solution correspond to the lambda's parameters, and how in both cases the initial value of the accumulator is {}:
def convert(lst):
return reduce(lambda acc, e: {e: acc}, reversed(lst), {})
Either way, the function convert works as expected:
mylist = ['a','b','c','d']
convert(mylist)
=> {'a': {'b': {'c': {'d': {}}}}}

mydict = dict()
currentDict = mydict
for el in mylist:
currentDict[el] = dict()
currentDict = currentDict[el]

Convert a list into a nested dictionary

For example I have
x = ['a','b','c']
I need to convert it to:
y['a']['b']['c'] = ''
Is that possible?
For the background, I have a config file which contains dotted notation that points to a place in some json data. I'd like to use the dotted notation string to access that specific data in the json file. For example, in the config:
path_to_data = "user.name.first_name"
I'd like my script to recognize that as:
json_data["user"]["name"]["first_name"]
so I can get the value of the first_name field. I converted the original string into a list, and now I don't know how to convert it to a nested dict.
EDIT: There is an existing data structure that I need to apply the dict with. Let's say:
m = {'a': {'b': {'c': 'lolcat'}}}
so that
m['a']['b']['c']
gives me 'lolcat'. If I get the right dictionary structure (as some of the replies did), I would still need to apply this to the existing dictionary 'm'.
So, again, I get this from a config file:
c = 'a.b.c'
That I converted to a list, thinking this will make things easier:
x = ['a','b','c']
Now I have a json-like data structure:
m = {'a': {'b': {'c': 'lolcat'}}}
So the nested dict generated from 'x' should be able to traverse 'm' so that
m['a']['b']['c']
gets me the cat.

li = ['a','b','c']
d = reduce(lambda x, y: {y:x}, reversed(li+['']))
print(d)
print(d['a']['b']['c'])

I guess you also want to include a value in the end. This works for that too:
def get_value(d, l):
if len(l) > 1:
return get_value(d[l[0]], l[1:])
return d[l[0]]
def add_keys(d, l, c=None):
if len(l) > 1:
d[l[0]] = _d = {}
d[l[0]] = d.get(l[0], {})
add_keys(d[l[0]], l[1:], c)
else:
d[l[0]] = c
def main():
d = {}
l1 = ['a', 'b', 'c', 'd']
c1 = 'letters'
l2 = [42, "42", (42,)]
c2 = 42
add_keys(d, l1, c1)
print d
add_keys(d, l2, c2)
print d
if __name__ == '__main__':
main()
It prints:
{'a': {'b': {'c': {'d': 'letters'}}}}
{'a': {'b': {'c': {'d': 'letters'}}}, 42: {'42': {(42,): 42}}}
letters
42
So it surely works. Recursion for the win.

>>> x = ['a','b','c']
>>> y={}
>>> y[x[-1]]=""
>>> x.pop(-1)
'c'
>>> for i in x[::-1]:
... y={i:y}
...
>>> y
{'a': {'b': {'c': ''}}}
>>> y['a']['b']['c']
''

This will work.
#!/usr/bin/python2
from __future__ import print_function
x = ['a','b','c']
def ltod(l):
rv = d = {}
while l:
i = l.pop(0)
d[i] = {}
d = d[i]
return rv
d = ltod(x)
print(d)
print(d["a"]["b"]["c"])
d["a"]["b"]["c"] = "text"
print(d["a"]["b"]["c"])
Outputs:
{'a': {'b': {'c': {}}}}
{}
text

Find below sample that is not very beautiful but quite simple:
path_to_data = "user.name.first_name"
keys = path_to_data.split('.')
t = []
for key in keys[::-1]: # just to iterate in reversed order
if not t:
t.append({k:{}})
else:
t[-1] = ({k: t[-1]})
#t[0] will contain your dictionary

A general solution would be to use collections.defaultdict to create a nested dictionary. Then override __setitem__ for whatever behavior you'd like. This example will do the string parsing as well.
from collections import defaultdict
class nesteddict(defaultdict):
def __init__(self):
defaultdict.__init__(self, nesteddict)
def __setitem__(self, key, value):
keys = key.split('.')
for key in keys[:-1]:
self = self[key]
defaultdict.__setitem__(self, keys[-1], value)
nd = nesteddict()
nd['a.b.c'] = 'lolcat'
assert nd['a']['b']['c'] == 'lolcat'