Develop Reference

How to get flattened list of all inner level nested keys in dictionary - python?

I'm trying to get a list of all keys in the nested level of my dictionary.
My dictionary resembles:
my_dict= {
'DICT':{
'level_1a':{
'level_2a':{}
},
'level_1b': {
'level_2b':{},
'level_2c':{}
}
}
My desired output should resemble:
['level_2a', 'level_2b', 'level_2c']
What I've tried:
[list(v) for k, v in json['DICT'].items()]
My current output:
[['level_2a'], ['level_2b', 'level_2c']]
I want my result to be fully flattened to a single-level list. I've tried flattening libraries but the result tends to appear as: ['level_2a', 'level_2blevel_2c'] which is incorrect. Not looking to make the code more complex by creating another method just to flatten this list.
Would appreciate some help, thank you!

Try:
my_dict = {
"DICT": {
"level_1a": {"level_2a": {}},
"level_1b": {"level_2b": {}, "level_2c": {}},
}
}
lst = [vv for v in my_dict["DICT"].values() for vv in v]
print(lst)
Prints:
['level_2a', 'level_2b', 'level_2c']

How to remove empty or None fields in deeply nested dictionary of unknown depth?

I have a dictionary of deeply nested dictionaries and I am trying to remove all k-v pairs that are None or "". The below dictionary d is an example of the input.
d = {
"1": {
"1_1": 'a real value',
"1_2": ""
},
"2": None,
"3": {
"3_1": {
"3_2": None
}
}
}
Normally, to remove empty fields in a dictionary, the command {k:v for k,v in d.items() if v} does the job. But in this case, I want to remove all the nested dictionaries whose values are empty or null, and I can't figure out how to do this. Any ideas?
After d passes through this transformation, all the empty dictionaries whose values are empty should be gone. It should look like this:
{
"1": {
"1_1": 'a real value',
}
}

You can write a function to recursively remove empty dictionaries. For example:
def return_non_empty(my_dict):
temp_dict = {}
for k, v in my_dict.items():
if v:
if isinstance(v, dict):
return_dict = return_non_empty(v)
if return_dict:
temp_dict[k] = return_dict
else:
temp_dict[k] = v
return temp_dict
For example, if d is:
d = {
"1": {
"1_1": 'a real value',
"1_2": ""
},
"2": None,
"3": {
"3_1": {
"3_2": None
}
}
}
then my custom function return_non_empty will return:
>>> return_non_empty(d)
{'1': {'1_1': 'a real value'}}

You can use recursion with a dictionary comprehension:
d = {'1': {'1_1': 'a real value', '1_2': ''}, '2': None, '3': {'3_1': {'3_2': None}}}
def get_d(d):
return {a:c for a, b in d.items() if (c:=(b if not isinstance(b, dict) else get_d(b)))}
print(get_d(d))
Output:
{'1': {'1_1': 'a real value'}}
Note: this solution uses Python's assignment expressions (:=) available in versions >= 3.8. For a solution that does not use this paradigm, please see #Anonymous's answer.

This python function recursively removes dictionary keys whose values are empty, None, False, or 0.
def remove_empty_keys(dictionary):
if isinstance(dictionary, dict):
for key, value in dictionary.copy().items():
if isinstance(value, dict):
remove_empty_keys(value)
if not value:
del(dictionary[key])
Similar to #Ajax1234's answer but this function acts directly on target dictionary rather than outputting a new one.

Pythonic way to get keys and values from nested dictionaries

just wondering, before I start to work on a function. I always like to hear some pythonic solutions.
I am trying to get keys and values from nested dictionaries:
for an example:
a = {'one': {'animal': 'chicken'},
'two': {'fish': {'sea':'shark'}}}
is there any pythonic way to get values from nested dictionary? Like get straight to value of 'fish'?
Thanks in advance

If you want to find all the items with the "fish" key in the nested dictionary, you can modify this answer flatten nested python dictionaries-compressing keys - answer #Imran
import collections
def get_by_key_in_nested_dict(d, key, parent_key='', sep='_'):
items = []
for k, v in d.items():
new_key = parent_key + sep + k if parent_key else k
if key==k:
items.append((new_key, v))
if isinstance(v, collections.MutableMapping):
items.extend(get_by_key_in_nested_dict(v, key, new_key, sep).items())
return dict(items)
with,
test = {
'one': {
'animal': 'chicken'
},
'two': {
'fish': {
'sea':'shark',
'fish':0
}
},
'fish':[1,2,3]
}
get_by_key_in_nested_dict(test,"fish")
You get all the items that have the key "fish"
{
'fish': [1, 2, 3],
'two_fish': {'fish': 0, 'sea': 'shark'},
'two_fish_fish': 0
}

How to remove all empty fields in a nested dict?

If I have a dict, which field's values may also be a dict or an array. How can I remove all empty fields in it?
"Empty field" means a field's value is empty array([]), None, or empty dict(all sub-fields are empty).
Example:
Input:
{
"fruit": [
{"apple": 1},
{"banana": None}
],
"veg": [],
"result": {
"apple": 1,
"banana": None
}
}
Output:
{
"fruit": [
{"apple": 1}
],
"result": {
"apple": 1
}
}

Use a recursive function that returns a new dictionary:
def clean_empty(d):
if isinstance(d, dict):
return {
k: v
for k, v in ((k, clean_empty(v)) for k, v in d.items())
if v
}
if isinstance(d, list):
return [v for v in map(clean_empty, d) if v]
return d
The {..} construct is a dictionary comprehension; it'll only include keys from the original dictionary if v is true, e.g. not empty. Similarly the [..] construct builds a list.
The nested (.. for ..) construct is a generator expression that allows the code to compactly filter empty objects after recursing.
Another way of constructing such a function is to use the #singledispatch decorator; you then write multiple functions, one per object type:
from functools import singledispatch
#singledispatch
def clean_empty(obj):
return obj
#clean_empty.register
def _dicts(d: dict):
items = ((k, clean_empty(v)) for k, v in d.items())
return {k: v for k, v in items if v}
#clean_empty.register
def _lists(l: list):
items = map(clean_empty, l)
return [v for v in items if v]
The above #singledispatch version does exactly the same thing as the first function but the isinstance() tests are now taken care of by the decorator implementation, based on the type annotations of the registered functions. I also put the nested iterators (the generator expression and map() function) into a separate variable to improve readability further.
Note that any values set to numeric 0 (integer 0, float 0.0) will also be cleared. You can retain numeric 0 values with if v or v == 0.
Demo of the first function:
>>> sample = {
... "fruit": [
... {"apple": 1},
... {"banana": None}
... ],
... "veg": [],
... "result": {
... "apple": 1,
... "banana": None
... }
... }
>>> def clean_empty(d):
... if isinstance(d, dict):
... return {
... k: v
... for k, v in ((k, clean_empty(v)) for k, v in d.items())
... if v
... }
... if isinstance(d, list):
... return [v for v in map(clean_empty, d) if v]
... return d
...
>>> clean_empty(sample)
{'fruit': [{'apple': 1}], 'result': {'apple': 1}}

If you want a full-featured, yet succinct approach to handling real-world data structures which are often nested, and can even contain cycles and other kinds of containers, I recommend looking at the remap utility from the boltons utility package.
After pip install boltons or copying iterutils.py into your project, just do:
from boltons.iterutils import remap
data = {'veg': [], 'fruit': [{'apple': 1}, {'banana': None}], 'result': {'apple': 1, 'banana': None}}
drop_falsey = lambda path, key, value: bool(value)
clean = remap(data, visit=drop_falsey)
print(clean)
# Output:
{'fruit': [{'apple': 1}], 'result': {'apple': 1}}
This page has many more examples, including ones working with much larger objects from Github's API.
It's pure-Python, so it works everywhere, and is fully tested in Python 2.7 and 3.3+. Best of all, I wrote it for exactly cases like this, so if you find a case it doesn't handle, you can bug me to fix it right here.

#mojoken - How about this to overcome the boolean problem
def clean_empty(d):
if not isinstance(d, (dict, list)):
return d
if isinstance(d, list):
return [v for v in (clean_empty(v) for v in d) if isinstance(v, bool) or v]
return {k: v for k, v in ((k, clean_empty(v)) for k, v in d.items()) if isinstance(v, bool) or v}

def not_empty(o):
# you can define what is empty.
if not (isinstance(o, dict) or isinstance(o, list)):
return True
return len(o) > 0
def remove_empty(o):
# here to choose what container you not need to recursive or to remove
if not (isinstance(o, dict) or isinstance(o, list)):
return o
if isinstance(o, dict):
return {k: remove_empty(v) for k, v in o.items() if not_empty(v)}
if isinstance(o, list):
return [remove_empty(v) for v in o if not_empty(v)]

def remove_empty_fields(data_):
"""
Recursively remove all empty fields from a nested
dict structure. Note, a non-empty field could turn
into an empty one after its children deleted.
:param data_: A dict or list.
:return: Data after cleaning.
"""
if isinstance(data_, dict):
for key, value in data_.items():
# Dive into a deeper level.
if isinstance(value, dict) or isinstance(value, list):
value = remove_empty_fields(value)
# Delete the field if it's empty.
if value in ["", None, [], {}]:
del data_[key]
elif isinstance(data_, list):
for index in reversed(range(len(data_))):
value = data_[index]
# Dive into a deeper level.
if isinstance(value, dict) or isinstance(value, list):
value = remove_empty_fields(value)
# Delete the field if it's empty.
if value in ["", None, [], {}]:
data_.pop(index)
return data_

How to delete entries in a dictionary with a given flag in python?

I have a dictionary, lets call it myDict, in Python that contains a set of similar dictionaries which all have the entry "turned_on : True" or "turned_on : False". I want to remove all the entries in myDict that are off, e.g. where "turned_on : False". In Ruby I would do something like this:
myDict.delete_if { |id,dict| not dict[:turned_on] }
How should I do this in Python?

You mean like this?
myDict = {"id1" : {"turned_on": True}, "id2" : {"turned_on": False}}
result = dict((a, b) for a, b in myDict.items() if b["turned_on"])
output:
{'id1': {'turned_on': True}}

Straight-forward way:
def delete_if_not(predicate_key, some_dict):
for key, subdict in some_dict.items():
if not subdict.get(predicate_key, True):
del some_dict[key]
Testing:
mydict = {
'test1': {
'turned_on': True,
'other_data': 'foo',
},
'test2': {
'turned_on': False,
'other_data': 'bar',
},
}
delete_if_not('turned_on', mydict)
print mydict
The other answers on this page so far create another dict. They don't delete the keys in your actual dict.

It's not clear what you want, but my guess is:
myDict = {i: j for i, j in myDict.items() if j['turned_on']}
or for older version of python:
myDict = dict((i, j) for i, j in myDict.iteritems() if j['turned_on'])

d = { 'id1':{'turned_on':True}, 'id2':{'turned_on':False}}
dict((i,j) for i, j in d.items() if not j['turned_on'])