Develop Reference

How to get a false-like value instead of error when retrieving non-existent elements in lists or dictionaries in Python

I would like to know how to, instead of getting an error, return a False-like value when trying to access a list out of range, for example, or trying to grab a key from a dictionary that doesn't exist. Does anyone know of a method?

for a dictionary you can use a defaultdict from the collections module I.E:
from collections import defaultdict
my_dict = defaultdict(lambda: False)
my_dict['test'] # This will be False
You can read more about defaultdict in the docs
For a list however it gets a bit more complicated, sublcassing list and overriding the __getitem__ method to wrap the call in a try/except clause and return a different value:
class defaultlist(list):
def __getitem__(self, key):
try:
return super().__getitem__(key)
except IndexError:
return False
my_list = defaultlist()
my_list[10] # This will be False

Dictionaries have a get method allowing you to do exactly that
test = {}
test.get("gold", False)
this yields False or anything else that you give get as a 2nd argument.

When you try to access a list out of range, you get this message on the last line of the console logs:
IndexError: list index out of range
so, to take an example:
test_list = [0]*1
print(test_list[1])
IndexError: list index out of range
So you need:
test_list = [0]*1
try:
print(test_list[1])
except IndexError:
value = False
print (value)
False

How to handle dictionary keys that do not exist in a loop?

I tried to parse json and use for loop in Python.
obj is json and as you know it is like dictionary in list in dictionary in dictionary...and so on.
So I parse json and make it as a for loop like this:
all = obj['products']
for a in obj:
recommendSellerScore.append(a['recommendSellerScore'])
However, the problem is 'recommendSellerScore' which is key does not exist in some lists.
What I want as a result is to print 'None' where recommendSellerScore does not exist, and print values where recommendSellerScore does exist.
for a in all:
if a['recommendSellerScore'] in all:
recommendSellerScore.append(a['recommendSellerScore'])
else:
continue
print('None')
I ran this code above, but got an Error because 'recommendSellerScore' does not exist.
for a in all:
if ['recommendSellerScore'] in all:
recommendSellerScore.append(a['recommendSellerScore'])
else:
continue
print('None')
This one worked, however, since 'recommendSellerScore' exists in list in dictionary, it was not iterable. (I guess)
Please fix my code, and any comments or advice will be appreciated!

Here is fixed version of your code (I took out the continue statement and the [] around the string):
for a in all:
if "recommendSellerScore" in a:
recommendSellerScore.append(a["recommendSellerScore"])
else:
print("None")

Maybe something like the code below works for you?
for a in all:
if a.get(['recommendSellerScore']) is not None:
recommendSellerScore.append(a['recommendSellerScore'])
else:
print("None")
# No need for continue here.

You can use get method of dictionaries to try to access the dictionary value, if the key is not present you can provide a default value, if you don't specify one, it will output None, for example:
key = 'recommendSellerScore'
a = [{key: 1}, {}]
recommendSellerScore = []
for d in a:
score = d.get(key)
if score is not None:
print(score)
recommendSellerScore.append(score)
else:
print('None')
Edit: as pointed by #kederrac if your data can have {key: None} values and you are interested in keeping those None values then it's best to ask if the key is truly present in the dictionary, for example:
key = 'recommendSellerScore'
a = [{key: 1}, {key: None}]
recommendSellerScore = []
for d in a:
if key in d.keys():
print(d[key])
recommendSellerScore.append(d[key])
else:
print('None')
It will print:
>>> 1
>>> None

Does an empty dictionary inside an empty list make the list Not-empty

I have this list:
li = [{}]
if li:
print 'yes'
#Output - yes
I, wrongly, thought that it would not output anything as it is an empty thing(falsiness). Now, how do I check for its emptiness?

If [{}] is your only case, then any([{}]) returns False. However, it will also return False for [0] which might not be what you want and it will return True for [[{}]].
If you want to check that a list contains only empty items recursively, try this:
def is_empty_recursive(collection):
try:
if len(collection) == 0:
return True
except TypeError:
# This will happen if collection is not a collection for example len(0) will cause this exception
return False
return all(is_empty_recursive(item) for item in collection)
Now you can use it like
li = [{}]
if not is_empty_recursive(li):
print("yes")

to check for emptiness, you could look at each thing contained in your list:
emptydict = [{}]
for obj in emptydict:
if obj:
print("yes")
else:
print("no")
to understand why sets operate a certain way, you could briefly review set theory(does a set of all sets contain itself?):
https://en.wikipedia.org/wiki/Universal_set

There are multiple ways depending on what you want to achieve. You could simply use:
li = []
print(li)
If nothing is returned there is nothing there.

Test if dictionary key exists, is not None and isn't blank

I have code that works but I'm wondering if there is a more pythonic way to do this. I have a dictionary and I want to see if:
a key exists
that value isn't None (NULL from SQL in this case)
that value isn't simply quote quote (blank?)
that value doesn't solely consist of spaces
So in my code the keys of "a", "b", and "c" would succeed, which is correct.
import re
mydict = {
"a":"alpha",
"b":0,
"c":False,
"d":None,
"e":"",
"g":" ",
}
#a,b,c should succeed
for k in mydict.keys():
if k in mydict and mydict[k] is not None and not re.search("^\s*$", str(mydict[k])):
print(k)
else:
print("I am incomplete and sad")
What I have above works, but that seems like an awfully long set of conditions. Maybe this simply is the right solution but I'm wondering if there is a more pythonic "exists and has stuff" or better way to do this?
UPDATE
Thank you all for wonderful answers and thoughtful comments. With some of the points and tips, I've updated the question a little bit as there some conditions I didn't have which should also succeed. I have also changed the example to a loop (just easier to test right?).

Try to fetch the value and store it in a variable, then use object "truthyness" to go further on with the value
v = mydict.get("a")
if v and v.strip():
if "a" is not in the dict, get returns None and fails the first condition
if "a" is in the dict but yields None or empty string, test fails, if "a" yields a blank string, strip() returns falsy string and it fails too.
let's test this:
for k in "abcde":
v = mydict.get(k)
if v and v.strip():
print(k,"I am here and have stuff")
else:
print(k,"I am incomplete and sad")
results:
a I am here and have stuff
b I am incomplete and sad # key isn't in dict
c I am incomplete and sad # c is None
d I am incomplete and sad # d is empty string
e I am incomplete and sad # e is only blanks
if your values can contain False, 0 or other "falsy" non-strings, you'll have to test for string, in that case replace:
if v and v.strip():
by
if v is not None and (not isinstance(v,str) or v.strip()):
so condition matches if not None and either not a string (everything matches) or if a string, the string isn't blank.

The get method for checking if a key exists is more efficient that iterating through the keys. It checks to see if the key exists without iteration using an O(1) complexity as apposed to O(n). My preferred method would look something like this:
if mydict.get("a") is not None and str(mydict.get("a")).replace(" ", "") != '':
# Do some work

You can use a list comprehension with str.strip to account for whitespace in strings.
Using if v is natural in Python to cover False-like objects, e.g. None, False, 0, etc. So note this only works if 0 is not an acceptable value.
res = [k for k, v in mydict.items() if (v.strip() if isinstance(v, str) else v)]
['a']

Here's a simple one-liner to check:
The key exists
The key is not None
The key is not ""
bool(myDict.get("some_key"))
As for checking if the value contains only spaces, you would need to be more careful as None doesn't have a strip() method.
Something like this as an example:
try:
exists = bool(myDict.get('some_key').strip())
except AttributeError:
exists = False

Well I have 2 suggestions to offer you, especially if your main issue is the length of the conditions.
The first one is for the check if the key is in the dict. You don't need to use "a" in mydict.keys() you can just use "a" in mydict.
The second suggestion to make the condition smaller is to break down into smaller conditions stored as booleans, and check these in your final condition:
import re
mydict = {
"a":"alpha",
"c":None,
"d":"",
"e":" ",
}
inKeys = True if "a" in mydict else False
isNotNone = True if mydict["a"] is not None else False
isValidKey = True if not re.search("^\s*$", mydict["a"]) else False
if inKeys and isNotNone and isValidKey:
print("I am here and have stuff")
else:
print("I am incomplete and sad")

it check exactly for NoneType not only None
from types import NoneType # dont forget to import this
mydict = {
"a":"alpha",
"b":0,
"c":False,
"d":None,
"e":"",
"g":" ",
}
#a,b,c should succeed
for k in mydict:
if type(mydict[k]) != NoneType:
if type(mydict[k]) != str or type(mydict[k]) == str and mydict[k].strip():
print(k)
else:
print("I am incomplete and sad")
else:
print("I am incomplete and sad")

cond is a generator function responsible for generating conditions to apply in a short-circuiting manner using the all function. Given d = cond(), next(d) will check if a exists in the dict, and so on until there is no condition to apply, in that case all(d) will evaluate to True.
mydict = {
"a":"alpha",
"c":None,
"d":"",
"e":" ",
}
def cond ():
yield 'a' in mydict
yield mydict ['a']
yield mydict ['a'].strip ()
if all (cond ()):
print("I am here and have stuff")
else:
print("I am incomplete and sad")

Elegant way to check if a nested key exists in a dict?

Is there are more readable way to check if a key buried in a dict exists without checking each level independently?
Lets say I need to get this value in a object buried (example taken from Wikidata):
x = s['mainsnak']['datavalue']['value']['numeric-id']
To make sure that this does not end with a runtime error it is necessary to either check every level like so:
if 'mainsnak' in s and 'datavalue' in s['mainsnak'] and 'value' in s['mainsnak']['datavalue'] and 'nurmeric-id' in s['mainsnak']['datavalue']['value']:
x = s['mainsnak']['datavalue']['value']['numeric-id']
The other way I can think of to solve this is wrap this into a try catch construct which I feel is also rather awkward for such a simple task.
I am looking for something like:
x = exists(s['mainsnak']['datavalue']['value']['numeric-id'])
which returns True if all levels exists.

To be brief, with Python you must trust it is easier to ask for forgiveness than permission
try:
x = s['mainsnak']['datavalue']['value']['numeric-id']
except KeyError:
pass
The answer
Here is how I deal with nested dict keys:
def keys_exists(element, *keys):
'''
Check if *keys (nested) exists in `element` (dict).
'''
if not isinstance(element, dict):
raise AttributeError('keys_exists() expects dict as first argument.')
if len(keys) == 0:
raise AttributeError('keys_exists() expects at least two arguments, one given.')
_element = element
for key in keys:
try:
_element = _element[key]
except KeyError:
return False
return True
Example:
data = {
"spam": {
"egg": {
"bacon": "Well..",
"sausages": "Spam egg sausages and spam",
"spam": "does not have much spam in it"
}
}
}
print 'spam (exists): {}'.format(keys_exists(data, "spam"))
print 'spam > bacon (do not exists): {}'.format(keys_exists(data, "spam", "bacon"))
print 'spam > egg (exists): {}'.format(keys_exists(data, "spam", "egg"))
print 'spam > egg > bacon (exists): {}'.format(keys_exists(data, "spam", "egg", "bacon"))
Output:
spam (exists): True
spam > bacon (do not exists): False
spam > egg (exists): True
spam > egg > bacon (exists): True
It loop in given element testing each key in given order.
I prefere this to all variable.get('key', {}) methods I found because it follows EAFP.
Function except to be called like: keys_exists(dict_element_to_test, 'key_level_0', 'key_level_1', 'key_level_n', ..). At least two arguments are required, the element and one key, but you can add how many keys you want.
If you need to use kind of map, you can do something like:
expected_keys = ['spam', 'egg', 'bacon']
keys_exists(data, *expected_keys)

You could use .get with defaults:
s.get('mainsnak', {}).get('datavalue', {}).get('value', {}).get('numeric-id')
but this is almost certainly less clear than using try/except.

Python 3.8 +
dictionary = {
"main_key": {
"sub_key": "value",
},
}
if sub_key_value := dictionary.get("main_key", {}).get("sub_key"):
print(f"The key 'sub_key' exists in dictionary[main_key] and it's value is {sub_key_value}")
else:
print("Key 'sub_key' doesn't exists or their value is Falsy")
Extra
A little but important clarification.
In the previous code block, we verify that a key exists in a dictionary but that its value is also Truthy.
Most of the time, this is what people are really looking for, and I think this is what the OP really wants. However, it is not really the most "correct" answer, since if the key exists but its value is False, the above code block will tell us that the key does not exist, which is not true.
So, I leet here a more correct answer:
dictionary = {
"main_key": {
"sub_key": False,
},
}
if "sub_key" in dictionary.get("main_key", {}):
print(f"The key 'sub_key' exists in dictionary[main_key] and it's value is {dictionary['main_key']['sub_key']}")
else:
print("Key 'sub_key' doesn't exists")

Try/except seems to be most pythonic way to do that.
The following recursive function should work (returns None if one of the keys was not found in the dict):
def exists(obj, chain):
_key = chain.pop(0)
if _key in obj:
return exists(obj[_key], chain) if chain else obj[_key]
myDict ={
'mainsnak': {
'datavalue': {
'value': {
'numeric-id': 1
}
}
}
}
result = exists(myDict, ['mainsnak', 'datavalue', 'value', 'numeric-id'])
print(result)
>>> 1

I suggest you to use python-benedict, a solid python dict subclass with full keypath support and many utility methods.
You just need to cast your existing dict:
s = benedict(s)
Now your dict has full keypath support and you can check if the key exists in the pythonic way, using the in operator:
if 'mainsnak.datavalue.value.numeric-id' in s:
# do stuff
Here the library repository and the documentation:
https://github.com/fabiocaccamo/python-benedict
Note: I am the author of this project

You can use pydash to check if exists: http://pydash.readthedocs.io/en/latest/api.html#pydash.objects.has
Or get the value (you can even set default - to return if doesn't exist): http://pydash.readthedocs.io/en/latest/api.html#pydash.objects.has
Here is an example:
>>> get({'a': {'b': {'c': [1, 2, 3, 4]}}}, 'a.b.c[1]')
2

The try/except way is the most clean, no contest. However, it also counts as an exception in my IDE, which halts execution while debugging.
Furthermore, I do not like using exceptions as in-method control statements, which is essentially what is happening with the try/catch.
Here is a short solution which does not use recursion, and supports a default value:
def chained_dict_lookup(lookup_dict, keys, default=None):
_current_level = lookup_dict
for key in keys:
if key in _current_level:
_current_level = _current_level[key]
else:
return default
return _current_level

The accepted answer is a good one, but here is another approach. It's a little less typing and a little easier on the eyes (in my opinion) if you end up having to do this a lot. It also doesn't require any additional package dependencies like some of the other answers. Have not compared performance.
import functools
def haskey(d, path):
try:
functools.reduce(lambda x, y: x[y], path.split("."), d)
return True
except KeyError:
return False
# Throwing in this approach for nested get for the heck of it...
def getkey(d, path, *default):
try:
return functools.reduce(lambda x, y: x[y], path.split("."), d)
except KeyError:
if default:
return default[0]
raise
Usage:
data = {
"spam": {
"egg": {
"bacon": "Well..",
"sausages": "Spam egg sausages and spam",
"spam": "does not have much spam in it",
}
}
}
(Pdb) haskey(data, "spam")
True
(Pdb) haskey(data, "spamw")
False
(Pdb) haskey(data, "spam.egg")
True
(Pdb) haskey(data, "spam.egg3")
False
(Pdb) haskey(data, "spam.egg.bacon")
True
Original inspiration from the answers to this question.
EDIT: a comment pointed out that this only works with string keys. A more generic approach would be to accept an iterable path param:
def haskey(d, path):
try:
functools.reduce(lambda x, y: x[y], path, d)
return True
except KeyError:
return False
(Pdb) haskey(data, ["spam", "egg"])
True

I had the same problem and recent python lib popped up:
https://pypi.org/project/dictor/
https://github.com/perfecto25/dictor
So in your case:
from dictor import dictor
x = dictor(s, 'mainsnak.datavalue.value.numeric-id')
Personal note:
I don't like 'dictor' name, since it doesn't hint what it actually does. So I'm using it like:
from dictor import dictor as extract
x = extract(s, 'mainsnak.datavalue.value.numeric-id')
Couldn't come up with better naming than extract. Feel free to comment, if you come up with more viable naming. safe_get, robust_get didn't felt right for my case.

Another way:
def does_nested_key_exists(dictionary, nested_key):
exists = nested_key in dictionary
if not exists:
for key, value in dictionary.items():
if isinstance(value, dict):
exists = exists or does_nested_key_exists(value, nested_key)
return exists

The selected answer works well on the happy path, but there are a couple obvious issues to me. If you were to search for ["spam", "egg", "bacon", "pizza"], it would throw a type error due to trying to index "well..." using the string "pizza". Like wise, if you replaced pizza with 2, it would use that to get the index 2 from "Well..."
Selected Answer Output Issues:
data = {
"spam": {
"egg": {
"bacon": "Well..",
"sausages": "Spam egg sausages and spam",
"spam": "does not have much spam in it"
}
}
}
print(keys_exists(data, "spam", "egg", "bacon", "pizza"))
>> TypeError: string indices must be integers
print(keys_exists(data, "spam", "egg", "bacon", 2)))
>> l
I also feel that using try except can be a crutch that we might too quickly rely on. Since I believe we already need to check for the type, might as well remove the try except.
Solution:
def dict_value_or_default(element, keys=[], default=Undefined):
'''
Check if keys (nested) exists in `element` (dict).
Returns value if last key exists, else returns default value
'''
if not isinstance(element, dict):
return default
_element = element
for key in keys:
# Necessary to ensure _element is not a different indexable type (list, string, etc).
# get() would have the same issue if that method name was implemented by a different object
if not isinstance(_element, dict) or key not in _element:
return default
_element = _element[key]
return _element
Output:
print(dict_value_or_default(data, ["spam", "egg", "bacon", "pizza"]))
>> INVALID
print(dict_value_or_default(data, ["spam", "egg", "bacon", 2]))
>> INVALID
print(dict_value_or_default(data, ["spam", "egg", "bacon"]))
>> "Well..."

Here's my small snippet based on #Aroust's answer:
def exist(obj, *keys: str) -> bool:
_obj = obj
try:
for key in keys:
_obj = _obj[key]
except (KeyError, TypeError):
return False
return True
if __name__ == '__main__':
obj = {"mainsnak": {"datavalue": {"value": "A"}}}
answer = exist(obj, "mainsnak", "datavalue", "value", "B")
print(answer)
I added TypeError because when _obj is str, int, None, or etc, it would raise that error.

I wrote a data parsing library called dataknead for cases like this, basically because i got frustrated by the JSON the Wikidata API returns as well.
With that library you could do something like this
from dataknead import Knead
numid = Knead(s).query("mainsnak/datavalue/value/numeric-id").data()
if numid:
# Do something with `numeric-id`

Using dict with defaults is concise and appears to execute faster than using consecutive if statements.
Try it yourself:
import timeit
timeit.timeit("'x' in {'a': {'x': {'y'}}}.get('a', {})")
# 0.2874350370002503
timeit.timeit("'a' in {'a': {'x': {'y'}}} and 'x' in {'a': {'x': {'y'}}}['a']")
# 0.3466246419993695

I have written a handy library for this purpose.
I am iterating over ast of the dict and trying to check if a particular key is present or not.
Do check this out.
https://github.com/Agent-Hellboy/trace-dkey

If you can suffer testing a string representation of the object path then this approach might work for you:
def exists(str):
try:
eval(str)
return True
except:
return False
exists("lst['sublist']['item']")

one can try to use this for checking whether key/nestedkey/value is in nested dict
import yaml
#d - nested dictionary
if something in yaml.dump(d, default_flow_style=False):
print(something, "is in", d)
else:
print(something, "is not in", d)

There are many great answers. here is my humble take on it. Added check for array of dictionaries as well. Please note that I am not checking for arguments validity. I used part Arnot's code above. I added this answer because a I got a use case that requires checking array or dictionaries in my data.
Here is the code:
def keys_exists(element, *keys):
'''
Check if *keys (nested) exists in `element` (dict).
'''
retval=False
if isinstance(element,dict):
for key,value in element.items():
for akey in keys:
if element.get(akey) is not None:
return True
if isinstance(value,dict) or isinstance(value,list):
retval= keys_exists(value, *keys)
elif isinstance(element, list):
for val in element:
if isinstance(val,dict) or isinstance(val,list):
retval=keys_exists(val, *keys)
return retval