I am reading a dictionary from external source, let's say
data = {'name': 'James', 'gender': 'male'}
And sometimes
data = {'name': 'James', 'gender': 'male', 'article': {'title':'abc'}}
And sometimes
data = {'name': 'James', 'gender': 'male', 'article': None}
I know that I can use .get(key, default) when I am not sure if articles exists in data:
articles = data.get('article', {}).get('title')
But sometimes, they provide the element with None value, so the above doesn't work and cause error, and need to become:
articles = data.get('article') or {}
But this requires me to break it into 2 statements instead of chaining up to get values from article as mentioned earlier.
Is there a more elegant way to do that, something like:
data.get('article', {}, ignore=[None])
or
data.get_ignore_none('article', {})
By default .get() will return None if the key doesn't exist. In your case you are returning an empty dictionary.
Now, I don't know what error is being raised, but I am sure its from get_stuff(article) rather than your list comprehension.
You have a few ways to solve this:
Modify get_stuff so that it takes the value directly, rather than each element. This way, you are just passing it [get_stuff(value) for value in data.get('articles')]. Now, in get_stuff, you simply do this:
def get_stuff(foo):
if not foo:
return None
for item in foo:
do stuff
return normal_results
Add a filter in your list comprehension:
[get_stuff(foo) for foo in data.get('articles') if data.get('articles')]
There's nothing wrong with using exceptions to break out early in this case. I'm assuming you want the title value, or None, no matter what the data is. The following function will work (for Python 3).
def get_title(d):
try:
return data.get("article").get("title")
except AttributeError:
return None
If the outer dictionary gets a None as value or by default it will raise the AttributeError on the None object which you just catch.
First off you seem to think that using an or expression to discard false-y results from data.get('article') can only be done in two statements like the following:
temp = data.get('article') or {}
articles = temp.get("title")
However you can just put brackets around the first expression and call .get("title") on it's return value directly:
articles = (data.get('article') or {}).get("title")
But I feel this is not particularly readable or efficient, when 'article' missing or None then you are creating a new mapping and checking it for "title" unnecessarily.
One possible solution is to use a function like the following:
def nested_get(mapping, *keys):
"""gets a value from a nested dictionary,
if any key along the way is missing or None then None is returned
will raise an AttributeError if a value in the chain is not a dictionary (support the .get method)"""
current = mapping
for item in keys:
current = current.get(item)
if current is None:
return None
return current
Then you would do nested_get(data, "article", "title") to try to get data["article"]["title"] without throwing errors if data["article"] is None or missing.
I tested this with the following code:
test_cases = [{'name': 'James', 'gender': 'male'},
{'name': 'James', 'gender': 'male', 'article': {'title':'abc'}},
{'name': 'James', 'gender': 'male', 'article': None}]
for case in test_cases:
print(case)
print(nested_get(case,"article","title"))
print()
#the follwing will raise an error since mapping["a"] would need to be a dict
nested_get({"a":5}, "a","b")
how about this
>>> data = {1:(42,23)}
>>> [x for x in data.get(1) or []]
[42, 23]
>>> [x for x in data.get(32) or []]
[]
use or to change to your default value in case you get None or something that eval to false
Edit:
In the same way you can or and brackets to get desired output in one line
articles = (data.get('article') or {}).get('title')
and with just that you handle all 3 cases.
you can also define get_ignore_none like for example
def get_ignore_none(data_dict, key, default=None):
if key in data_dict:
value = data_dict[key]
return value if value is not None else default
return default
Since you're loading this data from an external source, one option is a preprocessing step as soon as you load it:
from collections import Mapping
def remove_none(d):
for k, v in d.items():
if v is None:
del d[k]
if isinstance(v, Mapping):
remove_none(v)
data = load_data_from_somewhere()
remove_none(data)
Now you can just use get everywhere you need to:
articles = data.get('article', {}).get('title')
Related
i am using a dict like below and i want a function preferably which returns the key value, based on the value user provides.
test_dict={'Name':[abc,def],'Address':['Local','NonLocal']}
Used the below function:
def find_key_for(input_dict,value):
result=[]
for k,v in input_dict.items():
if value in v:
result.append(k)
return result
when i use the above function as below:
print(find_key_for(test-dict,'abc')) it returns []
while i expected Name
alternatively when i do (find_key_for(test-dict,'Local')) it returns [Address] which is expected.
why does it not work for Name? what changes can i make?
You need to un-indent the return statement:
def find_key_for(input_dict, value):
result = []
for k, v in input_dict.items():
if value in v:
result.append(k)
return result
The reason is because if you don't, and have it inside the for loop, your function will return after only looping once.
Finally, you can use a nested list comprehension in just one line:
test_dict = {'Name': ['abc', 'def'], 'Address': ['Local', 'NonLocal']}
def find_key_for(input_dict, value):
return [k for k in input_dict if value in input_dict[k]]
print(find_key_for(test_dict, 'abc'))
Or a lambda function:
test_dict = {'Name': ['abc', 'def'], 'Address': ['Local', 'NonLocal']}
find_key_for = lambda input_dict, value: [k for k in input_dict if value in input_dict[k]]
print(find_key_for(test_dict, 'abc'))
Output:
['Name']
Firstly, please be sure to format your code properly so it's more legible. You can do this by clicking the code {} up at the top of the screen.
Also, this code works completely fine for me. One thing that I noticed that could be an issue (but could also be you mistyping it here) was that in the list for 'Name', you have 'abc' and 'def' written without the parantheses, and you could accidentally be assigning it to an object rather than a string. Also, you wrote "test_dict" as "test-dict" later on.
I want to get the value a specific value '1222020' which has 'Expiration' as a key:
The 'Expiration' key can be placed at any position.
input :
my_list=[{'Key': 'Expiration', 'Value': '12122020'}, {'Key': 'Name', 'Value': 'Config Test 2'}]
my solution:
res = [sub['Value'] for sub in my_list if sub['Key'] =='Expiration' ]
print(res)
Sometimes the tag 'Expiration' is not present.
How to Handle that and avoid NoneType Object error
If you could re-organize your data like so,
custom_dict = {'Expiration': '12122020', 'Name': 'Config Test 2'}
Then, you could write the code like this,
def get_key_value_from_dictionary_search(dict_data, key_search_phrase, value_search_phrase):
for k,v in dict_data.items():
if k is key_search_phrase and v is value_search_phrase:
return k, v
_k, _v = get_key_value_from_dictionary_search(custom_dict, "Expiration", "12122020")
print("Key : {}\nValue : {}".format(_k, _v))
If the Expiration key isn't present, your res evaluates to an empty list. So if you just check for the presence on an empty list, you'll know if Expiration was in there to begin with.
def get_result(lst, default="99999999"):
res = [sub['Value'] for sub in lst if sub['Key'] == 'Expiration']
if res:
# there is something in the list, so return the first thing
return res[0]
else:
# the list is empty, so Expiration wasn't in lst
return default
print(get_result(my_list))
I want to find and return the minimal value of an id in a string, for example:
find_min_id([{"nonid": "-222", "id": 0}, {"id": -101}])
-101
find_min_id([{’id’: 63, 'id': 42}])
42
So far I have this:
def find_min_id(list):
return min(list)
but that gives:
{'id': -101}
and I only want the value of the lowest id.
Use the key parameter of min:
def find_min_id(l):
return min(l, key=lambda d: d.get("id", float('inf')))["id"]
This actually finds the min id, and does it without creating a new list.
The only thing is, the elements in your list might not had an 'id' key. For that reason I had to use .get("id", float('inf')). Thus the function will return inf if there is no id key, which might not be desirable. What min() does when given an empty list is it throws an exception, so we'd probably like to do the same if none of the dicts we pass it have an 'id' key. In that case the min of a generator appoach might indeed be better:
def find_min_id(l):
return min(d["id"] for d in l if "id" in d)
The other approach would be to check for inf as the result of min, but this is more cumbersome:
import math
def find_min_id(l):
res = min(l, key=lambda d: d.get("id", float('inf')))["id"]
if math.isinf(res):
raise ValueError("No dict has an 'id' key")
return res
Another approach, but works where there is no id in the dictionary, and when there is no dictionary with an id at all:
def find_min_id(lst):
ids = [d['id'] for d in lst if 'id' in d]
return min(ids) if ids else None
Without exceptions, and without running min for artificially extended list (i.e. the answer which puts maximum floats where an entry isn't an id-entry).
Following code return None if no id key is in list.
>>> data = [{"nonid": "-222", "id": 0}, {"id": -101}, {"nonid":-200}]
>>> min(filter(lambda x: x is not None, map(lambda x: x.get('id', None),
... data)) or [None])
-101
Here filter(lambda x: x is not None, ...) filters out dictionaries without id,
map(lambda x: x.get('id', None), data) gets all id's from data, and or [None] part treats case when no id key will be found in data.
You are trying to find the 'lowest' dictionaries. What we want is, to find the lowest 'id' value in the list.
def find_min_id(lst):
return min([d[key] for key in d for d in lst if key=="id"])
Also, avoid using list as a variable name, it overrides the built-in function list().
A little demo:
>>> def find_min_id(lst):
return min([d[key] for key in d for d in lst if key=="id"])
>>> find_min_id(lst)
-101
Hope this helps!
>>> ids = [{"nonid": "-222", "id": 0}, {"id": -101}]
>>> min([val for obj in ids for key, val in obj.items() if key == 'id'])
-101
>>> ids = [{'id': 63}, { 'id': 42}]
>>> min([val for obj in ids for key, val in obj.items() if key == 'id'])
42
Try the above.
You can make this into a function definition:
def find_lowest(ids):
return min([val for obj in ids for key, val in obj.items() if key == 'id'])
Working example.
Let me explain what I'm doing. Firstly, the min function takes in an iterable object as an argument. So, let me demonstrate:
>>> min([1,2,3,4,6,1,0])
0
So, what this means is this, we are essentially taking the minimum value of the list that we get from this, [val for obj in ids for key, val in obj.items() if key == 'id'].
Now, you might be wondering, well whats happening in there? It might be a little intimidating at first, but thats a list comprehension. Whats that you say? Well, in simple terms its a concise we in which we make a list:
Let me start with be first part, and no its not the beginning of the statement:
for obj in ids
What we are doing here, is iterating over all the dictionary objects in in side of ids. Now, we use that object here:
key, val in obj.items() if key == 'id'
Since object, is a dict, we use the items function to get a generator that gives a tuple of key, value pairs. In an object like this: {'id': 100}, the id would be they key and 100 would be the value. So, we are going over all the items in the dictionary object, and if the key happens to be id, then we append it to the list:
[val
Thats what the first part does. The first part of the list comprehension appends something to the final list, and that is val.
UPDATE:
If for some reason, the list does not containt anything with id as a key, then it will throw a ValueError as min does not accept an empty list, so to remedy this, we can check:
def find_lowest(ids):
_ret = [val for obj in ids for key, val in obj.items() if key == 'id']
if _ret:
return min(_ret)
else:
return None
list is a built-in type in Python. don't use it as an identifier
def find_min_id(my_list)
id_list = []
for record in my_list:
if 'id' in record:
id_list.append(record['id'])
return min(id_list)
I need a function to change one item in composite dictionary.
I've tried something like..
def SetItem(keys, value):
item = self.dict
for key in keys:
item = item[key]
item = value
and
SetItem(['key1', 'key2'], 86)
It should be equivalent to self.dict['key1']['key2'] = 86, but this function has no effect.
Almost. You actually want to do something like:
def set_keys(d, keys, value):
item = d
for key in keys[:-1]:
item = item[key]
item[keys[-1]] = value
Or recursively like this:
def set_key(d, keys, value):
if len(keys) == 1:
d[keys[0]] = value
else:
set_key(d[keys[0]], keys[1:], value)
Marcin's right though. You would really want to incorporate something more rigorous, with some error handling for missing keys/missing dicts.
setItem = lambda self,names,value: map((lambda name: setattr(self,name,value)),names)
You don't have a self parameter
Just use the line of working code you have.
If you insist, here's a way:
def setitem(self, keys, value):
reduce(dict.get, # = lambda dictionary, key: dictionary[key]
keys[:-1], self.dictionary)[keys[-1]] = value
Obviously, this will break if the list of keys hits a non-dict value. You'll want to handle that. In fact, an explicit loop would probably be better for that reason, but you get the idea.
An idea involving recursion and EAFP, both of which I always like:
def set_item(d, keys, value):
key = keys.pop(0)
try:
set_item(d[key], keys, value)
# IndexError happens when the pop fails (empty list), KeyError happens when it's not a dict.
# Assume both mean we should finish recursing
except (IndexError, KeyError):
d[key] = value
Example:
>>> d = {'a': {'aa':1, 'ab':2}, 'b':{'ba':1, 'bb':2}}
>>> set_item(d, ['a', 'ab'], 50)
>>> print d
{'a': {'aa': 1, 'ab': 50}, 'b': {'ba': 1, 'bb': 2}}
Edit: As Marcin points out below, this will not work for arbitrarily nested dicts since Python has a recursion limit. It's also not for highly performance-sensitive situations (recursion in Python generally isn't). Nonetheless, outside of these two situations I find this to be somewhat more explicit than something involving reduce or lambda.
Here is my issue:
I am doing an LDAP search in Python. The search will return a dictionary object:
{'mail':['user#mail.com'],'mobile':['07852242242'], 'telephoneNumber':['01112512152']}
As you can see, the returned dictionary contains list values.
Sometimes, no result will be found:
{'mail':None, 'mobile':None, 'telephoneNumber':['01112512152']}
To extract the required values, I am using get() as to avoid exceptions if the dictionary item does not exist.
return {"uname":x.get('mail')[0], "telephone":x.get('telephoneNumber')[0], "mobile":x.get('mobile')[0]}
I want to return my own dictionary as above, with just the string values, but Im struggling to find an efficient way to check that the lists are not None and keep running into index errors or type errors:
(<type 'exceptions.TypeError'>, TypeError("'NoneType' object is unsubscriptable",)
Is there a way to use a get() method on a list, so that if the list is None it wont throw an exception???
{"uname":x.get('mail').get(0)}
What is the most efficient way of getting the first value of a list or returning None without using:
if isinstance(x.get('mail'),list):
or
if x.get('mail') is not None:
If you want to flatten your dictionary, you can just do:
>>> d = {'mail':None, 'mobile':None, 'telephoneNumber':['01112512152']}
>>>
>>> dict((k,v and v[0] or v) for k,v in d.items())
{'mail': None, 'mobile': None, 'telephoneNumber': '01112512152'}
If you'd also like to filter, cutting off the None values, then you could do:
>>> dict((k,v[0]) for k,v in d.items() if v)
{'telephoneNumber': '01112512152'}
Try the next:
return {"uname":(x.get('mail') or [None])[0], ...
It is a bit unreadable, so you probably want to wrap it into some helper function.
You could do something like this:
input_dict = {'mail':None, 'mobile':None, 'telephoneNumber':['01112512152']}
input_key_map = {
'mail': 'uname',
'telephoneNumber': 'telephone',
'mobile': 'mobile',
}
dict((new_name, input_dict[old_name][0])
for old_name, new_name in input_key_map.items() if input_dict.get(old_name))
# would print:
{'telephone': '01112512152'}
I'm not sure if there's a straightforward way to do this, but you can try:
default_value = [None]
new_x = dict((k, default_value if not v else v) for k, v in x.iteritems())
And use new_x instead of x.