I am stack in something really simple, but I cannot find the right way. I have a dictionary like this one:
mydic= {'a': {'mylist':[..,..,..]}, 'b': {'mylist':[..,..,..]}}
I am trying to iterate through mylist and create a new subdictionary with the results of a function.
for i in mydic:
for j in mydic[i]['mylist']:
mydic[i]['thenewkey'][j] = myfunction(j)
The find dictionary would be like this:
mydic= {'a': {'mylist':[..,..,..], 'thenewkey':{'..':'..', '..':'..'}}, 'b': {'mylist':[..,..,..],'thenewkey':{'..':'..', '..':'..'}}}
But when I run the code, I have a key error on thenewkey. Any idea?
You have to create first mydic[i]['thenewkey'] before assigning to mydic[i]['thenewkey'][j].
As already said, you have to create the thenewkey entry before you can add items to it, i.e. you'd have to add mydic[i]['thenewkey'] = {} to your outer loop. You could use a defaultdict(dict) to make Python automatically insert missing entries, but since you have both list and dict entries, this does not seem like a good idea.
That said, using a dictionary comprehension makes it a bit shorter and IMHO much more readable:
for i in mydic:
mydic[i]['thenewkey'] = {j: myfunction(j) for j in mydic[i]['mylist']}
Related
I am computing returns from data in a dictionary. My keys are dates and for every key I have a dataframe with data to compute my returns. To compute the returns I need data today and yesterday (t and t-1), hence I want to initiate from the second observation (key).
Since I do not have much experience my initial thought was to execute like this:
dict_return = {}
for t, value in dict_data.items()[1:]:
returns = 'formula'
dict_returns[t] = returns
Which gave me the error:
TypeError: 'dict_items' object is not subscriptable
Searching for an answer, the only discussion I could find was skipping the first item, e.g. like this:
from itertools import islice
for key, value in islice(largeSet.items(), 1, None):
Is there a simple approach to skip the first key?
Thank you
If you are in Python 3 you need to use a list, Dict_ items ([‘No surfacing ‘,’flippers’]) returns a dict_ The items object is no longer of the list type and does not support index, this is why the list type can be used
I can think of 2 options, both require an extra step:
Option 1: Create a second dict without your key and loop over that
loop_dict = dict_data.pop(<key_to_remove>)
Then loop over loop_dict as you have done above.
Option 2: Create a list of keys from your dict and loop over that
keys = dict_data.keys()
loop_keys = keys[1:]
for key in loop_keys:
Etc
If you pass a reference to your dictionary to list() you will get a list of the dictionary's keys. This is because dictionaries are iterable. In your code you're not interested in the key's value so:
dict_data = {'a': 1, 'b': 2} # or whatever
dict_data[list(dict_data)[1]] = 3
print(dict_data)
Output:
{'a': 1, 'b': 3}
What is the best way to replace the first character of all keys in a dictionary?
old_cols= {"~desc1":"adjustment1","~desc23":"adjustment3"}
I am trying to get
new_cols= {"desc1":"adjustment1","desc23":"adjustment3"}
I have tried:
for k,v in old_cols.items():
new_cols[k]=old_cols.pop(k[1:])
old_cols = {"~desc1":"adjustment1", "~desc23":"adjustment3"}
new_cols = {}
for k, v in old_cols.items():
new_key = k[1:]
new_cols[new_key] = v
Here it is with a dictionary comprehension:
old_cols= {"~desc1":"adjustment1","~desc23":"adjustment3"}
new_cols = {k.replace('~', ''):old_cols[k] for k in old_cols}
print(new_cols)
#{'desc1': 'adjustment1', 'desc23': 'adjustment3'}
There are many ways to do this with list-comprehension or for-loops. What is important is to understand is that dictionaries are mutable. This basically means that you can either modify the existing dictionary or create a new one.
If you want to create a new one (and I would recommend it! - see 'A word of warning ...' below), both solutions provided in the answers by #Ethem_Turgut and by #pakpe do the trick. I would probably wirte:
old_dict = {"~desc1":"adjustment1","~desc23":"adjustment3"}
# get the id of this dict, just for comparison later.
old_id = id(old_dict)
new_dict = {k[1:]: v for k, v in old_dict.items()}
print(f'Is the result still the same dictionary? Answer: {old_id == id(new_dict)}')
Now, if you want to modify the dictionary in place you might loop over the keys and adapt the key/value to your liking:
old_dict = {"~desc1":"adjustment1","~desc23":"adjustment3"}
# get the id of this dict, just for comparison later.
old_id = id(old_dict)
for k in [*old_dict.keys()]: # note the weird syntax here
old_dict[k[1:]] = old_dict.pop(k)
print(f'Is the result still the same dictionary? Answer: {old_id == id(old_dict)}')
A word of warning for the latter approach:
You should be aware that you are modifying the keys while looping over them. This is in most of the cases problematic and can even lead to a RuntimeError if you loop directly over old_dict. I avoided this by explicitly unpacking the keys into a list and the looping over that list with [*old_dict.keys()].
Why can modifying keys while looping over them be problematic? Imagine for example that you have the keys '~key1' and 'key1' in your old_dict. Now when your loop handles '~key1' it will modify it to 'key1' which already exists in old_dict and thus it will overwrite the value of 'key1' with the value from '~key1'.
So, only use the latter approach if you are certain to not produce issues like the example mentioned here before. If you are uncertain, simply create a new dictionary!
I have a for loop that goes through two lists and combines them in dictionary. Keys are strings (web page headers) and values are lists (containing links).
Sometimes I get the same key from the loop that already exists in the dictionary. Which is fine. But the value is different (new links) and I'd like to update the key's value in a way where I append the links instead of replacing them.
The code looks something like that below. Note: issue_links is a list of URLs
for index, link in enumerate(issue_links):
issue_soup = BeautifulSoup(urllib2.urlopen(link))
image_list = []
for image in issue_soup.findAll('div', 'mags_thumb_article'):
issue_name = issue_soup.findAll('h1','top')[0].text
image_list.append(the_url + image.a['href'])
download_list[issue_name] = image_list
Currently the new links (image_list) that belong to the same issue_name key get overwritten. I'd like instead to append them. Someone told me to use collections.defaultdict module but I'm not familiar with it.
Note: I'm using enumerate because the index gets printed to the console (not included in the code).
Something like this:
from collections import defaultdict
d = defaultdict(list)
d["a"].append(1)
d["a"].append(2)
d["b"].append(3)
Then:
print(d)
defaultdict(<class 'list'>, {'b': [3], 'a': [1, 2]})
if download_list.has_key(issume_name):
download_list[issume_name].append(image_list)
else:
download_list[issume_name] = [image_list]
is it right?If you have the same key, append the list.
Say I want to write a function which will return an arbitrary value from a dict, like: mydict['foo']['bar']['baz'], or return an empty string if it doesn't. However, I don't know if mydict['foo'] will necessarily exist, let alone mydict['foo']['bar']['baz'].
I'd like to do something like:
safe_nested(dict, element):
try:
return dict[element]
except KeyError:
return ''
But I don't know how to approach writing code that will accept the lookup path in the function. I started going down the route of accepting a period-separated string (like foo.bar.baz) so this function could recursively try to get the next sub-dict, but this didn't feel very Pythonic. I'm wondering if there's a way to pass in both the dict (mydict) and the sub-structure I'm interested in (['foo']['bar']['baz']), and have the function try to access this or return an empty string if it encounters a KeyError.
Am I going about this in the right way?
You should use the standard defaultdict: https://docs.python.org/2/library/collections.html#collections.defaultdict
For how to nest them, see: defaultdict of defaultdict, nested or Multiple levels of 'collection.defaultdict' in Python
I think this does what you want:
from collections import defaultdict
mydict = defaultdict(lambda: defaultdict(lambda: defaultdict(str)))
You might also want to check out addict.
>>> from addict import Dict
>>> addicted = Dict()
>>> addicted.a = 2
>>> addicted.b.c.d.e
{}
>>> addicted
{'a': 2, 'b': {'c': {'d': {'e': {}}}}}
It returns an empty Dict, not an empty string, but apart from that it looks like it does what you ask for in the question.
Here's a common situation when compiling data in dictionaries from different sources:
Say you have a dictionary that stores lists of things, such as things I like:
likes = {
'colors': ['blue','red','purple'],
'foods': ['apples', 'oranges']
}
and a second dictionary with some related values in it:
favorites = {
'colors':'yellow',
'desserts':'ice cream'
}
You then want to iterate over the "favorites" object and either append the items in that object to the list with the appropriate key in the "likes" dictionary or add a new key to it with the value being a list containing the value in "favorites".
There are several ways to do this:
for key in favorites:
if key in likes:
likes[key].append(favorites[key])
else:
likes[key] = list(favorites[key])
or
for key in favorites:
try:
likes[key].append(favorites[key])
except KeyError:
likes[key] = list(favorites[key])
And many more as well...
I generally use the first syntax because it feels more pythonic, but if there are other, better ways, I'd love to know what they are. Thanks!
Use collections.defaultdict, where the default value is a new list instance.
>>> import collections
>>> mydict = collections.defaultdict(list)
In this way calling .append(...) will always succeed, because in case of a non-existing key append will be called on a fresh empty list.
You can instantiate the defaultdict with a previously generated list, in case you get the dict likes from another source, like so:
>>> mydict = collections.defaultdict(list, likes)
Note that using list as the default_factory attribute of a defaultdict is also discussed as an example in the documentation.
Use collections.defaultdict:
import collections
likes = collections.defaultdict(list)
for key, value in favorites.items():
likes[key].append(value)
defaultdict takes a single argument, a factory for creating values for unknown keys on demand. list is a such a function, it creates empty lists.
And iterating over .items() will save you from using the key to get the value.
Except defaultdict, the regular dict offers one possibility (that might look a bit strange): dict.setdefault(k[, d]):
for key, val in favorites.iteritems():
likes.setdefault(key, []).append(val)
Thank you for the +20 in rep -- I went from 1989 to 2009 in 30 seconds. Let's remember it is 20 years since the Wall fell in Europe..
>>> from collections import defaultdict
>>> d = defaultdict(list, likes)
>>> d
defaultdict(<class 'list'>, {'colors': ['blue', 'red', 'purple'], 'foods': ['apples', 'oranges']})
>>> for i, j in favorites.items():
d[i].append(j)
>>> d
defaultdict(<class 'list'>, {'desserts': ['ice cream'], 'colors': ['blue', 'red', 'purple', 'yellow'], 'foods': ['apples', 'oranges']})
All of the answers are defaultdict, but I'm not sure that's the best way to go about it. Giving out defaultdict to code that expects a dict can be bad. (See: How do I make a defaultdict safe for unexpecting clients? ) I'm personally torn on the matter. (I actually found this question looking for an answer to "which is better, dict.get() or defaultdict") Someone in the other thread said that you don't want a defaultdict if you don't want this behavior all the time, and that might be true. Maybe using defaultdict for the convenience is the wrong way to go about it. I think there are two needs being conflated here:
"I want a dict whose default values are empty lists." to which defaultdict(list) is the correct solution.
and
"I want to append to the list at this key if it exists and create a list if it does not exist." to which my_dict.get('foo', []) with append() is the answer.
What do you guys think?