I have a list of dictionaries like so:
[{'a':'21'},{},{'b':20'},{'c':'89'},{}]
What's the most efficient way to purge empty dictionaries from this list, end result being:
[{'a':'21'},{'b':'20'},{'c':'89'}]
I'm trying:
new_list_of_dictionaries = []
for dictionary in list_of_dictionaries:
if dictionary:
new_list_of_dictionaries.append(dictionary)
return new_list_of_dictionaries
I don't suppose this can be done in O(1) or something?
Just use a list comprehension, and filter on the boolean truth. An empty dictionary is considered false:
return [d for d in list_of_dictionaries if d]
In Python 2, you could also use the filter() function, using None as the filter:
return filter(None, list_of_dictionaries)
In Python 3 that returns an iterator, not a list, so you'd have to call list() on that (so return list(filter(None, ...))), at which point the list comprehension is simply more readable. Of course, if you don't actually need to have random access to the result (so direct index access to result[whatever]), then an iterator might still be a good idea anyway.
Note that this has to take O(N) time, you have to test each and every dictionary. Even if lists had some kind of automaticly updated map that lets you get the indices of the dictionaries that are empty in O(1) time, removing items from a list requires moving later entries forward.
Comprehension or filter (Python2, Python3):
return filter(None, list_of_dictionaries)
# Python3, if you prefer a list over an iterator
return list(filter(None, list_of_dictionaries))
None as filter function will filter out all non-truthy elements, which in the case of empty collections makes it quite concise.
could use a list comprehension?
myList = [{'a':'21'},{},{'b':'20'},{'c':'89'},{}]
result = [x for x in myList if x]
I did it in this way
d = [{'a': '21'}, {}, {'b': 20}, {'c': '89'}, {}]
new_d = []
for item in d:
check = bool(item)
if not check:
del item
else:
new_d.append(item)
print(new_d)
[{'a': '21'}, {'b': 20}, {'c': '89'}]
Related
i have a dictionary
dictionary = {
1:[[1,2],[3,4],[5,6],[7,8],[1,2]],
2:[[5,6],[7,8],[1,2]],
3:[3,4],[5,6],[3,4]]
}
How can i remove duplicate list in each value of the dictionary?
output = {
1:[[3,4],[5,6],[7,8],[1,2]],
2:[[5,6],[7,8],[1,2]],
3:[3,4],[5,6]]
}
How can i remove all duplicates?
output = [[1,2],[3,4],[5,6],[7,8]]
i have tried doing for loops, like so:
for i in dictionary.values():
for j in i:
for k in i:
if j == k:
i.remove(k)
but i'm just a beginner so i'm not getting any results...
The usual way to do this is to leverage a set, which is like a dictionary that has only keys and no values. Dictionaries (and sets) rely on their keys to be "hashable," which means that you can feed the key through some hash function and get the same result every time. In Python you can call this hash function with hash(some_object), which internally invokes some_object.__hash__().
The problem with this approach is that lists are not hashable. No mutable objects (things you can change with methods like list.append or set.add or dict.union or etc) are. This means you must either check equality by hand, or mutate it into some form that is hashable, use the set, and then mutate it back. I think the latter is probably your best bet.
To that end, let's use a tuple. Tuples are just like lists except they are not idiomatically homogenous (so mixing types is common, not just technically allowed) and their order has semantic meaning. Consider an ordered pair on a plane -- it would matter deeply if the order flipped: (1, 4) is not the same point as (4, 1). They are, however, immutable and hashable.
d = {1: [[1,2],[3,4],[5,6],[7,8],[1,2]],
2: [[5,6],[7,8],[1,2]],
3: [[3,4],[5,6],[3,4]]}
# we'll use a set comprehension here because it's concise
uniques = {tuple(sublst) for lst in d.values() for sublst in lst}
result = [list(tup) for tup in uniques] # then just change them back to lists
Note that the conversion to set and back does lose all ordering. If ordering is important then you'll have to do something like iterate through every sub list, convert it to tuple, check to see if it's already been seen, and if not add it to the seen set and append it to your final list.
d = {1: [[1,2],[3,4],[5,6],[7,8],[1,2]],
2: [[5,6],[7,8],[1,2]],
3: [[3,4],[5,6],[3,4]]}
seen = set()
result = []
for lst in d.values():
for sublst in lst:
tup = tuple(sublst)
if tup not in seen:
seen.add(tup)
result.append(sublst)
Below is the code:
s= "Name1=Value1;Name2=Value2;Name3=Value3"
dict(item.split("=") for item in s.split(";"))
I would like to understand how this works. Will it perform for loop first or will it split first?
List of dictionary
s1= "Name1=Value1,Name2=Value2,Name3=Value3;Name1=ValueA,Name2=ValueB,Name3=ValueC"
If you have python installed, I recommend using its interactive repl
With the repl you can run the parts of your program step by step:
s.split(";") will give you ['Name1=Value1', 'Name2=Value2', 'Name3=Value3']
['Name1=Value1', 'Name2=Value2', 'Name3=Value3']
item.split("=") for item in s.split(";") will give you a python generator that iterates on the the list from step 1 and split it off like into smaller lists like this:
[['Name1', 'Value1'], ['Name2', 'Value2'], ['Name3', 'Value3']]
Finally dict(...) on the pairs will turn them into key-value pairs in a python dictionary like this:
{'Name1': 'Value1', 'Name2': 'Value2', 'Name3': 'Value3'}
dict is being passed a generator expression, which produces a sequence of lists by first calling s.split(";"), then yielding the result of item.split("=") for each value in the result of the first split. A more verbose version:
s = "..."
d = dict()
name_value_pairs = s.split(";")
for item in name_value_pairs:
name_value = item.split("=")
d.update([name_value])
I use d.update rather than something simpler like d[x] = y because both dict and d.update can accept the same kind of sequence of key/value pairs as arguments.
From here, we can reconstruct the original by eliminating one temporary variable at a time, from
s = "..."
d = dict()
for item in s.split(";"):
name_value = item.split("=")
d.update(name_value)
to
s = "..."
d = dict()
for item in s.split(";"):
d.update([item.split("=")])
to
s = "..."
d = dict(item.split("=") for item in s.split(";"))
If you write it like that, you might understand better what's happening.
s= "Name1=Value1;Name2=Value2;Name3=Value3"
semicolon_sep = s.split(";")
equal_sep = [item.split("=") for item in semicolon_sep]
a = dict(equal_sep)
print(a["Name1"])
First, it splits the text from wherever there is a semicolon. In this way, we create a list with three elements as "semicolon_sep":
>>> print(semicolon_sep)
['Name1=Value1', 'Name2=Value2', 'Name3=Value3']
Then, it makes a loop over this list to separate each item wherever there is "=". In this way, we have 2 columns for each item (Name and Value). By putting this list (equal_sep) in dict() we change the list to a dictionary.
I need to build a list of dictionaries which is resulted from for loop with if statement, where each dictionary key is the same. How can I do it? Thanks in advance
list = []
dict = {}
for item in some_other_list
if item == 0:
dict.update({'the_key_which_is_always_same_word': item.value})
else:
dict.update({'the_key_which_is_always_same_word': item.value})
the result of list after for loop should look like:
[{'the_key_which_is_always_same_word': 'value_1'},
{'the_key_which_is_always_same_word': 'value_2'},
{'the_key_which_is_always_same_word': 'value_3'}]
The logic is all there, but I don't think you are adding items to the right object.
dict.update updates the dictionary that you've created. And by "updates", it will join the two dictionaries together. If the key already exists, it will update that key with the given value. If not, it will create a new key with that value. So, you are constantly updating your dict variable, not the list. To add values to the end of a list, you should use append.
list = []
for item in some_other_list
dict = {}
if item == 0:
dict['the_key_which_is_always_same_word'] = item.value
else:
dict['the_key_which_is_always_same_word'] = item.value
list.append(dict)
What this will do is, for each iteration of the loop, create a new dictionary. If item is 0, we will write item.value to a given key. Otherwise, we write it to a different key. I'm assuming that in your actual code, the two keys are different. If they aren't, then this if/else block is pointless.
After this, we append it to the end of the list.
List Comprehension
This could also be done in one line with list comprehension if you want:
[{"the_key_which_is_always_same_word":i} if i == 0 else {"the_key_which_is_always_same_word":i} for i in some_other_list]
If i==0, the first dictionary will be placed in the list, otherwise, the second one will be.
You need to change dict.update to list.append to get the result.
Actually, using list comprehensions would make it more clear:
list = [{'the_key_which_is_always_same_word': item} for item in some_other_list]
If you want to generate different dict according to the value of item, you can write this:
list = [{'key_one': item} if item == 0 else {'key_two': item} for item in some_other_list]
You have to first convert your list to a set. Because if you dont it, it will append the same dictionary again and again in the list. Which is kind of pointless.
l1 = [int(a) for a in list("10003204")]
l2 = set(l1)
l3 = []
for a in l2:
c = {}
if a == 0:
c['if_key']=a
l3.append(c)
else:
c['else_key']=a
l3.append(c)
print l1
print l2
print l3
Output:
[1, 0, 0, 0, 3, 2, 0, 4]
set([0, 1, 2, 3, 4])
[{'if_key': 0}, {'else_key': 1}, {'else_key': 2}, {'else_key': 3}, {'else_key': 4}]
And if you want your key to be same in all conditions then why are you using if else condition? You can simply do this:
for a in l2:
c = {}
c['same_key_always']=a
l3.append(c)
I am trying to remove a dictionary from a list if it already exists but it doesn't seem to be working. Can anyone see what I am doing wrong or advise me what I should be doing
new_dict = {'value': 'some value', 'key': 'someKey'}
if new_dict in my_list:
my_list.remove(new_dict)
new_list is a list of dictionaries where new_dict is definitely in
If new_dict is "definitely" in my_list, then my_list.remove(new_dict) should do the trick (i.e., no need for the if new_dict in my_list, that just slows it down).
my_list = [1,{'value':'some value', 'key' :'somekey'}, 2, {'z':'z', 'x': 'x'}]
new_dict = {'value':'some value', 'key' :'somekey'}
#new_dict = {'z':'z', 'x': 'x'}
differ = 0
matched = 0
for element in my_list:
if type(element) is types.DictType and matched != 0:
differ = 0
# check if dictionary keys match
if element.viewkeys() == new_dict.viewkeys():
# check if dictionary values match
for key in element.keys():
if element[key] != new_dict[key]:
differ = 1
matched = 1
if differ != 1:
my_list.remove(new_dict)
print my_list
It worked for both of the dictionaries for me.
In most cases it is clever to build a new list:
new_list = [ dd for dd in my_list if not dd is new_dict ]
This is typical for a functional programming style, as it avoids side effects. Imagine if you use your solution in a function or method. In most cases you need a modified list only for internal purposes, then modifying an input parameter is dangerous.
Your problem may come from the fact that removing from a list while iterating over the same list is not safe. What you want to do is something like:
copied_list = my_list[:]
if new_dict in copied_list:
my_list.remove(new_dict)
This way, you iterate over a copy of the list and remove from the original.
This may not be the cause of your problem though. It would be interesting to see:
how you build my_list
what you do with my_list after the loop, i.e. how do you realise your dictionary was not removed
I'm going through a whole bunch of tuples with a many-to-many correlation, and I want to make a dictionary where each b of (a,b) has a list of all the a's that correspond to a b. It seems awkward to test for a list at key b in the dictionary, then look for an a, then append a if it's not already there, every single time through the tuple digesting loop; but I haven't found a better way yet. Does one exist? Is there some other way to do this that's a lot prettier?
See the docs for the setdefault() method:
setdefault(key[, default])
If key is
in the dictionary, return its value.
If not, insert key with a value of
default and return default. default
defaults to None.
You can use this as a single call that will get b if it exists, or set b to an empty list if it doesn't already exist - and either way, return b:
>>> key = 'b'
>>> val = 'a'
>>> print d
{}
>>> d.setdefault(key, []).append(val)
>>> print d
{'b': ['a']}
>>> d.setdefault(key, []).append('zee')
>>> print d
{'b': ['a', 'zee']}
Combine this with a simple "not in" check and you've done what you're after in three lines:
>>> b = d.setdefault('b', [])
>>> if val not in b:
... b.append(val)
...
>>> print d
{'b': ['a', 'zee', 'c']}
Assuming you're not really tied to lists, defaultdict and set are quite handy.
import collections
d = collections.defaultdict(set)
for a, b in mappings:
d[b].add(a)
If you really want lists instead of sets, you could follow this with a
for k, v in d.iteritems():
d[k] = list(v)
And if you really want a dict instead of a defaultdict, you can say
d = dict(d)
I don't really see any reason you'd want to, though.
Use collections.defaultdict
your_dict = defaultdict(list)
for (a,b) in your_list:
your_dict[b].append(a)
you can sort your tuples O(n log n) then create your dictionary O(n)
or simplier O(n) but could impose heavy load on memory in case of many tuples:
your_dict = {}
for (a,b) in your_list:
if b in your_dict:
your_dict[b].append(a)
else:
your_dict[b]=[a]
Hmm it's pretty much the same as you've described. What's awkward about that?
You could also consider using an sql database to do the dirty work.
Instead of using an if, AFAIK it is more pythonic to use a try block instead.
your_list=[('a',1),('a',3),('b',1),('f',1),('a',2),('z',1)]
your_dict={}
for (a,b) in your_list:
try:
your_dict[b].append(a)
except KeyError:
your_dict[b]=[a]
print your_dict
I am not sure how you will get out of the key test, but once they key/value pair has been initialized it is easy :)
d = {}
if 'b' not in d:
d['b'] = set()
d['b'].add('a')
The set will ensure that only 1 of 'a' is in the collection. You need to do the initial 'b' check though to make sure the key/value exist.
Dict get method?
It returns the value of my_dict[some_key] if some_key is in the dictionary, and if not - returns some default value ([] in the example below):
my_dict[some_key] = my_dict.get(some_key, []).append(something_else)
There's another way that's rather efficient (though maybe not as efficient as sets) and simple. It's similar in practice to defaultdict but does not require an additional import.
Granted that you have a dict with empty (None) keys, it means you also create the dict keys somewhere. You can do so with the dict.fromkeys method, and this method also allows for setting a default value to all keys.
keylist = ['key1', 'key2']
result = dict.fromkeys(keylist, [])
where result will be:
{'key1': [], 'key2': []}
Then you can do your loop and use result['key1'].append(..) directly