Related
I have a nested list. In the list, it has a date and a name. I want the count the users on the date. For example,In '2019-10-11', there are 5 users. But sam user already exists, so i count it as 4. Here's the list
a =[['sam', '2019-10-11'], ['ram', '2019-10-12'], ['king', '2019-10-11'],
['queen', '2019-10-10'], ['ram', '2019-10-12'], ['kumar', '2019-10-11'],
['jeba', '2019-10-11'], ['sam', '2019-10-10'], ['sam', '2019-10-11'],
['howl', '2019-10-14']]
Output Required:
a = [['2019-10-10','2019-10-11','2019-10-12','2019-10-14'],[2,4,1,1]]
You can use the code given below it iterates over the array and count keys in a json b
a =[['sam', '2019-10-11'], ['ram', '2019-10-12'], ['king', '2019-10-11'],
['queen', '2019-10-10'], ['ram', '2019-10-12'], ['kumar', '2019-10-11'],
['jeba', '2019-10-11'], ['sam', '2019-10-10'], ['sam', '2019-10-11'],
['howl', '2019-10-14']]
b = {}
for iter in a:
if iter[1] in b:
if(iter[0] not in b[iter[1]]):
b[iter[1]][iter[0]] = 1
else:
b[iter[1]] = {iter[0]: 1}
print(b)
the result of above code will be:
{'2019-10-11': {'sam': 1, 'king': 1, 'kumar': 1, 'jeba': 1}, '2019-10-12': {'ram': 1}, '2019-10-10': {'queen': 1, 'sam': 1}, '2019-10-14': {'howl': 1}}
Of course you can convert it into array structure like this
for key in b:
print(key, len(b[key]))
res[0].append(key)
res[1].append(len(b[key]))
the result would be
[['2019-10-11', '2019-10-12', '2019-10-10', '2019-10-14'], [4, 1, 2, 1]]
You could use defaultdict from collections like,
>>> a =[['sam', '2019-10-11'], ['ram', '2019-10-12'], ['king', '2019-10-11'],
... ['queen', '2019-10-10'], ['ram', '2019-10-12'], ['kumar', '2019-10-11'],
... ['jeba', '2019-10-11'], ['sam', '2019-10-10'], ['sam', '2019-10-11'],
... ['howl', '2019-10-14']]
>>>
>>> from collections import defaultdict
>>> d = defaultdict(set) # using `set` to not care about duplicate `name` on same `date`, otherwise use `list` instead of `set`
>>> # and use `d[date].append(name)`
>>> for name, date in a:
... d[date].add(name) # maybe, two users can be there for the same date ?
...
>>> data = [(k,len(v)) for k,v in sorted(d.items())]
>>> data
[('2019-10-10', 2), ('2019-10-11', 4), ('2019-10-12', 1), ('2019-10-14', 1)]
>>> list(zip(*data)) # returns a `list` of `tuple`s which usually should be fine for any iteration
[('2019-10-10', '2019-10-11', '2019-10-12', '2019-10-14'), (2, 4, 1, 1)]
>>> required = [list(x) for x in zip(*data)] # if you really want `list` of `list`s
>>> required
[['2019-10-10', '2019-10-11', '2019-10-12', '2019-10-14'], [2, 4, 1, 1)]]
from collections import Counter
list(zip(*Counter(list(zip(*set(zip(*zip(*a)))))[1]).items()))
Out[54]: [('2019-10-11', '2019-10-10', '2019-10-12', '2019-10-14'), (4, 2, 1, 1)]
this can be simplified as:
list(zip(*Counter([j for _, j in set([tuple(i) for i in a])]).items()))
[('2019-10-11', '2019-10-10', '2019-10-12', '2019-10-14'), (4, 2, 1, 1)]
another way to go about this without importing modules:
b = set(zip(*zip(*a)))
d = {}
for name,date in b:
d[date] = d.get(date,0)+1
list(zip(*d.items()))
[('2019-10-11', '2019-10-10', '2019-10-12', '2019-10-14'), (4, 2, 1, 1)]
if you need sorted list of lists, you could do:
[list(i) for i in zip(*sorted(d.items(),key=lambda x:x[0]))]
Out[112]: [['2019-10-10', '2019-10-11', '2019-10-12', '2019-10-14'], [2, 4, 1, 1]]
I am just starting to learn a bit of Python. I am working with some basic dictionaries. I have a dictionary, which I make a copy of. I then take another dictionary and take the values away from my copy:
teams_goals = {'chelsea' : 2, 'man u' : 3,'Arsenal':1,'Man city':2}
print teams_goals
test_teams = {'chelsea' : 1, 'man u' : 1,'Arsenal':1}
teams_goals_copy = teams_goals.copy()
for team in test_teams:
for j in range(test_teams[team]):
teams_goals_copy[team]= teams_goals_copy[team]- 1
print teams_goals_copy
This leaves me with a dictionary that has some zero values. What I want is a method of removing items from the dictionary when the they are equal to zero.
I found this previous thread here; seems like this used to work on a previous version, but I do not understand the workaround.
A dictionary can have zero values actually, if you want to remove them, just do it in your algorithm.
teams_goals = {'chelsea' : 2, 'man u' : 3,'Arsenal':1,'Man city':2}
print teams_goals
test_teams = {'chelsea' : 1, 'man u' : 1,'Arsenal':1}
teams_goals_copy = teams_goals.copy()
for team in test_teams:
for j in range(test_teams[team]):
teams_goals_copy[team]= teams_goals_copy[team]- 1
if teams_goals_copy[team] == 0:
del(teams_goals_copy[team])
print teams_goals_copy
What you need is called comprehension. It exists for sets, lists, and dictionaries.
As you see here, you can iterate on members of your set, and choose only the ones you need. The output of the command is the ones selected, and you can use it as your copy.
Copy/pasting from that page, you have:
>>> print {i : chr(65+i) for i in range(4)}
{0 : 'A', 1 : 'B', 2 : 'C', 3 : 'D'}
>>> print {k : v for k, v in someDict.iteritems()} == someDict.copy()
1
>>> print {x.lower() : 1 for x in list_of_email_addrs}
{'barry#zope.com' : 1, 'barry#python.org' : 1, 'guido#python.org' : 1}
>>> def invert(d):
... return {v : k for k, v in d.iteritems()}
...
>>> d = {0 : 'A', 1 : 'B', 2 : 'C', 3 : 'D'}
>>> print invert(d)
{'A' : 0, 'B' : 1, 'C' : 2, 'D' : 3}
>>> {(k, v): k+v for k in range(4) for v in range(4)}
... {(3, 3): 6, (3, 2): 5, (3, 1): 4, (0, 1): 1, (2, 1): 3,
(0, 2): 2, (3, 0): 3, (0, 3): 3, (1, 1): 2, (1, 0): 1,
(0, 0): 0, (1, 2): 3, (2, 0): 2, (1, 3): 4, (2, 2): 4, (
2, 3): 5}
You can read about other comprehensions (list and set) here.
Now in your case, here's a minumal example:
>>> my_dict = {'chelsea' : 2, 'man u' : 3,'Arsenal':1,'Man city':0}
>>> print(my_dict)
{'man u': 3, 'Man city': 0, 'Arsenal': 1, 'chelsea': 2}
>>> second_dict = {x:y for x,y in my_dict.items() if y > 0}
>>> print(second_dict)
{'man u': 3, 'Arsenal': 1, 'chelsea': 2}
Hello I have a counterdict that contains data like this :
{1301: Counter({'total': 18,
'inDevelopment': 13,
'isDuplicate': 2,
'inAnalysis': 2,
'inQuest': 1}),
1302: Counter({'total': 15,
'inDevelopment': 9,
'inQuest': 1,
'inValidation': 1,
'inAnalysis': 1,
'ongoing' : 3})}
How can I retrieve its values in a list without repetition.
I mean I would like to extract all the existing values, but instead of getting them all, I would like to have them NOT duplicated, so instead of this :
[' inDevelopment','isDuplicate','inAnalysis', 'inQuest','total', 'inDevelopment','inQuest', 'inValidation','inAnalysis', 'ongoing']
The output would be like this :
['total','inDevelopment','isDuplicate','inAnalysis','inQuest','inValidation','ongoing']
Any help would be appreciated, thanks!
You can union Counter objects using | operator:
>>> from collections import Counter
>>> a = Counter('123')
>>> b = Counter('44144')
>>> a
Counter({'2': 1, '3': 1, '1': 1})
>>> b
Counter({'4': 4, '1': 1})
>>> a | b
Counter({'4': 4, '2': 1, '3': 1, '1': 1})
>>> list(a | b)
['2', '3', '1', '4']
In Python 2.x
>>> from collections import Counter
>>> d = {1301: Counter({'total': 18,
...
... "ongoing" : 3})}
>>> list(reduce(lambda a,b:a|b, d.values()))
['inAnalysis', 'inQuest', 'inDevelopment', ' inDevelopment', 'inValidation', 'ongoing', 'isDuplicate', 'total']
In Python 3.x
>>> from collections import Counter
>>> from functools import reduce
>>> d = ...
>>> list(reduce(lambda a,b:a|b, d.values()))
['inValidation', 'total', ' inDevelopment', 'inDevelopment', 'isDuplicate', 'ongoing', 'inQuest', 'inAnalysis']
UPDATE
You can also use set.union:
>>> list(set().union(*d.values()))
['inValidation', 'inDevelopment', 'isDuplicate', 'total', 'ongoing', 'inAnalysis', 'inQuest', ' inDevelopment']
This work in both Python 2.x/3.x in one code.
you can use np.unique with import numpy as np and
>>> d = {1301: Counter({'total': 18,
'inDevelopment': 13,
'isDuplicate': 2, 'inAnalysis': 2,
'inQuest': 1}),
1302: Counter({'total': 15, 'inDevelopment': 9,
'inQuest': 1,
'inValidation': 1,
'inAnalysis': 1,
"ongoing" : 3})}
gives
>>> np.unique(list(d[1301]|d[1302]))
array(['inAnalysis', 'inDevelopment', 'inQuest', 'inValidation',
'isDuplicate', 'ongoing', 'total'],
dtype='|S13')
I have an array containing an even number of integers. The array represents a pairing of an identifier and a count. The tuples have already been sorted by the identifier. I would like to merge a few of these arrays together. I have thought of a few ways to do it but they are fairly complicated and I feel there might be an easy way to do this with python.
IE:
[<id>, <count>, <id>, <count>]
Input:
[14, 1, 16, 4, 153, 21]
[14, 2, 16, 3, 18, 9]
Output:
[14, 3, 16, 7, 18, 9, 153, 21]
It would be better to store these as dictionaries than as lists (not just for this purpose, but for other use cases, such as extracting the value of a single ID):
x1 = [14, 1, 16, 4, 153, 21]
x2 = [14, 2, 16, 3, 18, 9]
# turn into dictionaries (could write a function to convert)
d1 = dict([(x1[i], x1[i + 1]) for i in range(0, len(x1), 2)])
d2 = dict([(x2[i], x2[i + 1]) for i in range(0, len(x2), 2)])
print d1
# {16: 4, 153: 21, 14: 1}
After that, you could use any of the solutions in this question to add them together. For example (taken from the first answer):
import collections
def d_sum(a, b):
d = collections.defaultdict(int, a)
for k, v in b.items():
d[k] += v
return dict(d)
print d_sum(d1, d2)
# {16: 7, 153: 21, 18: 9, 14: 3}
collections.Counter() is what you need here:
In [21]: lis1=[14, 1, 16, 4, 153, 21]
In [22]: lis2=[14, 2, 16, 3, 18, 9]
In [23]: from collections import Counter
In [24]: dic1=Counter(dict(zip(lis1[0::2],lis1[1::2])))
In [25]: dic2=Counter(dict(zip(lis2[0::2],lis2[1::2])))
In [26]: dic1+dic2
Out[26]: Counter({153: 21, 18: 9, 16: 7, 14: 3})
or :
In [51]: it1=iter(lis1)
In [52]: it2=iter(lis2)
In [53]: dic1=Counter(dict((next(it1),next(it1)) for _ in xrange(len(lis1)/2)))
In [54]: dic2=Counter(dict((next(it2),next(it2)) for _ in xrange(len(lis2)/2)))
In [55]: dic1+dic2
Out[55]: Counter({153: 21, 18: 9, 16: 7, 14: 3})
Use collections.Counter:
import itertools
import collections
def grouper(n, iterable, fillvalue=None):
args = [iter(iterable)] * n
return itertools.izip_longest(fillvalue=fillvalue, *args)
count1 = collections.Counter(dict(grouper(2, lst1)))
count2 = collections.Counter(dict(grouper(2, lst2)))
result = count1 + count2
I've used the itertools library grouper recipe here to convert your data to dictionaries, but as other answers have shown you there are more ways to skin that particular cat.
result is a Counter with each id pointing to a total count:
Counter({153: 21, 18: 9, 16: 7, 14: 3})
Counters are multi-sets and will keep track of the count of each key with ease. It feels like a much better data structure for your data. They support summing, as used above, for example.
All of the previous answers look good, but I think that the JSON blob should be properly formed to begin with or else (from my experience) it can cause some serious problems down the road during debugging etc. In this case with id and count as the fields, the JSON should look like
[{"id":1, "count":10}, {"id":2, "count":10}, {"id":1, "count":5}, ...]
Properly formed JSON like that is much easier to deal with, and probably similar to what you have coming in anyway.
This class is a bit general, but certainly extensible
from itertools import groupby
class ListOfDicts():
def init_(self, listofD=None):
self.list = []
if listofD is not None:
self.list = listofD
def key_total(self, group_by_key, aggregate_key):
""" Aggregate a list of dicts by a specific key, and aggregation key"""
out_dict = {}
for k, g in groupby(self.list, key=lambda r: r[group_by_key]):
print k
total=0
for record in g:
print " ", record
total += record[aggregate_key]
out_dict[k] = total
return out_dict
if __name__ == "__main__":
z = ListOfDicts([ {'id':1, 'count':2, 'junk':2},
{'id':1, 'count':4, 'junk':2},
{'id':1, 'count':6, 'junk':2},
{'id':2, 'count':2, 'junk':2},
{'id':2, 'count':3, 'junk':2},
{'id':2, 'count':3, 'junk':2},
{'id':3, 'count':10, 'junk':2},
])
totals = z.key_total("id", "count")
print totals
Which gives
1
{'count': 2, 'junk': 2, 'id': 1}
{'count': 4, 'junk': 2, 'id': 1}
{'count': 6, 'junk': 2, 'id': 1}
2
{'count': 2, 'junk': 2, 'id': 2}
{'count': 3, 'junk': 2, 'id': 2}
{'count': 3, 'junk': 2, 'id': 2}
3
{'count': 10, 'junk': 2, 'id': 3}
{1: 12, 2: 8, 3: 10}
Is there a method of logically merging multiple dictionaries if they have common strings between them? Even if these common strings match between values of one dict() to a key of another?
I see a lot of similar questions on SO but none that seem to address my specific issue of relating multiple keys in "lower level files" to those in higher keys/values(level1dict)
Say we have:
level1dict = { '1':[1,3], '2':2 }
level2dict = { '1':4, '3':[5,9], '2':10 }
level3dict = { '1':[6,8,11], '4':12, '2':13, '3':[14,15], '5':16, '9':17, '10':[18,19,20]}
finaldict = level1dict
When I say logically I mean, in level1dict 1=1,3 and in level2dict 1=4 and 3=5,9 so overall (so far) 1 = 1,3,4,5,9 (sorting not important)
The result I would like to get to is
#.update or .append or .default?
finaldict = {'1':[1,3,4,5,9,6,8,11,12,14,15,16,17] '2':[2,10,18,19,20]}
Answered: Thank you Ashwini Chaudhary and Abhijit for the networkx module.
This is a problem of connected component subgraphs and can be best determined if you want to use networkx. Here is a solution to your problem
>>> import networkx as nx
>>> level1dict = { '1':[1,3], '2':2 }
>>> level2dict = { '1':4, '3':[5,9], '2':10 }
>>> level3dict = { '1':[6,8,11], '4':12, '2':13, '3':[14,15], '5':16, '9':17, '10':[18,19,20]}
>>> G=nx.Graph()
>>> for lvl in level:
for key, value in lvl.items():
key = int(key)
try:
for node in value:
G.add_edge(key, node)
except TypeError:
G.add_edge(key, value)
>>> for sg in nx.connected_component_subgraphs(G):
print sg.nodes()
[1, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 16, 17]
[2, 10, 13, 18, 19, 20]
>>>
Here is how you visualize it
>>> import matplotlib.pyplot as plt
>>> nx.draw(G)
>>> plt.show()
A couple of notes:
It's not convenient that some values are numbers and some are lists. Try converting numbers to 1-item lists first.
If the order is not important, you'll be better off using sets instead of lists. They have methods for all sorts of "logical" operations.
Then you can do:
In [1]: dict1 = {'1': {1, 3}, '2': {2}}
In [2]: dict2 = {'1': {4}, '2': {10}, '3': {5, 9}}
In [3]: dict3 = {'1': {6, 8, 11}, '2': {13}, '4': {12}}
In [4]: {k: set.union(*(d[k] for d in (dict1, dict2, dict3)))
for k in set.intersection(*(set(d.keys()) for d in (dict1, dict2, dict3)))}
Out[4]: {'1': set([1, 3, 4, 6, 8, 11]), '2': set([2, 10, 13])}
In [106]: level1dict = { '1':[1,3], '2':2 }
In [107]: level2dict = { '1':4, '3':[5,9], '2':10 }
In [108]: level3dict = { '1':[6,8,11], '4':12, '2':13, '3':[14,15], '5':16, '9':17, '10':[18,19,20]}
In [109]: keys=set(level2dict) & set(level1dict) & set(level3dict) #returns ['1','2']
In [110]: dic={}
In [111]: for key in keys:
dic[key]=[]
for x in (level1dict,level2dict,level3dict):
if isinstance(x[key],int):
dic[key].append(x[key])
elif isinstance(x[key],list):
dic[key].extend(x[key])
.....:
In [112]: dic
Out[112]: {'1': [1, 3, 4, 6, 8, 11], '2': [2, 10, 13]}
# now iterate over `dic` again to get the values related to the items present
# in the keys `'1'` and `'2'`.
In [122]: for x in dic:
for y in dic[x]:
for z in (level1dict,level2dict,level3dict):
if str(y) in z and str(y) not in dic:
if isinstance(z[str(y)],(int,str)):
dic[x].append(z[str(y)])
elif isinstance(z[str(y)],list):
dic[x].extend(z[str(y)])
.....:
In [123]: dic
Out[123]:
{'1': [1, 3, 4, 6, 8, 11, 5, 9, 14, 15, 12, 16, 17],
'2': [2, 10, 13, 18, 19, 20]}