Related
I have a little bit of a logic game. I have list with following dicts in each listnode.
{1: [1,{'X': -0.48595, 'Y': 0.0, 'Z': 0.56283},
2,{'X': -0.48595, 'Y': 0.0, 'Z': -0.6}],
2: [2,{'X': -0.48595, 'Y': 0.0, 'Z': -0.6},
4,{'X': 1.14756, 'Y': 0.0, 'Z': -0.6}],
3: [4,{'X': 1.14756, 'Y': 0.0, 'Z': -0.6},
9,{'X': 1.14756, 'Y': 0.0, 'Z': 0.8}]}
What I want? List with nodes of lists as below:
[{'Id': 1, 'Nodes': [{'Id': 1, 'Position': {'X': -0.48595, 'Y': 0.0, 'Z': 0.56283}},
{'Id': 2, 'Position': {'X': -0.48595, 'Y': 0.0, 'Z': -0.6}}]},
{'Id': 2, 'Nodes': [{'Id': 2, 'Position': {'X': -0.48595, 'Y': 0.0, 'Z': -0.6}},
{'Id': 4, 'Position': {'X': 1.14756, 'Y': 0.0, 'Z': -0.6}}]},
{'Id': 3, 'Nodes': [{'Id': 4, 'Position': {'X': 1.14756, 'Y': 0.0, 'Z': -0.6}},
{'Id': 9, 'Position': {'X': 1.14756, 'Y': 0.0, 'Z': 0.8}}]}]
How can I make that transformation? I'm keep trying modifying my code, but everytime something's wrong.
mem_list = []
for x in range(len(list_temp)):
mem_loc_list = []
for key in list_temp[x]:
nodes_list = []
member = {}
member['Id'] = key
node_dict = {}
for y in list_temp[x][key]:
if type(y) == int:
node_dict['Id'] = y
else:
node_dict['Position'] = y
nodes_list.append(node_dict)
member['Nodes'] = nodes_list
mem_loc_list.append(member)
Using list comprehension you can do it like this:
[[{"Id": k, "Nodes": [{"Id": v[i], "Position": v[i+1]} for i in range(0, len(v), 2)]}
for k,v in data.items()] for data in list_temp]
Say i have a list of dictionaries that all have the same keys. and i want to regroup these into several lists such that the values for certain attributes of my choosing are equal. here is an example:
Suppose i have the following list of dictionaries:
[ {'a': 0.0, 'b': 0.2, 'c': 0.1},
{'a': 0.1, 'b': 0.7, 'c': 0.2},
{'a': 0.0, 'b': 0.2, 'c': 0.3},
{'a': 0.1, 'b': 0.7, 'c': 0.4},
{'a': 0.0, 'b': 0.7, 'c': 0.5},
{'a': 0.0, 'b': 0.7, 'c': 0.6}]
and i want to cluster it according the the keys a and b. Then the output would be the following list of list of dictionaries:
[[{'a': 0.0, 'b': 0.2, 'c': 0.1},
{'a': 0.0, 'b': 0.2, 'c': 0.3}]
[{'a': 0.1, 'b': 0.7, 'c': 0.2},
{'a': 0.1, 'b': 0.7, 'c': 0.4}]
[{'a': 0.0, 'b': 0.7, 'c': 0.5},
{'a': 0.0, 'b': 0.7, 'c': 0.6}]]
What is the best way of achieving this?
Sort this firstly, then use itertools.groupby.You may could try this below:
from itertools import groupby
t = [{'a': 0.0, 'b': 0.2, 'c': 0.1},
{'a': 0.1, 'b': 0.7, 'c': 0.2},
{'a': 0.0, 'b': 0.2, 'c': 0.3},
{'a': 0.1, 'b': 0.7, 'c': 0.4},
{'a': 0.0, 'b': 0.7, 'c': 0.5},
{'a': 0.0, 'b': 0.7, 'c': 0.6}]
print([[*j] for i, j in groupby(sorted(t, key=lambda x: (x['a'], x['b'])), key=lambda x: (x['a'], x['b']))])
Result:
[[{'a': 0.0, 'b': 0.2, 'c': 0.1}, {'a': 0.0, 'b': 0.2, 'c': 0.3}], [{'a': 0.0, 'b': 0.7, 'c': 0.5}, {'a': 0.0, 'b': 0.7, 'c': 0.6}], [{'a': 0.1, 'b': 0.7, 'c': 0.2}, {'a': 0.1, 'b': 0.7, 'c': 0.4}]]
If you want to create a function to receive muitlple keys, you could try:
from itertools import groupby
def group_by(*args):
return [[*j] for i, j in groupby(sorted(t, key=itemgetter(*args)), key=itemgetter(*args))]
t = [{'a': 0.0, 'b': 0.2, 'c': 0.1},
{'a': 0.1, 'b': 0.7, 'c': 0.2},
{'a': 0.0, 'b': 0.2, 'c': 0.3},
{'a': 0.1, 'b': 0.7, 'c': 0.4},
{'a': 0.0, 'b': 0.7, 'c': 0.5},
{'a': 0.0, 'b': 0.7, 'c': 0.6}]
print(group_by('a', 'b'))
I have a list of dictionaries which have a date string within them. I would like to remove a single entry of two if there is a matching hour and minute for that record.
Here is some sample data, as you can see the first two dictionaries have 14:21 in them, I would only like one of those dictionaries and the other to be removed.
I'm not sure how to even start with this one, is it possible?
[{'x': '2018-06-19 14:21:22', 'y': 80},
{'x': '2018-06-19 14:21:26', 'y': 86},
{'x': '2018-06-19 14:24:02', 'y': 89},
{'x': '2018-06-19 14:24:07', 'y': 95},
{'x': '2018-06-19 14:25:10', 'y': 127}]
This is one approach using a simple iteration and a check list.
Demo:
checkVal = set()
data = [{'x': '2018-06-19 14:21:22', 'y': 80}, {'x': '2018-06-19 14:21:26', 'y': 86}, {'x': '2018-06-19 14:24:02', 'y': 89}, {'x': '2018-06-19 14:24:07', 'y': 95}, {'x': '2018-06-19 14:25:10', 'y': 127}, {'x': '2018-06-19 14:25:14', 'y': 138}, {'x': '2018-06-19 14:28:04', 'y': 91}, {'x': '2018-06-19 14:28:08', 'y': 83}, {'x': '2018-06-19 14:30:11', 'y': 92}, {'x': '2018-06-19 14:30:16', 'y': 99}, {'x': '2018-06-19 14:31:21', 'y': 80}, {'x': '2018-06-19 14:31:26', 'y': 90}, {'x': '2018-06-19 14:34:03', 'y': 131}, {'x': '2018-06-19 14:34:07', 'y': 137}, {'x': '2018-06-19 14:35:28', 'y': 98}, {'x': '2018-06-19 14:35:32', 'y': 91}, {'x': '2018-06-19 14:37:11', 'y': 86}, {'x': '2018-06-19 14:37:16', 'y': 92}, {'x': '2018-06-19 14:39:02', 'y': 111}, {'x': '2018-06-19 14:39:06', 'y': 118}, {'x': '2018-06-19 14:42:03', 'y': 95}, {'x': '2018-06-19 14:42:08', 'y': 104}, {'x': '2018-06-19 14:43:04', 'y': 165}, {'x': '2018-06-19 14:43:09', 'y': 168}, {'x': '2018-06-19 14:45:11', 'y': 89}, {'x': '2018-06-19 14:45:15', 'y': 94}, {'x': '2018-06-19 14:47:11', 'y': 133}, {'x': '2018-06-19 14:47:16', 'y': 146}, {'x': '2018-06-19 14:49:16', 'y': 134}, {'x': '2018-06-19 14:49:21', 'y': 146}, {'x': '2018-06-19 14:52:05', 'y': 157}, {'x': '2018-06-19 14:52:09', 'y': 169}, {'x': '2018-06-19 14:54:13', 'y': 66}, {'x': '2018-06-19 14:54:17', 'y': 63}, {'x': '2018-06-19 14:55:09', 'y': 95}, {'x': '2018-06-19 14:55:14', 'y': 90}, {'x': '2018-06-19 14:58:02', 'y': 112}, {'x': '2018-06-19 14:58:07', 'y': 119}, {'x': '2018-06-19 14:59:09', 'y': 98}, {'x': '2018-06-19 14:59:13', 'y': 91}]
res = []
for i in data:
if i["x"][:-3] not in checkVal:
res.append(i)
checkVal.add(i["x"][:-3])
print(res)
Output:
[{'y': 80, 'x': '2018-06-19 14:21:22'}, {'y': 89, 'x': '2018-06-19 14:24:02'}, {'y': 127, 'x': '2018-06-19 14:25:10'}, {'y': 91, 'x': '2018-06-19 14:28:04'}, {'y': 92, 'x': '2018-06-19 14:30:11'}, {'y': 80, 'x': '2018-06-19 14:31:21'}, {'y': 131, 'x': '2018-06-19 14:34:03'}, {'y': 98, 'x': '2018-06-19 14:35:28'}, {'y': 86, 'x': '2018-06-19 14:37:11'}, {'y': 111, 'x': '2018-06-19 14:39:02'}, {'y': 95, 'x': '2018-06-19 14:42:03'}, {'y': 165, 'x': '2018-06-19 14:43:04'}, {'y': 89, 'x': '2018-06-19 14:45:11'}, {'y': 133, 'x': '2018-06-19 14:47:11'}, {'y': 134, 'x': '2018-06-19 14:49:16'}, {'y': 157, 'x': '2018-06-19 14:52:05'}, {'y': 66, 'x': '2018-06-19 14:54:13'}, {'y': 95, 'x': '2018-06-19 14:55:09'}, {'y': 112, 'x': '2018-06-19 14:58:02'}, {'y': 98, 'x': '2018-06-19 14:59:09'}]
You already have an answer, but for a very efficient solution use the itertools unique_everseen recipe. It's also safer since it will throw a useful error if the input date isn't valid.
from datetime import datetime
from itertools import filterfalse
input_ = [{'x': '2018-06-19 14:21:22', 'y': 80},
{'x': '2018-06-19 14:21:26', 'y': 86},
{'x': '2018-06-19 14:24:02', 'y': 89},
{'x': '2018-06-19 14:24:07', 'y': 95},
{'x': '2018-06-19 14:25:10', 'y': 127}]
def unique_everseen(iterable, key=None):
"""List unique elements, preserving order. Remember all elements ever seen.
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
"""
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element
def hour_and_min(dct):
fmt = '%Y-%m-%d %H:%M:%S'
d = datetime.strptime(dct['x'], fmt)
return d.hour, d.minute # add `, d.year, d.month, d.day` if you care about these
output = list(unique_everseen(input_, key=hour_and_min))
And output is:
[{'x': '2018-06-19 14:21:22', 'y': 80},
{'x': '2018-06-19 14:24:02', 'y': 89},
{'x': '2018-06-19 14:25:10', 'y': 127}]
I have a list of dictionary items
[{'x': 0, 'y': 0}, {'x': 1, 'y': 0}, {'x': 2, 'y': 2}]
I want to have an array of "array of dictionaries" with all the maximum permutation order of the list for example for the above array it would be (3 factorial ways)
[[{'x': 0, 'y': 0}, {'x': 1, 'y': 0}, {'x': 2, 'y': 2}],
[{'x': 0, 'y': 0}, {'x': 2, 'y': 2}, {'x': 1, 'y': 0}],
[{'x': 1, 'y': 0}, {'x': 0, 'y': 0}, {'x': 2, 'y': 2}],
[{'x': 1, 'y': 0}, {'x': 2, 'y': 2}, {'x': 0, 'y': 0}],
[{'x': 2, 'y': 2}, {'x': 1, 'y': 0}, {'x': 0, 'y': 0}],
[{'x': 2, 'y': 2}, {'x': 0, 'y': 0}, {'x': 1, 'y': 0}]]
itertools can do permutations
#!python2
import itertools
yourlist = [{'x': 0, 'y': 0}, {'x': 1, 'y': 0}, {'x': 2, 'y': 2}]
for seq in itertools.permutations(yourlist):
print seq
'''
({'y': 0, 'x': 0}, {'y': 0, 'x': 1}, {'y': 2, 'x': 2})
({'y': 0, 'x': 0}, {'y': 2, 'x': 2}, {'y': 0, 'x': 1})
({'y': 0, 'x': 1}, {'y': 0, 'x': 0}, {'y': 2, 'x': 2})
({'y': 0, 'x': 1}, {'y': 2, 'x': 2}, {'y': 0, 'x': 0})
({'y': 2, 'x': 2}, {'y': 0, 'x': 0}, {'y': 0, 'x': 1})
({'y': 2, 'x': 2}, {'y': 0, 'x': 1}, {'y': 0, 'x': 0})
'''
Despite the comments, if you are still messed with how to solve your issue, consider the following.
Strategy: Make use of permutations from itertoolswhich returns a list of tuples in this case. Then, iterating through to convert list of tuples to list of lists to match with your required output.
Here is how you could do:
>>> import itertools
>>> lst = [{'x': 0, 'y': 0}, {'x': 1, 'y': 0}, {'x': 2, 'y': 2}]
>>> [list(elem) for elem in list(itertools.permutations(lst))]
[[{'x': 0, 'y': 0}, {'x': 1, 'y': 0}, {'x': 2, 'y': 2}],
[{'x': 0, 'y': 0}, {'x': 2, 'y': 2}, {'x': 1, 'y': 0}],
[{'x': 1, 'y': 0}, {'x': 0, 'y': 0}, {'x': 2, 'y': 2}],
[{'x': 1, 'y': 0}, {'x': 2, 'y': 2}, {'x': 0, 'y': 0}],
[{'x': 2, 'y': 2}, {'x': 0, 'y': 0}, {'x': 1, 'y': 0}],
[{'x': 2, 'y': 2}, {'x': 1, 'y': 0}, {'x': 0, 'y': 0}]]
I have a list containing dictionaries:
[{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb4340', 'y': u'osgb4000'},
{'x': u'osgb4020', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'}]
I wish to count the incidents of each dict and create a new field count
The desired outcome looks like this:
[{'x': u'osgb32', 'y': u'osgb4000', 'count': 3},
{'x': u'osgb4340', 'y': u'osgb4000', 'count': 1},
{'x': u'osgb4020', 'y': u'osgb4000', 'count': 1}]
I am unsure how to match dicts.
This is a job for collections.Counter. But first you have to convert your dicts to actual tuples, as dicts are not hashable and thus can not be used as keys in a Counter object:
>>> dicts = [{'x': u'osgb32', 'y': u'osgb4000'},
... {'x': u'osgb4340', 'y': u'osgb4000'},
... {'x': u'osgb4020', 'y': u'osgb4000'},
... {'x': u'osgb32', 'y': u'osgb4000'},
... {'x': u'osgb32', 'y': u'osgb4000'}]
>>> collections.Counter(tuple(d.items()) for d in dicts)
Counter({(('y', u'osgb4000'), ('x', u'osgb32')): 3,
(('y', u'osgb4000'), ('x', u'osgb4020')): 1,
(('y', u'osgb4000'), ('x', u'osgb4340')): 1})
Then, you can turn those back into dicts with the added "count" key:
>>> c = collections.Counter(tuple(d.items()) for d in dicts)
>>> [dict(list(k) + [("count", c[k])]) for k in c]
[{'count': 1, 'x': u'osgb4020', 'y': u'osgb4000'},
{'count': 3, 'x': u'osgb32', 'y': u'osgb4000'},
{'count': 1, 'x': u'osgb4340', 'y': u'osgb4000'}]
You can use Counter and frozenset for this:
from collections import Counter
l = [{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb4340', 'y': u'osgb4000'},
{'x': u'osgb4020', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'}]
c = Counter(frozenset(d.items()) for d in l)
[dict(k, count=v) for k, v in c.items()] # [{'y': u'osgb4000', 'x': u'osgb4340', 'count': 1}, {'y': u'osgb4000', 'x': u'osgb32', 'count': 3}, {'y': u'osgb4000', 'x': u'osgb4020', 'count': 1}]
You can achieve that easily with code below
items = [{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb4340', 'y': u'osgb4000'},
{'x': u'osgb4020', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'}]
result = {}
counted_items = []
for item in items:
key = item['x'] + '_' + item['y']
result[key] = result.get(key, 0) + 1
for key, value in result.iteritems():
y, x = key.split('_')
counted_items.append({'x': x, 'y': y, 'count': value})
print counted_items # [{'y': u'osgb32', 'x': u'osgb4000', 'count': 3}, {'y': u'osgb4340', 'x': u'osgb4000', 'count': 1}, {'y': u'osgb4020', 'x': u'osgb4000', 'count': 1}]
Another option is to use counter. There are plenty of answers of how to dial with collections.Counter :)
Good Luck!
You can pass your list of dicts as the data arg to DataFrame ctor:
In [74]:
import pandas as pd
data = [{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb4340', 'y': u'osgb4000'},
{'x': u'osgb4020', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'},
{'x': u'osgb32', 'y': u'osgb4000'}]
df = pd.DataFrame(data)
df
Out[74]:
x y
0 osgb32 osgb4000
1 osgb4340 osgb4000
2 osgb4020 osgb4000
3 osgb32 osgb4000
4 osgb32 osgb4000
you can then groubpy on the cols and call size to get a count:
In [76]:
df.groupby(['x','y']).size()
Out[76]:
x y
osgb32 osgb4000 3
osgb4020 osgb4000 1
osgb4340 osgb4000 1
dtype: int64
and then call to_dict:
In [77]:
df.groupby(['x','y']).size().to_dict()
Out[77]:
{('osgb32', 'osgb4000'): 3,
('osgb4020', 'osgb4000'): 1,
('osgb4340', 'osgb4000'): 1}
You can wrap the above into a list:
In [79]:
[df.groupby(['x','y']).size().to_dict()]
Out[79]:
[{('osgb32', 'osgb4000'): 3,
('osgb4020', 'osgb4000'): 1,
('osgb4340', 'osgb4000'): 1}]
You can reset_index, rename the column and pass arg orient='records':
In [94]:
df.groupby(['x','y']).size().reset_index().rename(columns={0:'count'}).to_dict(orient='records')
Out[94]:
[{'count': 3, 'x': 'osgb32', 'y': 'osgb4000'},
{'count': 1, 'x': 'osgb4020', 'y': 'osgb4000'},
{'count': 1, 'x': 'osgb4340', 'y': 'osgb4000'}]