Given a list of dictionaries:
data = {
"data": [
{
"categoryOptionCombo": {
"id": "A"
},
"dataElement": {
"id": "123"
}
},
{
"categoryOptionCombo": {
"id": "B"
},
"dataElement": {
"id": "123"
}
},
{
"categoryOptionCombo": {
"id": "C"
},
"dataElement": {
"id": "456"
}
}
]
}
I would like to display the dataElement where the count of distinct categoryOptionCombo is larger than 1.
e.g. the result of the function would be an iterable of IDs:
[123]
because the dataElement with id 123 has two different categoryOptionCombos.
tracker = {}
for d in data['data']:
data_element = d['dataElement']['id']
coc = d['categoryOptionCombo']['id']
if data_element not in tracker:
tracker[data_element] = set()
tracker[data_element].add(coc)
too_many = [key for key,value in tracker.items() if len(value) > 1]
How can I iterate the list of dictionaries preferably with a comprehension? This solution above is not pythonic.
One approach:
import collections
counts = collections.defaultdict(set)
for d in data["data"]:
counts[d["dataElement"]["id"]].add(d["categoryOptionCombo"]["id"])
res = [k for k, v in counts.items() if len(v) > 1]
print(res)
Output
['123']
This approach creates a dictionary mapping dataElements to the different types of categoryOptionCombo:
defaultdict(<class 'set'>, {'123': {'B', 'A'}, '456': {'C'}})
Almost a one-liner:
counts = collections.Counter( d['dataElement']['id'] for d in data['data'] )
print( counts )
Output:
Counter({'123': 2, '456': 1})
No need for sets, you can just remember each data element's first coc or mark it as having 'multiple'.
tracker = {}
for d in data['data']:
data_element = d['dataElement']['id']
coc = d['categoryOptionCombo']['id']
if tracker.setdefault(data_element, coc) != coc:
tracker[data_element] = 'multiple'
too_many = [key for key,value in tracker.items() if value == 'multiple']
(If the string 'multiple' can be a coc id, then use multiple = object() and compare with is).
I am trying to update transaction ID from the following json:
{
"locationId": "5115",
"transactions": [
{
"transactionId": "1603804404-5650",
"source": "WEB"
} ]
I have done following code for the same, but it does not update the transaction id, but it inserts the transaction id to the end of block:-
try:
session = requests.Session()
with open(
"sales.json",
"r") as read_file:
payload = json.load(read_file)
payload["transactionId"] = random.randint(0, 5)
with open(
"sales.json",
"w") as read_file:
json.dump(payload, read_file)
Output:-
{
"locationId": "5115",
"transactions": [
{
"transactionId": "1603804404-5650",
"source": "WEB"
} ]
}
'transactionId': 1
}
Expected Outut:-
{
"locationId": "5115",
"transactions": [
{
"transactionId": "1",
"source": "WEB"
} ]
This would do it, but only in your specific case:
payload["transactions"][0]["transactionId"] = xxx
There should be error handling for cases like "transactions" key is not int the dict, or there are no records or there are more than one
also, you will need to assign =str(your_random_number) not the int if you wish to have the record of type string as the desired output suggests
If you just want to find the transactionId key and you don't know exactly where it may exist. You can do-
from collections.abc import Mapping
def update_key(key, new_value, jsondict):
new_dict = {}
for k, v in jsondict.items():
if isinstance(v, Mapping):
# Recursive traverse if value is a dict
new_dict[k] = update_key(key, new_value, v)
elif isinstance(v, list):
# Traverse through all values of list
# Recursively traverse if an element is a dict
new_dict[k] = [update_key(key, new_value, innerv) if isinstance(innerv, Mapping) else innerv for innerv in v]
elif k == key:
# This is the key to replace with new value
new_dict[k] = new_value
else:
# Just a regular value, assign to new dict
new_dict[k] = v
return new_dict
Given a dict-
{
"locationId": "5115",
"transactions": [
{
"transactionId": "1603804404-5650",
"source": "WEB"
} ]
}
You can do-
>>> update_key('transactionId', 5, d)
{'locationId': '5115', 'transactions': [{'transactionId': 5, 'source': 'WEB'}]}
Yes because transactionId is inside transactions node. So your code should be like:
payload["transactions"][0].transactionId = random.randint(0, 5)
or
payload["transactions"][0]["transactionId"] = random.randint(0, 5)
The required output of my nested loops is json, how to get there?
The input list structure looks like list = [[name, version, id],[name, version, id], ...]
list_1 = [
['mipl-abnd','v1.0.2','eelp234'],
['mipl-avfd','v1.1.5','32fv423'],
['mipl-awsd','v9.0.2','234eelp'],
['mipl-tgfd','v3.0.0','124fdge'],
['mipl-hrdss','v1.0.2','543rfd3'],
['mipl-oldss','v1.0.2','eelp234']]
list_2 = [
['mipl-abnd','v1.0.2','eelp234'],
['mipl-avfd','v1.1.6','3254323'],
['mipl-awsd','v9.0.2','234eelp'],
['mipl-tgfd','v3.0.0','124fdge'],
['mipl-hrdss','v1.0.2','543rfd3'],
['mipl-newss','v1.0.2','eelp234']]
This is the code I used to get a final list:
def get_difference(l1,l2):
l1 = get_ordered_list(file1.read())
l2 = get_ordered_list(file2.read())
d1 = {k:[v1,v2] for k,v1,v2 in l1}
d2 = {k:[v1,v2] for k,v1,v2 in l2}
result = []
for k,v in d2.items():
if k in d1:
v1 = d1[k]
if v1[0] != v[0]:
result.append({k,v1[0],v[0], v1[1],v[1]})
else:
result.append({k,'new',v[0],'new', v[1]})
for k,v in d1.items():
if k not in d2:
result.append({k,v[0],'deprecated', v[1], 'deprecated'})
res_json = json.dumps(result)
return res_json
Current Output :
result = [['mipl-avfd', 'v1.1.5', 'v1.1.6','32fv423', '3254323'], ['mipl-oldss','v1.0.2', 'deprecated','eelp234', 'deprecated'], ['mipl-newss', 'new','v1.0.2','new', 'eelp234']]
Required Output(I want to write it to an easily readable JSON which can be later made into a table) :
{diff = {"name" : "mipl-avfd",
"old-version" : "v1.1.5",
"new-version" : "v1.1.6",
"old-id" : "32fv423",
"new-id" : "3254323"
},
{"name" : "mipl-oldss",
"old-version" : "v1.0.2",
"new-version" : "deprecated",
"old-id" : "eelp234",
"new-id" : "deprecated"
},
{"name" : "mipl-newss",
"old-version" : "new",
"new-version" : "v1.0.2",
"old-id" : "eelp234",
"new-id" : "new"
}
}
I hope I understand your question right. You have "old" list_1 and "new" list_2 and you want to construct flat list how the versions change (I assume, in list_1 you have old versions):
import json
from itertools import groupby
list_1 = [
['mipl-abnd','v1.0.2','eelp234'],
['mipl-avfd','v1.1.5','32fv423'],
['mipl-awsd','v9.0.2','234eelp'],
['mipl-tgfd','v3.0.0','124fdge'],
['mipl-hrdss','v1.0.2','543rfd3'],
['mipl-oldss','v1.0.2','eelp234']]
list_2 = [
['mipl-abnd','v1.0.2','eelp234'],
['mipl-avfd','v1.1.6','3254323'],
['mipl-awsd','v9.0.2','234eelp'],
['mipl-tgfd','v3.0.0','124fdge'],
['mipl-hrdss','v1.0.2','543rfd3'],
['mipl-newss','v1.0.2','eelp234']]
s = sorted(list_1 + list_2, key=lambda k: k[0])
out = []
for v, g in groupby(s, lambda k: k[0]):
g = list(g)
if len(g) == 2:
out.append({
'name': v,
'old-version': g[0][1],
'new-version': g[1][1],
'old-id': g[0][2],
'new-id': g[1][2],
})
else:
if g[0] in list_1:
out.append({
'name': v,
'old-version': g[0][1],
'new-version': 'deprecated',
'old-id': g[0][2],
'new-id': 'deprecated',
})
else:
out.append({
'name': v,
'old-version': 'new',
'new-version': g[0][1],
'old-id': 'new',
'new-id': g[0][2],
})
print(json.dumps(out, indent=4))
Prints:
[
{
"name": "mipl-abnd",
"old-version": "v1.0.2",
"new-version": "v1.0.2",
"old-id": "eelp234",
"new-id": "eelp234"
},
{
"name": "mipl-avfd",
"old-version": "v1.1.5",
"new-version": "v1.1.6",
"old-id": "32fv423",
"new-id": "3254323"
},
{
"name": "mipl-awsd",
"old-version": "v9.0.2",
"new-version": "v9.0.2",
"old-id": "234eelp",
"new-id": "234eelp"
},
{
"name": "mipl-hrdss",
"old-version": "v1.0.2",
"new-version": "v1.0.2",
"old-id": "543rfd3",
"new-id": "543rfd3"
},
{
"name": "mipl-newss",
"old-version": "new",
"new-version": "v1.0.2",
"old-id": "new",
"new-id": "eelp234"
},
{
"name": "mipl-oldss",
"old-version": "v1.0.2",
"new-version": "deprecated",
"old-id": "eelp234",
"new-id": "deprecated"
},
{
"name": "mipl-tgfd",
"old-version": "v3.0.0",
"new-version": "v3.0.0",
"old-id": "124fdge",
"new-id": "124fdge"
}
]
What you said is a json is not a valid json. Also, json is a string - you want a dict structure. You don't have to dump it into a json string.
Why do you give l1 and l2 as arguments to the function when you overwrite them in the first lines?
file1 and file2 are not defined in the function. Also, for reading files you should use with to properly close the file.
First, you need to declare the keys (labels) somewhere:
keys = ["name", "old-version", "old-id", "new-id"]
Then, instead of appending a list, you append a dict.
Thankfully, dicts can be easily made from lists of tuples - and we can merge keys and your current lists into lists of tuples easily, e.g.:
dict(zip(keys, [k,v1[0],v[0], v1[1],v[1]]))
So it now looks like this:
for k,v in d2.items():
if k in d1:
v1 = d1[k]
if v1[0] != v[0]:
result.append(dict(zip(keys, [k,v1[0],v[0], v1[1],v[1]])))
else:
result.append(dict(zip(keys, [k,'new',v[0],'new', v[1]])))
for k,v in d1.items():
if k not in d2:
result.append(dict(zip(keys, [k,v[0],'deprecated', v[1], 'deprecated'])))
Assume that I have a pandas dataframe called df similar to:
source tables
src1 table1
src1 table2
src1 table3
src2 table1
src2 table2
I'm currently able to output a JSON file that iterates through the various sources, creating an object for each, with the code below:
all_data = []
for src in df['source']:
source_data = {
src: {
}
}
all_data.append(source_data)
with open('data.json', 'w') as f:
json.dump(all_data, f, indent = 2)
This yields the following output:
[
{
"src1": {}
},
{
"src2": {}
}
]
Essentially, what I want to do is also iterate through those list of sources and add the table objects corresponding to each source respectively. My desired output would look similar to as follows:
[
{
"src1": {
"table1": {},
"table2": {},
"table3": {}
}
},
{
"src2": {
"table1": {},
"table2": {}
}
}
]
Any assistance on how I can modify my code to also iterate through the tables column and append that to the respective source values would be greatly appreciated. Thanks in advance.
Is this what you're looking for?
data = [
{k: v}
for k, v in df.groupby('source')['tables'].agg(
lambda x: {v: {} for v in x}).items()
]
with open('data.json', 'w') as f:
json.dump(data, f, indent=2)
There are two layers to the answer here. To group the tables by source, use groupby first with an inner comprehension. You can use a list comprehension to assemble your data in this specific format overall.
[
{
"src1": {
"table1": {},
"table2": {},
"table3": {}
}
},
{
"src2": {
"table1": {},
"table2": {}
}
}
]
Example using .apply with arbitrary data
df['tables2'] = 'abc'
def func(g):
return {x: y for x, y in zip(g['tables'], g['tables2'])}
data = [{k: v} for k, v in df.groupby('source').apply(func).items()]
data
# [{'src1': {'table1': 'abc', 'table2': 'abc', 'table3': 'abc'}},
# {'src2': {'table1': 'abc', 'table2': 'abc'}}]
Note that this will not work with pandas 1.0 (probably because of a bug)
can someone help me learn how to loop through the items listed below?
I am getting a dict inside a list.
Trying to learn how to get each item by itself
This is the results that I get when I connect:
[
{
"Index": "NASDAQ",
"ExtHrsChange": "-0.22",
"LastTradePrice": "972.92",
"LastTradeWithCurrency": "972.92",
}
]
Current code:
for line in quotes:
(key, value) = line.split()
if "LastTradePrice" in key:
print key
You can use a for loop to take every dictionary in the list and then use the builtin items() method to split the keys and values from the dictionary:
l = [
{
"Index": "NASDAQ",
"ExtHrsChange": "-0.22",
"LastTradePrice": "972.92",
"LastTradeWithCurrency": "972.92",
}
]
for i in l:
if "LastTradePrice" in i:
for a, b in i.items():
print a, b
Your data looks to be a string in json format except for an extra comma at the end of the dictionary list (maybe you manually typed this?). Use the json module to parse it and then iterate over the list of dictionaries:
raw_data = '''\
[
{
"Index": "NASDAQ",
"ExtHrsChange": "-0.22",
"LastTradePrice": "972.92",
"LastTradeWithCurrency": "972.92"
}
]'''
import json
data = json.loads(raw_data)
for item in data:
for key,value in item.items():
print(key,value)
Index NASDAQ
ExtHrsChange -0.22
LastTradeWithCurrency 972.92
LastTradePrice 972.92
For Python2 try iteritems():
dict_list = [
{
"Index": "NASDAQ",
"ExtHrsChange": "-0.22",
"LastTradePrice": "972.92",
"LastTradeWithCurrency": "972.92"
}
]
for entry in dict_list:
for key, value in entry.iteritems():
print key, value