Trying to parse a Json structure in python and Adding a new value with key 'cat':
data = []
for x in a:
for y in x['Hp'].values():
for z in y:
for k in z['abc']['xyz']:
for m in data:
det = m['response']
// Some processing with det whose output is stored in s
k['cat'] = s
print x
However when x is print only the last value is being appended onto the whole dictionary, wheras there are different values for s.
Its obvious that the 'cat' key is being overwritten everytime the loop rounds,but can't find a way to make it right
Below is a sample Json structure:
{
"_id" : ObjectId("asdasda156121s"),
"Hp" : {
"bermud" : [
{
"abc" : {
"gfh" : 1,
"fgh" : 0.0,
"xyz" : [
{
"kjl" : "0",
"bnv" : 0,
}
],
"xvc" : "bv",
"hgth" : "INnn",
"sdf" : 0,
}
}
},
{
"abc" : {
"gfh" : 1,
"fgh" : 0.0,
"xyz" : [
{
"kjl" : "0",
"bnv" : 0,
}
],
"xvc" : "bv",
"hgth" : "INnn",
"sdf" : 0,
}
}
},
..
If you want to store all values change
k['cat'] = s
to
if 'cat' in k.keys():
k['cat'] += s
else:
k['cat'] = s
If you want to store only the first one change
k['cat'] = s
to
if 'cat' not in k.keys():
k['cat'] = s
Related
Here is a part of my Jason file, and I want to read "information" under "runs" -> "results" -> "properties"
I am trying the following:
with open(inputFile, "r") as readFile:
data = json.load(readFile)
print(type(data))
print("Run data type is: ",type(data['runs']))
#print("properties data type is: ", type(data['runs']['properties']))
# error: print("results data type is: ", type(data['runs']['properties']))TypeError: list indices must be integers or slices, not str
for info in data['runs']:
res = info.get('results',{})
#res = info.get('results', {}).get('properties', None)
#Error: AttributeError: 'list' object has no attribute 'get'
#inf = info.get('properties')
print(res)
All the parts that I have commented is not working. and I added also the error message
how can i read "information" in a loop?
{
"$schema" : "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.4.json",
"version" : "2.1.0",
"runs" : [ {
"tool" : { ...},
"artifacts" : [ ...],
"results" : [ {
"ruleId" : "DECL_MISMATCH",
"ruleIndex" : 0,
"message" : {
"text" : "XXXXX"
},
"level" : "error",
"baselineState" : "unchanged",
"rank" : 100,
"kind" : "fail",
"properties" : {
"tags" : [ "databaseId", "metaFamily", "family", "group", "information", "severity", "status", "comment", "justified", "assignedTo", "ticketKey", "color" ],
"databaseId" : 54496,
"metaFamily" : "Defect",
"family" : "Defect",
"group" : "Programming",
"information" : "Impact: High",
"severity" : "Unset",
"status" : "Unreviewed",
"comment" : "",
"justified" : false,
"color" : "RED"
},
"locations" : [ {
"physicalLocation" : {
"artifactLocation" : {
"index" : 0
}
},
"logicalLocations" : [ {
"fullyQualifiedName" : "File Scope",
"kind" : "function"
} ]
} ]
} ]
} ]
}
While you're trying to access the key properties which is inside a list, you have to set the index number. In this json you've posted the index number can be 0. So the code probably should be like this:
with open(inputFile, "r") as readFile:
data = json.load(readFile)
print(type(data))
print("Run data type is: ",type(data['runs']))
#print("properties data type is: ", type(data['runs']['properties']))
# error: print("results data type is: ", type(data['runs']['properties']))TypeError: list indices must be integers or slices, not str
for info in data['runs']:
# res = info.get('results',{})
res = info.get('results', {})[0].get('properties', None)
#inf = info.get('properties')
print(res)
for run in data['runs']:
for result in run['results']:
properties = result['properties']
print("information = {}".format(properties['information']))
I am creating a JSON file of a nested dictionary. My code is currently as follows:
myfamily = {
"child1" : {
"name" : "Emil"
},
"child2" : {
"name" : "Tobias"
},
"child3" : {
"name" : "Linus"
}
}
names = []
for i in myfamily.values():
print(type(i))
print(i)
s = json.dumps(i)
names.append(s)
df_family = pd.DataFrame()
df_family['Child'] = myfamily.keys()
df_family['Name'] = values
text = df_family.to_json(orient='records')
print(text)
This leads to the following output:
[{"Child":"child1","Name":"{\"2022\": 50, \"2023\": 50, \"2024\": 0}"},{"Child":"child2","Name":"{\"2022\": 50, \"2023\": 50, \"2024\": 50}"},{"Child":"child3","Name":"{\"2022\": 0, \"2023\": 100, \"2024\": 0}"}]
So my question is, why are these slashes added and is this the correct way to create a JSON text format of a nested dictionary?
import json
myfamily = {
"child1" : {
"name" : "Emil"
},
"child2" : {
"name" : "Tobias"
},
"child3" : {
"name" : "Linus"
}
}
def nested_json(dict_t,fist_key="Child"):
list_t= []
for key,val in myfamily.items():
nested_key = next(iter( val.keys()))
list_t+= [{
fist_key:key,
nested_key:val[nested_key]
}]
return json.dumps(list_t)
nested_json(myfamily)
I am trying to iterate through a nested dictionary. I want to display all the values associated with the key: "city_name". This is my piece of code.
nested_dictionary = {"responseCode": 0,
"responseDesc": [
{
"city_id": 1,
"city_name": "Mumbai",
"total_trips": 0
},
{
"city_id": 2,
"city_name": "Delhi",
"total_trips": 0
}
]
}
temp = "city_name"
for i in nested_dictionary.keys():
print(i)
if i == "responseDesc":
x = [v[temp] for k, v in nested_dictionary.items() if temp in v]
print("The extracted values : " + str(x))
Each time i try to run it throws the type error. I am not able to figure out where is the value of x becoming an integer?
Any help would be appreciated. Thanks in advance.
You are trying to access to whole dict instead of included list of cities "responseDesc"
I think you was going to get somethinkg like this:
nested_dictionary = {"responseCode": 0,
"responseDesc": [
{
"city_id": 1,
"city_name": "Mumbai",
"total_trips": 0
},
{
"city_id": 2,
"city_name": "Delhi",
"total_trips": 0
}
]
}
temp = "city_name"
x = [desc[temp] for desc in nested_dictionary['responseDesc'] if temp in desc]
print("The extracted values : " + str(x))
This is what you need. You need to fetch the value using the key.
x = [v[temp] for v in nested_dictionary[i] if temp in v]
print("The extracted values : " + str(x))
how i make a list from the values inside urban only for type gasolina?
{ ... "fuelUse" : {
"urban" : [
{
"value" : 6.2,
"unit" : "km/l",
"type" : "alcool"
},
{
"value" : 8.9,
"unit" : "km/l",
"type" : "gasolina"
}
],
},
...."fuelUse" : {
"urban" : [
{
"value" : 7.8,
"unit" : "km/l",
"type" : "alcool"
},
{
"value" : 10.4,
"unit" : "km/l",
"type" : "gasolina"
}
],
}
}
the output like: list = [ 8.9 , 10.4 ]
i tried to iterate in that way, but hav key error: 1
for c in cars:
for a in c['fuelUse']['urban']:
list.append(a[1]['value'])
try
list.append(a['value'])
instead of
list.append(a[1]['value'])
Since a is not a list, it is a single object, there is no need for further indexing.
If you would like the value of the second element, which type is gasolina, from each urban, you should loop through them, not the object's inside.
for c in cars:
for a in c['fuelUse']['urban']:
if a['type'] == 'gasolina':
list.append(a['value'])
I am not quite sure as you did not provide the entire data structure but according to your try it could be like this:
output = [x.get("value") for car in cars for x in car.get("fuelUse").get("urban") if x.get("type") == "gasolina"]
I have a mongo collection, i want get total value of 'number_of_ad_clicks' by given sitename, timestamp and variant id. Because we have large data so it would be better use map/reduce. Could any guys give me any suggestion?
Here is my collection json format
{ "_id" : ObjectId( "4e3c280ecacbd1333b00f5ff" ),
"timestamp" : "20110805",
"variants" : { "94" : { "number_of_ad_clicks" : 41,
"number_of_search_keywords" : 9,
"total_duration" : 0,
"os" : { "os_2" : 2,
"os_1" : 1,
"os_0" : 0 },
"countries" : { "ge" : 6,
"ca" : 1,
"fr" : 8,
"uk" : 4,
"us" : 6 },
"screen_resolutions" : { "(320, 240)" : 1,
"(640, 480)" : 5,
"(1024, 960)" : 5,
"(1280, 768)" : 5 },
"widgets" : { "widget_1" : 1,
"widget_0" : 0 },
"languages" : { "ua_uk" : 8,
"ca_en" : 2,
"ca_fr" : 2,
"us_en" : 5 },
"search_keywords" : { "search_keyword_8" : 8,
"search_keyword_5" : 5,
"search_keyword_4" : 4,
"search_keyword_7" : 7,
"search_keyword_6" : 6,
"search_keyword_1" : 1,
"search_keyword_3" : 3,
"search_keyword_2" : 2 },
"number_of_pageviews" : 18,
"browsers" : { "browser_4" : 4,
"browser_0" : 0,
"browser_1" : 1,
"browser_2" : 2,
"browser_3" : 3 },
"keywords" : { "keyword_5" : 5,
"keyword_4" : 4,
"keyword_1" : 1,
"keyword_0" : 0,
"keyword_3" : 3,
"keyword_2" : 2 },
"number_of_keyword_clicks" : 83,
"number_of_visits" : 96 } },
"site_name" : "fonter.com",
"number_of_variants" : 1 }
Here is my try. but failed.
He is my try.
m = function() {
emit(this.query, {variants: this.variants});
}
r = function(key , vals) {
var clicks = 0 ;
for(var i = 0; i < vals.length(); i++){
clicks = vals[i]['number_of_ad_clicks'];
}
return clicks;
}
res = db.variant_daily_collection.mapReduce(m, r, {out : "myoutput", "query":{"site_name": 'fonter.com', 'timestamp': '20110805'}})
db.myoutput.find()
could somebody any suggestion?
Thank you very much, i try you solution but nothing return.
I invoke the mapreduce in the following, is there any thing wrong?
res = db.variant_daily_collection.mapReduce(map, reduce, {out : "myoutput", "query":{"site_name": 'facee.com', 'timestamp': '20110809', 'variant_id': '305'}})
db.myoutput.find()
The emit function emits both a key and a value.
If you are used to SQL think of key as your GROUP BY and value as your SUM(), AVG(), etc..
In your case you want to "group by": site_name, timestamp and variant id. It looks like you may have more than one variant, so you will need to loop through the variants, like this:
map = function() {
for(var i in variants){
var key = {};
key.timestamp = this.timestamp;
key.site_name = this.site_name;
key.variant_id = i; // that's the "94" string.
var value = {};
value.clicks = this.variants[i].number_of_ad_clicks;
emit(key, value);
}
}
The reduce function will get an array of values each one like this { clicks: 41 }. The function needs to return one object that looks the same.
So if you get values = [ {clicks:21}, {clicks:10}, {clicks:5} ] you must output {clicks:36}.
So you do something like this:
reduce = function(key , vals) {
var returnValue = { clicks: 0 }; // initializing to zero
for(var i = 0; i < vals.length(); i++){
returnValue.clicks += vals[i].clicks;
}
return returnValue;
}
Note that the value from map has the same shape as the return from reduce.