Search for value in json file to get number (python) - python

{
"meta": {
"code": 200
},
"response": {
"holidays": [
{
"name": "New Year's Day",
"description": "New Year\u2019s Day is celebrated many countries such as in India on the January 1 in the Gregorian calendar.",
"country": {
"id": "in",
"name": "India"
},
"date": {
"iso": "2021-01-01",
"datetime": {
"year": 2021,
"month": 1,
"day": 1
}
},
"type": [
"Optional holiday"
],
"locations": "All",
"states": "All"
},
{
"name": "Lohri",
"description": "Lohri is a restricted holiday in India",
"country": {
"id": "in",
"name": "India"
},
"date": {
"iso": "2021-01-13",
"datetime": {
"year": 2021,
"month": 1,
"day": 13
}
},
"type": [
"National holiday"
],
"locations": "All",
"states": "All"
}
]
}
}
This is my json file saved as dates.json.
I want to search for New Year's day value in the key name and then get the value of the iso key.
How can i do that?
I am a beginner in python and json. so please tell me how to do this
Is there any search algorith or a library that can help me?

One way to do it is using the modules json and jsonpath-rw-ext.
Use json module to read the json file and jsonpath-rw-ext to parse/filter.
https://github.com/sileht/python-jsonpath-rw-ext
I made it work for your example. Take a look at this.
#!/usr/bin/env python3
import json
import jsonpath_rw_ext
with open('dates.json') as json_file:
data = json.load(json_file)
result = jsonpath_rw_ext.parse("$..holidays[?(#.name=='New Year\\'s Day')]").find(data)
print([x.value for x in result])
To get the iso value use the code below.
#!/usr/bin/env python3
import json
import jsonpath_rw_ext
with open('dates.json') as json_file:
data = json.load(json_file)
result = jsonpath_rw_ext.parse("$..holidays[?(#.name=='New Year\\'s Day')].date.iso").find(data)
print(result[0].value)

Related

filter data based on a condition in json

I am working on a requirement where I need to filter data if a condition is satisfied from json data into a data frame in python. When I use the below code I run into following error. I am trying to filter data based on a random condition here, I am checking if the Country Code is US then I need data frame to be populated with all the records where country is USA.
Code:
import json
data = {
"demographic": [
{
"id": 1,
"country": {
"code": "AU",
"name": "Australia"
},
"state": {
"name": "New South Wales"
},
"location": {
"time_zone": {
"name": "(UTC+10:00) Canberra, Melbourne, Sydney",
"standard_name": "AUS Eastern Standard Time",
"symbol": "AUS Eastern Standard Time"
}
},
"address_info": {
"address_1": "",
"address_2": "",
"city": "",
"zip_code": ""
}
},
{
"id": 2,
"country": {
"code": "AU",
"name": "Australia"
},
"state": {
"name": "New South Wales"
},
"location": {
"time_zone": {
"name": "(UTC+10:00) Canberra, Melbourne, Sydney",
"standard_name": "AUS Eastern Standard Time",
"symbol": "AUS Eastern Standard Time"
}
},
"address_info": {
"address_1": "",
"address_2": "",
"city": "",
"zip_code": ""
}
},
{
"id": 3,
"country": {
"code": "US",
"name": "United States"
},
"state": {
"name": "Illinois"
},
"location": {
"time_zone": {
"name": "(UTC-06:00) Central Time (US & Canada)",
"standard_name": "Central Standard Time",
"symbol": "Central Standard Time"
}
},
"address_info": {
"address_1": "",
"address_2": "",
"city": "",
"zip_code": "60611"
}
}
]
}
jd = json.loads(data)
df = [cnt for cnt in jd["demographic"] if cnt["country"]["code"] == "US"]
print(df)
Error:
TypeError: string indices must be integers
You don't need to parse a json string into python dict, cause data var is already a python dict!
Remove this line
jd = json.loads(data)
This is your code:
df = [cnt for cnt in data["demographic"] if cnt["country"]["code"] == "US"]
print(df)

Nested json files - Python

Good afternoon all,
I've been reading through the various posts regarding reading .json files using pandas but so far I've not been sucessful extract.
I need to read a specific 'score' in the json file of which I'll then iterate through all the json files I have as the label would be the same.
In the below how would I read the 'score'? I've tried using the normalise function but regardless of the agruement I put in I cannot get any closer.
Part of the json file:
"template_id": "template_fe61177cb0eb4642901b1eae9488fbb4",
"audit_id": "audit_1a0e9ef4a7914286808accb3dcb0700b",
"archived": false,
"created_at": "2022-10-07T08:00:14.021Z",
"modified_at": "2022-10-07T08:05:56.594Z",
"audit_data": {
"score": 10,
"total_score": 11,
"score_percentage": 90.909,
"name": "7 Oct 2022 / Test",
"duration": 240,
"authorship": {
"device_id": "user_65c3799b0f1a48549cacbceca244e1db",
"owner": "test",
"owner_id": "user_65c3799b0f1a48549cacbceca244e1db",
"author": "test",
"author_id": "user_65c3799b0f1a48549cacbceca244e1db"
},
"date_completed": "2022-10-07T08:05:55.860Z",
"date_modified": "2022-10-07T08:05:56.594Z",
"date_started": "2022-10-07T08:00:13.000Z",
"site": {
"name": "Blue Warehouse"
}
},
"template_data": {
"authorship": {
"device_id": "user_4bb896b5308341f7a7543a32f6c1f3ec",
"owner": "test",
"owner_id": "user_4bb896b5308341f7a7543a32f6c1f3ec",
"author": "test",
"author_id": "user_4bb896b5308341f7a7543a32f6c1f3ec"
},
"metadata": {
"description": "",
"name": "RCS",
"image": {
"date_created": "2022-04-12T13:27:18.852Z",
"file_ext": "png",
"label": "Go \u0026 See icon.PNG",
"media_id": "cf944a4b-7589-47e6-b42a-8d17f06b7031",
"href": "https://1"
}
},
"response_sets": {
"5b69aee5-0532-46a4-b2f5-d020d4d5381d": {
"id": "5b69aee5-0532-46a4-b2f5-d020d4d5381d",
"type": "question",
"responses": [
{
"id": "ef4abf51-3361-46f5-ba04-70c23c85ca20",
"label": "Good",
"colour": "19,133,95",
***"score": 1,***
"enable_score": true
},
Thanks for your help.
Rob.
This is done without pandas
import json
with open("my_file.json", 'r') as f:
my_dict = json.load(f)
score = my_dict["response_sets"]["5b69aee5-0532-46a4-b2f5-d020d4d5381d"]["responses"][0]["score"]

Looking to generically convert JSON file to CSV in Python

Tried solution shared in link :: Nested json to csv - generic approach
This worked for Sample 1 , but giving only a single row for Sample 2.
is there a way to have generic python code to handle both Sample 1 and Sample 2.
Sample 1 ::
{
"Response": "Success",
"Message": "",
"HasWarning": false,
"Type": 100,
"RateLimit": {},
"Data": {
"Aggregated": false,
"TimeFrom": 1234567800,
"TimeTo": 1234567900,
"Data": [
{
"id": 11,
"symbol": "AAA",
"time": 1234567800,
"block_time": 123.282828282828,
"block_size": 1212121,
"current_supply": 10101010
},
{
"id": 12,
"symbol": "BBB",
"time": 1234567900,
"block_time": 234.696969696969,
"block_size": 1313131,
"current_supply": 20202020
},
]
}
}
Sample 2::
{
"Response": "Success",
"Message": "Summary succesfully returned!",
"Data": {
"11": {
"Id": "3333",
"Url": "test/11.png",
"value": "11",
"Name": "11 entries (11)"
},
"122": {
"Id": "5555555",
"Url": "test/122.png",
"Symbol": "122",
"Name": "122 cases (122)"
}
},
"Limit": {},
"HasWarning": False,
"Type": 50
}
Try this, you need to install flatten_json from here
import sys
import csv
import json
from flatten_json import flatten
data = json.load(open(sys.argv[1]))
data = flatten(data)
with open('foo.csv', 'w') as f:
out = csv.DictWriter(f, data.keys())
out.writeheader()
out.writerow(data)
Output
> cat foo.csv
Response,Message,Data_11_Id,Data_11_Url,Data_11_value,Data_11_Name,Data_122_Id,Data_122_Url,Data_122_Symbol,Data_122_Name,Limit,HasWarning,Type
Success,Summary succesfully returned!,3333,test/11.png,11,11 entries (11),5555555,test/122.png,122,122 cases (122),{},False,50
Note: False is incorrect in Json, you need to change it to false

Create new key value in JSON data using Python / Pandas?

I'm trying to work with the Campaign Monitor API, posting JSON data through the API to update subscriber lists. I'm currently one change away from being able to send data,
Right now, my JSON data looks like this
{
"EmailAddress": "subscriber1#example.com",
"Name": "New Subscriber One",
"CustomFields": [
{
"Key": "website",
"Value": "http://example.com"
},
{
"Key": "interests",
"Value": "magic"
},
{
"Key": "interests",
"Value": "romantic walks"
},
{
"Key": "age",
"Value": "",
"Clear": true
}
],
},
{
"EmailAddress": "subscriber2#example.com",
"Name": "New Subscriber Two",
},
{
"EmailAddress": "subscriber3#example.com",
"Name": "New Subscriber Three",
}
}
I still need to add a new key value at the beginning of the JSON payload, incorporating the 'Subscribers' : my_json_data. How would I go about easily adding on the Subscribers key and placing my full and current json data into a list?
Final result should look like
{
'Subscribers' : [
{
"EmailAddress": "subscriber1#example.com",
"Name": "New Subscriber One",
"CustomFields": [
{
"Key": "website",
"Value": "http://example.com"
},
{
"Key": "interests",
"Value": "magic"
},
{
"Key": "interests",
"Value": "romantic walks"
},
{
"Key": "age",
"Value": "",
"Clear": true
}
],
},
{
"EmailAddress": "subscriber2#example.com",
"Name": "New Subscriber Two",
},
{
"EmailAddress": "subscriber3#example.com",
"Name": "New Subscriber Three",
}
}
]
}
I've tried to approach this with creating a new dictionary however when I convert that back to JSON I get more issues and headaches. Is there any easy way to keep everything as a JSON formatted dataset and add in the leading 'Subscribers' key?
this should do it assuming you've got a valid JSON.
your_new_json = {}
your_new_json['Subscribers'] = [your_current_json]

Paging using json file in python

I have this json file which includes information about pages I have retrieving using this
python code:
import facebook # pip install facebook-sdk
import json
import codecs
# Create a connection to the Graph API with your access token
ACCESS_TOKEN = ''#my access token
g = facebook.GraphAPI(ACCESS_TOKEN)
s=g.request('search', { 'q' : '&','type' : 'page', 'limit' : 5000 , 'locale' : 'ar_AR' })
f = open("sampels.txt", 'w')
f.write(json.dumps(s, indent=1))
f.close()
#########################################################
this is my samples json file a snapshot of it:
{
"paging": {
"next": "https://graph.facebook.com/search?limit=5000&type=page&q=%26&locale=ar_AR&access_token=CAACEdEose0cBAIRlSOXkyk1lIMUIWViAoz5lf5t0pSdsu6lg5ZANJuYMIPZCy5N9KFQoLnpi1oxD8tNIaabWackCO31UYaAGkb38IPHxI33ldbRQDXJ02CtJrwE8NI4mZAz20OznLfuCpypDbxNYF3p9XauZCtoywoS9KJwAgW8NYgZCpD4ZBKfCBR5jjXnbcZD&offset=5000&__after_id=92240239536"
},
"data": [
{
"category": "\u0627\u0644\u062a\u0639\u0644\u064a\u0645",
"name": "The London School of Economics and Political Science - LSE",
"category_list": [
{
"id": "108051929285833",
"name": "\u0627\u0644\u0643\u0644\u064a\u0629 \u0648\u0627\u0644\u062c\u0627\u0645\u0639\u0629"
},
{
"id": "187751327923426",
"name": "\u0645\u0646\u0638\u0645\u0629 \u062a\u0639\u0644\u064a\u0645\u064a\u0629"
}
],
"id": "6127898346"
},
{
"category": "\u0628\u0636\u0627\u0626\u0639 \u0627\u0644\u0628\u064a\u0639 \u0628\u0627\u0644\u062a\u062c\u0632\u0626\u0629 \u0648\u0628\u0636\u0627\u0626\u0639 \u0627\u0644\u0645\u0633\u062a\u0647\u0644\u0643\u064a\u0646",
"name": "Stop & Shop",
"category_list": [
{
"id": "169207329791658",
"name": "\u0645\u062d\u0644 \u0628\u0642\u0627\u0644\u0629"
}
],
"id": "170000993071234"
},
{
"category": "\u0628\u0636\u0627\u0626\u0639 \u0627\u0644\u0628\u064a\u0639 \u0628\u0627\u0644\u062a\u062c\u0632\u0626\u0629 \u0648\u0628\u0636\u0627\u0626\u0639 \u0627\u0644\u0645\u0633\u062a\u0647\u0644\u0643\u064a\u0646",
"name": "C&A",
"category_list": [
{
"id": "186230924744328",
"name": "\u0645\u062a\u062c\u0631 \u0645\u0644\u0627\u0628\u0633"
}
],
"id": "109345009145382"
},
{
"category": "\u0645\u0646\u0638\u0645\u0629 \u063a\u064a\u0631 \u0631\u0628\u062d\u064a\u0629",
"name": "Rock and Roll Hall of Fame + Museum",
"category_list": [
{
"id": "396255327112122",
"name": "\u0645\u062a\u062c\u0631 \u0645\u0648\u0633\u064a\u0642\u0649"
},
now what I want to do is to get the next field in order to get my next 5000 page into my database.
I have tried a lot but I couldn't figure out away to do this can any body tell me how to take the next URL and how pass it to the request function I have in my code.?
g.request() returns a Python dict which gives you access to the paging/next url. Why do you dump it as json and write it to file instead of just using it ?

Categories

Resources