Develop Reference

Using Pandas to convert csv to Json

I want to convert a CSV to a JSON format using pandas. I am a tester and want to send some events to Event Hub for that I want to maintain a CSV file and update my records/data using the CSV file. I created a CSV file by reading a JSON using pandas for reference. Now when I am again converting the CSV into JSON using pandas< the data is not getting displayed in the correct format. Can you please help.
Step 1: Converted JSON to CSV using pandas:
df = pd.read_json('C://Users//DAMALI//Desktop/test.json')
df.to_csv('C://Users//DAMALI//Desktop/test.csv')
Step2: Now if I try to convert the JSON again to CSV, it's not getting converted in the same format as earlier:
df = pd.read_csv('C://Users//DAMALI//Desktop/test.csv')
df.to_json('C://Users//DAMALI//Desktop/test1.json')
Providing JSON below:
{
"body": {
"deviceId": "UDM",
"registrationDate": "12/11/2019",
"testRegistration": false,
"serialNumber": "25",
"articleNumber": "R91",
"deviceName": "UDM-test",
"locationId": "lc0",
"sapSoldToId": "1138474",
"crmDomainAccountId": "1234566",
"crmAccountDetails": {
"accountName": "ProjectX",
"accountId": "Instal",
"region": "AP"
},
"productLine": "UD",
"state": "registered",
"installerName": "ABC Rooms",
"installationAddress": {
"street": "Benelu",
"zipCode": "850",
"city": "Kortr",
"state": "OVL",
"country": "Belgi"
},
"customerDetails": {
"name": "John D",
"contactName": "John Doe",
"phone": "+32 999999999",
"email": "john.doe#test.com"
},
"wallConnect": {
"wallSize": "Width 5 x Height 4",
"wallOrientation": "LANDSCAPE",
"displayType": "BVD-D55M21H321A1C300",
"softwareVersion": "1.13.1.1.3"
},
"projector": {
"name": "UDX 40K-123456789",
"subType": "UDX 40K"
},
"featureLicense": ["UDX-aa00213a-5719-440e-a3b5", "UDX-aa00a-571"],
"cloudServiceLicense": ["EN04d5-4d2a-9131-875ad37c5883", "E15-4d2a-9131-875ad37c5154"],
"metadata": {
"cusQuesAns": [{
"ques": "End ucal industry",
"ans": "Hosity",
"key": "CUST_ANSWER"
},
{
"ques": "End user video wall application",
"ans": "Simulation & Virtual Reality",
"key": "CUSSECOND_ANSWER"
}
]
},
"frequency": "realtime",
"subDevices": [{
"deviceType": "DISPLAY",
"serialNumber": "68960",
"articleNumber": "R792",
"wallConnect": {
"displayFMWVersion": "3.0.0",
"displayVariant": "KVD21H331A1C300"
}
}]
},
"properties": {
"drs": {
"type": "salesforce-lm"
}
},
"systemProperties": {
"user-id": "data-cvice",
"message-id": "1b1012cc-9b18c192"
}
}

Try this for converting CSV to JSON
import pandas as pd
df = pd.read_csv (r'Fayzan-Bhatti\test.csv')
df.to_json (r'Fayzan-Bhatti\new_test.json')

Exchange of 2 json data values which has different keys

I want to exchange 2 json data's value. But keys of these datas are different from each other. I don't know how can I exchange data value between them.
sample json1: A
{
"contact_person":"Mahmut Kapur",
"contact_people": [
{
"email": "m#gmail.com",
"last_name": "Kapur"
}
],
"addresses": [
{
"city": "istanbul",
"country": "CA",
"first_name": "Mahmut",
"street1": "adres 1",
"zipcode": "34678",
"id": "5f61f72b8348230004f149fd"
}
]
"created_at": "2020-09-16T07:29:47.244-04:00",
"updated_at": "2020-09-16T07:32:50.567-04:00",
}
sample json2: B
The values in this example are: Represents the keys in the A json.
{
"Customer":{
"DisplayName":"contact_person",
"PrimaryEmailAddr":{
"Address":"contact_people/email"
},
"FamilyName":"contact_people/last_name",
"BillAddr":{
"City":"addresses/city",
"CountrySubDivisionCode":"addresses/country",
"Line1":"addresses/street1",
"PostalCode":"addresses/zipcode",
"Id":"addresses/id"
},
"GivenName":"addresses/first_name",
"MetaData":{
"CreateTime":"created_at",
"LastUpdatedTime":"updated_at"
}
}
}
The outcome needs to be:
{
"Customer":{
"DisplayName":"Mahmut Kapur",
"PrimaryEmailAddr":{
"Address":"m#gmail.com"
},
"FamilyName":"Kapur",
"BillAddr":{
"City":"istanbul",
"CountrySubDivisionCode":"CA",
"Line1":"adres 1",
"PostalCode":"34678",
"Id":"5f61f72b8348230004f149fd"
},
"GivenName":"Mahmut",
"MetaData":{
"CreateTime":"2020-09-16T07:29:47.244-04:00",
"LastUpdatedTime":"2020-09-16T07:32:50.567-04:00"
}
}
}
So the important thing here is to match the keys. I hope I was able to explain my problem.

This code can do the work for you. I dont know if someone can make this code shorter for you. It basically searches for dict and list till the leaf level and acts accordingly.
a={
"contact_person":"Mahmut Kapur",
"contact_people": [
{
"email": "m#gmail.com",
"last_name": "Kapur"
}
],
"addresses": [
{
"city": "istanbul",
"country": "CA",
"first_name": "Mahmut",
"street1": "adres 1",
"zipcode": "34678",
"id": "5f61f72b8348230004f149fd"
}
],
"created_at": "2020-09-16T07:29:47.244-04:00",
"updated_at": "2020-09-16T07:32:50.567-04:00",
}
b={
"Customer":{
"DisplayName":"contact_person",
"PrimaryEmailAddr":{
"Address":"contact_people/email"
},
"FamilyName":"contact_people/last_name",
"BillAddr":{
"City":"addresses/city",
"CountrySubDivisionCode":"addresses/country",
"Line1":"addresses/street1",
"PostalCode":"addresses/zipcode",
"Id":"addresses/id"
},
"GivenName":"addresses/first_name",
"MetaData":{
"CreateTime":"created_at",
"LastUpdatedTime":"updated_at"
}
}
}
c={}
for keys in b:
if isinstance(b[keys], dict):
for items in b[keys]:
if isinstance(b[keys][items], dict):
for leaf in b[keys][items]:
if "/" in b[keys][items][leaf]:
getter=b[keys][items][leaf].split("/")
b[keys][items][leaf]=a[getter[0]][0][getter[1]]
else:
b[keys][items][leaf]=a[b[keys][items][leaf]]
else:
if "/" in b[keys][items]:
getter=b[keys][items].split("/")
b[keys][items]=a[getter[0]][0][getter[1]]
else:
b[keys][items]=a[b[keys][items]]
else:
if "/" in b[keys]:
getter=b[keys].split("/")
b[keys]=a[getter[0]][0][getter[1]]
else:
b[keys]=a[b[keys]]
print(json.dumps(b,indent=4))

Retrieve data from json file using python

I'm new to python. I'm running python on Azure data bricks. I have a .json file. I'm putting the important fields of the json file here
{
"school": [
{
"schoolid": "mr1",
"board": "cbse",
"principal": "akseal",
"schoolName": "dps",
"schoolCategory": "UNKNOWN",
"schoolType": "UNKNOWN",
"city": "mumbai",
"sixhour": true,
"weighting": 3,
"paymentMethods": [
"cash",
"cheque"
],
"contactDetails": [
{
"name": "picsa",
"type": "studentactivities",
"information": [
{
"type": "PHONE",
"detail": "+917597980"
}
]
}
],
"addressLocations": [
{
"locationType": "School",
"address": {
"countryCode": "IN",
"city": "Mumbai",
"zipCode": "400061",
"street": "Madh",
"buildingNumber": "80"
},
"Location": {
"latitude": 49.313885,
"longitude": 72.877426
},
I need to create a data frame with schoolName as one column & latitude & longitude are others two columns. Can you please suggest me how to do that?

you can use the method json.load(), here's an example:
import json
with open('path_to_file/file.json') as f:
data = json.load(f)
print(data)

use this
import json # built-in
with open("filename.json", 'r') as jsonFile:
Data = jsonFile.load()
Data is now a dictionary of the contents exp.
for i in Data:
# loops through keys
print(Data[i]) # prints the value
For more on JSON:
https://docs.python.org/3/library/json.html
and python dictionaries:
https://www.programiz.com/python-programming/dictionary#:~:text=Python%20dictionary%20is%20an%20unordered,when%20the%20key%20is%20known.

Parsing JSON with Python, trouble with an array of arrays

I've got a JSON structure that looks like the following
{
"PersonInformation": {
"PhysicalStatus": "",
"OpenDetainers": [],
"StartDate": "",
"FacilityLog": [],
"CustStatus": "",
"EndDate": ""
},
"IdentityList": [
{
"CreationDate": "01/01/1999",
"PersonNames": [
{
"Suffix": "",
"FirstName": "Johnny",
"LastName": "Appleseed",
"MiddleName": ""
},
{
"Suffix": "",
"FirstName": "Foo",
"LastName": "Bar",
"MiddleName": ""
}
],
"PlaceOfBirthList": [
{
"City": "Boston",
"State": "MA",
"CountryCode": ""
}
]
}
]
}
I can parse the outer array like so, but I'm having trouble figuring out how to loop through one of the child arrays, like "PersonNames"
So I can do this
myjson = json.loads(json_data)
print myjson['PersonInformation']['PhysicalStatus']
for identity_list in myjson['IdentityList']:
print identity_list['CreationDate']
Which returns
OK
01/01/1999
as expected, but I don't know how to take it to the next level to traverse into and loop through "PersonNames"
Thanks for the assistance

You can iterate through the sub-list under the PersonNames key like this:
for identity in myjson['IdentityList']:
for person in identity['PersonNames']:
print person['FirstName'], person['LastName']

python json(how to read mulpite id in comple

I have json file i what to read all the values
data=""" {"employees":[
{"firstName":"John", "lastName":"Doe"},
{"firstName":"Anna", "lastName":"Smith"},
{"firstName":"Peter", "lastName":"Jones"}
]}
{
"maps":[
{"id":"apple","iscategorical":"0"},
{"id":"ball","iscategorical":"0"}
],
"mask":{"id1":"aaaaa"},
"mask":{"id1":"bbb"},
"mask":{"id1":"cccccc"},
"om_points":"value",
"parameters":
{"id":"valore"}
}"""
out = json.loads(data)
how to get all values
firstname
lastname
mask.id1
map.id
output:
[(firstname_vaues,lastname_values,mask.id1,map.id)
(firstname_vaues,lastname_values,mask.id1,map.id) ......]
please help me

First thing, there are two json objects in your data string. So you cannot use json.loads(data). You can seperate them by a charcter like ";" . Then split the string and use json.loads on each of them.Use following code.
import json
data=""" {
"employees": [{
"firstName": "John",
"lastName": "Doe"
}, {
"firstName": "Anna",
"lastName": "Smith"
}, {
"firstName": "Peter",
"lastName": "Jones"
}]
};{
"maps": [{
"id": "apple",
"iscategorical": "0"
}, {
"id": "ball",
"iscategorical": "0"
}],
"mask": {
"id1": "aaaaa"
},
"mask": {
"id1": "bbb"
},
"mask": {
"id1": "cccccc"
},
"om_points": "value",
"parameters": {
"id": "valore"
}
}"""
splitdata = data.split(';')
datatop = json.loads(splitdata[0])
databottom = json.loads(splitdata[1])
Then you can access required fields as follows
print(datatop['employees'][0]['firstName'])
print(datatop['employees'][0]['lastName'])
print(databottom['mask']['id1'])
print(databottom['maps'][0]['id'])