Trying to grab certain values from a json file and then 're-create' a new json file (sort of like a conversion). In the code below. I do the following:
define function that returns an dictionary
for each item in json, if function returns results, add results to list located inside parentObj dictionary
oldFile.json:
{
"elements": [
{
"fieldValues": [
{
"id": "101",
"value": "John"
},
{
"id": "102",
"value": "Doe"
}
]
},
{
"fieldValues": [
{
"id": "101",
"value": "Jane"
},
{
"id": "102",
"value": "Doe"
}
]
}
]
}
file.py
import json
import os
output = {}
parentObj = {}
parentObj['entries'] = []
def grabVals(iCounter):
# legend is a pre-determined dictionary matching ids with names (like first/last)
for myKey in subResults['elements'][iCounter]['fieldValues']:
if myKey['id'] in legend:
if 'value' in myKey:
newEntry = {legend[myKey['id']]: myKey['value']}
output.update(newEntry) # first adds 'John', then 'Doe'
# sample output below; next iteration would be 'Jane Doe'
# {"First": "John", "Last": "Doe"}
return output
subResults = json.loads(oldFile.json)
formCount = len(subResults['elements']) # subResults is the json above file. Grab total number of entries
for counter in range(0, formCount):
if convertTime(formEntryStamp, formEntryID) == 'keep': # self defined function (returns keep or None)
parentObj['entries'].append(grabVals(counter))
else:
pass
export = json.dumps(parent_obj, indent=4, sort_keys=False) # create new json based of dictionary
f = open("finished.json", "w")
f.write(export)
f.close()
Expected data in finished.json
{
"entries": [
{
"First": "John",
"Last": "Doe"
},
{
"First": "Jane",
"Last": "Doe"
}
]
}
Actual data in finished.json:
{
"entries": [
{
"First": "Jane",
"Last": "Doe"
},
{
"First": "Jane",
"Last": "Doe"
}
]
}
My question: How do I permanently write to parentObj? When output is changed in the function, the value inside parentObj is overwritten with new value. Does this have something to do mututable/immutable objects? Please let me know any further clarification is required.
Related links are similar, but refer to lists, whereas my is an object/dictionary:
Link 1
Link 2
After doing some reading on mutation of objects in python (link here), the code below solved my problem. Similar to what Juanpa said in the comments, I mutated the variable that was being re-used in my function (output). I assume with the code below, I am creating a copy thus leaving the original untouched.
def grabVals(iCounter, output=None):
if output == None:
output = {}
for myKey in subResults['elements'][iCounter]['fieldValues']:
if myKey['id'] in legend:
if 'value' in myKey:
newEntry = {legend[myKey['id']]: myKey['value']}
output.update(newEntry)
return output
Related
I have a dict like below with hundreds of "assets". I would like to get the key="href" and print the url but because all "assets" are in a list and the first key of "assets" is changing I havent found a way to get there. Thanks for helping!
d = {"features": [
{
"assets": {
"swissbuildings3d_2_2018-07_1064-24_2056_5728.dxf.zip": {
"checksum:multihash":
"1220A94A04BD19E190139FAD49E6174DE82987221F7330DDEB7F6943AEAC3D7C4C78",
"created": "2021-02-10T17:51:31.618859Z",
"href":
"https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-24/swissbuildings3d_2_2018-07_1064-24_2056_5728.dxf.zip",
"proj:epsg": 2056,
"type": "application/x.dxf+zip",
"updated": "2021-02-10T17:51:31.618877Z"
}
}
},
{
"assets": {
"swissbuildings3d_2_2018-07_1064-42_2056_5728.dxf.zip": {
"checksum:multihash":
"1220EA3AFCCDE8648CB60CDF17AF679458233DE2E6052CBBB91F058CBCA651191F6D",
"created": "2021-02-10T17:51:33.722985Z",
"href":
"https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-42/swissbuildings3d_2_2018-07_1064-42_2056_5728.dxf.zip",
"proj:epsg": 2056,
"type": "application/x.dxf+zip",
"updated": "2021-02-10T17:51:33.723005Z"}
}
}
]}
Try this one.
output_list = []
for data_dict in d['features']:
for key, value_dict in data_dict['assets'].items():
output_list.append(value_dict['href'])
print(output_list)
Output:
['https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-24/swissbuildings3d_2_2018-07_1064-24_2056_5728.dxf.zip', 'https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-42/swissbuildings3d_2_2018-07_1064-42_2056_5728.dxf.zip']
If the dictionary in your example is assigned to a variable called d, this works:
result = [(next(iter(x['assets'])), x['assets'][next(iter(x['assets']))]['href']) for x in d['features']]
print(result)
Output:
[('swissbuildings3d_2_2018-07_1064-24_2056_5728.dxf.zip', 'https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-24/swissbuildings3d_2_2018-07_1064-24_2056_5728.dxf.zip'), ('swissbuildings3d_2_2018-07_1064-42_2056_5728.dxf.zip', 'https://data.geo.admin.ch/ch.swisstopo.swissbuildings3d_2/swissbuildings3d_2_2018-07_1064-42/swissbuildings3d_2_2018-07_1064-42_2056_5728.dxf.zip')]
If what you shared wasn't in fact a dictionary, but a .json file, this is how to get it all the same:
import json
with open('mydata.json') as f:
d = load.loads(f.read())
I have a large JSON file with the following structure, with different depth embedded in a node. I want to delete different levels based on assigned depth.
So far I tried to cut some level manually but still it doesn't remove correctly as I remove them based on index and each time indexes shift
content = json.load(file)
content_copy = content.copy()
for j, i in enumerate(content):
if 'children' in i:
for xx, x in enumerate(i['children']):
if 'children' in x:
for zz, z in enumerate(x['children']):
if 'children' in z:
del content_copy[j]['children'][xx]['children'][zz]
Input:
[
{
"name":"1",
"children":[
{
"name":"3",
"children":"5"
},
{
"name":"33",
"children":"51"
},
{
"name":"13",
"children":[
{
"name":"20",
"children":"30"
},
{
"name":"40",
"children":"50"
}
]
}
]
},
{
"name":"2",
"children":[
{
"name":"7",
"children":"6"
},
{
"name":"3",
"children":"521"
},
{
"name":"193",
"children":"292"
}
]
}
]
Output:
In which in 'name':13, its children were removed.
[
{
"name": "1",
"children": [
{
"name": "3",
"children": "5"
},
{
"name": "33",
"children": "51"
},
{
"name": "13"
}
]
},
{
"name": "2",
"children": [
{
"name": "7",
"children": "6"
},
{
"name": "3",
"children": "521"
},
{
"name": "193",
"children": "292"
}
]
}
]
Not a python answer, but in the hope it's useful to someone, here is a one liner using jq tool:
<file jq 'del(.[][][]?[]?[]?)'
It simply deletes all elements that has a depth more than the 5.
The question mark ? is used to avoid iterating over elements that would have a depth less than 3.
One way to prune is pass depth+1 in a recursive function call.
You are asking for different behaviors for different types. If the grandchild is just a string, you want to keep it, but if it is a list then you want to prune. This seems inconsistent, 13 should have children ["20", "30"] but then they wouldn't be the same node structure, so I can see your dilemma.
I would convent them to a tree of node objects, and then just prune nodes, but to get the exact output you listed, I can just selectively prune based on whether child is a string or list.
import pprint
import json
data = """[{
"name": "1", "children":
[
{"name":"3","children":"5"},{"name":"33","children":"51"},
{"name":"13","children":[{"name":"20","children":"30"},
{"name":"40","children":"50"}]}
]
},
{
"name": "2", "children":
[
{"name":"7","children":"6"},
{"name":"3","children":"521"},
{"name":"193","children":"292"}
]
}]"""
content = json.loads(data)
def pruned_nodecopy(content, prune_level, depth=0):
if not isinstance(content, list):
return content
result = []
for node in content:
node_copy = {'name': node['name']}
if 'children' in node:
children = node['children']
if not isinstance(children, list):
node_copy['children'] = children
elif depth+1 < prune_level:
node_copy['children'] = pruned_nodecopy(node['children'], prune_level, depth+1)
result.append(node_copy)
return result
content_copy = pruned_nodecopy(content, 2)
pprint.pprint (content_copy)
Note that this is specifically copying the attributes you use. I had to make it use hard-coded attributes because you're asking for specific (and different) behaviors on those.
I'm having an issue which I can't figure out the solution of. I'm trying to build a JSON object. I'm working with some VMware API and I'm building a JSON object for usernames and nesting VM information inside. I'm having trouble building the nested objects. See below and I'll explain further. Note, annotation is used as a tag to identify the owner of the virtual machine.
owner_logged_in = "johndoe"
service_instance = connect.SmartConnectNoSSL(host='10.0.0.202', user='', pwd='')
atexit.register(connect.Disconnect, service_instance)
content = service_instance.RetrieveContent()
container = content.rootFolder # starting point to look into
viewType = [vim.VirtualMachine] # object types to look for
recursive = True # whether we should look into it recursively
containerView = content.viewManager.CreateContainerView(container, viewType, recursive)
children = containerView.view
virtual_machines = []
vm_username = {}
vm_container = {}
for child in children:
summary = child.summary
annotation = summary.config.annotation
if owner_logged_in == annotation:
children = []
children.append({'ip': summary.guest.ipAddress,'power': summary.runtime.powerState})
vm_container['name'] = summary.config.name
vm_username[owner_logged_in] = vm_container
vm_container['properties'] = children
jsonvalues = json.dumps(vm_username)
#debug#
print(jsonvalues)
#debug#
The returned results are as follows:
{"johndoe": {"name": "centos01", "properties": [{"ip": "10.0.0.201", "power": "poweredOn"}]}}
{"johndoe": {"name": "dc01", "properties": [{"ip": "10.0.0.200", "power": "poweredOn"}]}}
I need to somehow combine these two into 1 object that I can store in a value which then I'm using a DJANGO web app iterate through the value to build a table using some javascript (tabullar.js). I can handle that part but what I'm struggling with is coming up with a way to make 1 object. A user might have more than one virtual machines and I need the properties of each but make the 'johndoe' be the parent.
Essentially I need it formatted to look like this to properly convert it to a table using the tabular.js.
{
"johndoe":[
{
"name":"centos01",
"properties":[
{
"ip":"10.0.0.201",
"power":"poweredOn"
}
]
},
{
"name":"dc01",
"properties":[
{
"ip":"10.0.0.200",
"power":"poweredOn"
}
]
}
]
}
Any help would greatly be appreciated!
try follwing:
from collections import defaultdict
virtual_machines = []
vm_username = defaultdict(list)
vm_container = defaultdict(list)
for child in children:
summary = child.summary
annotation = summary.config.annotation
if owner_logged_in == annotation:
children = []
children.append({'ip': summary.guest.ipAddress,'power': summary.runtime.powerState})
vm_container['name'] = summary.config.name
vm_username[owner_logged_in].append(vm_container)
vm_container['properties'] = children
jsonvalues = json.dumps(vm_username, indent=4)
#debug#
print(jsonvalues)
#debug#
Instead of printing if you can just append it to a list the following solution should work.
import json
d_list=[
{"johndoe": {"name": "centos01", "properties": [{"ip": "10.0.0.201", "power": "poweredOn"}]}},
{"johndoe": {"name": "dc01", "properties": [{"ip": "10.0.0.200", "power": "poweredOn"}]}},
{"janedoe": {"name": "centos02", "properties": [{"ip": "10.0.0.201", "power": "poweredOn"}]}},
{"janedoe": {"name": "dc02", "properties": [{"ip": "10.0.0.200", "power": "poweredOn"}]}}
]
d_new={name:[v for x in d_list for k,v in x.items() if k ==name] for name in set(list(y)[0] for y in d_list)}
# for printing output properly
print(json.dumps(d_new,indent=4))
Output
{
"johndoe": [
{
"name": "centos01",
"properties": [
{
"ip": "10.0.0.201",
"power": "poweredOn"
}
]
},
{
"name": "dc01",
"properties": [
{
"ip": "10.0.0.200",
"power": "poweredOn"
}
]
}
],
"janedoe": [
{
"name": "centos02",
"properties": [
{
"ip": "10.0.0.201",
"power": "poweredOn"
}
]
},
{
"name": "dc02",
"properties": [
{
"ip": "10.0.0.200",
"power": "poweredOn"
}
]
}
]
}
I have a JSON file which has some key-value pairs in Arrays. I need to update/replace the value for key id with a value stored in a variable called Var1
The problem is that when I run my python code, it adds the new key-value pair in outside the inner array instead of replacing:
PYTHON SCRIPT:
import json
import sys
var1=abcdefghi
with open('C:\\Projects\\scripts\\input.json', 'r+') as f:
json_data = json.load(f)
json_data['id'] = var1
f.seek(0)
f.write(json.dumps(json_data))
f.truncate()
INPUT JSON:
{
"channel": "AT",
"username": "Maintenance",
"attachments": [
{
"fallback":"[Urgent]:",
"pretext":"[Urgent]:",
"color":"#D04000",
"fields":[
{
"title":"SERVERS:",
"id":"popeye",
"short":false
}
]
}
]
}
OUTPUT:
{
"username": "Maintenance",
"attachments": [
{
"color": "#D04000",
"pretext": "[Urgent]:",
"fallback": "[Urgent]:",
"fields": [
{
"short": false,
"id": "popeye",
"title": "SERVERS:"
}
]
}
],
"channel": "AT",
"id": "abcdefghi"
}
Below will update the id inside fields :
json_data['attachments'][0]['fields'][0]['id'] = var1
I am new to python and I would like to understand how to access an array in a json object without referring to its name.
The given json object has the below structure
import json
input_json = {
"records": [
{
"values": {
"col1": "1"
},
"no": 1,
},
{
"values": {
"col1": "2"
},
"no": 2,
}
],
"number_of_records": 2
}
myVar = json.load(input_json)
for i in myVar['records']: # How do I replace this line?
print i['values']['col1']
I need to loop through the objects inside the 'records' array. How can I fetch the array without using myVar['records']?
Note that the code cannot depend on the order of the json attributes too. The only thing guaranteed is that the json string would have only one array in it.
input_json = {
"records": [
{
"values": {
"col1": "1"
},
"no": 1,
},
{
"values": {
"col1": "2"
},
"no": 2,
}
],
"number_of_records": 2
}
for anything in input_json:
if isinstance(input_json[anything], list):
for values in input_json[anything]:
print(values['values']['col1'])
You can also further nest the for loop if you don't know the 'values' and 'col1' names.