I have code that uses a JSON file as an input - each entry in the JSON file is a unique configuration for a simulation run. Below is a simplified sample of one row (expanded) of the JSON file. You can have thousands of these rows each with different unique values.
{
"1": {
"description": "unique configuration 1",
"attribute to change": 1750,
"object type 1": {
"object name": {
"number": 10,
"object attribute 1": 5
}
},
"object type 2": {
"object name": {
"number": 5,
"object attribute 1": 50
}
}
}
}
It works well. However whenever I wish to make a change to the configuration files I need to do it manually, which if you have thousands of entries, can be tedious. I wish to be able to load a default JSON structure (which is the above) and automatically create the variations required.
i have created the below code which is nearly there.
def generate_config_file(input_data, attribute, new_value):
for key, value in input_data.items():
if isinstance(value, dict):
if attribute in value:
value[attribute] = new_value
else:
generate_config_file(value, attribute, new_value)
elif key == attribute:
input_data[attribute] = new_value
file_name = input('Enter file name: ')
if len(file_name) < 1:
file_name = 'default structure.JSON'
id_num = 1
out_file = open('new config file.JSON', "a")
# so here create a new file with multiple rows
# where "attribute to change" is the attribute modified
# here between 5 and 95 in steps of 5
for i in range(5, 100, 5):
with open(file_name) as data_file:
data = json.load(data_file)
# give the new row a unique id_num
data[str(id_num)] = data.pop('1')
generate_config_file(data[str(id_num)], 'attribute to change', i)
json.dump(data, out_file, sort_keys=True, indent=4)
id_num += 1
out_file.close()
I would like the output to look like below (except you would have 19 rows). I have collapsed to the top level but within each row (1,2,3....19) the structure should match the default above. The only difference between the rows being the value associated with the attribute to be changed.
{
"1": {},
"2": {},
"3": {}
}
However it produces:
{
"1": {}
}{
"2": {}
}{
"3": {}
}
I've tried various things. Such as converting the output to a string and trying to strip out the extra {} and replacing with a '. I have also tried when dumping each section of output o remove the outer {} and replace. Neither worked and I am now not sure what to try.
Any help appreciated.
What you are doing is dumping the json data in the for loop, which will always dump a dictionary, which will always produce the data you are getting now, to get around this. I suggest you create a new dictionary ( new_data ) like this,
new_data = {}
# so here create a new file with multiple rows
# where "attribute to change" is the attribute modified
# here between 5 and 95 in steps of 5
for i in range(5, 100, 5):
with open(file_name) as data_file:
data = json.load(data_file)
# give the new row a unique id_num
data[str(id_num)] = data.pop('1')
generate_config_file(data[str(id_num)], 'attribute to change', i)
new_data[str(id_num)] = data[str(id_num)]
#json.dump(data, out_file, sort_keys=True, indent=4)
id_num += 1
json.dump(new_data, out_file, sort_keys=True, indent=4)
and dump it afterwards
You are talking about "rows" but you expect a dictionary structure like this, (that's a valid JSON file):
{
"1": {},
"2": {},
"3": {}
}
So I think is better to forget about "rows" and think always in terms of dictionary key value pairs, mainly because "rows" aren't part of JSON standard, check also the validator.
Using inside a loop:
json.dump(data, out_file, sort_keys=True, indent=4)
having opened the output file in incremental mode here:
out_file = open('new config file.JSON', "a")`)
translates to stacking multiple objects into the output.json text file,
that creates an invalid JSON file, like the one you pointed out.
In order to avoid this you may write your dictionary structure to file all at once, to do this you can change the second part of your example code like this:
# parse old config
with open(file_name, "r") as data_file:
data = json.load(data_file)
# set the new value of attribute_to_change for the first object in json
i = 5
# loop through top level object or what you call rowss
for key in sorted(data.keys()):
# update the attribute with recursive function on each top level
# object with an increasing value i
generate_config_file(data[key], 'attribute to change', i)
i += 5
# if you have 19 objects inside your root object in input json
# the value will span from 5 to 95
# save the whole modified "data" dictionary in one shot
out_file_name = 'new config file.JSON'
with open(out_file_name, "w") as out_file:
json.dump(data, out_file, sort_keys=True, indent=4)
Related
self.channel_ticket = await category.create_text_channel(f'övgü-{user.name}', overwrites=overwrites)
with open('data.json', 'a+') as f:
f.seek(0)
json.dump(str(user_id), f, indent = 4 )
here, it writes user ID that created channel.
Output :
"332115091338297344""323516141777715202"
it writes the ID of more than one user, but how can I make it type from one under the other instead of side by side
As quamrana say, json is a specific format where you can't just add a simple string.
A good way to store id in json format shoul be to create an array of id in your json like that :
{
"ids": [
1,
2,
3
]
}
To do that init your json file with the id's array empty.
Then in python, get the array, append new id and rewrites all the file.
new_id = 444
with open('data.json', 'r+') as f:
# read json datas
json_datas = json.load(f)
# get ids array
ids = json_datas["ids"]
# add new id
ids.append(new_id)
# rebuild our json
json_new_datas = json.dumps({"id":ids})
f.seek(0)
# write json into file
f.write(json_new_datas)
.json is essentially equivalent to a dictionary in python.
JSON
{
"name" : "Li",
"age" : 20,
"stupid" : true
}
PYTHON DICT
about_li = {
"name" : "Li",
"age" : 20,
"stupid" : True
}
So what you can do is (and this is keeping in mind the last question you asked) store the values as True or False with the key being their ID.
import json
id_json = open("id.json", "r+")
ids = json.loads(id_json)
try:
if not ids[str(user_id)]:
# Create your channel and send the dm
ids[str(user_id)] = True
else:
# Say the channel is already open
except:
ids[str(user_id)] = False
id_json.write(json.dumps(ids))
id_json.close()
I have a JSON file with 20 objects, each containing a resource parameter with an associated value. I would like to extract the value of resource for each object, and save that value as a line in a txt file.
The structure of the JSON is:
"objects": [
{"created": "2020-10-04", "name": "john", "resource": "api/john/",}
{"created": "2020-10-04", "name": "paul", "resource": "api/paul/",}
{"created": "2020-10-04", "name": "george", "resource": "api/george/",}
{"created": "2020-10-04", "name": "ringo", "resource": "api/ringo/",}
]
So far, I have got the following code, however this can only get the resource value from the first object, and does not let me write it to a txt file using Python.
with open(input_json) as json_file:
data = json.load(json_file)
resource = (data["objects"][1]["resource"])
values = resource.items()
k = {str(key): str(value) for key, value in values}
with open ('resource-list.txt', 'w') as resource_file:
resource_file.write(k)
You have to use lists:
txtout=""
with open(input_json) as json_file:
data = json.load(json_file)
objects = data["objects"]
for jobj in objects:
txtout = txtout + jobj["resource"] + "\n"
with open ('resource-list.txt', 'w') as resource_file:
resource_file.write(txtout)
hi there new Pythonista!
well the thing you missed here is the part where you iterate over your json object.
with open(input_json) as json_file:
data = json.load(json_file)
resource = (data["objects"][1]["resource"]) # right here you simply took the second object (which is the [1] position)
a decet fix would be:
with open(input_json) as json_file:
data = json.load(json_file)
all_items = [] # lets keep here all resource values
for item in data["objects"]: # iterate over entire items
all_items.append(item["resource"]) # push the necessary info
# lets concat every item to one string since it's only made of 20, it will not make our buffer explode
to_write = "\n".join(all_items)
with open("resource-list.txt", "w") as f:
f.write(to_write)
and we’re done!
I have the following json object (Say car_details.json):
{
"name":"John",
"age":30,
"cars":
[
{
"car_model": "Mustang",
"car_brand": "Ford"
},
{
"car_model": "cx-5",
"car_brand": "Mazda"
}
}
I want to change the value of car_model from cx-5 to cx-9 through python code.
I am providing the json path to this element, through an external file. The json-path expression is basically represented as a string. Something like this:
'cars[2].car_model'
And the new value is also provided through an external file as a string:
'cx-9'
Now how do I parse through car_details.json using the jsonpath expression, and change its value to the one provided as string, and finally return the modified json object
P.S I want to do this through python code
This is an approach without using json module. Load your data in variable. Then iterate over cars key/values. If you find the key that is the value you are looking for set it to new value.
Also note: you need to close your array block, otherwise your above json is not valid. Generally I use an online json parser to check if my data is valid etc. (may be helpful in future).
data = {
"name":"John",
"age":30,
"cars":
[
{
"car_model": "Mustang",
"car_brand": "Ford"
},
{
"car_model": "cx-5",
"car_brand": "Mazda"
}
]
}
for cars in data['cars']:
for key, value in cars.items():
if key == "car_model" and value == "cx-5":
cars[key] = "cx-9"
print(data)
If you want to load your json object from a file, let's assume it is called "data.json" and is in the same directory as the python script you are going to run:
import json
with open('data.json') as json_data:
data = json.load(json_data)
for cars in data['cars']:
for key, value in cars.items():
if key == "car_model" and value == "cx-5":
cars[key] = "cx-9"
print(data)
Now if you'd like to write the content to the original file or new file, in this case I am writing to a file called "newdata.json":
import json
import re
with open('data.json') as json_data:
data = json.load(json_data)
print(data)
with open('external.txt') as f:
content = f.read()
print(content)
for cars in data['cars']:
for key, value in cars.items():
if key == "car_model" and value == "cx-5":
cars[key] = content
with open('newdata.json', 'w') as outfile:
json.dump(data, outfile)
I have json file something like this one.
{
"SomethingA": {
"SomethingB": {
"SomethingC": {
"C-property": "something",
"C-property2": {}
}
}
}
}
I want to add some new data top of the the "Something C" as "NEWSomethingC"
so It should be
{
"SomethingA": {
"SomethingB": {
"NEWSomethingC": {
"NEWC-property": "NEWsomething",
"NEWC-property2": {}
},
"SomethingC": {
"C-property": "something",
"C-property2": {}
}
}
}
}
Okay, here is the problem. I can't add new value top of the keys. Always, NEWSomethingC is going to appear below the SomethingC.
The code I use for adding...
with open(credantials.init['config'], 'r+') as f:
data = json.load(f)
try:
old_data = data['SomethingA'][SomethingB]
append_data = data['SomethingA'][SomethingB]
old_data = {NEWSomethingC :{'C-property':something, 'C-Property2':{}}}
except KeyError:
print ('There is no key you want to search here')
append_data.update(old_data)
print(append_data)
f.seek(0)
json.dump(data,f, indent=4)
f.truncate()
As already pointed out dictionaries in python are unorderd. Therefore we have to use OrderedDict
As explained in this answer we can use the object_pairs_hook
argument in json.loads() to load as OrderdDicts. Then we can add a new dictionary to our "OrderdJsonDictionary" and use the move_to_end function to move our added dictionary to the begin
with open(credantials.init['config'], 'r+') as f:
data = json.load(f, object_pairs_hook=OrderedDict)
new_data = {'newc':{
"NEWC-property": "NEWsomething",
"NEWC-property2": {}
}
}
data["SomethingA"]["SomethingB"].update(new_data)
# last=False moves to the beginning
data["SomethingA"]["SomethingB"].move_to_end(list(new_data.keys())[0], last=False)
f.seek(0)
json.dump(data,f, indent=4)
f.truncate()
So what you would want to do is read the data, search for the data to the point where you wish to make an insertion.
1. Write that data to a new file
2. add your new insertion to the new file
3. add the rest of the file contents to the new file
4. delete the old file
So in order to write to file you would want to insert the following into your code.
outfile = open('file.json')
json.dump(data, outfile)
outfile.close()
I'm trying to append an existing JSON file. When I overwrite the entire JSON file then everything works perfect. The problem that I have been unable to resolve is in the append. I'm completely at a loss at this point.
{
"hashlist": {
"QmVZATT8jWo6ncQM3kwBrGXBjuKfifvrE": {
"description": "Test Video",
"url": ""
},
"QmVqpEomPZU8cpNezxZHG2oc3xQi61P2n": {
"description": "Cat Photo",
"url": ""
},
"QmYdWb4CdFqWGYnPA7V12bX7hf2zxv64AG": {
"description": "test.co",
"url": ""
}
}
}%
Here is the code that I'm using where data['hashlist'].append(entry) receive AttributeError: 'dict' object has no attribute 'append'
#!/usr/bin/python
import json
import os
data = []
if os.stat("hash.json").st_size != 0 :
file = open('hash.json', 'r')
data = json.load(file)
# print(data)
choice = raw_input("What do you want to do? \n a)Add a new IPFS hash\n s)Seach stored hashes\n >>")
if choice == 'a':
# Add a new hash.
description = raw_input('Enter hash description: ')
new_hash_val = raw_input('Enter IPFS hash: ')
new_url_val = raw_input('Enter URL: ')
entry = {new_hash_val: {'description': description, 'url': new_url_val}}
# search existing hash listings here
if new_hash_val not in data['hashlist']:
# append JSON file with new entry
# print entry
# data['hashlist'] = dict(entry) #overwrites whole JSON file
data['hashlist'].append(entry)
file = open('hash.json', 'w')
json.dump(data, file, sort_keys = True, indent = 4, ensure_ascii = False)
file.close()
print('IPFS Hash Added.')
pass
else:
print('Hash exist!')
Usually python errors are pretty self-explanatory, and this is a perfect example. Dictionaries in Python do not have an append method. There are two ways of adding to dictionaries, either by nameing a new key, value pair or passing an iterable with key, value pairs to dictionary.update(). In your code you could do:
data['hashlist'][new_hash_val] = {'description': description, 'url': new_url_val}
or:
data['hashlist'].update({new_hash_val: {'description': description, 'url': new_url_val}})
The first one is probably superior for what you are trying to do, because the second one is more for when you are trying to add lots of key, value pairs.
You can read more about dictionaries in Python here.