I have a JSON file I'm trying to manipulate in python but it seems the json formating is not correct:
{{"ticket":{"id":"123", "name":"bill"}},
{"ticket":{"id":"1234", "name":"james"}}}
When i try to format it using a json formatter it gives me Error multiple root elements
How can I fix it?
Update: with the suggestion from funqkey i updated the script:
import json
with open('ticketData8242020-6152021.json', 'r') as f:
data = f.read()
data = json.loads(data)
There is something wrong with the file. I will attempt to remove the ticket object references from the file. to fix it. Thanks everyone.
The problems here include
ticket needs to be in quotes
When you have multiple objects, you need a list, not a dict
You can't have an object with multiple "ticket" keys.
I SUSPECT what you want is a list of objects, like this:
[{"id":"123", "name":"bill"}, {"id":"1234", "name":"james"}]
Or maybe a list of objects with one entry each, as funqkey suggested:
[{"ticket":{"id":"123", "name":"bill"}}, {"ticket":{"id":"1234", "name":"james"}}]
# Should look like this [{"ticket": {"id": "123", "name": "bill"}}, {"ticket": {"id": "1234", "name": "james"}}]
import json
with open('ticketData8242020-6152021.json', 'r') as f:
data = f.read()
data = json.loads(data)
In JSON, the keys should be quoted using ". Therefore
{{ticket:{"id":"123", "name":"bill"}}, {ticket:{"id":"1234", "name":"james"}}}
is not a valid JSON. The corrected version is
{{"ticket":{"id":"123", "name":"bill"}}, {"ticket":{"id":"1234", "name":"james"}}}
You can validate your JSON online: JSON Online Validator and Formatter - JSON Lint
Related
recently I have been working on a project, and I needed to append a list of dictionaries to my existing JSON file. But it behaves somewhat strangely.
Here is what I have:
def write_records_to_json(json_object):
with open("tracker.json", "r+") as f:
json_file = json.load(f)
json_file.append(json_object)
print(json_file)
This is the object I'm trying to append(The object is formatted this way):
[
{
"file": "dnc_complaint_numbers_2021-12-03.csv",
"date": "2021-12-03"
}
]
And this is what I get(Pay attention to the end):
Excuse me please, for not having it more readable.
[{'file': 'dnc_complaint_numbers_2021-12-01.csv', 'date': '2021-12-01'}, {'file': 'dnc_complaint_numbers_2021-12-02.csv', 'date': '2021-12-02'}, '[\n {\n "file": "dnc_complaint_numbers_2021-12-03.csv",\n "date": "2021-12-03"\n }\n]']
Can someone tell me why is that and how to fix it? Thanks a lot.
From your code and output, we can infer that json_object refers to a string. This string contains JSON. json_file is not JSON, it is a list that is deserialised from JSON.
If you want to add json_object to json_file you should first deserialise the former:
json_file.extend(json.loads(json_object))
You also want to use extend instead of append here, so it is on the same level as the rest of the data.
Can't solve how to convert JSON to python so that all data be in an array.
I used code to extract JSON, but the problem is that to extract strings from each new JSON data set is a new issue due to the inequality of the number of columns.
import json
data = open('acndata_sessions.json')
json.load(data)
I also tried to use https://app.quicktype.io/, but the function result is:
data_from_dict(json.loads(json_string)) doesn't work.
Data set: json
This question seems to have been asked before. Convert JSON array to Python list
They use 'json.loads' instead of 'json.load' which you use. Both are functions, but they are different.
I think your looking for something like this.
import json
with open('myfile.json','r') as jsonFile:
pythonJSON=json.load(jsonFile)
jsonFile.close()
print(pythonJSON.keys())
The json.loads() is used when you have a string type. If the example above doesn't work with just json.load() try it with json.loads().
json.load already gives you a dictionary. You just have to use it and iterate through _items
import json
data = open('acndata_sessions.json')
data_dict = json.load(data)
# Load items from dictionary
items = data_dict['_items']
# Iterate in items
for item in items:
# Print the item
print(item)
# Or if you want to further iterate on user inputs present in this item
user_inputs = item['userInputs']
# Check if there are any user inputs present in this item
if user_inputs:
for user_input in user_inputs:
print(user_input)
Try checking this question. You can parse a json file like this:
import json
# read file
with open('example.json', 'r') as myfile:
data=myfile.read()
# parse file
obj = json.loads(data)
# show values
print(str(obj['_items']))#returns a dict
print(str(obj['_meta']))
New to reading and writing with Python and JSON, so go easy on me, but would appreciate any help!
OK so I am using Pycharm and am reading in a large JSON file with the following code and able to print it out:
import json
from pprint import pprint
with open('Files/InstrumentPublished.json') as data_file:
data = json.load(data_file)
pprint(data)
Within the JSON file I am reading there are 4 fields I want to look at: A_ID, B_ID, C_ID and D_ID. All these fields should have the same value, ie (they are located in different places throughout the JSON file:
"A_ID": "12345",
......
"B_ID": "12345",
.......
"C_ID": "12345",
........
"D_ID": "12345",
Basically I need to add the following logic when reading the file:
Identify the following fields in the JSON file: A_ID, B_ID, C_ID and D_ID
Read/consume their value and compare them
If all values are the equal print true.
If all values are not equal, print false.
I have searched around the internet but can only find reading and writing to a JSON using Python but nothing of comparing individual fields.
Like I said any help would be appreciated!
Thanks!
Kirsty
When you read a JSON in Python it becomes a dictionary, then you can access each key value with the square bracket
Here is the code
import json
from pprint import pprint
with open('Files/InstrumentPublished.json') as data_file:
data = json.load(data_file)
all_ids_equal = data['A_ID'] == data['B_ID'] == data['C_ID'] == data['D_ID']
print(all_ids_equal)
I collected some tweets from the twitter API and stored it to mongodb, I tried exporting the data to a JSON file and didn't have any issues there, until I tried to make a python script to read the JSON and convert it to a csv. I get this traceback error with my code:
json.decoder.JSONDecodeError: Extra data: line 367 column 1 (char 9745)
So, after digging around the internet I was pointed to check the actual JSON data in an online validator, which I did. This gave me the error of:
Multiple JSON root elements
from the site https://jsonformatter.curiousconcept.com/
Here are pictures of the 1st/2nd object beginning/end of the file:
or a link to the data here
Now, the problem is, I haven't found anything on the internet of how to handle that error. I'm not sure if it's an error with the data I've collected, exported, or if I just don't know how to work with it.
My end game with these tweets is to make a network graph. I was looking at either Networkx or Gephi, which is why I'd like to get a csv file.
Robert Moskal is right. If you can address the issue at source and use --jsonArray flag when you use mongoexport then it will make the problem easier i guess. If you can't address it at source then read the below points.
The code below will extract you the individual json objects from the given file and convert them to python dictionaries.
You can then apply your CSV logic to each individual dictionary.
If you are using csv module then I would say use unicodecsv module as it would handle the unicode data in your json objects.
import json
with open('path_to_your_json_file', 'rb') as infile:
json_block = []
for line in infile:
json_block.append(line)
if line.startswith('}'):
json_dict = json.loads(''.join(json_block))
json_block = []
print json_dict
If you want to convert it to CSV using pandas you can use the below code:
import json, pandas as pd
with open('path_to_your_json_file', 'rb') as infile:
json_block = []
dictlist=[]
for line in infile:
json_block.append(line)
if line.startswith('}'):
json_dict = json.loads(''.join(json_block))
dictlist.append(json_dict)
json_block = []
df = pd.DataFrame(jsonlist)
df.to_csv('out.csv',encoding='utf-8')
If you want to flatten out the json object you can use pandas.io.json.json_normalize() method.
Elaborating on #MYGz suggestion to use --jsonArray
Your post doesn't show how you exported the data from mongo. If you use the following via the terminal, you will get valid json from mongodb:
mongoexport --collection=somecollection --db=somedb --jsonArray --out=validfile.json
Replace somecollection, somedb and validfile.json with your target collection, target database, and desired output filename respectively.
The following: mongoexport --collection=somecollection --db=somedb --out=validfile.json...will NOT give you the results you are looking for because:
By default mongoexport writes data using one JSON document for every
MongoDB document. Ref
A bit late reply, and I am not sure it was available the time this question was posted. Anyway, now there is a simple way to import the mongoexport json data as follows:
df = pd.read_json(filename, lines=True)
mongoexport provides each line as a json objects itself, instead of the whole file as json.
Assuming I have a configuration txt file with this content:
{"Mode":"Classic","Encoding":"UTF-8","Colors":3,"Blue":80,"Red":90,"Green":160,"Shortcuts":[],"protocol":"2.1"}
How can i change a specific value like "Red":90 to "Red":110 in the file without changing its original format?
I have tried with configparser and configobj but as they are designed for .INI files I couldn't figure out how to make it work with this custom config file. I also tried splitting the lines searching for the keywords witch values I wanted to change but couldn't save the file the same way it was before. Any ideas how to solve this? (I'm very new in Python)
this looks like json so you could:
import json
obj = json.load(open("/path/to/jsonfile","r"))
obj["Blue"] = 10
json.dump(obj,open("/path/to/mynewfile","w"))
but be aware that a json dict does not have an order.
So the order of the elements is not guaranteed (and normally it's not needed) json lists have an order though.
Here's how you can do it:
import json
d = {} # store your data here
with open('config.txt','r') as f:
d = json.loads(f.readline())
d['Red']=14
d['Green']=15
d['Blue']=20
result = "{\"Mode\":\"%s\",\"Encoding\":\"%s\",\"Colors\":%s,\
\"Blue\":%s,\"Red\":%s,\"Green\":%s,\"Shortcuts\":%s,\
\"protocol\":\"%s\"}"%(d['Mode'],d['Encoding'],d['Colors'],
d['Blue'],d['Red'],d['Green'],
d['Shortcuts'],d['protocol'])
with open('config.txt','w') as f:
f.write(result)
f.close()
print result