how to convert using python below payloads:
{
"abc": {
"i": "1212",
"j": "add"
}
}
to
"{\n\n \"abc\": {\n \"i\": \"1212\",\n \"j\": \"add\"\n }\n}"
You can use the optional indent parameter of json.dump and json.dumps to add newlines and indent to the generated string.
>>> import json
>>> payload = {
... "assessmentstatus": {
... "id": "D37002079003",
... "value": "In-Progress"
... }
... }
...
>>> json.dumps(payload)
'{"assessmentstatus": {"id": "D37002079003", "value": "In-Progress"}}'
>>> json.dumps(payload, indent=0)
'{\n"assessmentstatus": {\n"id": "D37002079003",\n"value": "In-Progress"\n}\n}'
>>> json.dumps(payload, indent=2)
'{\n "assessmentstatus": {\n "id": "D37002079003",\n "value": "In-Progress"\n }\n}'
With properly displayed whitespace:
>>> print(json.dumps(payload, indent=2))
{
"assessmentstatus": {
"id": "D37002079003",
"value": "In-Progress"
}
}
It seems you actually want the string including the enclosing " and with all the " within the string escaped. This is surprisingly tricky using Python's repr, as it always tries to use either ' or " as the outer quotes so that the quotes do not have to be escaped.
What seems to work, though, is to just json.dumps the JSON string again:
>>> json.dumps(json.dumps(payload, indent=2))
'"{\\n \\"assessmentstatus\\": {\\n \\"id\\": \\"D37002079003\\",\\n \\"value\\": \\"In-Progress\\"\\n }\\n}"'
>>> print(json.dumps(json.dumps(payload, indent=2)))
"{\n \"assessmentstatus\": {\n \"id\": \"D37002079003\",\n \"value\": \"In-Progress\"\n }\n}"
Im assuming it has newlines since its in a file
with open('file.json') as f: data = json.load(f)
Then do what you want with the data
You can save data to a file in JSON format like this
import json
data = {
"assessmentstatus": {
"id": "D37002079003",
"value": "In-Progress"
}
}
with open("file.txt", "w") as file:
json.dump(data, file, indent=4)
To retrieve this data you can do
import json
with open("file.txt") as file:
data = json.load(file)
print(data)
JSON is a way to store store data in a string format and be able to parse that data to convert it to actual data.
So trying to convert "{\n\n "assessmentstatus": {\n "id": "D37002079003",\n "value": "In-Progress"\n }\n}" on your own is the wrong way to go at this, since JSON is great at doing this.
Related
I am trying to read some json with the following format. A simple pd.read_json() returns ValueError: Trailing data. Adding lines=True returns ValueError: Expected object or value. I've tried various combinations of readlines() and load()/loads() so far without success.
Any ideas how I could get this into a dataframe?
{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
}
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}
The sample you have above isn't valid JSON. To be valid JSON these objects need to be within a JS array ([]) and be comma separated, as follows:
[{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
},
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}]
I just tried on my machine. When formatted correctly, it works
>>> pd.read_json('data.json')
content source title url
0 kdjfsfkjlffsdkj {'name': 'jfkldsjf'} dsldkjfslj vkljfklgjkdlgj
1 djlskgfdklgjkfgj {'name': 'ldfjkdfjs'} lfsjdfklfldsjf lkjlfggdflkjgdlf
Another solution if you do not want to reformat your files.
Assuming your JSON is in a string called my_json you could do:
import json
import pandas as pd
splitted = my_json.split('\n\n')
my_list = [json.loads(e) for e in splitted]
df = pd.DataFrame(my_list)
Thanks for the ideas internet. None quite solved the problem in the way I needed (I had lots of newline characters in the strings themselves which meant I couldn't split on them) but they helped point the way. In case anyone has a similar problem, this is what worked for me:
with open('path/to/original.json', 'r') as f:
data = f.read()
data = data.split("}\n")
data = [d.strip() + "}" for d in data]
data = list(filter(("}").__ne__, data))
data = [json.loads(d) for d in data]
with open('path/to/reformatted.json', 'w') as f:
json.dump(data, f)
df = pd.read_json('path/to/reformatted.json')
If you can use jq then solution is simpler:
jq -s '.' path/to/original.json > path/to/reformatted.json
im trying to print the first object of this json file but it only prints the first char of it.
This is my code:
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
new_data = json.dumps(data, indent = 2)
print(str(new_data[0]))
result i was hoping for:
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}
actual result:
[
json.dump the first element of the response:
import json
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
first_elem = json.dumps(data[0], indent=2)
print(first_elem)
Apparently response.json() already is a dictionary.
So if you try first_element = data[0] you would get what you are looking for.
And then, if you want to make it pretty :
json.dumps(first_element, indent = 2)
If you wish a JSON object to behave like a dictionary , take a look at
json.loads
https://docs.python.org/2/library/json.html
Also :
What's the best way to parse a JSON response from the requests library?
json.dumps results in a string.
You are printing the first word of the string by doing [0]
For the output that you want, do:
print(new_data)
I have a list of json objects that I would like to write to a json file. Example of my data is as follows:
{
"_id": "abc",
"resolved": false,
"timestamp": "2017-04-18T04:57:41 366000",
"timestamp_utc": {
"$date": 1492509461366
},
"sessionID": "abc",
"resHeight": 768,
"time_bucket": ["2017-year", "2017-04-month", "2017-16-week", "2017-04-18-day", "2017-04-18 16-hour"],
"referrer": "Standalone",
"g_event_id": "abc",
"user_agent": "abc"
"_id": "abc",
} {
"_id": "abc",
"resolved": false,
"timestamp": "2017-04-18T04:57:41 366000",
"timestamp_utc": {
"$date": 1492509461366
},
"sessionID": "abc",
"resHeight": 768,
"time_bucket": ["2017-year", "2017-04-month", "2017-16-week", "2017-04-18-day", "2017-04-18 16-hour"],
"referrer": "Standalone",
"g_event_id": "abc",
"user_agent": "abc"
}
I would like to wirte this to a json file. Here's the code that I am using for this purpose:
with open("filename", 'w') as outfile1:
for row in data:
outfile1.write(json.dumps(row))
But this gives me a file with only 1 long row of data. I would like to have a row for each json object in my original data. I know there are some other StackOverflow questions that are trying to address somewhat similar situation (by externally inserting '\n' etc.), but it hasn't worked in my case for some reason. I believe there has to be a pythonic way to do this.
How do I achieve this?
The format of the file you are trying to create is called JSON lines.
It seems, you are asking why the jsons are not separated with a newline. Because write method does not append the newline.
If you want implicit newlines you should better use print function:
with open("filename", 'w') as outfile1:
for row in data:
print(json.dumps(row), file=outfile1)
Use the indent argument to output json with extra whitespace. The default is to not output linebreaks or extra spaces.
with open('filename.json', 'w') as outfile1:
json.dump(data, outfile1, indent=4)
https://docs.python.org/3/library/json.html#basic-usage
I'm new to JSON and Python, any help on this would be greatly appreciated.
I read about json.loads but am confused
How do I read a file into Python using json.loads?
Below is my JSON file format:
{
"header": {
"platform":"atm"
"version":"2.0"
}
"details":[
{
"abc":"3"
"def":"4"
},
{
"abc":"5"
"def":"6"
},
{
"abc":"7"
"def":"8"
}
]
}
My requirement is to read the values of all "abc" "def" in details and add this is to a new list like this [(1,2),(3,4),(5,6),(7,8)]. The new list will be used to create a spark data frame.
Open the file, and get a filehandle:
fh = open('thefile.json')
https://docs.python.org/2/library/functions.html#open
Then, pass the file handle into json.load(): (don't use loads - that's for strings)
import json
data = json.load(fh)
https://docs.python.org/2/library/json.html#json.load
From there, you can easily deal with a python dictionary that represents your json-encoded data.
new_list = [(detail['abc'], detail['def']) for detail in data['details']]
Note that your JSON format is also wrong. You will need comma delimiters in many places, but that's not the question.
I'm trying to understand your question as best as I can, but it looks like it was formatted poorly.
First off your json blob is not valid json, it is missing quite a few commas. This is probably what you are looking for:
{
"header": {
"platform": "atm",
"version": "2.0"
},
"details": [
{
"abc": "3",
"def": "4"
},
{
"abc": "5",
"def": "6"
},
{
"abc": "7",
"def": "8"
}
]
}
Now assuming you are trying to parse this in python you will have to do the following.
import json
json_blob = '{"header": {"platform": "atm","version": "2.0"},"details": [{"abc": "3","def": "4"},{"abc": "5","def": "6"},{"abc": "7","def": "8"}]}'
json_obj = json.loads(json_blob)
final_list = []
for single in json_obj['details']:
final_list.append((int(single['abc']), int(single['def'])))
print(final_list)
This will print the following: [(3, 4), (5, 6), (7, 8)]
I try to read this ascii file with this json content with the following function:
{ "directory": { "name": "/wiki", "files": { "file": [ { "name": "/wiki/a.txt", "digest": "97d37a2ff85fbe35e1bf8ad38934d8fb518a6a3fbeb9b0b9305ce98e992f9dd2 " },
{ "name": "/wiki/d.txt", "digest": "ef91ee1257c3faa49f86f343cfec66010e5810e99db9f42e88774f90cd5b95d9 " },] } } }
def readJsonFile(path):
with open(path) as json_file:
json_data = json.load(json_file)
return json_data
I get this error of no JSON object could be decoded:
ValueError: No JSON object could be decoded
I tried with json.loads and I get the error:
TypeError: expected string or buffer
Am I using the right function?
The data is not a valid json (It has a trailing ,).
But it's a valid python literal; you can use ast.literal_eval instead:
import ast
def readJsonFile(path):
with open(path) as json_file:
return ast.literal_eval(json_file.read())
Your json string is wrong, as validated by jsonint: