convert json to string with newline and escape in python - python

how to convert using python below payloads:
{
"abc": {
"i": "1212",
"j": "add"
}
}
to
"{\n\n \"abc\": {\n \"i\": \"1212\",\n \"j\": \"add\"\n }\n}"

You can use the optional indent parameter of json.dump and json.dumps to add newlines and indent to the generated string.
>>> import json
>>> payload = {
... "assessmentstatus": {
... "id": "D37002079003",
... "value": "In-Progress"
... }
... }
...
>>> json.dumps(payload)
'{"assessmentstatus": {"id": "D37002079003", "value": "In-Progress"}}'
>>> json.dumps(payload, indent=0)
'{\n"assessmentstatus": {\n"id": "D37002079003",\n"value": "In-Progress"\n}\n}'
>>> json.dumps(payload, indent=2)
'{\n "assessmentstatus": {\n "id": "D37002079003",\n "value": "In-Progress"\n }\n}'
With properly displayed whitespace:
>>> print(json.dumps(payload, indent=2))
{
"assessmentstatus": {
"id": "D37002079003",
"value": "In-Progress"
}
}
It seems you actually want the string including the enclosing " and with all the " within the string escaped. This is surprisingly tricky using Python's repr, as it always tries to use either ' or " as the outer quotes so that the quotes do not have to be escaped.
What seems to work, though, is to just json.dumps the JSON string again:
>>> json.dumps(json.dumps(payload, indent=2))
'"{\\n \\"assessmentstatus\\": {\\n \\"id\\": \\"D37002079003\\",\\n \\"value\\": \\"In-Progress\\"\\n }\\n}"'
>>> print(json.dumps(json.dumps(payload, indent=2)))
"{\n \"assessmentstatus\": {\n \"id\": \"D37002079003\",\n \"value\": \"In-Progress\"\n }\n}"

Im assuming it has newlines since its in a file
with open('file.json') as f: data = json.load(f)
Then do what you want with the data

You can save data to a file in JSON format like this
import json
data = {
"assessmentstatus": {
"id": "D37002079003",
"value": "In-Progress"
}
}
with open("file.txt", "w") as file:
json.dump(data, file, indent=4)
To retrieve this data you can do
import json
with open("file.txt") as file:
data = json.load(file)
print(data)
JSON is a way to store store data in a string format and be able to parse that data to convert it to actual data.
So trying to convert "{\n\n "assessmentstatus": {\n "id": "D37002079003",\n "value": "In-Progress"\n }\n}" on your own is the wrong way to go at this, since JSON is great at doing this.

Related

Reading json in python separated by newlines

I am trying to read some json with the following format. A simple pd.read_json() returns ValueError: Trailing data. Adding lines=True returns ValueError: Expected object or value. I've tried various combinations of readlines() and load()/loads() so far without success.
Any ideas how I could get this into a dataframe?
{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
}
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}
The sample you have above isn't valid JSON. To be valid JSON these objects need to be within a JS array ([]) and be comma separated, as follows:
[{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
},
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}]
I just tried on my machine. When formatted correctly, it works
>>> pd.read_json('data.json')
content source title url
0 kdjfsfkjlffsdkj {'name': 'jfkldsjf'} dsldkjfslj vkljfklgjkdlgj
1 djlskgfdklgjkfgj {'name': 'ldfjkdfjs'} lfsjdfklfldsjf lkjlfggdflkjgdlf
Another solution if you do not want to reformat your files.
Assuming your JSON is in a string called my_json you could do:
import json
import pandas as pd
splitted = my_json.split('\n\n')
my_list = [json.loads(e) for e in splitted]
df = pd.DataFrame(my_list)
Thanks for the ideas internet. None quite solved the problem in the way I needed (I had lots of newline characters in the strings themselves which meant I couldn't split on them) but they helped point the way. In case anyone has a similar problem, this is what worked for me:
with open('path/to/original.json', 'r') as f:
data = f.read()
data = data.split("}\n")
data = [d.strip() + "}" for d in data]
data = list(filter(("}").__ne__, data))
data = [json.loads(d) for d in data]
with open('path/to/reformatted.json', 'w') as f:
json.dump(data, f)
df = pd.read_json('path/to/reformatted.json')
If you can use jq then solution is simpler:
jq -s '.' path/to/original.json > path/to/reformatted.json

Why does print(new_data[0]) only print out the first char of the json file?

im trying to print the first object of this json file but it only prints the first char of it.
This is my code:
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
new_data = json.dumps(data, indent = 2)
print(str(new_data[0]))
result i was hoping for:
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}
actual result:
[
json.dump the first element of the response:
import json
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
first_elem = json.dumps(data[0], indent=2)
print(first_elem)
Apparently response.json() already is a dictionary.
So if you try first_element = data[0] you would get what you are looking for.
And then, if you want to make it pretty :
json.dumps(first_element, indent = 2)
If you wish a JSON object to behave like a dictionary , take a look at
json.loads
https://docs.python.org/2/library/json.html
Also :
What's the best way to parse a JSON response from the requests library?
json.dumps results in a string.
You are printing the first word of the string by doing [0]
For the output that you want, do:
print(new_data)

Writing multiple json objects to a json file

I have a list of json objects that I would like to write to a json file. Example of my data is as follows:
{
"_id": "abc",
"resolved": false,
"timestamp": "2017-04-18T04:57:41 366000",
"timestamp_utc": {
"$date": 1492509461366
},
"sessionID": "abc",
"resHeight": 768,
"time_bucket": ["2017-year", "2017-04-month", "2017-16-week", "2017-04-18-day", "2017-04-18 16-hour"],
"referrer": "Standalone",
"g_event_id": "abc",
"user_agent": "abc"
"_id": "abc",
} {
"_id": "abc",
"resolved": false,
"timestamp": "2017-04-18T04:57:41 366000",
"timestamp_utc": {
"$date": 1492509461366
},
"sessionID": "abc",
"resHeight": 768,
"time_bucket": ["2017-year", "2017-04-month", "2017-16-week", "2017-04-18-day", "2017-04-18 16-hour"],
"referrer": "Standalone",
"g_event_id": "abc",
"user_agent": "abc"
}
I would like to wirte this to a json file. Here's the code that I am using for this purpose:
with open("filename", 'w') as outfile1:
for row in data:
outfile1.write(json.dumps(row))
But this gives me a file with only 1 long row of data. I would like to have a row for each json object in my original data. I know there are some other StackOverflow questions that are trying to address somewhat similar situation (by externally inserting '\n' etc.), but it hasn't worked in my case for some reason. I believe there has to be a pythonic way to do this.
How do I achieve this?
The format of the file you are trying to create is called JSON lines.
It seems, you are asking why the jsons are not separated with a newline. Because write method does not append the newline.
If you want implicit newlines you should better use print function:
with open("filename", 'w') as outfile1:
for row in data:
print(json.dumps(row), file=outfile1)
Use the indent argument to output json with extra whitespace. The default is to not output linebreaks or extra spaces.
with open('filename.json', 'w') as outfile1:
json.dump(data, outfile1, indent=4)
https://docs.python.org/3/library/json.html#basic-usage

How do I read a json file into python?

I'm new to JSON and Python, any help on this would be greatly appreciated.
I read about json.loads but am confused
How do I read a file into Python using json.loads?
Below is my JSON file format:
{
"header": {
"platform":"atm"
"version":"2.0"
}
"details":[
{
"abc":"3"
"def":"4"
},
{
"abc":"5"
"def":"6"
},
{
"abc":"7"
"def":"8"
}
]
}
My requirement is to read the values of all "abc" "def" in details and add this is to a new list like this [(1,2),(3,4),(5,6),(7,8)]. The new list will be used to create a spark data frame.
Open the file, and get a filehandle:
fh = open('thefile.json')
https://docs.python.org/2/library/functions.html#open
Then, pass the file handle into json.load(): (don't use loads - that's for strings)
import json
data = json.load(fh)
https://docs.python.org/2/library/json.html#json.load
From there, you can easily deal with a python dictionary that represents your json-encoded data.
new_list = [(detail['abc'], detail['def']) for detail in data['details']]
Note that your JSON format is also wrong. You will need comma delimiters in many places, but that's not the question.
I'm trying to understand your question as best as I can, but it looks like it was formatted poorly.
First off your json blob is not valid json, it is missing quite a few commas. This is probably what you are looking for:
{
"header": {
"platform": "atm",
"version": "2.0"
},
"details": [
{
"abc": "3",
"def": "4"
},
{
"abc": "5",
"def": "6"
},
{
"abc": "7",
"def": "8"
}
]
}
Now assuming you are trying to parse this in python you will have to do the following.
import json
json_blob = '{"header": {"platform": "atm","version": "2.0"},"details": [{"abc": "3","def": "4"},{"abc": "5","def": "6"},{"abc": "7","def": "8"}]}'
json_obj = json.loads(json_blob)
final_list = []
for single in json_obj['details']:
final_list.append((int(single['abc']), int(single['def'])))
print(final_list)
This will print the following: [(3, 4), (5, 6), (7, 8)]

How to read an ascii file with json content: ValueError

I try to read this ascii file with this json content with the following function:
{ "directory": { "name": "/wiki", "files": { "file": [ { "name": "/wiki/a.txt", "digest": "97d37a2ff85fbe35e1bf8ad38934d8fb518a6a3fbeb9b0b9305ce98e992f9dd2 " },
{ "name": "/wiki/d.txt", "digest": "ef91ee1257c3faa49f86f343cfec66010e5810e99db9f42e88774f90cd5b95d9 " },] } } }
def readJsonFile(path):
with open(path) as json_file:
json_data = json.load(json_file)
return json_data
I get this error of no JSON object could be decoded:
ValueError: No JSON object could be decoded
I tried with json.loads and I get the error:
TypeError: expected string or buffer
Am I using the right function?
The data is not a valid json (It has a trailing ,).
But it's a valid python literal; you can use ast.literal_eval instead:
import ast
def readJsonFile(path):
with open(path) as json_file:
return ast.literal_eval(json_file.read())
Your json string is wrong, as validated by jsonint:

Categories

Resources