I'm trying to format json data to html using json2html.
The json data look like this:
json_obj = [{"Agent Status": "status1", "Last backup": "", "hostId": 1234567, "hostfqdn": "test1.example.com", "username": "user1"}, {"Agent Status": "status2", "Last backup": "", "hostId": 2345678, "hostfqdn": "test2.example.com", "username": "user2"}]
As already reported in post "json2html not a valid json list python", to make the code works, the json parameter must be a dictionary and not a list, so I'm calling it that way:
json_obj_in_html = json2html.convert(json = { "data" : json_obj })
However it does not format the json data to html only the first level of dictionary { "data" : json_obj }:
print json_obj_in_html
<table border="1"><tr><th>data</th><td>[{"Agent Status": "status1", "Last backup": "", "hostId": 1234567, "hostfqdn": "test1.example.com", "username": "user1"}, {"Agent Status": "status2", "Last backup": "", "hostId": 2345678, "hostfqdn": "test2.example.com", "username": "user2"}]</td></tr></table>
Note that the online convert tool provides the right output: http://json2html.varunmalhotra.xyz/
<table border="1"><tr><th>data</th><td><ul><table border="1"><tr><th>Agent Status</th><td>status1</td></tr><tr><th>Last backup</th><td></td></tr><tr><th>hostId</th><td>1234567</td></tr><tr><th>hostfqdn</th><td>test1.example.com</td></tr><tr><th>username</th><td>user1</td></tr></table><table border="1"><tr><th>Agent Status</th><td>status2</td></tr><tr><th>Last backup</th><td></td></tr><tr><th>hostId</th><td>2345678</td></tr><tr><th>hostfqdn</th><td>test2.example.com</td></tr><tr><th>username</th><td>user2</td></tr></table></ul></td></tr></table>
Any help would be very welcome.
Make sure that json_obj is an array of objects and not a string (str).
I put your code to a complete sample:
from json2html import *
json_obj = [{"Agent Status": "status1", "Last backup": "", "hostId": 1234567, "hostfqdn": "test1.example.com", "username": "user1"}, {"Agent Status": "status2", "Last backup": "", "hostId": 2345678, "hostfqdn": "test2.example.com", "username": "user2"}]
json_obj_in_html = json2html.convert(json = { "data" : json_obj })
print json_obj_in_html
With Python 2.7 and json2html 1.0.1 this leads to this result:
If you receive a result like
<table border="1"><tr><th>data</th><td>[{"Agent Status": "sta...
it is very likely that json_obj is a str and not an array of objects. You can check this by inserting a statement like
print type(json_obj)
before jsonhtml.convert. I assume that type(json_obj) returns a <type 'str'> in your case and that is why the JSON like string appears in your html. To get it right you have to modify your code in that way that type(json_obj) returns <type 'list'>.
My list of dictionaries anomaly_list was already in a json format, so trying to convert it using json.dumps(anomaly_list, sort_keys=True) was turning into a string, which was not what I wanted.
I solved the issue by leaving my list of dictionaries as it is and this code now works:
json_obj_in_html = ''
for j in anomalies_list:
json_obj_in_html += json2html.convert(json = j)
It outputs what I wanted.
#gus42: thanks, your feedback made me understand where the real pb was.
Related
I am trying to read some json with the following format. A simple pd.read_json() returns ValueError: Trailing data. Adding lines=True returns ValueError: Expected object or value. I've tried various combinations of readlines() and load()/loads() so far without success.
Any ideas how I could get this into a dataframe?
{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
}
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}
The sample you have above isn't valid JSON. To be valid JSON these objects need to be within a JS array ([]) and be comma separated, as follows:
[{
"content": "kdjfsfkjlffsdkj",
"source": {
"name": "jfkldsjf"
},
"title": "dsldkjfslj",
"url": "vkljfklgjkdlgj"
},
{
"content": "djlskgfdklgjkfgj",
"source": {
"name": "ldfjkdfjs"
},
"title": "lfsjdfklfldsjf",
"url": "lkjlfggdflkjgdlf"
}]
I just tried on my machine. When formatted correctly, it works
>>> pd.read_json('data.json')
content source title url
0 kdjfsfkjlffsdkj {'name': 'jfkldsjf'} dsldkjfslj vkljfklgjkdlgj
1 djlskgfdklgjkfgj {'name': 'ldfjkdfjs'} lfsjdfklfldsjf lkjlfggdflkjgdlf
Another solution if you do not want to reformat your files.
Assuming your JSON is in a string called my_json you could do:
import json
import pandas as pd
splitted = my_json.split('\n\n')
my_list = [json.loads(e) for e in splitted]
df = pd.DataFrame(my_list)
Thanks for the ideas internet. None quite solved the problem in the way I needed (I had lots of newline characters in the strings themselves which meant I couldn't split on them) but they helped point the way. In case anyone has a similar problem, this is what worked for me:
with open('path/to/original.json', 'r') as f:
data = f.read()
data = data.split("}\n")
data = [d.strip() + "}" for d in data]
data = list(filter(("}").__ne__, data))
data = [json.loads(d) for d in data]
with open('path/to/reformatted.json', 'w') as f:
json.dump(data, f)
df = pd.read_json('path/to/reformatted.json')
If you can use jq then solution is simpler:
jq -s '.' path/to/original.json > path/to/reformatted.json
im trying to print the first object of this json file but it only prints the first char of it.
This is my code:
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
new_data = json.dumps(data, indent = 2)
print(str(new_data[0]))
result i was hoping for:
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}
actual result:
[
json.dump the first element of the response:
import json
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
first_elem = json.dumps(data[0], indent=2)
print(first_elem)
Apparently response.json() already is a dictionary.
So if you try first_element = data[0] you would get what you are looking for.
And then, if you want to make it pretty :
json.dumps(first_element, indent = 2)
If you wish a JSON object to behave like a dictionary , take a look at
json.loads
https://docs.python.org/2/library/json.html
Also :
What's the best way to parse a JSON response from the requests library?
json.dumps results in a string.
You are printing the first word of the string by doing [0]
For the output that you want, do:
print(new_data)
I'm new to python and trying to practice aggregating data from an api call
i have this script
r = requests.get('https://jsonplaceholder.typicode.com/users')
print r.text
which returns an array of objects in this format
[{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}]
I've been playing around and tried this to see if I can access the first object
print r.text[0]
And it didn't work. So how do I do this with python
You need to parse the JSON text:
import json
array = json.loads(r)
print array[0]
request.text returns Http response body.
so If you want to get first propery of json,
you should convert string to json object.
This works
result = r.text
print(type) # prints str
import json
result = json.loads(result)
print(result[0]) # (...)
Can some one tell me what I am doing wrong ?I am Getting this error..
went through the earlier post of similar error. couldn't able to understand..
import json
import re
import requests
import subprocess
res = requests.get('https://api.tempura1.com/api/1.0/recipes', auth=('12345','123'), headers={'App-Key': 'some key'})
data = res.text
extracted_recipes = []
for recipe in data['recipes']:
extracted_recipes.append({
'name': recipe['name'],
'status': recipe['status']
})
print extracted_recipes
TypeError: string indices must be integers
data contains the below
{
"recipes": {
"47635": {
"name": "Desitnation Search",
"status": "SUCCESSFUL",
"kitchen": "eu",
"active": "YES",
"created_at": 1501672231,
"interval": 5,
"use_legacy_notifications": false
},
"65568": {
"name": "Validation",
"status": "SUCCESSFUL",
"kitchen": "us-west",
"active": "YES",
"created_at": 1522583593,
"interval": 5,
"use_legacy_notifications": false
},
"47437": {
"name": "Gateday",
"status": "SUCCESSFUL",
"kitchen": "us-west",
"active": "YES",
"created_at": 1501411588,
"interval": 10,
"use_legacy_notifications": false
}
},
"counts": {
"total": 3,
"limited": 3,
"filtered": 3
}
}
You are not converting the text to json. Try
data = json.loads(res.text)
or
data = res.json()
Apart from that, you probably need to change the for loop to loop over the values instead of the keys. Change it to something the following
for recipe in data['recipes'].values()
There are two problems with your code, which you could have found out by yourself by doing a very minimal amount of debugging.
The first problem is that you don't parse the response contents from json to a native Python object. Here:
data = res.text
data is a string (json formatted, but still a string). You need to parse it to turn it into it's python representation (in this case a dict). You can do it using the stdlib's json.loads() (general solution) or, since you're using python-requests, just by calling the Response.json() method:
data = res.json()
Then you have this:
for recipe in data['recipes']:
# ...
Now that we have turned data into a proper dict, we can access the data['recipes'] subdict, but iterating directly over a dict actually iterates over the keys, not the values, so in your above for loop recipe will be a string ( "47635", "65568" etc). If you want to iterate over the values, you have to ask for it explicitly:
for recipe in data['recipes'].values():
# now `recipe` is the dict you expected
I'm trying to call API's from various cryptocurrency exchanges in PYTHON.
This is the API JSON-string that gets returned by the following URL (https://api.mintpal.com/v1/market/stats/uro/BTC)
[
{
"market_id": "210",
"coin": "Uro",
"code": "URO",
"exchange": "BTC",
"last_price": "0.00399700",
"yesterday_price": "0.00353011",
"change": "+13.23",
"24hhigh": "0.00450000",
"24hlow": "0.00353010",
"24hvol": "6.561",
"top_bid": "0.00374001",
"top_ask": "0.00399700"
}
]
I'm interested in getting the "Last Price", I print it using the following code.
import urllib2
import json
url = 'https://api.mintpal.com/v1/market/stats/URO/BTC'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
for item in URO_data:
URO_last_price = item['last_price']
print URO_last_price
So far so good. However, I'm trying to do the same thing for the Bittrex exchange using the following URL (https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-uro)
The JSON String returned looks as follows:
{
"success": true,
"message": "",
"result": [
{
"MarketName": "BTC-URO",
"High": 0.00479981,
"Low": 0.00353505,
"Volume": 30375.93454693,
"Last": 0.00391656,
"BaseVolume": 120.61056568,
"TimeStamp": "2014-07-29T17:54:35.897",
"Bid": 0.00393012,
"Ask": 0.00395967,
"OpenBuyOrders": 182,
"OpenSellOrders": 182,
"PrevDay": 0.00367999,
"Created": "2014-05-15T05:46:29.917"
}
]
}
This JSON string has a different structure then the one before, and I can't use my first code to get the value "LAST". However, I can work around it by printing 'string index', but that's not a solution.
url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
URO_String = str(URO_data)
last_price = URO_String[79:89]
URO_LastPrice = float(last_price)
print last_price
I want to get the value of "Last" in the second JSON string.
The Json is best thought of as a bunch of layers, and you need to get the information you need by going layer by layer.
The first layer you need to get is "result" with:
URO_data["result"]
REsult is an array, so you need to get the first item in the array by index
URO_data["result"][0]
From that object, you need to get the Last entry
URO_data["result"][0]["Last"]
That should work, but I haven't tested it.
You can access it as bellow,
>>>
>>> url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
>>> json_obj = urllib2.urlopen(url)
>>> URO_data = json.load(json_obj)
>>> URO_data['result']['Last']
0.00396129
>>>