This question already has answers here:
How can I extract a single value from a nested data structure (such as from parsing JSON)?
(5 answers)
Closed 4 years ago.
I'm trying to get the zip code for a particular city using zippopotam.us. I have the following code which works, except when I try to access the post code key which returns TypeError: expected string or buffer
r = requests.get('http://api.zippopotam.us/us/ma/belmont')
j = r.json()
data = json.loads(j)
print j['state']
print data['places']['latitude']
Full JSON output:
{
"country abbreviation": "US",
"places": [
{
"place name": "Belmont",
"longitude": "-71.4594",
"post code": "02178",
"latitude": "42.4464"
},
{
"place name": "Belmont",
"longitude": "-71.2044",
"post code": "02478",
"latitude": "42.4128"
}
],
"country": "United States",
"place name": "Belmont",
"state": "Massachusetts",
"state abbreviation": "MA"
}
Places is a list and not a dictionary. This line below should therefore not work:
print(data['places']['latitude'])
You need to select one of the items in places and then you can list the place's properties. So to get the first post code you'd do:
print(data['places'][0]['post code'])
I did not realize that the first nested element is actually an array. The correct way access to the post code key is as follows:
r = requests.get('http://api.zippopotam.us/us/ma/belmont')
j = r.json()
print j['state']
print j['places'][1]['post code']
In your code j is Already json data and j['places'] is list not dict.
r = requests.get('http://api.zippopotam.us/us/ma/belmont')
j = r.json()
print j['state']
for each in j['places']:
print each['latitude']
I'm using this lib to access nested dict keys
https://github.com/mewwts/addict
import requests
from addict import Dict
r = requests.get('http://api.zippopotam.us/us/ma/belmont')
j = Dict(r.json())
print j.state
print j.places[1]['post code'] # only work with keys without '-', space, or starting with number
Related
I have a list of countries and their cities on one website. I take all names of countries and their capitals from this list, and want to put them in JSON file like this:
[
{
"country": "Afghanistan",
"city": "Kabul"
},
{
"country": "Aland Islands",
"city": "Mariehamn"
}
]
there's my code:
cells = soup.table('td')
count = 0
cities_list.write('[\n')
for cell in cells:
if count == len(cells)-2:
break
else:
cities_list.write(json.dumps({"country": "{}".format(cells[count].getText()),
"city": "{}".format(cells[count].next_sibling.getText())},
indent=2))
count += 2
cities.list_write('\n]')
And my problem is that objects are not separated by comma:
[
{
"country": "Afghanistan",
"city": "Kabul"
}{
"country": "Aland Islands",
"city": "Mariehamn"
}
]
How can I make my objects separated by comma, and also is it possible to do without writing '\n]' at the end and beginning?
Python obviously has no way to know that you are writing a list of objects when you are writing them one at a time ... so just don't.
cells = soup.table('td')
cities = []
for cell in cells[:-2]:
cities.append({"country": str(cells[count].getText()),
"city": str(cells[count].next_sibling.getText())})
json.dump(cities, cities_list)
Notice also how "{}".format(value) is just a clumsy way to write str(value) (or just value if value is already a string) and how json.dump lets you pass an open file handle to write to.
This question already has answers here:
Find all occurrences of a key in nested dictionaries and lists
(12 answers)
Closed 2 years ago.
So basically I am web scraping a site and for that I need "id" of all the location from a complicated json content:
https://hilfe.diakonie.de/hilfe-vor-ort/marker-json.php?ersteller=&kategorie=0&text=&n=54.14365551060835&e=19.704533281249986&s=48.00384435890099&w=1.2035567187499874&zoom=20000
I have tried dict.items method but i am getting only 2 values that are in start of the dict and then a list start:
res = requests.get(url).json()
json_obj = res.items()
for key, value in json_obj:
if key == "id":
print(value)
json = {
"count": 17652,
"items": [
{
"lat": 51.17450581504055,
"lng": 10.007757153036533,
"count": 17652,
"north": 54.1425475,
"east": 15.0019,
"south": 48.0039543,
"west": 5.952813,
"elements": [
{
"id": "5836de61a581c245ae48806b",
"o": 'null'
},
{
"id": "5836de62a581c245ae48814b",
"o": 'null'
},
{
"id": "5836de57a581c245ae487944",
"o": 'null'
},
{
"id": "5836de64a581c245ae4882a8",
"o": 'null'
},
{
"id": "5836de54a581c245ae48772a",
"o": 'null'
},
{
"id": "5836de57a581c245ae487945",
"o": 'null'
}
]
}
]
}
The id attribute is nested inside arrays in the elements attribute of objects, which are in turn nested inside an array in the items attribute of the response. Use a list comprehension with 2 loops to extract them:
res = requests.get(url).json()
ids = [ele["id"] for v in res["items"] for ele in v["elements"]]
for id in ids:
print(id)
The JSON consists of a root dictionary with two key-value pairs. One is count, which is an integer, the other is items, which maps to a list of a single item. This item is a dictionary, which has several key-value pairs, one of which is elements, which is a list of dictionaries, each containing an id:
import requests
url = "https://hilfe.diakonie.de/hilfe-vor-ort/marker-json.php?ersteller=&kategorie=0&text=&n=54.14365551060835&e=19.704533281249986&s=48.00384435890099&w=1.2035567187499874&zoom=20000"
response = requests.get(url)
response.raise_for_status()
elements = response.json()["items"][0]["elements"]
# print only the first ten ids
for element in elements[:10]:
print(element["id"])
Output:
5836de61a581c245ae48806b
5836de62a581c245ae48814b
5836de57a581c245ae487944
5836de64a581c245ae4882a8
5836de54a581c245ae48772a
5836de57a581c245ae487945
5836de61a581c245ae48806c
5836de64a581c245ae4882aa
5836de57a581c245ae487947
5836de62a581c245ae48814d
>>>
Same thing but different - using operator.itemgetter.
items = operator.itemgetter('items')
elements = operator.itemgetter('elements')
eyedees = operator.itemgetter('id')
data = elements(items(json)[0])
stuff = map(eyedees,data)
print(list(stuff))
Uses json from the example in the question.
im trying to print the first object of this json file but it only prints the first char of it.
This is my code:
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
new_data = json.dumps(data, indent = 2)
print(str(new_data[0]))
result i was hoping for:
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}
actual result:
[
json.dump the first element of the response:
import json
response = requests.get("http://jsonplaceholder.typicode.com/users")
data = response.json()
first_elem = json.dumps(data[0], indent=2)
print(first_elem)
Apparently response.json() already is a dictionary.
So if you try first_element = data[0] you would get what you are looking for.
And then, if you want to make it pretty :
json.dumps(first_element, indent = 2)
If you wish a JSON object to behave like a dictionary , take a look at
json.loads
https://docs.python.org/2/library/json.html
Also :
What's the best way to parse a JSON response from the requests library?
json.dumps results in a string.
You are printing the first word of the string by doing [0]
For the output that you want, do:
print(new_data)
I'm new to python and trying to practice aggregating data from an api call
i have this script
r = requests.get('https://jsonplaceholder.typicode.com/users')
print r.text
which returns an array of objects in this format
[{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere#april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
}]
I've been playing around and tried this to see if I can access the first object
print r.text[0]
And it didn't work. So how do I do this with python
You need to parse the JSON text:
import json
array = json.loads(r)
print array[0]
request.text returns Http response body.
so If you want to get first propery of json,
you should convert string to json object.
This works
result = r.text
print(type) # prints str
import json
result = json.loads(result)
print(result[0]) # (...)
I'm trying to call API's from various cryptocurrency exchanges in PYTHON.
This is the API JSON-string that gets returned by the following URL (https://api.mintpal.com/v1/market/stats/uro/BTC)
[
{
"market_id": "210",
"coin": "Uro",
"code": "URO",
"exchange": "BTC",
"last_price": "0.00399700",
"yesterday_price": "0.00353011",
"change": "+13.23",
"24hhigh": "0.00450000",
"24hlow": "0.00353010",
"24hvol": "6.561",
"top_bid": "0.00374001",
"top_ask": "0.00399700"
}
]
I'm interested in getting the "Last Price", I print it using the following code.
import urllib2
import json
url = 'https://api.mintpal.com/v1/market/stats/URO/BTC'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
for item in URO_data:
URO_last_price = item['last_price']
print URO_last_price
So far so good. However, I'm trying to do the same thing for the Bittrex exchange using the following URL (https://bittrex.com/api/v1.1/public/getmarketsummary?market=btc-uro)
The JSON String returned looks as follows:
{
"success": true,
"message": "",
"result": [
{
"MarketName": "BTC-URO",
"High": 0.00479981,
"Low": 0.00353505,
"Volume": 30375.93454693,
"Last": 0.00391656,
"BaseVolume": 120.61056568,
"TimeStamp": "2014-07-29T17:54:35.897",
"Bid": 0.00393012,
"Ask": 0.00395967,
"OpenBuyOrders": 182,
"OpenSellOrders": 182,
"PrevDay": 0.00367999,
"Created": "2014-05-15T05:46:29.917"
}
]
}
This JSON string has a different structure then the one before, and I can't use my first code to get the value "LAST". However, I can work around it by printing 'string index', but that's not a solution.
url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
json_obj = urllib2.urlopen(url)
URO_data = json.load(json_obj)
URO_String = str(URO_data)
last_price = URO_String[79:89]
URO_LastPrice = float(last_price)
print last_price
I want to get the value of "Last" in the second JSON string.
The Json is best thought of as a bunch of layers, and you need to get the information you need by going layer by layer.
The first layer you need to get is "result" with:
URO_data["result"]
REsult is an array, so you need to get the first item in the array by index
URO_data["result"][0]
From that object, you need to get the Last entry
URO_data["result"][0]["Last"]
That should work, but I haven't tested it.
You can access it as bellow,
>>>
>>> url = 'https://bittrex.com/api/v1.1/public/getticker?market=btc-uro'
>>> json_obj = urllib2.urlopen(url)
>>> URO_data = json.load(json_obj)
>>> URO_data['result']['Last']
0.00396129
>>>