Develop Reference

How to import API data using Pandas?

I am trying to pull some data from EIA API, below is what I tried but I'm getting the error on the first line of code:
AttributeError: 'str' object has no attribute 'text'
Any help would be much appreciated!
call_eia = requests.get = 'https://api.eia.gov/v2/nuclear-outages/facility-nuclear-outages/data?api_key=XXXXXXXX'
data_eia=pd.read_csv(StringIO(call_eia.text))

You haven't requested anything from the API. Look carefully at your first line:
call_eia = requests.get = 'https://api.eia.gov/v2/nuclear-outages/facility-nuclear-outages/data?api_key=XXXXXXXX'
# ^ ^
There are 2 = signs, so what you're really doing is assigning the URL string to both your call_eia variable and the get attribute of the requests module, overwriting the function that was there originally. Then, when you try to pass call_eia to pd.read_csv(), instead of passing a requests object, you're just passing a string, the URL.
Try
call_eia = requests.get('https://api.eia.gov/v2/nuclear-outages/facility-nuclear-outages/data?api_key=XXXXXXXX')
instead and your code should work.

Python web scraping nested dict key pairs - AttributeError

I'm attempting to scrape PGA stats from the API below.
url = 'https://statdata.pgatour.com/r/stats/current/02671.json?userTrackingId=exp=1594257225~acl=*~hmac=464d3dfcda2b2ccb384b77ac7241436f25b7284fb2eb0383184f48cbdff33cc4'
response = requests.get(url)
pga_stats = response.json()
I would like to only select the nested keys identified in this image. I've been able to traverse to the 'year' key with the below code, but I receive the following AttributeError for anything beyond that.
test = pga_stats.get('tours')[0].get('years')
(prints reduced dictionary)
test = pga_stats.get('tours')[0].get('years').get('stats')
'list' object has no attribute 'get'
My end goal is to write this player data to a csv file. Any suggestions would be greatly appreciated.

pga_stats.get('tours')[0].get('years') returns a list, not a dict. You actually want to use the get method on it's first element, like this:
test = pga_stats.get('tours')[0].get('years')[0].get('stats')

Can't Get Python To Parse JSON From Site

I'm trying to get my Python script to parse some data (the price) from a specific json file on a site, but I am unable to get it working.
It can extract the whole page fine, but it cannot extract certain data just by itself.
Here is the JSON I am trying to extract data from:
[{
"id": 1696146,
"name": "Genos",
"photo_url": "https://hobbydb-production.s3.amazonaws.com/processed_uploads/collectible_photo/collectible_photo/image/324461/1556082253-24867-7610/Genos_Vinyl_Art_Toys_60fb245b-1af9-4ad1-a5a2-c90d3e8291a6_medium.jpg",
"preorder": false,
"price": "$40.00",
"price_after_discount": "$40.00",
"seller_username": "BatmanPajamas",
"url": "https://www.hobbydb.com/marketplaces/2/cart/1696146"
}]
Here is the code I have got that allows me to get the entire json:
import urllib.request, json
withurllib.request.urlopen("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2") as url:
data = json.loads(url.read().decode())
print(data)
I have tried various pieces of code, but everytime I get:
TypeError: list indices must be integers or slices, not str
Any ideas how I can parse the price from this JSON?

The outer brackets ([]) indicate the response returns a list of items. So, you need to loop over the indices of the list, then you can access what you're trying to access. Here's how I do it with requests
import requests
resp = requests.get("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2")
#requests has built-in support for json, so no need to import json module
for product in resp.json():
print(product["price"])

To iterate over json array:
for item in data:
for keys in item.keys():
print(item[keys])
to display only price
for item in data:
print(item['price'])

I think the problem you are having is because this JSON object starts with an array (which will be a list once we load it as a Python object). First, you need to use the json library from the standard lib. Then, you have to access the object using the list index, then the dict keys.
Try this:
import urllib.request, json
with urllib.request.urlopen("https://www.hobbydb.com/api/collectibles/for_sale_search?limit=5&original_site_id=10748&market_id=2") as url:
data = json.loads(url.read().decode())
print(data)
toy = data[0]
price = toy['price']
Also, keep in mind that the with keyword creates a context for parsing the JSON data, so once your script moves on to code outside of this context, you won't be able to access your price variable any longer, so you might want to assign or set that value to to another variable created outside of that context.

Python http request and loop over contents of JSON

I'm trying to learn Python and have following problem:
I get an error while running this as it cannot see the 'name' attribute in data.
It works when I grab one by one items from JSON. However when I want to do it in a loop it fails.
I assume my error is wrong request. That it cannot read JSON correctly and see attributes.
import requests
import json
def main():
req = requests.get('http://pokeapi.co/api/v2/pokemon/')
print("HTTP Status Code: " + str(req.status_code))
print(req.headers)
json_obj = json.loads(req.content)
for i in json_obj['name']:
print(i)
if __name__ == '__main__':
main()

You want to access the name attribute of the results attribute in your json_object like this:
for pokemon in json_obj['results']:
print (pokemon['name'])
I was able to guess that you want to access the results keys because I have looked at the result of
json_obj.keys()
that is
dict_keys(['count', 'previous', 'results', 'next'])

Because all pokemons are saved in a list which is under keyword results, so you firstly need to get that list and then iterate over it.
for result in json_obj['results']:
print(result['name'])

A couple things: as soon mentioned, iterating through json_obj['name'] doesn't really make sense - use json_obj['results'] instead.
Also, you can use req.json() which is a method that comes with the requests library by default. That will turn the response into a dictionary which you can then iterate through as usual (.iteritems() or .items(), depending if you're using Python 2 or 3).

How to access json in python

I am trying to access json object in python and I am running through different errors
this is the data
value = '{"0":{"created":"05-16-13","counter":3},"1":{"created":"05-17-13","counter":1},"2":{"created":"05-18-13","counter":1}}'
I will like to get
"05-16-13","counter":3
"05-18-13","counter":1
I did
for info in value:
print info['counter']
I keep getting a type error, any help?
TypeError: string indices must be integers, not str

Use json.loads to convert it into a Python dictionary:
import json
value = '{"0":{"created":"05-16-13","counter":3},"1":{"created":"05-17-13","counter":1},"2":{"created":"05-18-13","counter":1}}'
d = json.loads(value)
for key, info in d.items():
print info['counter']
The error you were getting before was because string objects should be indexed by integers.
Let's take a completely different string and see why:
'abcd'[0] # 'a'
'abcd'['xyx'] # What does this even mean? TypeError!
'{"0":{"created":"05-16-13","counter":3}"}'['couter'] # TypeError for the same reasons.

There is a json library that you can import and use in Python. You can see docs for Python 3 here and Docs for Python 2 here.
import json
value = '{"0":{"created":"05-16-13","counter":3},"1":{"created":"05-17-13","counter":1},"2":{"created":"05-18-13","counter":1}}'
value = json.loads(value)
print(value[0])

Because value is a string. You should parse the json in it to access to its elements:
import json
value = json.loads('{"0":{"created":"05-16-13","counter":3},"1":{"created":"05-17-13","counter":1},"2":{"created":"05-18-13","counter":1}}')
for info in value.items():
print info['counter']