I am trying to write my first API query with python. I am calling an extremely simple dataset. (http://api.open-notify.org/astros.json). This displays information about the number of people in space.
I can return the number, but I want to try and display the names. So far I have:
import requests
response = requests.get("http://api.open-notify.org/astros.json")
data = response.json()
print(data["number"])
Any help would be greatly appreciated.
If you want to get names or crafts just do this:
print("Fist name: ",data["people"][0]["name"])
print("Fist craft: ",data["people"][0]["craft"])
Eventually you can put it in a for loop like this:
for i in range(len(data["people"])):
print(data["people"][i]["name"])
You should iterate by data['people'] and then get the name:
for people in data['people']:
print(people['name'])
Related
This is my first time using Python and I'm tasked with the following: print a list of cities from this JSON: http://jsonplaceholder.typicode.com/users
I'm trying to print out a list that should read:
Gwenborough
Wisokyburgh
McKenziehaven
South Elvis
etc.
This is the code I have so far:
import json
import requests
response = requests.get("https://jsonplaceholder.typicode.com/users")
users = json.loads(response.text)
print(users)
When I run $python3 -i api.py (file is named api.py) I'm able to print the list from the JSON file in my terminal. However I have been stuck trying to figure out how to print the cities only. I'm assuming it would look something like users.address.city but any attempt at figuring out the code has resulted in the following error: AttributeError: 'list' object has no attribute 'address'.
Any help you could provide would be greatly appreciated. Thanks!
As users is a list, it should be:
print(users[0]['address']['city'])
This is how you can access nested properties in JSON response.
You can also loop over the users and print their city in the same format.
for user in users:
print(user['address']['city'])
You can get city name with user['address']['city']
and use loop to get all city names
like this
for user in users:
print(user['address']['city'])
output :
Gwenborough
Wisokyburgh
McKenziehaven
South Elvis
Roscoeview
South Christy
Howemouth
Aliyaview
Bartholomebury
Lebsackbury
[Program finished]
first of all i get this, why your loading(response.text) , instead requests package has a built in .json() method which is what you want to access nested data . so you could do something like this
response = requests.get("https://jsonplaceholder.typicode.com/users")
data = response.json()
# optional
print(data)
* loop through the addresses to get all the cities
for dt in data['address']:
# do what you want with the data returned
I currently want to scrape some data from an amazon page and I'm kind of stuck.
For example, lets take this page.
https://www.amazon.com/NIKE-Hyperfre3sh-Athletic-Sneakers-Shoes/dp/B01KWIUHAM/ref=sr_1_1_sspa?ie=UTF8&qid=1546731934&sr=8-1-spons&keywords=nike+shoes&psc=1
I wanted to scrape every variant of shoe size and color. That data can be found opening the source code and searching for 'variationValues'.
There we can see sort of a dictionary containing all the sizes and colors and, below that, in 'asinToDimentionIndexMap', every product code with numbers indicating the variant from the variationValues 'dictionary'.
For example, in asinToDimentionIndexMap we can see
"B01KWIUH5M":[0,0]
Which means that the product code B01KWIUH5M is associated with the size '8M US' (position 0 in variationValues size_name section) and the color 'Teal' (same idea as before)
I want to scrape both the variationValues and the asinToDimentionIndexMap, so i can associate the IndexMap numbers to the variationValues one.
Another person in the site (thanks for the help btw) suggested doing it this way.
script = response.xpath('//script/text()').extract_frist()
import re
# capture everything between {}
data = re.findall(script, '(\{.+?\}_')
import json
d = json.loads(data[0])
d['products'][0]
I can sort of understand the first part. We get everything that's a 'script' as a string and then get everything between {}. The issue is what happens after that. My knowledge of json is not that great and reading some stuff about it didn't help that much.
Is it there a way to get, from that data, 2 dictionaries or lists with the variationValues and asinToDimentionIndexMap? (maybe using some regular expressions in the middle to get some data out of a big string). Or explain a little bit what happens with the json part.
Thanks for the help!
EDIT: Added photo of variationValues and asinToDimensionIndexMap
I think you are close Manuel!
The following code will turn your scraped source into easy-to-select boxes:
import json
d = json.loads(data[0])
JSON is a universal format for storing object information. In other words, it's designed to interpret string data into object data, regardless of the platform you are working with.
https://www.w3schools.com/js/js_json_intro.asp
I'm assuming where you may be finding things a challenge is if there are any errors when accessing a particular "box" inside you json object.
Your code format looks correct, but your access within "each box" may look different.
Eg. If your 'asinToDimentionIndexMap' object is nested within a smaller box in the larger 'products' object, then you might access it like this (after running the code above):
d['products'][0]['asinToDimentionIndexMap']
I've hacked and slash a little bit so you can better understand the structure of your particular json file. Take a look at the link below. On the right-hand side, you will see "which boxes are within one another" - which is precisely what you need to know for accessing what you need.
JSON Object Viewer
For example, the following would yield "companyCompliancePolicies_feature_div":
import json
d = json.loads(data[0])
d['updateDivLists']['full'][0]['divToUpdate']
The person helping you before outlined a general case for you, but you'll need to go in an look at structure this way to truly find what you're looking for.
variationValues = re.findall(r'variationValues\" : ({.*?})', ' '.join(script))[0]
asinVariationValues = re.findall(r'asinVariationValues\" : ({.*?}})', ' '.join(script))[0]
dimensionValuesData = re.findall(r'dimensionValuesData\" : (\[.*\])', ' '.join(script))[0]
asinToDimensionIndexMap = re.findall(r'asinToDimensionIndexMap\" : ({.*})', ' '.join(script))[0]
dimensionValuesDisplayData = re.findall(r'dimensionValuesDisplayData\" : ({.*})', ' '.join(script))[0]
Now you can easily convert them to json as use them combine as you wish.
i'm currently working on a project that involves getting data from RiotGames API and would apprciate some help in this problem I can't seem to solve.
The code:
import requests
r = requests.get(url="https://na.api.pvp.net/api/lol/na/v1.3/stats/by-summoner/7250/ranked?season=SEASON2016&api_key=ABC420")
json_data = r.json()
get_summoner_data = (json_data["champions"])
print(get_summoner_data)
And this is the response
{"summonerId":7250,"modifyDate":1479970436000,"champions":[{"id":111,"stats":{"totalSessionsPlayed":2,"totalSessionsLost":2,"totalSessionsWon":0,"totalChampionKills":8,"totalDamageDealt":211063,"totalDamageTaken":42772,"mostChampionKillsPerSession":7,"totalMinionKills":318,"totalDoubleKills":1,"totalTripleKills":0,"totalQuadraKills":0,"totalPentaKills":0,"totalUnrealKills":0,"totalDeathsPerSession":15,"totalGoldEarned":21574,"mostSpellsCast":0,"totalTurretsKilled":1,"totalPhysicalDamageDealt":33376,"totalMagicDamageDealt":174433,"totalFirstBlood":0,"totalAssists":11,"maxChampionsKilled":7,"maxNumDeaths":8}},{"id":110,"stats":{"totalSessionsPlayed":3,"totalSessionsLost":3,"totalSessionsWon":0,"totalChampionKills":12,"totalDamageDealt":463303,"totalDamageTaken":49704,"mostChampionKillsPerSession":8,"totalMinionKills":675,"totalDoubleKills":0,"totalTripleKills":0,"totalQuadraKills":0,"totalPentaKills":0,"totalUnrealKills":0,"totalDeathsPerSession":15,"totalGoldEarned":35203,"mostSpellsCast":0,"totalTurretsKilled":2,"totalPhysicalDamageDealt":438295,"totalMagicDamageDealt":22391,"totalFirstBlood":0,"totalAssists":28,"maxChampionsKilled":8,"maxNumDeaths":6}}
So the respone is basically data of the players performance with different champions, in this case defined by their id's.
But let's say someone is playing the champion with the id 111, now I want to get all data connected to that id and then be able to print out certain parts ex: MaxDeaths and MaxGold.
But if no data for that id is found I would have a message like "Data not found".
As you can see here there is data for the champion-id 111, but how do I call to get only that data?
Side note for better understading:
A player is in a game with 10 other players, 5 on each team. Every single person is playing different champions. I want to see if the player has any prior experience with that champion (All that data), and will therefore check if the champion-id is in the summary. If it is, I will use that data to make some sort of performance analyse.
for i in json_data["champions"]:
if i["id"]=="111":
print("found")
print(i["stats"]["maxNumDeaths"])
else:
print("not found")
So there are a lot of solutions, you can make something like this:
def get_champion_info(data, id):
champions = [c for c in data if c['id'] == id]
if len(champions):
return champions[0]
return "Data not found"
champion_info = get_champion_info(json_data["champions"], 111)
But it's not a good solution to return different types of data (string and dict from one function)
In the API documentation, there is a method which I think answer your question:
GET /api/lol/{region}/v1.2/champion/{id}
So I guess that you could do:
r = requests.get(url="https://na.api.pvp.net/api/lol/na/v1.3/champion/{id}")
Of course, put the champion-id and your api key.
So I'm trying to learn Python here, and would appreciate any help you guys could give me. I've written a bit of code that asks one of my favorite websites for some information, and the api call returns an answer in a dictionary. In this dictionary is a list. In that list is a dictionary. This seems crazy to me, but hell, I'm a newbie.
I'm trying to assign the answers to variables, but always get various error messages depending on how I write my {},[], or (). Regardless, I can't get it to work. How do I read this return? Thanks in advance.
{
"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true
}
Edited because I put in the wrong sample code.
You need to show your code, but the de-facto way of doing this is by using the requests module, like this:
import requests
url = 'http://www.example.com/api/v1/something'
r = requests.get(url)
data = r.json() # converts the returned json into a Python dictionary
for item in data['answer']:
print(item['widgets'])
Assuming that you are not using the requests library (see Burhan's answer), you would use the json module like so:
data = '{"answer":
[{"widgets":16,
"widgets_available":16,
"widgets_missing":7,
"widget_flatprice":"156",
"widget_averages":15,
"widget_cost":125,
"widget_profit":"31",
"widget":"90.59"}],
"result":true}'
import json
data = json.loads(data)
# Now you can use it as you wish
data['answer'] # and so on...
First I will mention that to access a dictionary value you need to use ["key"] and not {}. see here an Python dictionary syntax.
Here is a step by step walkthrough on how to build and access a similar data structure:
First create the main dictionary:
t1 = {"a":0, "b":1}
you can access each element by:
t1["a"] # it'll return a 0
Now lets add the internal list:
t1["a"] = ["x",7,3.14]
and access it using:
t1["a"][2] # it'll return 3.14
Now creating the internal dictionary:
t1["a"][2] = {'w1':7,'w2':8,'w3':9}
And access:
t1["a"][2]['w3'] # it'll return 9
Hope it helped you.
I've crawled a tracklist of 36.000 songs, which have been played on the Danish national radio station P3. I want to do some statistics on how frequently each of the genres have been played within this period, so I figured the discogs API might help labeling each track with genre. However, the documentation for the API doesent seem to include an example for querying the genre of a particular song.
I have a CSV-file with with 3 columns: Artist, Title & Test(Test where i want the API to label each song with the genre).
Here's a sample of the script i've built so far:
import json
import pandas as pd
import requests
import discogs_client
d = discogs_client.Client('ExampleApplication/0.1')
d.set_consumer_key('key-here', 'secret-here')
input = pd.read_csv('Desktop/TEST.csv', encoding='utf-8',error_bad_lines=False)
df = input[['Artist', 'Title', 'Test']]
df.columns = ['Artist', 'Title','Test']
for i in range(0, len(list(df.Artist))):
x = df.Artist[i]
g = d.artist(x)
df.Test[i] = str(g)
df.to_csv('Desktop/TEST2.csv', encoding='utf-8', index=False)
This script has been working with a dummy file with 3 records in it so far, for mapping the artist of a given ID#. But as soon as the file gets larger(ex. 2000), it returns a HTTPerror when it cannot find the artist.
I have some questions regarding this approach:
1) Would you recommend using the search query function in the API for retrieving a variable as 'Genre'. Or do you think it is possible to retrieve Genre with a 'd.' function from the API?
2) Will I need to aquire an API-key? I have succesfully mapped the 3 records without an API-key so far. Looks like the key is free though.
Here's the guide I have been following:
https://github.com/discogs/discogs_client
And here's the documentation for the API:
https://www.discogs.com/developers/#page:home,header:home-quickstart
Maybe you need to re-read the discogs_client examples, i am not an expert myself, but a newbie trying to use this API.
AFAIK, g = d.artist(x) fails because x must be a integer not a string.
So you must first do a search, then get the artist id, then d.artist(artist_id)
Sorry for no providing an example, i am python newbie right now ;)
Also have you checked acoustid for
It's a probably a rate limit.
Read the status code of your response, you should find an 429 Too Many Requests
Unfortunately, if that's the case, the only solution is to add a sleep in your code to make one request per second.
Checkout the api doc:
http://www.discogs.com/developers/#page:home,header:home-rate-limiting
I found this guide:
https://github.com/neutralino1/discogs_client.
Access the api with your key and try something like:
d = discogs_client.Client('something.py', user_token=auth_token)
release = d.release(774004)
genre = release.genres
If you found a better solution please share.