Develop Reference

Not sure of the Print Structure with YouTube v3 API

So I was creating a script to list information from Google's V3 YouTube API and I used the structure that was shown on their Site describing it, so I'm pretty sure I'm misunderstanding something.
I tried using the structure that was shown to print JUST the Video's Title as a test
and was expecting that to print, however it just throws an error. Error is below
Here's what I wrote below
import sys, json, requests
vidCode = input('\nVideo Code Here: ')
url = requests.get(f'https://youtube.googleapis.com/youtube/v3/videos?part=snippet%2CcontentDetails%2Cstatistics&id={vidCode}&key=(not sharing the api key, lol)')
text = url.text
data = json.loads(text)
if "kind" in data:
print(f'Video URL: youtube.com/watch?v={vidCode}')
print('Title: ', data['snippet.title'])
else:
print("The video could not be found.\n")
This did not work, however if I change snippet.title to just something like etag the print is successful.
I take it this is because the Title is further down in the JSON List.
I've also tried doing data['items'] which did work, but I also don't want to output a massive chunk of unformatted information, it's not pretty lol.
Another test I did was data['items.snippet.title'] to see if that was what I was missing, also no, that didn't work.
Any idea what I'm doing wrong?

you need to access the keys in the dictionary separately.
import sys, json, requests
vidCode = input('\nVideo Code Here: ')
url = requests.get(f'https://youtube.googleapis.com/youtube/v3/videos?part=snippet%2CcontentDetails%2Cstatistics&id={vidCode}&key=(not sharing the api key, lol)')
text = url.text
data = json.loads(text)
if "kind" in data:
print(f'Video URL: youtube.com/watch?v={vidCode}')
print('Title: ', data['items'][0]['snippet']['title'])
else:
print("The video could not be found.\n")
To be clear, you need to access the 'items' value in the dictionary which is a list, get the first item from that list, then get the 'snippet' sub object, then finally the title.

How to access specific element of API Get with Python's Request Package?

Hi I am currently using Python 3.X to run the following code:
import requests
jurisdiction = 'us'
name = 'netflix'
limit = 1
r = requests.get('https://opencorporates.com/reconcile?query={%22query%22:%22' + name + '%22,%20%22limit%22:' + str(limit) + ',%20%22jurisdiction_code%22:%22' + jurisdiction + '%22}')
print(type(r))
print(r.text)
The output of this is
<class 'requests.models.Response'>
{"result":[{"id":"/companies/gb/12022722","name":"AMAZON-UK LIMITED","type":[{"id":"/organization/organization","name":"Organization"}],"score":69.0,"match":false,"uri":"https://opencorporates.com/companies/gb/12022722"}],"duration":157.957621}
I want to be able to access the company name from the response and then add it to a list. So I can iterate over a bunch of names/jurisdictions and have a list at the end that I can export to csv (or whatever).
I think that using a to.json or json.dump or something like that might work but I am not sure exactly how to do this? I am open to importing more packages like pandas etc, should the need arise.

Import json and you can read the company names like this:-
import json
data = json.loads(r.text)
#initialize your list
namesList = []
for s in data['result']:
name = s['name']
namesList.append(name)

You can try this.
data = {"result":[{"id":"/companies/gb/12022722","name":"AMAZON-UK LIMITED","type":[{"id":"/organization/organization","name":"Organization"}],"score":69.0,"match":False,"uri":"https://opencorporates.com/companies/gb/12022722"}],"duration":157.957621}
for i in data['result']:
print(i['name'])

scraping data from json after using requests

i am trying to extract specific data from requested json file
so after passing Authorization and using requests.get i got my request , i think it is called dictionary for python coders and called json for javascript coders
it containt too much information that i dont need and i would like to extract one or two only
for example {"bio" : " hello world " }
and that json file contains more that one " bio "
for example i am scraping 100 accounts and i would like to extract all " bio " in one code
so i tried this :
from bs4 import BeautifulSoup
import requests
headers = {"Authorization" : "xxxx"}
req = requests.get('website', headers = headers)
data = req.text
soup = BeautifulSoup(data,'html.parser')
titles = soup.find_all('span',{'class':'bio'})
for title in titles :
print(title.text)
and didnt work , i tried multiple ideas with no success
if possible please write me a code that i can understande since iam trying to learn more about my mistakes
thanks

The Aphid library I created is perfect for this.
from command-prompt
py -m pip install Aphid
Then its just as easy as loading your json data and searching it with aphid.
import json
import Aphid
resp = requests.get(yoururl)
data = json.loads(resp.text)
results = Aphid.findall(data, 'bio')
results is now equal to a list of tuples(key, value), of every occurence of the 'bio' key.

After you get your request either:
you get a simple json file (in which case you import it to python using json) or
you get an html file from which you can extract the json code (using BeautifulSoup) which in turn you will parse using json library.

python unable to parse JSON Data

I am unable to parse the JSON data using python.
A webpage url is returning JSON Data
import requests
import json
BASE_URL = "https://www.codechef.com/api/ratings/all"
data = {'page': page, 'sortBy':'global_rank', 'order':'asc', 'itemsPerPage':'40' }
r = requests.get(BASE_URL, data = data)
receivedData = (r.text)
print ((receivedData))
when I printed this, I got large text and when I validated using https://jsonlint.com/ it showed VALID JSON
Later I used
import requests
import json
BASE_URL = "https://www.codechef.com/api/ratings/all"
data = {'page': page, 'sortBy':'global_rank', 'order':'asc', 'itemsPerPage':'40' }
r = requests.get(BASE_URL, data = data)
receivedData = (r.text)
print (json.loads(receivedData))
When I validated the large printed text using https://jsonlint.com/ it showed INVALID JSON
Even if I don't print and directly use the data. It is working properly. So I am sure even internally it is not loading correctly.
is python unable to parse the text to JSON properly?

in short, json.loads converts from a Json (thing, objcet, array, whatever) into a Python object - in this case, a Json Dictionary. When you print that, it will print as a itterative and therefore print with single quotes..
Effectively your code can be expanded:
some_dictionary = json.loads(a_string_which_is_a_json_object)
print(some_dictionary)
to make sure that you're printing json-safe, you would need to re-encode with json.dumps

When you use python's json.loads(text) it returns a python dictionary. When you print that dictionary out it is not in json format.
If you want a json output you should use json.dumps(json_object).

JSON read from sparkcore to python

I have searched the web but couldn't find a suitable answer so I will try and ask here.
I am experimenting with a spark core and parsing data through JSON. I have already managed to read the data and print it with the following code:
import urllib, json
from pprint import pprint
url = "https://api.spark.io/v1/devices/mycore/result?access_token=accesstoken"
response = urllib.urlopen(url);
data = json.loads(response.read())
pprint(data)
And now I am trying to print the value I am sending with this code:
data["result"]["data1"]
I found the above in another topic but I am probably to unexperienced to properly apply it to my own code.
This is what python prints:
{u'cmd': u'VarReturn',
u'coreInfo': {u'connected': True,
u'deviceID': u'1111111111111111111',
u'last_app': u'',
u'last_handshake_at': u'2015-03-09T12:28:20.271Z',
u'last_heard': u'2015-03-09T12:56:42.780Z'},
u'name': u'result',
u'result': u'{"data1":2869}'}
the error I get says the following: TypeError: string indices must be integers
I used the example code from this topic:
https://community.spark.io/t/example-logging-and-graphing-data-from-your-spark-core-using-google/2929
I hope I am clear, can anyone enlighten me?

Try to print out data["result"]. From python print you have provided, the output should be '{"data1":2869}', which is another json object.
Try something like this:
import urllib, json
from pprint import pprint
url = "https://api.spark.io/v1/devices/mycore/result?access_token=accesstoken"
response = urllib.urlopen(url);
data = json.loads(response.read())
pprint(data)
new_data = json.loads(data["result"])
print new_data["data1"]

The contents of data["result"] is a unicode string. The string contains something that looks like a JSON doc / Python dictionary (see the single quotes around the whole construction):
>>> data["result"]
u'{"data1":2869}'