Develop Reference

Google API OAuth 2 sign in something went wrong with new OAuth 2 client

I am trying to make a discord bot that interacts with the google API specifically the Google Classroom API, so therefore I made a new project from the google console and created a new OAuth client for a web application. I enabled the Classroom API as well and selected all the scopes that I wanted to use:
['https://www.googleapis.com/auth/classroom.course-work.readonly',
'https://www.googleapis.com/auth/classroom.student-submissions.students.readonly',
'https://www.googleapis.com/auth/classroom.courses.readonly']
Then I set up my python programme using Google's example (At first I wrote my own using the documentation but got the same result). When I run the example code everything goes fine, it opens the browser and asks me to select my account, I select my school account and when it loads and I expect an Authorization screen to pup up to ask me if I allow the requested data it says something went wrong with no error messages at all. I have downloaded the correct credentials.json folder from the google dashboard and used it in my programme.
I will also provide the simplified code that I wrote maybe it's a problem there.
import pickle
import os
from google_auth_oauthlib.flow import Flow, InstalledAppFlow
from googleapiclient.discovery import build
from google.auth.transport.requests import Request
CLIENTSECRETPATH = "credentials.json"
APISERVICENAME = "classroom"
APIVERSION = "v1"
SCOPES = ['https://www.googleapis.com/auth/classroom.course-work.readonly', 'https://www.googleapis.com/auth/classroom.student-submissions.students.readonly', 'https://www.googleapis.com/auth/classroom.courses.readonly']
cred = None
if os.path.exists("toke.pickle"):
with open("tiken.pickle", "rb") as token:
cred = pickle.load(token)
if not cred or not cred.valid:
if cred and cred.expired and cred.refresh_token:
cred.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(CLIENTSECRETPATH, SCOPES)
cred = flow.run_local_server()
with open("token.pickle", "wb") as token:
pickle.dump(cred, token)
try:
service = build(APISERVICENAME, APIVERSION, credentials=cred)
except Exception as e:
print(e)
Edit:
I tried to change some settings on the google console and randomly decided to click publish as the project was still in the testing state, after doing this I could sign in with no errors. But that still doesn't explain why it didn't work when it was in a testing status, I added my school e-mail address to the test users list and made sure I did everything right for testing.
something went wrong screenshot

I can't reply to the above issue (need more reputation) but can confirm that I am seeing the same behaviour. What makes it even more weird is that the issue only presents itself when the user attempts to perform the OAuth integration with an account that is already signed in. The user is presented with a generic “Sorry, something went wrong there. Try again.” error before even seeing the required scopes list. However if the user is not logged into their account, and logs in as part of the OAuth integration, then there is no error and integration can be completed successfully.
The fact that this issue doesn't affect users who aren't logged in shows that the setup (callback API, credentials.json, etc.) is all correct. I believe this issue has been introduced in the last month or so.

I am seeing a similar Issue to Alex's post. Adding what I've seen so far in hopes to help debug.
When the OAuth flow is initiated in a session that has authenticated accounts (using the Account Picker), the Oauth flow fails with /unknownerror in the forward URL and the user is presented with the generic "Sorry, something went wrong there. Try again."
But, If the OAuth flow is initiated in a session where the User needs to sign-in to their Google account, the flow works as expected.
I do suspect this to be an Error with a Test App and Test Accounts. But hoping this will get fixed or to find some workaround.
I have found that the scopes have an affect as well. With basic scopes (profile,email) the error flow does not occur. But once you add another more restricted scope, the error flow returns.

This does not solve the problem when one actually needs the restricted scopes, nor does it allow for testing the app, which is particularly important for restricted scopes.
To make testing work, it seems that non-Chromium browsers will simply fail. You must use a Chromium-based browser to get the authorization code.
After many tries getting this error with the latest Firefox (version 102) I installed the snap of Midori and it worked with no problems. Now that the app is authorized I can switch back to Firefox.

How do I download YouTube subtitles via Python Shell?

I've been trying to work out how to download YouTube captions via the API but the official instructions are tailored towards command line code whereas I'm trying to do this in Python Shell.
Currently I've been following this page to no avail - https://developers.google.com/youtube/v3/docs/captions/list
What seems to trip me up is the Storage and args related pieces of code which even after much googling doesn't make any sense to me.
See the storage code below:
storage = Storage("%s-oauth2.json" % sys.argv[0])
#nowhere on the page does it refer to this oauth2.json file
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run_flow(flow, storage, args)
#if credentials is none why would storage still be needed as an argument?
In the second half of the page, it's all adding arguments to args.parser which seems to be command line stuff that I don't want to use as I'm working from Python Shell.
I've also been reading the official page on getting authorization OAuth 2.0 for YouTube API so I can rewrite this code myself but I can't get past this part: https://developers.google.com/api-client-library/python/auth/web-app
Step 5: Exchange authorization code for refresh and access tokens
After the web server receives the authorization code, it can exchange the authorization code for an access token.
On your callback page, use the google-authlibrary to verify the authorization server response. Then, use the flow.fetch_tokenmethod to exchange the authorization code in that response for an access token:
state = flask.session['state']
this is error I'm getting from the above code statement:
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
Essentially from lots of digging regarding the Oauth 2.0 process, I think I am unable to get the flow.fetch_token to swap my authorization code for the access token.
I've tested all my credentials so they are fine, it's getting it authorized that I'm struggling with.
This is the code so far I'm writing as a simplified version of the Captions API code on the first link:
import google.oauth2.credentials
import google_auth_oauthlib.flow
flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file("//client_secret_file.json", scopes=["https://www.googleapis.com/auth/youtube.force-ssl"])
flow.redirect_uri = "http://localhost:8080"
authorization_url, state = flow.authorization_url(access_type="offline", include_granted_scopes="true")
flask.redirect(authorization_url)
[Response 929 bytes [302 FOUND]]
state = flask.session['state']
[RuntimeError]
Summary:
How can I download YouTube captions from code in Python Shell? I've set up my credentials, just can't get the web server to give me the access token so I can execute the rest of the code. I would use the code on the captions.list link but I cannot get past the Storage code statement and args.arguments which is written for command line.

The following solution doesn't work anymore, I recommend switching to my other StackOverflow answer.
With Python 3 you can easily grab YouTube video subtitles in XML format by accessing the URL: https://video.google.com/timedtext?lang=en&v=VIDEO_ID
Here is another URL listing available subtitles available for a given YouTube video: http://video.google.com/timedtext?type=list&v=VIDEO_ID

How to Create Python Code with Google API Client

I have code below that was given to me to list Google Cloud Service Accounts for a specific Project.
import os
from googleapiclient import discovery
from gcp import get_key
"""gets all Service Accounts from the Service Account page"""
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = get_key()
service = discovery.build('iam', 'v1')
project_id = 'projects/<google cloud project>'
request = service.projects().serviceAccounts().list(name=project_id)
response = request.execute()
accounts = response['accounts']
for account in accounts:
print(account['email'])
This code works perfectly and prints the accounts as I need them. What I'm trying to figure out is:
Where can I go to see how to construct code like this? I found a site that has references to the Python API Client, but I can't seem to figure out how to make the code above from it. I can see the Method to list the Service Accounts, but it's still not giving me enough information.
Is there somewhere else I should be going to educate myself. Any information you have is appreciated so I don't pull out the rest of my hair.
Thanks, Eric

Let me share with you this documentation page, where there is a detailed explanation on how to build a script such as the one you shared, and what does each line of code mean. It is extracted from the documentation of ML Engine, not IAM, but it is using the same Python Google API Client Libary, so just ignore the references to ML and the rest will be useful for you.
In any case, here it is a commented version of your code, so that you understand it better:
# Imports for the Client API Libraries and the key management
import os
from googleapiclient import discovery
from gcp import get_key
# Look for an environment variable containing the credentials for Google Cloud Platform
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = get_key()
# Build a Python representation of the REST API
service = discovery.build('iam', 'v1')
# Define the Project ID of your project
project_id = 'projects/<google cloud project>'
"""Until this point, the code is general to any API
From this point on, it is specific to the IAM API"""
# Create the request using the appropriate 'serviceAccounts' API
# You can substitute serviceAccounts by any other available API
request = service.projects().serviceAccounts().list(name=project_id)
# Execute the request that was built in the previous step
response = request.execute()
# Process the data from the response obtained with the request execution
accounts = response['accounts']
for account in accounts:
print(account['email'])
Once you understand the first part of the code, the last lines are specific to the API you are using, which in this case is the Google IAM API. In this link, you can find detailed information on the methods available and what they do.
Then, you can follow the Python API Client Library documentation that you shared in order to see how to call the methods. For instance, in the code you shared, the method used depends on service, which is the Python representation of the API, and then goes down the tree of methods in the last link as in projects(), then serviceAccounts() and finally the specificlist() method, which ends up in request = service.projects().serviceAccounts().list(name=project_id).
Finally, just in case you are interested in the other available APIs, please refer to this page for more information.
I hope the comments I made on your code were of help, and that the documentation shared makes it easier for you to understand how a code like that one could be scripted.

You can use ipython having googleapiclient installed - with something like:
sudo pip install --upgrade google-api-python-client
You can go to interactive python console and do:
from googleapiclient import discovery
dir(discovery)
help(discovery)
dir - gives all entries that object has - so:
a = ''
dir(a)
Will tell what you can do with string object. Doing help(a) will give help for string object. You can do dipper:
dir(discovery)
# and then for instance
help(discovery.re)
You can call your script in steps, and see what is result print it, do some research, having something - do %history to printout your session, and have solution that can be triggered as a script.

Is it possible to store authentication token in a separate file?

I am building a command line tool using python that interfaces with an RESTful api. The API uses oauth2 for authentication. Rather than asking for access_token every time user runs the python tool. Can I store the access_token in some way so that I can use it till its lifespan? If it is then how safe it is.

You can store the access token in a file on your user's desktop.
You can do so using a storage. Assuming you use oauth2client:
# Reading credentials
store = oauth2client.file.Storage(cred_path)
credentials = store.get()
# Writing credentials
creds = client.AccessTokenCredentials(access_token, user_agent)
creds.access_token = access_token
creds.refresh_token = refresh_token
creds.client_id = client_id
creds.client_secret = client_secret
# For some reason it does not save all the credentials,
# so write them to a json file manually instead
with open(credential_path, "w") as f:
f.write(creds.to_json)
In terms of security, I would not see much of a threat here as these access tokens will be on a user's desktop. If someone wants to get their access token, they would need to have read access to that file for that time frame. However, if they can already do that, they most likely also can use your script to send them a copy of the user's access token every time it is authenticated. But take my word lightly as I'm not a professional in that area. See information security stack exchange.
A post in information security stack exchange did talk about this:
these tokens give access to some fairly privileged information about your users.
However, the question was addressed to a database instead.
In conclusion, you can keep it in a file. (But take my word with a grain of salt)

Do you want to store it on the service side or locally?
Since your tool interfaces RESTful API, which is stateless, meaning that no information is stored between different requests to API, you actually need to provide access token every time your client accesses any of the REST endpoints. I am maybe missing some of the details in your design, but access tokens should be used only for authorization, since your user is already authenticated if he has a token. This is why tokens are valid only for a certain amount of time, usually 1 hour.
So you need to provide a state either by using cookie (web interface) or storing the token locally (Which is what you meant). However, you should trigger the entire oauth flow every time a user logs in to your client (authenticating user and providing a new auth token) otherwise you are not utilizing the benefits of oauth.

Using Google Calendar API v 3 with Python

Can someone please give me a clear explanation of how to get the Google Calendar API v3 working with the Python Client? Specifically, the initial OAuth stage is greatly confusing me. All I need to do is access my own calendar, read it, and make changes to it. Google provides this code for configuring my app:
import gflags
import httplib2
from apiclient.discovery import build
from oauth2client.file import Storage
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.tools import run
FLAGS = gflags.FLAGS
# Set up a Flow object to be used if we need to authenticate. This
# sample uses OAuth 2.0, and we set up the OAuth2WebServerFlow with
# the information it needs to authenticate. Note that it is called
# the Web Server Flow, but it can also handle the flow for native
# applications
# The client_id and client_secret are copied from the API Access tab on
# the Google APIs Console
FLOW = OAuth2WebServerFlow(
client_id='YOUR_CLIENT_ID',
client_secret='YOUR_CLIENT_SECRET',
scope='https://www.googleapis.com/auth/calendar',
user_agent='YOUR_APPLICATION_NAME/YOUR_APPLICATION_VERSION')
# To disable the local server feature, uncomment the following line:
# FLAGS.auth_local_webserver = False
# If the Credentials don't exist or are invalid, run through the native client
# flow. The Storage object will ensure that if successful the good
# Credentials will get written back to a file.
storage = Storage('calendar.dat')
credentials = storage.get()
if credentials is None or credentials.invalid == True:
credentials = run(FLOW, storage)
# Create an httplib2.Http object to handle our HTTP requests and authorize it
# with our good Credentials.
http = httplib2.Http()
http = credentials.authorize(http)
# Build a service object for interacting with the API. Visit
# the Google APIs Console
# to get a developerKey for your own application.
service = build(serviceName='calendar', version='v3', http=http,
developerKey='YOUR_DEVELOPER_KEY')
But (a) it makes absolutely no sense to me; the comment explanations are terrible, and (b) I don't know what to put in the variables. I've registered my program with Google and signed up for a Service Account key. But all that gave me was an encrypted key file to download, and a client ID. I have no idea what a "developerKey" is, or what a "client_secret" is? Is that the key? If it is, how do I get it, since it is actually contained in an encrypted file? Finally, given the relatively simple goals of my API use (i.e., it's not a multi-user, multi-access operation), is there a simpler way to be doing this? Thanks.

A simple (read: way I've done it) way to do this is to create a web application instead of a service account. This may sound weird since you don't need any sort of web application, but I use this in the same way you do - make some queries to my own calendar/add events/etc. - all from the command line and without any sort of web-app interaction. There are ways to do it with a service account (I'll tinker around if you do in fact want to go on that route), but this has worked for me thus far.
After you create a web application, you will then have all of the information indicated above (side note: the sample code above is based on a web application - to use a service account your FLOW needs to call flow_from_clientsecrets and further adjustments need to be made - see here). Therefore you will be able to fill out this section:
FLOW = OAuth2WebServerFlow(
client_id='YOUR_CLIENT_ID',
client_secret='YOUR_CLIENT_SECRET',
scope='https://www.googleapis.com/auth/calendar',
user_agent='YOUR_APPLICATION_NAME/YOUR_APPLICATION_VERSION')
You can now fill out with the values you see in the API console (client_id = the entire Client ID string, client_secret = the client secret, scope is the same and the user_agent can be whatever you want). As for the service line, developerKey is the API key you can find under the Simple API Access section in the API console (label is API key):
service = build(serviceName='calendar', version='v3', http=http,
developerKey='<your_API_key>')
You can then add in a simple check like the following to see if it worked:
events = service.events().list(calendarId='<your_email_here>').execute()
print events
Now when you run this, a browser window will pop up that will allow you to complete the authentication flow. What this means is that all authentication will be handled by Google, and the authentication response info will be stored in calendar.dat. That file (which will be stored in the same directory as your script) will contain the authentication info that the service will now use. That is what is going here:
storage = Storage('calendar.dat')
credentials = storage.get()
if credentials is None or credentials.invalid == True:
credentials = run(FLOW, storage)
It checks for the existence of valid credentials by looking for that file and verifying the contents (this is all abstracted away from you to make it easier to implement). After you authenticate, the if statement will evaluate False and you will be able to access your data without needing to authenticate again.
Hopefully that shines a bit more light on the process - long story short, make a web application and use the parameters from that, authenticate once and then forget about it. I'm sure there are various points I'm overlooking, but hopefully it will work for your situation.

Google now has a good sample application that gets you up and running without too much fuss. It is available as the "5 minute experience - Quickstart" on their
Getting Started page.
It will give you a URL to visit directly if you are working on a remote server without a browser.