Not very experience with API authentication but I cannot figure out how to read a Google Sheet share with me from Python.
I've tried:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope = ['https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name('/Users/alexiseggermont/Downloads/Accountable-7b29dac08324.json', scope)
gc = gspread.authorize(credentials)
wks = gc.open("mysheet").sheet1
This gives me ImportError: cannot import name 'opentype' on line 2.
I then tried:
import oauth2client.client, oauth2client.file, oauth2client.tools
import gspread
flow = oauth2client.client.OAuth2WebServerFlow(client_id, client_secret, 'https://spreadsheets.google.com/feeds')
storage = oauth2client.file.Storage('credentials.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
import argparse
flags = argparse.ArgumentParser(parents=[oauth2client.tools.argparser]).parse_args([])
credentials = oauth2client.tools.run_flow(flow, storage, flags)
gc = gspread.authorize(credentials)
# when this cell is run, your browser will take you to a Google authorization page.
# this authorization is complete, the credentials will be cached in a file named credentials.dat
This opens a window asking if I want to give my app access to my sheets, and I click yes. But then sheet = gc.open("mysheet").sheet1 gives me a permission error:
APIError: {
"error": {
"errors": [
{
"domain": "global",
"reason": "insufficientPermissions",
"message": "Insufficient Permission"
}
],
"code": 403,
"message": "Insufficient Permission"
}
}
The only suggestion I find to solve this error is to change the 'scope' variable, but there is no scope variable used in that code, so I am confused.
You can add several scopes using spaces. So can you try the following modification?
From :
flow = oauth2client.client.OAuth2WebServerFlow(client_id, client_secret, 'https://spreadsheets.google.com/feeds')
To :
flow = oauth2client.client.OAuth2WebServerFlow(client_id, client_secret, 'https://spreadsheets.google.com/feeds https://www.googleapis.com/auth/drive')
Note :
Before you run the modified script, please remove credentials.dat. By this, credentials.dat is created using new scopes.
In my environment, I confirmed that when the scopes is only https://spreadsheets.google.com/feeds, the same error occurs. When the scopes are https://spreadsheets.google.com/feeds https://www.googleapis.com/auth/drive, no error occurs.
there is also Gspread-Pandas library itself, where the docs have a simple intro to the Oauth part, and in a simpler way:
https://github.com/aiguofer/gspread-pandas
Before using, you will need to download Google client credentials for your app.
Client Credentials To allow a script to use Google Drive API we need
to authenticate our self towards Google. To do so, we need to create a
project, describing the tool and generate credentials. Please use your
web browser and go to Google console and :
Choose Create Project in popup menu on the top. A dialog box appears,
so give your project a name and click on Create button. On the
left-side menu click on API Manager. A table of available APIs is
shown. Switch Drive API and click on Enable API button. Do the same
for Sheets API. Other APIs might be switched off, for our purpose. On
the left-side menu click on Credentials. In section OAuth consent
screen select your email address and give your product a name. Then
click on Save button. In section Credentials click on Add credentials
and switch OAuth 2.0 client ID. A dialog box Create Cliend ID appears.
Select Application type item as Other. Click on Create button. Click
on Download JSON icon on the right side of created OAuth 2.0 client
IDs and store the downloaded file on your file system. Please be
aware, the file contains your private credentials, so take care of the
file in the same way you care of your private SSH key; i.e. move
downloaded JSON to ~/.config/gspread_pandas/google_secret.json (or you
can configure the directory and file name by directly calling
gspread_pandas.conf.get_config Thanks to similar project df2gspread
for this great description of how to get the client credentials.
User Credentials Once you have your client credentials, you can have
multiple user credentials stored in the same machine. This can be
useful when you have a shared server (for example with a Jupyter
notebook server) with multiple people that may want to use the
library. The first parameter to Spread must be the key identifying a
user's credentials. The first time this is called for a specific key,
you will have to authenticate through a text based OAuth prompt; this
makes it possible to run on a headless server through ssh or through a
Jupyter notebook. After this, the credentials for that user will be
stored (by default in ~/.config/gspread_pandas/creds or you can
manually set it in GSPREAD_PANDAS_CONFIG_DIR env var) and the tokens
will berefreshed automatically any time the tool is used.
Users will only be able to interact with Spreadsheets that they have
access to.
Handling Authentication In the backend, the library is leveraging
Google's oauth2client
<http://oauth2client.readthedocs.io/en/latest/__ to handle
authentication. It conveniently stores everything as described above
so that you don't have to worry about boiler plate code to handle
auth. However, if you need to customize how you handle authentication
you can do so in a few different ways. You can change the directory
where everything is stored using the GSPREAD_PANDAS_CONFIG_DIR env
var. You can also generate your own
oauth2client.client.OAuth2Credentials and pass them in when
instanciating a Client or Spread object. For other ways to customize
authentication, see gspread_pandas.conf.get_config and
gspread_pandas.conf.get_creds
Related
I am trying to do a personal project related to gathering data from YouTube, and I am having a heck of a time using the API methods that require one to authenticate with OAuth2. The ones you can use an API key with, I'm all set on, and that part of my little app is working great. But downloading the transcript of a video requires OAuth2.
As a starting point to figuring OAuth2 out, I got example Python code generated from the Try It sidebar on the Youtube API Reference page for the Captions Download method. I modified it to include the caption ID for one of the videos I am interested in, the JSON secrets file for the project's OAuth 2.0 Client ID of type desktop that I downloaded from the Google Cloud Console, and the file path where I wanted it to put the transcript.
Unfortunately, it does not work! I've included below (with possibly-secret info replaced with placeholders) the code I was using, the contents of the secrets JSON file, and the contents of the console with the error message.
The code includes a call to flow.run_console() which is marked in PyCharm as deprecated. When I hover my mouse over it for a couple seconds, this pops up above the documentation about the function:
New clients will be unable to use `InstalledAppFlow.run_console` starting on Feb 28, 2022. All clients will be unable to use this method starting on Oct 3, 2022. Use `InstalledAppFlow.run_local_server` instead. For details on the OOB flow deprecation, see https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html?m=1#disallowed-oob
At that site, I read about the security risks that made them deprecate this. Unfortunately, I could not figure out from that site how to replace it. Using credentials = flow.run_local_server() instead of credentials = flow.run_console() produced the exact same KeyError. It seems like the other options are all intended to be hosted on a website or used for mobile apps. I only ever want to use this app on my computer specifically.
It does seem to work as far as that line, or at least, it gives me a URL that lets me click, authorize my project to use my Youtube account, and paste the code back in. It's after I paste in the code that it fails. The error it gives me is actually about the credentials file not having a key with the name client_secret -- which, indeed, is missing from my credentials file. Maybe they stopped including that in general and never updated the example-code-generator on the API reference? When searching for other answers, I see a lot of references to creating an OAuth2 ClientID with the type of "Installed / other" but that is no longer an option in the type dropdown, as far as I can see.
Or, maybe it's related to the fact that my app is still listed with Google as in "testing" mode? I don't intend to release it to the world, I'm the only one who will ever use this app. It's staying on my computer, so I don't have a website to give Google so that they can check out my privacy policy or whatever other requirements they need for a released app.
Error message when running code:
C:\Users\myUserName\PycharmProjects\MyProjectName\venv\Scripts\python.exe C:/Users/acrai/PycharmProjects/MyProjectName/example_download_captions.py
flow_before
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=xxxxxxxxxxxxxxx.apps.googleusercontent.com&redirect_uri=urnxxxxxxoauth%3A2.0%3Aoob&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube.force-ssl&state=xxxxxxxxxx&prompt=consent&access_type=offline
Enter the authorization code: 4/1AX4blahblahblah-I clicked the link above, authorized with my Youtube account, and pasted in the code it gave me here!
Traceback (most recent call last):
File "C:\Users\myUserName\PycharmProjects\MyProjectName\example_download_captions.py", line 54, in <module>
main()
File "C:\Users\myUserName\PycharmProjects\MyProjectName\example_download_captions.py", line 36, in main
credentials = flow.run_console()
File "C:\Users\myUserName\PycharmProjects\MyProjectName\venv\lib\site-packages\google_auth_oauthlib\flow.py", line 439, in run_console
self.fetch_token(code=code)
File "C:\Users\myUserName\PycharmProjects\MyProjectName\venv\lib\site-packages\google_auth_oauthlib\flow.py", line 298, in fetch_token
kwargs.setdefault("client_secret", self.client_config["client_secret"])
KeyError: 'client_secret'
My code, very much based on the example code generated from the Try It sidebar on the Youtube API Reference page for the Captions Download method:
# https://developers.google.com/youtube/v3/docs/captions/download
# -*- coding: utf-8 -*-
# Sample Python code for youtube.captions.download
# NOTE: This sample code downloads a file and can't be executed via this
# interface. To test this sample, you must run it locally using your
# own API credentials.
# See instructions for running these code samples locally:
# https://developers.google.com/explorer-help/code-samples#python
import io
import os
import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors
from googleapiclient.http import MediaIoBaseDownload
scopes = ["https://www.googleapis.com/auth/youtube.force-ssl"]
def main():
# Disable OAuthlib's HTTPS verification when running locally.
# *DO NOT* leave this option enabled in production.
os.environ["OAUTHLIB_INSECURE_TRANSPORT"] = "1"
api_service_name = "youtube"
api_version = "v3"
client_secrets_file = "C:\\Users\\myUserName\\Documents\\MyProjectDirectoryName\\client_secret_xxxxxxxx.apps.googleusercontent.com.json"
result_file = 'C:\\Users\\myUserName\\Documents\\MyProjectDirectoryName\\Video Transcripts\\vid_example_transcript.txt'
test_caption_id = 'VxPQ0cBI7BNjG-6Kpo8Wi1H6VXAziegGD0Be0NlMJCg='
# Get credentials and create an API client
flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(
client_secrets_file, scopes)
print("flow_before")
credentials = flow.run_console()
print("flow_after")
youtube = googleapiclient.discovery.build(
api_service_name, api_version, credentials=credentials)
request = youtube.captions().download(
id=test_caption_id
)
fh = io.FileIO(result_file, "wb")
download = MediaIoBaseDownload(fh, request)
complete = False
while not complete:
status, complete = download.next_chunk()
if __name__ == "__main__":
main()
What is in the JSON secrets file:
{"installed": {
"client_id": "abcd-1234.apps.googleusercontent.com",
"project_id": "MyProjectName",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"redirect_uris": [
"http://localhost"
]
}}
Here's how I obtained this file:
I went to https://console.cloud.google.com/apis/api/youtube.googleapis.com/credentials?project=MyProjectName
Under "OAuth 2.0 Client IDs" on that page is the credential I made with the same name as the project name, the type is Desktop.
At the far right of that line with the credential is an Actions column, I hit the Download icon, and then in the box that pops up, I hit the Download JSON button.
So, does anyone know how I can change my code to make it work?
I'm working on a simple "bot" using the Google Chat API and I'm unable to send a message in a chat group.
In the Google Cloud Console I created a project, enabled the Google Chat API, generated OAuth credentials and in the OAuth consent screen I added this scope:
.../auth/chat
by manually entering:
https://www.googleapis.com/auth/chat
EDIT:
Initially I tried to enter the documentation suggested scope:
https://www.googleapis.com/auth/chat.bot
but it was not accepted and I got and error for an invalid scope from the cloud console interface. Also to double check this, when I tried to put it in a request I got the same error in the browser, so I came up with the working scope simply by trial and error.
Then I downloaded the credentials from the Credentials section:
client_secret_XXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com.json
Locally I'm using a Google.py python wrapper around the google python modules, located here:
https://learndataanalysis.org/google-py-file-source-code/
I made a slight change in this file to use console authentication instead of browser:
flow.run_local_server() -> flow.run_console()
My code looks like this:
from Google import Create_Service
CLIENT_SECRET_FILE = 'client_secret_XXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com.json'
API_NAME = 'chat'
API_VERSION = 'v1'
SCOPES = ['https://www.googleapis.com/auth/chat']
service = Create_Service(CLIENT_SECRET_FILE, API_NAME, API_VERSION, SCOPES)
text = 'My message'
space = 'spaces/XXXXXXXXXX'
chat_metadata = {
'text': text,
}
service.spaces().messages().create(parent=space, body=chat_metadata).execute()
The Create_Service call guides me through a console authorization, e.g.:
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=...
Enter the authorization code:
After I visit the link, I receive a token, which I enter in the console and I get this Python error:
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://chat.googleapis.com/v1/spaces/XXXXXXX/messages?alt=json returned "Request had insufficient authentication scopes.". Details: "[{'#type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'ACCESS_TOKEN_SCOPE_INSUFFICIENT', 'domain': 'googleapis.com', 'metadata': {'service': 'chat.googleapis.com', 'method': 'google.chat.v1.ChatService.CreateMessage'}}]">
Apparently I supplied the chat scope with the request and also, as I mentioned, I made sure it is added to the OAuth consent screen. I couldn't find any other chat scopes.
Is there some other step I'm missing or are there any other scopes, which I should use for creating messages?
BTW, because I noticed that in the Google Cloud Console this scope's description is "View and manage chat conversations and user state" and doesn't explicitly mention "creating", I tried other calls like: service.spaces().get(), or service.spaces().members().list(). However I still got the same error.
I should also mention that the same approach works fine with the Google Drive API for creating files, folders, creating permissions, etc.
I appreciate any help with this.
Always check the documentation for the method it will tell you what scope of permissions it needs.
In this case Method: spaces.messages.create needs the following scope.
Which is not the one you are using.
When writing an Azure Function in Python, I would expect to be able to access the host and function keys from the environment. Is this possible? All the examples I've seen do it by calling a get request, which seems like a lot of code to access something that I've set through the website.
This question is very similar, but not language specific.
It sounds like you want to get the response of the Host API admin/host/keys of Azure Functions as below, so please refer to Azure Functions wiki page Key management API
Here is my sample code.
# App Credentials, to get it see the figures below
username = "<your username like `$xxxxx`>"
password = "<your password>"
functionapp_name = "<your function app name>"
api_url = f"https://{functionapp_name}.scm.azurewebsites.net/api"
site_url = f"https://{functionapp_name}.azurewebsites.net"
import base64
import requests
auth_info = f"{username}:{password}"
base64_auth = base64.b64encode(str.encode(auth_info)).decode()
print(base64_auth)
jwt_resp = requests.get(f"{api_url}/functions/admin/token", headers={"Authorization": f"Basic {base64_auth}"})
jwt = jwt_resp.text.replace("\"", "", -1)
print(jwt)
keys_resp = requests.get(f"{site_url}/admin/host/keys", headers={"Authorization": f"Bearer {jwt}"})
print(keys_resp.text)
It works and its result as below.
For getting the username and password of App Credentials, please see the figures below.
Fig 1. On Azure portal, open the Platform features tab of your Function App and click the Deployment Center link
Fig 2. Select the FTP option in the first step of SOURCE CONTROL and click the Dashboard button to copy the values of Username and Password, but just use the part of Username with $ prefix as username variable in my script. Ofcouse, you also can use them in tab User Credentials tab.
Also, you can refer to my answer for the similar SO thread Unable to access admin URL of Azure Functions using PowerShell, and my figures below come from that.
Update: For using Azure Function for Python in container, please refer to the figure below to get the deployment credentials.
In a django application, I try to have RW access to a google calendar which I own myself.
Tried several ways with a service account & client secrets, but all resulting in authentication errors.
The API explorer works, but it requests consent in a popup window, which is obviously not acceptable.
Documentation on google OAuth2 describes several scenarios. Probably "web server application" applies here? It says:
"The authorization sequence begins when your application redirects a
browser to a Google URL; the URL includes query parameters that
indicate the type of access being requested. Google handles the user
authentication, session selection, and user consent. The result is an
authorization code, which the application can exchange for an access
token and a refresh token."
Again, we do not want a browser redirection, we want direct access to the google calendar.
So question is: how can a django server access a google calendar, on which I have full rights, view events and add events using a simple server stored key or similar mechanism?
With help of DalmTo and this great article, I got RW access to a google calendar working from python code. I will summarize the solution here.
Here are the steps:
First of all register for a google service account: Service accounts are pre-authorized accounts that avoid you need to get consent or refresh keys every time:
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
(The part on G-suite can be ignored)
Download the service account credentials and store them safely. Your python code will need access to this file.
Go to your google calendar you want to get access to.
e.g. https://calendar.google.com/calendar/r/month
On the right side you see your calendars. Create an additional one for testing (since we'll write to it soon). Then point to this new calendar: click the 3 dots next to it and edit the sharing settings. Add the service account email address to the share under "share with specific people". (you can find the service account email address in the file downloaded previously under "client_email")
In the same screen, note the "calendar ID", you'll need it in below code.
Now you service account has the RW rights to the calendar.
Add at least one event to the calendar using the web UI (https://calendar.google.com/calendar/r/month) so we can read and change it from below code.
Then use following python code to read the calendar and change an event.
from google.oauth2 import service_account
import googleapiclient.discovery
SCOPES = ['https://www.googleapis.com/auth/calendar']
SERVICE_ACCOUNT_FILE = '<path to your service account file>'
CAL_ID = '<your calendar ID>'
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = googleapiclient.discovery.build('calendar', 'v3', credentials=credentials)
events_result = service.events().list(calendarId=CAL_ID).execute()
events = events_result.get('items', [])
event_id = events[0]['id']
event = events[0]
service.events().update(calendarId=CAL_ID, eventId=event_id, body={"end":{"date":"2018-03-25"},"start":{"date":"2018-03-25"},"summary":"Kilroy was here"}).execute()
And there you go... read an event and updated the event.
I am trying to access my own docs & spreadsheets via onedrive's api. I have:
import requests
client_id = 'my_id'
client_secret = 'my_secret'
scopes = 'wl.offline_access%20wl.signin%20wl.basic'
response_type = 'token' # also have tried "code"
redirect_uri = 'https://login.live.com/oauth20_desktop.srf'
base_url = 'https://apis.live.net/v5.0/'
r = requests.get('https://login.live.com/oauth20_authorize.srf?client_id=%s&scope=%s&response_type=%s&redirect_uri=%s' % (client_id, scopes, response_type, redirect_uri))
print r.text
(For my client I've also tried both "Mobile or desktop client app:" set to "Yes" and "No")
This will return the html for the user to manually click on. Since the user is me and it's my account how do I access the API without user interaction?
EDIT #1:
For those confused on what I'm looking for it would be the equivalent of Google's Service Account (OAuth2): https://console.developers.google.com/project
You cannot "bypass" the user interaction.
However you are very close to getting it to work. If you want to gain an access token in python you have to do it through the browser. You can use the web browser library to open the default web browser. It will look something like this (your app must be a desktop app):
import webbrowser
webbrowser.open("https://login.live.com/oauth20_authorize.srf?client_id=foo&scope=bar&response_type=code&redirect_uri=https://login.live.com/oauth20_desktop.srf")
This will bring you to the auth page, sign in and agree to the terms (it will differ depending on scope). It will direct you to a page where the url looks like:
https://login.live.com/oauth20_desktop.srf?code=<THISISTHECODEYOUWANT>&lc=foo
Copy this code from the browser and have your python script take it as input.
You can then make a request as described here using the code you received from the browser.
You will receive a response described here