I'm trying to move a Python Jupyter scraper script (and json cred file) from my laptop to Google Colab.
I've made a connection between Google Colab and Google Drive.
I've stored the (.ipynb) script and credential JSON file on Google Drive.
However I can't make the connection between the 2 (gdrive json cred file and colab) to make it work.
Here below the part of the script concerning the credentials handling:
# Sheet key
# 1i1bmMt-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_d7Eo
import gspread
import pandas as pd
import requests
from bs4 import BeautifulSoup
from oauth2client.service_account import ServiceAccountCredentials
# Access credentials for google sheet and access the google sheet
scope = ["https://spreadsheets.google.com/feeds",
"https://www.googleapis.com/auth/spreadsheets",
"https://www.googleapis.com/auth/drive.file",
"https://www.googleapis.com/auth/drive"]
# Copy your path to your credential JSON file.
PATH_TO_CREDENTIAL = '/Users/user/json-keys/client_secret.json'
# Initiate your credential
credentials = ServiceAccountCredentials.from_json_keyfile_name(PATH_TO_CREDENTIAL, scope)
# Authorize your connection to your google sheet
gc = gspread.authorize(credentials)
I receive FileNotFoundError: and credential erros
Hope someone can help me with this, thanks
You try to put the file to the same directory to test it first. Make sure that the file is okay and can run successfully.
Here's the source code for reference:
If client_secret.json is in the same directory as the file you're running, then the correct syntax is:
import os
DIRNAME = os.path.dirname(__file__)
credentials = ServiceAccountCredentials.from_json_keyfile_name(os.path.join(DIRNAME, 'client_secret.json'), scope)
If the above test is okay, then try to move the file to your target directory '/Users/user/json-keys/client_secret.json' and try to create a symbolic link in the current directory to link the client_secret.json file. Then, run the program with the above code to test it again. Make sure it has no problem when putting the file to that directory. It's a workaround.
I used this case for reference to this:
Django not recognizing or seeing JSON file
Related
I am getting all the data present in google sheet using code below,
i want to write all these data to the pdf file and download that.
import gspread
import sys
print(sys.path)
import os
#sys.path.append('/usr/lib/python3/dist-packages')
from oauth2client.service_account import ServiceAccountCredentials
scope = [
'https://www.googleapis.com/auth/spreadsheets',
'https://www.googleapis.com/auth/drive'
]
path = os.path.abspath('cred.json')
credentials=ServiceAccountCredentials.from_json_keyfile_name('cred.json',scope)
client=gspread.authorize(credentials)
sheet=client.open('xyz').sheet1
data=sheet.get_all_records()
print(data)
I believe your goal as follows.
You want to export Google Spreadsheet of xyz as a PDF file using gspread with python and the service acccount.
Modification points:
Unfortunately, it seems that in the current stage, the Spreadsheet cannot be directly export as a PDF file using gspread. So in this case, requests library and the endpoint for exporting Spreadsheet to PDF are used.
When the points are reflected to your script, it becomes as follows.
Modified script:
import gspread
import sys
print(sys.path)
import os
#sys.path.append('/usr/lib/python3/dist-packages')
from oauth2client.service_account import ServiceAccountCredentials
scope = [
'https://www.googleapis.com/auth/spreadsheets',
'https://www.googleapis.com/auth/drive'
]
path = os.path.abspath('cred.json')
creds=ServiceAccountCredentials.from_json_keyfile_name('cred.json',scope)
client=gspread.authorize(creds)
# I added below script
spreadsheet_name = 'xyz'
spreadsheet = client.open(spreadsheet_name)
url = 'https://docs.google.com/spreadsheets/export?format=pdf&id=' + spreadsheet.id
headers = {'Authorization': 'Bearer ' + creds.create_delegated("").get_access_token().access_token}
res = requests.get(url, headers=headers)
with open(spreadsheet_name + ".pdf", 'wb') as f:
f.write(res.content)
Note:
In this modified script, it supposes that you hav ealready been able to get values from Google Spreadsheet using Sheets API. Please be careful this.
If an error related to Drive API, please enable Drive API at the API console.
If an error related to the service account, please modify create_delegated("") to create_delegated("email of the service account").
Set-up
I have a local folder – access_google_drive – which contains the .py script used to access my Google Drive account via the Google Drive API.
The script looks like this,
def connect_google_drive_api():
import os
# use Gdrive API to access Google Drive
os.chdir('/Users/my/fol/ders/access_google_drive')
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LocalWebserverAuth() # client_secrets.json need to be in the same directory as the script
drive = GoogleDrive(gauth)
return drive
In the access_google_drive I also have the client_secrets.json file.
Issue
This set-up worked fine, until yesterday. Since yesterday, I see the following error,
Failed to find "code" in the query parameters of the redirect.
Try command-line authentication
Traceback (most recent call last):
File "<ipython-input-8-48c5ad9148cf>", line 2, in <module>
gauth.LocalWebserverAuth() # client_secrets.json need to be in the same directory as the script
File "/opt/anaconda3/lib/python3.7/site-packages/pydrive/auth.py", line 115, in _decorated
code = decoratee(self, *args, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/pydrive/auth.py", line 241, in LocalWebserverAuth
raise AuthenticationError('No code found in redirect')
AuthenticationError: No code found in redirect
Question
I have no idea why I'm seeing this error. Both script and file are in the same folder. I haven't edited the script nor the .json.
Did I miss an update? Are the stars not aligned?
Who can help me out?
Consider the following code that uses the PyDrive module:
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LocalWebserverAuth()
drive = GoogleDrive(gauth)
file = drive.CreateFile({'title': 'test.txt'})
file.Upload()
file.SetContentString('hello')
file.Upload()
file.SetContentString('')
file.Upload() # This throws an exception.
Creating file and changing its contents works fine until I try to erase the contents by setting the content string to an empty one. Doing so throws this exception:
pydrive.files.ApiRequestError
<HttpError 400 when requesting
https://www.googleapis.com/upload/drive/v2/files/{LONG_ID}?alt=json&uploadType=resumable
returned "Bad Request">
When I look at my Drive, I see the test.txt file successfully created with text hello in it. However I expected that it would be empty.
If I change the empty string to any other text, the file is changed twice without errors. Though this doesn't clear the contents so it's not what I want.
When I looked up the error on the Internet, I found this issue on PyDrive github that may be related though it remains unsolved for almost a year.
If you want to reproduce the error, you have to create your own project that uses Google Drive API following this tutorial from the PyDrive docs.
How can one erase the contents of a file through PyDrive?
Issue and workaround:
When resumable=True is used, it seems that the data of 0 byte cannot be used. So in this case, it is required to upload the empty data without using resumable=True. But when I saw the script of PyDrive, it seems that resumable=True is used as the default. Ref So in this case, as a workaround, I would like to propose to use the requests module. The access token is retrieved from gauth of PyDrive.
When your script is modified, it becomes as follows.
Modified script:
import io
import requests
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LocalWebserverAuth()
drive = GoogleDrive(gauth)
file = drive.CreateFile({'title': 'test.txt'})
file.Upload()
file.SetContentString('hello')
file.Upload()
# file.SetContentString()
# file.Upload() # This throws an exception.
# I added below script.
res = requests.patch(
"https://www.googleapis.com/upload/drive/v3/files/" + file['id'] + "?uploadType=multipart",
headers={"Authorization": "Bearer " + gauth.credentials.token_response['access_token']},
files={
'data': ('metadata', '{}', 'application/json'),
'file': io.BytesIO()
}
)
print(res.text)
References:
PyDrive
Files: update
I'm using Python 3 running Pycharm and the module gspread to read google sheet files in my google drive. Well, I've followed all steps from this link about how to read a file. Unfortunately, my code here below doesn't work yet.
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope =['https://docs.google.com/spreadsheets/d/1xnaOZMd2v93tY28h_hsuMnZYXC9YqCfFpQX70lwpN94/edit?usp=sharing']
credentials = ServiceAccountCredentials.from_json_keyfile_name('distech-c1e26e7150b2.json',scope)
gc = gspread.authorize(credentials)
wks = gc.open("POC").sheet1
for temp in wks:
print(temp)
How could I read the google sheet file using this module guys? thanks so much
I got it after a deep research I realize two things.
the scope in my code was wrong cause the scope is just one provided by Google API to grant right access permissions under spreadsheet.
The right scope for me was: scope =['https://spreadsheets.google.com/feeds']
the opener it's just to open the spreadsheet that will return the worksheets within my file.
So solutions thanks to #Pedro Lobito in his post here.
Solution:
I had the same issue with just a couple of spreadsheets on my account, the problem was solved by:
Opening the json key file (projectname-ca997255dada.json)
Find the value of client_email , i.e.: client_email": "278348728734832-compute#developer.gserviceaccount.com
Share your sheet(s) with that email
Now my code looks like:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope =['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name('xxx.json',scope)
gc = gspread.authorize(credentials)
spreadsheet = gc.open("POC")
wks = spreadsheet.worksheet('test1')
wks2 = spreadsheet.worksheet('test2')
out = list()
out = wks.col_values(1)
for temp in out:
print(out)
I've been working on trying to integrate google sheets with django, i'm trying to use gspread. I can see the data using python filename.py, but when I run python manage.py runserver, I keep getting this error:
IOError: [Errno 2] No such file or directory: 'key.json'
It's not recognizing for seeing my json file for some reason, i've also tried using 'key' without the .json, no luck. I've been googling here, any ideas here? Here's my code below
*************************** code below *******************************
import gspread
import json
from oauth2client.service_account import ServiceAccountCredentials
import os
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name('key.json', scope)
gc = gspread.authorize(credentials)
wks = gc.open("RAMP - Master").sheet1
print wks
cell_list = wks.range('A1:B7')
print cell_list
If key.json is in the same directory as the file you're running, then the correct syntax is:
import os
DIRNAME = os.path.dirname(__file__)
credentials = ServiceAccountCredentials.from_json_keyfile_name(
os.path.join(DIRNAME, 'key.json'),
scope
)