Take data from private google spreadsheet - python

I have a Google Spreadsheet which I'm sharing with several person. I want to built a script to search for some rows and take cells values and process a program locally afterwards. I was thinking going with python, as it seems Google provide a good API for it.
Have someone an example on how to connect to Google Spreadsheet ? I read the api, but I don't get how does the OAuth 2.0 thing works...
Many thanks :)

To perform read and write operations on a Google Spreadsheet, OAuth 2.0 will not be necessary. As shown in these samples for reading as well as writing to a Google Spreadsheet, in order to be able to access the SpreadSheet you must include the username-password of the related account in your in your code. And then using them in the following manner (as shown in the writing sample) to access the Spreadsheet:
spr_client = gdata.spreadsheet.service.SpreadsheetsService()
spr_client.email = email
spr_client.password = password
spr_client.source = 'Example Spreadsheet Writing Application'
spr_client.ProgrammaticLogin()
Also, do remember to import gdata.spreadsheet.service and any other related library that you might need for your code. And if you're new, this would also be a good place to start. Hope this helps!

Related

How to save a googlesheet chart as image by Python

I'm using Googlesheet API with Python and I can get access to the sheet and the cells now. However, I don't know how to get the chart in the sheet.
client = gspread.service_account_from_dict(creds)
workbook = client.open('HR - 8/16-8/31 Data')
sheet = workbook.get_worksheet(0)
H1 = sheet.acell('B3').value
I found this question:How to download charts in PNG from google sheet mentioned I can use the getCharts() function, but it is for JavaScript only. If there a similar function in Python?
Currently the API doesn't have a method to do this. The charts overview documentation explains how to manipulate and create them, but not how to export them. Reading the data also only gives you a JSON representation of it, not an image. It seems that the Apps Script getCharts() leverages other server-side functions that are not in the regular API.
This is documented as a feature request in Google's issue tracker here, so you can +1 it if you want. In that thread a possible workaround was posted. If you publish your file you can build a URL if you know the chartID to generate it as an image:
https://docs.google.com/spreadsheets/d/e/<publish-id>/pubchart?oid=<chart-id>&format=image
Gspread doesn't seem to have methods to do this so you'll have to use the Google APIs. In their Python Quickstart you can find a sample to set up authorization, and you can use spreadsheets.get(), which gives you all the data from the spreadsheet including the chart IDs. If you only have a single chart that you want to export periodically then you can just get the ID once from the UI and just retrieve it with Python. The caveat is that you have to publish the Sheet which you don't want to do with sensitive information.
As another alternative you could build an Apps Script Web App which uses the getCharts() method in the answer that you linked, and just send a POST message from your Python app and have Apps Script return the image in its response.

Accessing a file from google drive within Python

I am working on a machine learning task and have saved a Keras model and want to deploy it to Github (so that I can host a web demo using Streamlit and/or Flask). However, the model file is so large (> 1 GB), that I cannot upload it to Github for free.
My thought process regarding an alternative is to upload it to a cloud service such as google drive (or dropbox, box etc.) then using some sort of Python module to access it from there.
So my question is, can I upload a pickle file containing a pickled Keras model to Google Drive and then access that object from a Python script? If so, how would I go about doing so?
Thank you!
I believe you can, you'll need to pip oauth2client & gspread. To access the data you would need to enable API manager on your google drive and get credentials in the form of a json file. Then you would need to share the file with the email in the credentials giving it permission. You could then port over the information as you needed to, I'm not sure how Keras works but this would be the first step.
Another important factor is that Google api is very touch when it comes to requests that are coming to fast, to overcome this put in sleep commands between each one, but if you do that this method may become way to slow for your idea.
scope = ["https://spreadsheets.google.com/feeds", 'https://www.googleapis.com/auth/spreadsheets',
"https://www.googleapis.com/auth/drive.file", "https://www.googleapis.com/auth/drive"]
creds = ServiceAccountCredentials.from_json_keyfile_name("Your json file here.json", scope)
client = gspread.authorize(creds)
sheet = client.open("your google sheets name or whatever").sheet1 # Open the spreadhseet
data = sheet.get_all_records() # you can call all the information with this.
I understand that you require a way to upload and download large files* from Drive using Python. If I understood your situation correctly, then you can achieve your goals easily by using the Drive API as #TimothyChen commented. First I highly recommend you to follow the Drive API Python Quickstart tutorial to create a working example. Later, you could modify it to use Files.create() and Files.get() to upload/download files as needed. Don't hesitate to ask me more questions if you have doubts.
*Please, keep in mind that there is a 5 TB size limit in Drive.

Dump data from python cron job to google spreadsheets

I am creating a CSV which contains the report as a result of a cronjob. I want to share this CSV via Google spreadsheets - the report itself is versioned, so I would just dump the CSV contents into the same worksheet in the same spreadsheet each and every single time.
I have found gspread which looked very promising but unfortunately gives me NoValidUrlKeyFound Errors. The Python example for interacting with the Spreadsheets API v4 (to be found here) requires interactivity because of the OAuth flow.
Can someone point me in the right direction? Ideally I would just do:
client = spreadsheet.client(credential_file)
client.open_spreadheet(url_or_something).open_worksheet(0).dump(csv)
print("Done")
I have found the answer in a Github issue:
Using gspread, follow the official guide to receive an oauth token. You will end up with a JSON file holding the credentials.
In order to access a spreadsheet with these credentials however, you need to share the spreadsheet with the account represented by them. In the JSON file there is a property client_email. Share the document with that e-mail address and voila, you'll be able to open it!

Python gspread and Oauth2 for server-side applications returns no spreadsheets

I'm trying to read a bunch of spreadsheets within my organization that have been shared with my account and import the records into a database. I'd like to use OAuth2.0 instead of just using raw text passwords, but I'm having trouble getting this to work. My code right now is this:
scope = ['https://spreadsheets.google.com/feeds', 'https://docs.google.com/feeds']
credentials = SignedJwtAssertionCredentials('5***2#developer.gserviceaccount.com',SIGNED_KEY,scope)
gc = gspread.authorize(credentials)
print gc.openall()
This returns:
$ python read_sheets.py
[]
My account probably has a hundred spreadsheets shared with it, so I'm not sure why this is happening. One idea I had is that the service account email address doesn't have the sheets shared with it, but when I switch to the actual email address of the account, I get oauth2client.client.AccessTokenRefreshError: invalid_grant as an error. It seems like maybe I'm missing a step, but I've followed both the gspread instructions and the google oauth2 for service accounts and don't see what else I need to do. Appreciate the help

Using Python, how can I read plain text from a Google Doc?

I am attempting to read the raw text/content of a Google Doc (just a plain document, not a spreadsheet or presentation) from within a Python script, but so far have had little success.
Here's what I've tried:
import gdata.docs.service
client = gdata.docs.service.DocsService()
client.ClientLogin('email', 'password')
q = gdata.docs.service.DocumentQuery()
q.AddNamedFolder('email', 'Folder Name')
feed = client.Query(q.ToUri())
doc = feed.entry[0] # extract one of the documents
However, this variable doc, which is of type gdata.docs.DocumentListEntry, doesn't seem to contain any content, just meta information about the document.
Am I doing something wrong here? Can somebody point me in the right direction? Thank you!
UPDATE (Mar 2019) Good news! The Google Docs REST API is now available. More info about it from my SO answer to a similar question, but to get you going, here's the official Python "quickstart" sample showing you how to get the title of a Google Doc in plain text.
Both the Apps Script and Drive REST API solutions originally answered below are still valid and are alternate ways to get the contents of a Google Doc. (The Drive API works on both Python 2 & 3, but Apps Script is JavaScript-only.)
Bottom-line: if you want to download the entire Doc in plain text, the Drive API solution is best. If you want to programmatically CRUD different parts of a Doc, then you must use either the Docs API or Apps Script.
(Feb 2017) The code in the OP and the only other answer are both now out-of-date as ClientLogin authentication was deprecated back in 2012(!), and GData APIs are the previous generation of Google APIs. While not all GData APIs have been deprecated, all newer Google APIs do not use the Google Data protocol.
There isn't a REST API available (at this time) for Google Docs documents, although there is an "API-like" service provided by Google Apps Script, the JavaScript-in-the-cloud solution which provides programmatic access to Google Docs (via its DocumentService object), including Docs add-ons.
To read plain text from a Google Doc, considered file-level access, you would use the Google Drive API instead. Examples of using the Drive API:
Exporting a Google Sheet as CSV (blog post)
"Poor man's plain text to PDF" converter (blog post) (*)
(*) - TL;DR: upload plain text file to Drive, import/convert to Google Docs format, then export that Doc as PDF. Post above uses Drive API v2; this follow-up post describes migrating it to Drive API v3, and here's a developer video combining both "poor man's converter" posts.
The solution to the OP is to perform similar operations as what you see in both posts above but ensure you're using the text/plain export MIMEtype. For other import/export formats to/from Drive, see this related question SO answer as well as the downloading files from Drive docs page. Here's some pseudocode that searches for Google Docs documents called "Hello World" in my Drive folder and displays the contents of the first matching file found on-screen (assuming DRIVE is your API service endpoint):
from __future__ import print_function
NAME = 'Hello World'
MIME = 'text/plain'
# using Drive API v3; if using v2, change 'pageSize' to 'maxResults',
# 'name=' to 'title=', and ".get('files')" to ".get('items')"
res = DRIVE.files().list(q="name='%s'" % NAME, pageSize=1).execute().get('files')
if res:
fileID = res[0]['id'] # 1st matching "Hello World" name
res = DRIVE.files().export(fileId=fileID, mimeType=MIME).execute()
if res:
print(res.decode('utf-8')) # decode bytes for Py3; NOP for Py2
If you need more than this, see these videos on how to setup using Google APIs, OAuth2 authorization, and creating a Drive service endpoint to list your Drive files, plus a corresponding blog post for all three.
To learn more about how to use Google APIs with Python in general, check out my blog as well as a variety of Google developer videos (series 1 and series 2) I'm producing.
A DocumentQuery doesn't return you all the documents with their contents—that would take forever. It just returns a list of documents, with metadata about each. (Actually, IIRC you can get a preview page this way, so if your document is only one page that might be enough…)
You then need to download the content in a separate request. The content element has a type (the MIME type) and a src (the URL to the actual data). You can just download that src, and parse it. However, you can override the default type by adding an exportFormat parameter, so you don't need to do any parsing.
See the section Downloading documents and files in the docs, which has an example showing how to download a document and specify a format. (It's in .NET rather than Python, and it uses HTML rather than plain text, but you should be able to figure it out.)

Categories

Resources