Develop Reference

google api (sheets) Request had insufficient authentication scopes

i want to read an write data from a sheet, reading works fine but writing doesn't. i use all the scopes mentioned in the documentation: https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.values/append
data_writer(1,1,1)
code:
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Sheets API
SCOPES = 'https://www.googleapis.com/auth/spreadsheets'+"https://www.googleapis.com/auth/drive.file"+"https://www.googleapis.com/auth/drive"
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
service = apiclient.discovery.build('sheets', 'v4', http=creds.authorize(Http()))
# Call the Sheets API
SPREADSHEET_ID = '1JwVOqtUCWBMm_O6esIb-9J4TgqAmMIdYm9sf5y-A7EM'
RANGE_NAME = 'Jokes!A:C'
# How the input data should be interpreted.
value_input_option = 'USER_ENTERED'
# How the input data should be inserted.
insert_data_option = 'INSERT_ROWS'
def data_reader():
#reading data
read = service.spreadsheets().values().get(spreadsheetId=SPREADSHEET_ID,range=RANGE_NAME).execute()
#reading values
values = read.get('values', [])
if not values:
print('No data found.')
else:
for row in values:
print(row[2])
continue
def data_writer(score,num_comments,mystring):
value_range_body = {
"score":score,
"num_comments":num_comments,
"joke":mystring
}
request = service.spreadsheets().values().append(spreadsheetId=SPREADSHEET_ID, range=RANGE_NAME, valueInputOption=value_input_option, insertDataOption=insert_data_option, body=value_range_body)
response = request.execute()

SCOPES must be of type list
SCOPES = ['https://www.googleapis.com/auth/spreadsheets', "https://www.googleapis.com/auth/drive.file", "https://www.googleapis.com/auth/drive"]
Side note: you have
.../auth/drive
And
.../auth/drive.file
/drive.file limits the API to work with only drive.file API but /drive opens up to all of the drive API. So you should pick one that fits your needs.
Side note 2:
Based on the link you’ve provided, it mentions you need at least one of the API’s to work with spreadsheets, so you may not need all of them either.

First off you should only need https://www.googleapis.com/auth/drive as it gives full access to a users drive account including reading and writing sheets.
list of Sheet scopes
list of drive scopes
If you have already run your code once and authenticated your user then changed the scopes in your code. Remember that you will need to run your code again and re-authenticate the user to gain the access granted by the new scopes.
Side note:
SCOPES = 'https://www.googleapis.com/auth/spreadsheets'+"https://www.googleapis.com/auth/drive.file"+"https://www.googleapis.com/auth/drive"
Is just going to be one long string you need to separate them with a space or as the other answer states use an array.
SCOPES = 'https://www.googleapis.com/auth/spreadsheets ' + "https://www.googleapis.com/auth/drive.file " + "https://www.googleapis.com/auth/drive"

How to access Google Apps API from appengine?

What's the actual task?
I'm trying to access google Apps API (Drive API, Sheets API) from my google appengine standard project. So when a form is submitted, it has to create a new google sheet and write the contents of the form to the sheet and then the sheet has to be stored in the google drive associated with the google service account or authorized email account (ie, email account we gave in the authorized email section)
What I actually tried?
I have used google_auth library to get the appengine's default credentials and I have enabled both drive, sheets API on my gae project console.
from googleapiclient import discovery
from google.auth import app_engine
from google.auth.transport.requests import AuthorizedSession
import httplib2
def creat_sample_sheet():
credentials = app_engine.Credentials()
http = AuthorizedSession(credentials)
discoveryUrl = ('https://sheets.googleapis.com/$discovery/rest?'
'version=v4')
service = discovery.build('sheets', 'v4', http=http,
discoveryServiceUrl=discoveryUrl)
result = service.spreadsheets().values()
print result
But it's not working.. Here is the tracback I got...
File "/base/data/home/apps/vidyalay/1.397393333574060152/alumni_registration_dateycollege/sheet_handler.py" in creat_sample_sheet
30. discoveryServiceUrl=discoveryUrl)
File "/base/data/home/apps/vidyalay/1.397393333574060152/lib/oauth2client/_helpers.py" in positional_wrapper
133. return wrapped(*args, **kwargs)
File "/base/data/home/apps/vidyalay/1.397393333574060152/lib/googleapiclient/discovery.py" in build
222. cache)
File "/base/data/home/apps/vidyalay/1.397393333574060152/lib/googleapiclient/discovery.py" in _retrieve_discovery_doc
269. resp, content = http.request(actual_url)
Exception Type: TypeError at /alumni_dateycollege/sheet/
Exception Value: request() takes at least 3 arguments (2 given)
Don't know I'm on the right path..
Update
By following this link mentioned by Daniel works for me. But I don't know how to view the created spreadsheet .
Here is my attempt to download the created sheet.
service.spreadsheets().get(spreadsheetId=SHEET_ID, alt='media')
But this creates an get request to https://sheets.googleapis.com/v4/spreadsheets/1awnM7z_aomHx833Z5S_Z-agFusaidmgcCa0FJIFyGE8?alt=json
url. But I actually want to pass media to alt parameter instead of json. I tried the above, but it won't work.

I managed to solve the problem on my own.
I have used my google service account to create spreadsheet on the service account's google drive (sheet API takecare of storing the sheet to google drive). Then I gave writer role permission for that particular file to my own gmail id. Now I can be able to view that particular form sheet in my google drive.
from googleapiclient import discovery
import cloudstorage as gcs
# from oauth2client import client
# from oauth2client.contrib import appengine
# from google.appengine.api import memcache
import httplib2
from google.appengine.api import memcache
from oauth2client.contrib.appengine import AppAssertionCredentials
import logging
import os
import io
from googleapiclient.http import MediaIoBaseDownload
logger = logging.getLogger(__name__)
credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/spreadsheets')
drive_credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/drive')
http = credentials.authorize(httplib2.Http(memcache))
drive_http = drive_credentials.authorize(httplib2.Http(memcache))
discoveryUrl = ('https://sheets.googleapis.com/$discovery/rest?'
'version=v4')
service = discovery.build('sheets', 'v4', http=http,
discoveryServiceUrl=discoveryUrl)
def callback(request_id, response, exception):
if exception:
# Handle error
print exception
else:
print "Permission Id: %s" % response.get('id')
def list_drive_files(drive_service):
results = drive_service.files().list(
pageSize=10).execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print('{0} ({1})'.format(item['name'], item['id']))
def give_file_permission(drive_service, file_id):
batch = drive_service.new_batch_http_request(callback=callback)
user_permission = {
'type': 'user',
'role': 'writer',
'emailAddress': 'foobar#gmail.com' # email address of the user you want to give permission
}
batch.add(drive_service.permissions().create(
fileId=file_id,
body=user_permission,
fields='id',
))
batch.execute()
def creat_sample_sheet():
# data = {'properties': {'title': 'Academic Sheet'}}
# res = sheet_service.spreadsheets().create(body=data).execute()
# SHEET_ID = res['spreadsheetId']
SHEET_ID = '1awnM7z_aomHx833Z5S_Z-agFusaidmgcCa0FJIFyGE8'
sheet = service.spreadsheets().get(spreadsheetId=SHEET_ID, includeGridData=True)
drive_service = discovery.build('drive', 'v3', http=drive_http)
list_drive_files(drive_service)
# Sharing a file
# file_id = '1bgvJdXG0eg2JGaNlIcdtde_XJlg2gAUT_DOzHi75zys'
def write_to_sheet(form):
logger.info('Inside write to sheet')
first_name = form.cleaned_data.get('first_name', '')
sur_name = form.cleaned_data.get('sur_name', '')
email = form.cleaned_data.get('e_mail', '')
phone_no = form.cleaned_data.get('mobile_phone', '')
year_of_passing = form.cleaned_data.get('year_of_passing', '')
present_occupation = form.cleaned_data.get('present_occupation', '')
present_city = form.cleaned_data.get('present_city', '')
courses_attended = ', '.join([str(i) for i in form.cleaned_data.get('courses_attended', '')])
volunteer = form.cleaned_data.get('volunteer', '')
fields = [ 'first_name', 'sur_name', 'e_mail', 'mobile_phone', 'year_of_passing', 'present_occupation', 'present_city', 'courses_attended' , 'volunteer' ]
# data = {'properties': {'title': 'Form Sheet'}}
# # create sheet
# res = service.spreadsheets().create(body=data).execute()
# # sheet_id = res['spreadsheetId']
sheet_id = '1bgvJdXG0eg2JGaNlIcdtde_XJlg2gAUT_DOzHi75zys'
# print sheet_id
# update sheet
data = {'values': [[first_name, sur_name, email, phone_no, year_of_passing, present_occupation,
present_city, courses_attended, volunteer]]}
service.spreadsheets().values().append(spreadsheetId=sheet_id,
range='A1', body=data, valueInputOption='RAW').execute()
# Getting rows
# rows = service.spreadsheets().values().get(spreadsheetId=sheet_id,
# range='Sheet1').execute().get('values', [])
# for row in rows:
# print row

How to mark messages as read using gmail api as i parse?

In python how do i mark messages as 'read' as i parse it from gmail api ?
Also how do i save the values to the database after parsing?
This is the code so far to get the content of each message.
from __future__ import print_function
import httplib2
import os
import re
import MySQLdb
from email.utils import parsedate_tz,mktime_tz,formatdate
from requests.adapters import HTTPAdapter
import datetime
from datetime import date,timedelta
import time
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
import json
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/gmail.readonly'
CLIENT_SECRET_FILE = 'client_server.json'
APPLICATION_NAME = 'Gmail API Python Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'gmail-python-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
da=date.fromordinal(730920)
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('gmail', 'v1', http=http)
today=date.today()
print (today)
yesterday=today-timedelta(1)
print (yesterday)
response = service.users().messages().list(userId='me',q='in:inbox is:unread newer_than:1d').execute()
messages=[]
store=[]
message1=[]
test2=[]
da=[]
if 'messages' in response:
messages.extend(response['messages'])
fo = open("fooa.txt", "wb")
for i in range(len(messages)):
store=messages[i]['id']
message = service.users().messages().get(userId='me',id=store,format='metadata',metadataHeaders=['from','date']).execute()
fo.write(message['snippet'].encode('utf-8')+"")
From=message['payload']['headers'][0]['value']
fo.write(From+"");
da=message['payload']['headers'][1]['value']
fo.write(da+"\n");
for line in open("fooa.txt"):
print(line)
fo.close()
a=open("fooa.txt","r")
for wo in a:
match=re.findall(r':[\w]+',wo)
for word in match:
print(word.replace(':',' '))
db = MySQLdb.connect("localhost","testuser","mysql23","db1" )
cursor = db.cursor()
sql = """INSERT INTO customers((LeadName, CITY, SERVICE,CUSTOMER, MOBILE, EMAIL)
VALUES (, , , , )"""
try:
cursor.execute(sql)
db.commit()
except:
db.rollback()
db.close()
if __name__ == '__main__':
main()
Need help please!

You need to modify the message in a separate request, and remove the UNREAD-label.
POST https://www.googleapis.com/gmail/v1/users/me/messages/1533cb4d7dac1633/modify?access_token={ACCESS_TOKEN}
{
"removeLabelIds": [
"UNREAD"
]
}

If you're trying to mark the message as read, you will have to do something like:
gmail_service
.users()
.messages()
.modify(userId='me', id=message_id, body={'removeLabelIds': ['UNREAD']})
.execute()

What you can do with the Gmail API depends on what scope you've granted OAuth. Here's what you need to do:
Change SCOPES = 'https://www.googleapis.com/auth/gmail.readonly' to this:
SCOPES = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.modify'
]
Explanation: since marking an email as "unread" is a message modification, you need the 'gmail.modify' scope in order to use this API request. To see the full list of scopes, refer to Choose Auth Scopes.
Delete your token.pickle file and rerun the script. You'll need to do this in order to reauthorize the OAuth app with the new scopes, which will then generate a new token.pickle file.
Here's the code to mark an email as read in Python:
service.users().messages().modify(userId='me', id=message['id'], body={
'removeLabelIds': ['UNREAD']
}).execute()

New Python Gmail API - Only Retrieve Messages from Yesterday

I've been updating some scripts to the new Python Gmail API. However, I am confused as how to update the following so that I only retrieve messages from yesterday. Can anyone show me how to do this?
The only way I can currently see is to loop through all messages and only parse those with epochs in the correct time range. However, that seems horribly inefficient if I have 1000's of messages. There must be a more efficient way to do this.
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
import os
import httplib2
import email
from apiclient.http import BatchHttpRequest
import base64
from bs4 import BeautifulSoup
import re
import datetime
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/gmail.readonly'
CLIENT_SECRET_FILE = '/Users/sokser/Downloads/client_secret.json'
APPLICATION_NAME = 'Gmail API Python Quickstart'
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'gmail-python-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def visible(element):
if element.parent.name in ['style', 'script', '[document]', 'head', 'title']:
return False
elif re.match('<!--.*-->', str(element)):
return False
return True
def main():
"""Shows basic usage of the Gmail API.
Creates a Gmail API service object and outputs a list of label names
of the user's Gmail account.
"""
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('gmail', 'v1', http=http)
#Get yesterdays date and the epoch time
yesterday = datetime.date.today() - datetime.timedelta(1)
unix_time= int(yesterday.strftime("%s"))
messages = []
message = service.users().messages().list(userId='me').execute()
for m in message['messages']:
#service.users().messages().get(userId='me',id=m['id'],format='full')
message = service.users().messages().get(userId='me',id=m['id'],format='raw').execute()
epoch = int(message['internalDate'])/1000
msg_str = str(base64.urlsafe_b64decode(message['raw'].encode('ASCII')),'utf-8')
mime_msg = email.message_from_string(msg_str)
#print(message['payload']['parts'][0]['parts'])
#print()
mytext = None
for part in mime_msg.walk():
mime_msg.get_payload()
#print(part)
#print()
if part.get_content_type() == 'text/plain':
soup = BeautifulSoup(part.get_payload(decode=True))
texts = soup.findAll(text=True)
visible_texts = filter(visible,texts)
mytext = ". ".join(visible_texts)
if part.get_content_type() == 'text/html' and not mytext:
mytext = part.get_payload(decode=True)
print(mytext)
print()
if __name__ == '__main__':
main()

You can pass queries to the messages.list method that searches for messages within a date range. You can actually use any query supported by Gmail's advanced search.
You do this, which will just return messages.
message = service.users().messages().list(userId='me').execute()
But can do this to search for messages sent yesterday, by passing the q keyword argument, and a query specifying the before: and after: keywords.
from datetime import date, timedelta
today = date.today()
yesterday = today - timedelta(1)
# do your setup...
user_id = 'user email address'
# Dates have to formatted in YYYY/MM/DD format for gmail
query = "before: {0} after: {1}".format(today.strftime('%Y/%m/%d'),
yesterday.strftime('%Y/%m/%d'))
response = service.users().messages().list(userId=user_id,
q=query).execute()
# Process the response for messages...
You can also try this against their GMail messages.list reference page.

Not able to download google spreadsheet by google drive API using python

I am trying to download a spreadsheet file from my drive to my computer.
I am able to authenticate, get list of files and even get meta-data successfully.
But when I try to download the file, I get the following error :
downloading file starts
An error occurred: <HttpError 400 when requesting https://www.googleapis.com/dri
ve/v2/files/1vJetI_p8YEYiKvPVl0LtXGS5uIAx1eRGUupsXoh7UbI?alt=media returned "The
specified file does not support the requested alternate representation.">
downloading file ends
I couldn't get any such problem or question on SO and the other methods or solutions provided on SO for downloading the spreadsheet are outdated.Those have been deprecated by Google .
Here is the code, I am using to download the file :
import httplib2
import os
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
from apiclient import errors
from apiclient import http
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
#SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly'
SCOPES = 'https://www.googleapis.com/auth/drive'
CLIENT_SECRET_FILE = 'client_secrets.json'
APPLICATION_NAME = 'Drive API Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'drive-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatability with Python 2.6
credentials = tools.run(flow, store)
print 'Storing credentials to ' + credential_path
return credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v2', http=http)
file_id = '1vJetI_p8YEYiKvPVl0LtXGS5uIAx1eRGUupsXoh7UbI'
print "downloading file starts"
download_file(service, file_id)
print "downloading file ends "
def download_file(service, file_id):
local_fd = open("foo.csv", "w+")
request = service.files().get_media(fileId=file_id)
media_request = http.MediaIoBaseDownload(local_fd, request)
while True:
try:
download_progress, done = media_request.next_chunk()
except errors.HttpError, error:
print 'An error occurred: %s' % error
return
if download_progress:
print 'Download Progress: %d%%' % int(download_progress.progress() * 100)
if done:
print 'Download Complete'
return
if __name__ == '__main__':
main()

Google spreadsheets don't have media. Instead they have exportLinks. Get the file metadata, then look in the exportlinks and pick an appropriate URL.

This code worked for me. I only had to download client_secret.json from google developers dashboard and keep in the same directory as python script.
And in the list_of_lists variable I got a list with each row as list.
import gspread
import json
from oauth2client.client import SignedJwtAssertionCredentials
json_key = json.load(open('client_secret.json'))
scope = ['https://spreadsheets.google.com/feeds']
credentials = SignedJwtAssertionCredentials(json_key['client_email'], json_key['private_key'], scope)
gc = gspread.authorize(credentials)
sht1 = gc.open_by_key('<id_of_sheet>')
worksheet_list = sht1.worksheets()
worksheet = sht1.sheet1
list_of_lists = worksheet.get_all_values()
for row in list_of_lists :
print row