Develop Reference

Python update google sheet from local csv problem

I am a beginner in python.
I have successfully put together the script to update my google sheet from some hard coded values
Test1 = [["1/1/2020",4000],["4/4/2020",3000],["7/12/2020",'salah4-tiga1000']]
But when I try to update from a local csv file,
Test2 = pd.read_csv(ordersCSV).to_json()
I can see the csv file data running in my terminal, but it is not updating the google sheet.
Can you help me to see what I missed please.
Thank you in advance.
from googleapiclient.discovery import build
from google.oauth2.credentials import Credentials
from google.oauth2 import service_account
from oauth2client.transport import request
from pprint import pprint
from numpy import greater
import pandas as pd
from pandas.core import series
from pandas.core.indexes.base import Index
from pyasn1.type.univ import Null
SERVICE_ACCOUNT_FILE = 'drfruit4.json'
SCOPES = ['https://www.googleapis.com/auth/spreadsheets']
creds = None
creds = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
SPREADSHEET_ID = '1rpgVUAmLojG1_U2SRQ_x9vQGI16mEYiOCGcZLrIRZOQ'
service = build('sheets', 'v4', credentials=creds)
# Call the Sheets API
sheet =service.spreadsheets()
result = sheet.values().get(spreadsheetId=SPREADSHEET_ID,
range="Detail_Exp_SiteGiant_via_Python!A1:Z1100").execute()
values = result.get('values',[])
Test1 = [["1/1/2020",4000],["4/4/2020",3000],["7/12/2020",'salah4-tiga1000']]
ordersCSV = r'Ready4Sofia.csv'
Test2 = pd.read_csv(ordersCSV).to_json()
request = sheet.values().update(spreadsheetId=SPREADSHEET_ID,
range="Detail_Exp_SiteGiant_via_Python!A1", valueInputOption="USER_ENTERED", body={'values':Test2}).execute()

I have found the answer here .
Thanks to Stack Overflow and all the contributors.

Read formula in the Google Sheets cells using Python

I am trying to download a Google Sheets document as a Microsoft Excel document using Python. I have been able to accomplish this task using the Python module googleapiclient.
However, the Sheets document may contain some formulas which are not compatible with Microsoft Excel (https://www.dataeverywhere.com/article/27-incompatible-formulas-between-excel-and-google-sheets/).
When I use the application I created on any Google Sheets document that used any of these formulas anywhere, I get a bogus Microsoft Excel document as output.
I would like to read the cell values in the Google Sheets document before downloading it as a Microsoft Excel document, just to prevent any such errors from happening.
The code I have written thus far is attached below:
import sys
import os
from googleapiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools
SCOPES = "https://www.googleapis.com/auth/drive.readonly"
store = file.Storage("./credentials/credentials.json")
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets("credentials/client_secret.json",
SCOPES)
creds = tools.run_flow(flow, store)
DRIVE = discovery.build("drive", "v3", http = creds.authorize(Http()))
print("Usage: tmp.py <name of the spreadsheet>")
FILENAME = sys.argv[1]
SRC_MIMETYPE = "application/vnd.google-apps.spreadsheet"
DST_MIMETYPE = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
files = DRIVE.files().list(
q = 'name="%s" and mimeType="%s"' % (FILENAME, SRC_MIMETYPE),
orderBy = "modifiedTime desc,name").execute().get("files", [])
if files:
fn = '%s.xlsx' % os.path.splitext(files[0]["name"].replace(" ", "_"))[0]
print('Exporting "%s" as "%s"... ' % (files[0]['name'], fn), end = "")
data = DRIVE.files().export(fileId=files[0]['id'], mimeType=DST_MIMETYPE).execute()
if data:
with open(fn, "wb") as f:
f.write(data)
print("Done")
else:
print("ERROR: Could not download file")
else:
print("ERROR: File not found")

If you want to use python to export something from google docs, then the simplest way is to let googles own server do the job for you.
I was doing a little webscraping on google sheets, and I made this little program which will do the job for you. You just have to insert the id of the document you want to download.
I put in a temporary id, so anyone can try it out.
import requests
ext = 'xlsx' #csv, ods, html, tsv and pdf can be used as well
key = '1yEoHh7WL1UNld-cxJh0ZsRmNwf-69uINim2dKrgzsLg'
url = f'https://docs.google.com/spreadsheets/d/{key}/export?format={ext}'
res = requests.get(url)
with open(f'file.{ext}', 'wb') as f:
f.write(res.content)
That way conversion will most certainly always be correct, because this is the same a clicking the export button inside the browser version of google sheets.
If you are planning to work with the data inside python, then I recommend using csv format instead of xlsx, and then create the necessary formulas inside python.

I think the gspread library might be what you are looking for. https://gspread.readthedocs.io/en/latest/
Here's a code sample:
import tenacity
import gspread
from oauth2client.service_account import ServiceAccountCredentials
#tenacity.retry(wait=tenacity.wait_exponential()) # If you exceed the Google API quota, this waits to retry your request
def loadGoogleSheet(spreadsheet_name):
# use creds to create a client to interact with the Google Drive API
print("Connecting to Google API...")
scope = [
'https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive'
]
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)
spreadsheet = client.open(spreadsheet_name)
return spreadsheet
def readGoogleSheet(spreadsheet):
sheet = spreadsheet.sheet1 # Might need to loop through sheets or whatever
val = sheet.cell(1, 1).value # This just gets the value of the first cell. The docs I linked to above are pretty helpful on all the other stuff you can do
return val
test_spreadsheet = loadGoogleSheet('Copy of TLO Summary - Template DO NOT EDIT')
test_output = readGoogleSheet(test_spreadsheet)
print(test_output)

Python - how to read Sharepoint excel sheet specific worksheet

In Python I am utilizing Office 365 REST Python Client library to access and read an excel workbook that contains many sheets.
While the authentication is successful, I am unable to append the right path of sheet name to the file name in order to access the 1st or 2nd worksheet by its name, which is why the output from the sheet is not JSON, rather IO Bytes which my code is not able to process.
My end goal is to simply access the specific work sheet by its name 'employee_list' and transform it into JSON or Pandas Data frame for further usage.
Code snippet below -
import io
import json
import pandas as pd
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.runtime.auth.user_credential import UserCredential
from office365.runtime.http.request_options import RequestOptions
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.files.file import File
from io import BytesIO
username = 'abc#a.com'
password = 'abcd'
site_url = 'https://sample.sharepoint.com/sites/SAMPLE/_layouts/15/Doc.aspx?OR=teams&action=edit&sourcedoc={739271873}'
# HOW TO ACCESS WORKSHEET BY ITS NAME IN ABOVE LINE
ctx = ClientContext(site_url).with_credentials(UserCredential(username, password))
request = RequestOptions("{0}/_api/web/".format(site_url))
response = ctx.execute_request_direct(request)
json_data = json.loads(response.content) # ERROR ENCOUNTERED JSON DECODE ERROR SINCE DATA IS IN BYTES

You can access it by sheet index, check the following code....
import xlrd
loc = ("File location")
wb = xlrd.open_workbook(loc)
sheet = wb.sheet_by_index(0)
# For row 0 and column 0
print(sheet.cell_value(1, 0))

You can try to add the component 'sheetname' to the url like so.
https://site/lib/workbook.xlsx#'Sheet1'!A1

It seems that URL constructed to access data is not correct. You should test full URL in your browser as working and then modify code to get going. You may try this with some changes, I have verified that URL formed with this logic would return JSON data.
import io
import json
import pandas as pd
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.runtime.auth.user_credential import UserCredential
from office365.runtime.http.request_options import RequestOptions
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.files.file import File
from io import BytesIO
username = 'abc#a.com'
password = 'abcd'
site_url = 'https://sample.sharepoint.com/_vti_bin/ExcelRest.aspx/RootFolder/ExcelFileName.xlsx/Model/Ranges('employee_list!A1%7CA10')?$format=json'
# Replace RootFolder/ExcelFileName.xlsx with actual path of excel file from the root.
# Replace A1 and A10 with actual start and end of cell range.
ctx = ClientContext(site_url).with_credentials(UserCredential(username, password))
request = RequestOptions(site_url)
response = ctx.execute_request_direct(request)
json_data = json.loads(response.content)
Source: https://learn.microsoft.com/en-us/sharepoint/dev/general-development/sample-uri-for-excel-services-rest-api

The update I'm using (Office365-REST-Python-Client==2.3.11) allows simpler access to an Excel file in the SharePoint repository.
# from original_question import pd,\
# username,\
# password,\
# UserCredential,\
# File,\
# BytesIO
user_credentials = UserCredential(user_name=username,
password=password)
file_url = ('https://sample.sharepoint.com'
'/sites/SAMPLE/{*recursive_folders}'
'/sample_worksheet.xlsx')
## absolute path of excel file on SharePoint
excel_file = BytesIO()
## initiating binary object
excel_file_online = File.from_url(abs_url=file_url)
## requesting file from SharePoint
excel_file_online = excel_file_online.with_credentials(
credentials=user_credentials)
## validating file with accessible credentials
excel_file_online.download(file_object=excel_file).execute_query()
## writing binary response of the
## file request into bytes object
We now have a binary copy of the Excel file as BytesIO named excel_file. Progressing, reading it as pd.DataFrame is straight-forward like usual Excel file stored in local drive. Eg.:
pd.read_excel(excel_file) # -> pd.DataFrame
Hence, if you are interested in a specific sheet like 'employee_list', you may preferably read it as
employee_list = pd.read_excel(excel_file,
sheet_name='employee_list')
# -> pd.DataFrame
or
data = pd.read_excel(excel_file,
sheet_name=None) # -> dict
employee_list = data.get('employee_list')
# -> [pd.DataFrame, None]

I know you stated you can't use a BytesIO object, but for those coming here who are reading the file in as a BytesIO object like I was looking for, you can use the sheet_name arg in pd.read_excel:
url = "https://sharepoint.site.com/sites/MySite/MySheet.xlsx"
sheet_name = 'Sheet X'
response = File.open_binary(ctx, relative_url)
bytes_file_obj = io.BytesIO()
bytes_file_obj.write(response.content)
bytes_file_obj.seek(0)
df = pd.read_excel(bytes_file_obj, sheet_name = sheet_name) //call sheet name

JSON to Google sheets using Python

I am trying to find a way to transfer the values from JSON to a Google Sheet.
The values in JSON file are something like this {"someone_name1#gmail.com": 4, "someone_name2.com": 4} and they keep updating in the different run of the script. However, I am getting an error when its trying to put the value in the sheet.
sheet4.update_cells(1, 1, results)
TypeError: update_cells() takes from 2 to 3 positional arguments but 4 were given
Here is the code below. Any ideas what am I doing wrong and how can I fix this? I tried researching this but not able to find a suitable answer. As I am new to coding and python, I am unable to figure this one out. Any help appreciated. :)
# all functions imported
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import json
from collections import Counter
# login & open sheet sheets
scope = ["https://spreadsheets.google.com/feeds", 'https://www.googleapis.com/auth/spreadsheets',
"https://www.googleapis.com/auth/drive.file", "https://www.googleapis.com/auth/drive"]
credentials = ServiceAccountCredentials.from_json_keyfile_name('myfile-b16b15370c5b.json', scope)
client = gspread.authorize(credentials)
sheet4 = client.open('Dashboard').worksheet('Sheet4') # Open the spreadsheet
counter_file_path = "counter.json"
with open(counter_file_path, "r") as f:
email_stats = json.load(f)
results = []
for key in email_stats:
results.append([key, email_stats[key]])
sheet4.update_cells(1, 1, results)

gspread reading a google sheet file using python 3

I'm using Python 3 running Pycharm and the module gspread to read google sheet files in my google drive. Well, I've followed all steps from this link about how to read a file. Unfortunately, my code here below doesn't work yet.
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope =['https://docs.google.com/spreadsheets/d/1xnaOZMd2v93tY28h_hsuMnZYXC9YqCfFpQX70lwpN94/edit?usp=sharing']
credentials = ServiceAccountCredentials.from_json_keyfile_name('distech-c1e26e7150b2.json',scope)
gc = gspread.authorize(credentials)
wks = gc.open("POC").sheet1
for temp in wks:
print(temp)
How could I read the google sheet file using this module guys? thanks so much

I got it after a deep research I realize two things.
the scope in my code was wrong cause the scope is just one provided by Google API to grant right access permissions under spreadsheet.
The right scope for me was: scope =['https://spreadsheets.google.com/feeds']
the opener it's just to open the spreadsheet that will return the worksheets within my file.
So solutions thanks to #Pedro Lobito in his post here.
Solution:
I had the same issue with just a couple of spreadsheets on my account, the problem was solved by:
Opening the json key file (projectname-ca997255dada.json)
Find the value of client_email , i.e.: client_email": "278348728734832-compute#developer.gserviceaccount.com
Share your sheet(s) with that email
Now my code looks like:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope =['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name('xxx.json',scope)
gc = gspread.authorize(credentials)
spreadsheet = gc.open("POC")
wks = spreadsheet.worksheet('test1')
wks2 = spreadsheet.worksheet('test2')
out = list()
out = wks.col_values(1)
for temp in out:
print(out)