I'm using below code,fileID is getting generated but file is not uploaded. I tried debugged but it shows same output as code output. I have placed the service account json file and test.csv in the folder where the python code is placed.
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
def uploadGdrive():
SCOPES = ['https://www.googleapis.com/auth/drive.file']
SERVICE_ACCOUNT_FILE = './iron-tea-266706-8a1f9ee69710.json'
folder_id = '15xc7s-dFnpW6pjtjtUH4kT1BnZ46ffFh'
#file_id = '1IcbtXGW1ZMmv64SqqXJZCsFrwMlE3NiQcDZR98XuFX8'
file_location='./test.csv'
creds = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = build('drive', 'v3', credentials=creds)
file_metadata = {
'name': 'Master Summary',
'mimeType': 'application/vnd.google-apps.spreadsheet'
}
media = MediaFileUpload(
'{}'.format(file_location),
mimetype='text/csv',
resumable=True)
#This is to update existing file
#file = service.files().update(body=file_metadata,media_body=media,fileId='{}'.format(file_id)).execute()
#print(file.get('id'))
#this is to create new file
file = service.files().create(body=file_metadata,
media_body=media,
fields='id').execute()
print('File ID: %s' % file.get('id'))
if __name__ == '__main__':
uploadGdrive()
A service account is not you. A service account is a dummy user, it has its own drive account. Files it uploads are owned by it.
Your file is probably being uploaded to the service accounts drive account. or where ever '15xc7s-dFnpW6pjtjtUH4kT1BnZ46ffFh' is assuming that the service account has access to write to that directory. You can see this by doing a file.list and seeing which files it currently has access to.
If the file to create was successful it will return a file.id as part of the response you can check that as well.
You could create a directory on your own google drive account share that directory with the service account. Then have the service account upload files into that directory then you would be able to see them on your own account. You would also need to grant yourself permissions to the file when its uploaded. Look into permissions.create.
Alternatively you could have the service account upload the files to its own account and then add permissions for you to the file so that you would be able to see it. Look into permissions.create.
Related
I have a lots of data about my employes in a spread sheet. I uploaded those data to my google drive using Drive API. I successfully uploaded the file with python script. But now all the files owner is service account that I created in Drive API.
I tried to change the owner to my current gmail account using python script. I can't change the owner, the script will stop with "You can't change the owner of a file owned by a service account." Error. How can I change the owner to default gmail account, because the all files are just shared with me, not own by me. If I disable to delete the service account, all the files are gone. I need to disable it after I uploaded all the files from my website to drive. What can I do?
my code:
import os
import pandas as pd
import json
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from googleapiclient.http import MediaIoBaseUpload
from google.oauth2 import service_account
import requests
import io
# Define the ID of the file containing the credentials
json_file_id = 'json file location'
# Download the file
with open(json_file_id) as json_file:
json_content = json.load(json_file)
SCOPES = ['https://www.googleapis.com/auth/drive']
# create the credentials object
creds = service_account.Credentials.from_service_account_info(json_content, scopes=SCOPES)
service = build('drive', 'v3', credentials=creds)
file_id = 'file id as service account owner'
new_owner_email = 'current_account_gmail'
permission = {
'type': 'user',
'role': 'owner',
'emailAddress': new_owner_email,
'transferOwnership': True,
}
service.permissions().create(fileId=file_id, body=permission, transferOwnership=True, fields='id').execute()
I want to find a free cloud storage service with free API, that could help me back up some files automatically.
I want to write some script (for example python) to upload files automatically.
I investigated OneDrive and GoogleDrive. OneDrive API is not free, GoogleDrive API is free while it need human interactive authorization before using API.
For now I'm simply using email SMTP protocol to send files as email attachments, but there's a max file size limition, which will fail me in the future, as my file size is growing.
Is there any other recommendations ?
I believe your goal as follows.
You want to upload a file using Drive API with the service account.
You want to achieve your goal using python.
At first, in your situation, how about using google-api-python-client? In this answer, I would like to explain the following flow and the sample script using google-api-python-client.
Usage:
1. Create service account.
Please create the service account and download a JSON file. Ref
2. Install google-api-python-client.
In order to use the sample script, please install google-api-python-client.
$ pip install google-api-python-client
3. Prepare a folder.
Please create a new folder in your Google Drive. And, please share the created folder with the email of your service account. Because the Google Drive of your account is different from the Drive of service account. By sharing the folder with the service account, the file can be uploaded to the folder in your Google Drive using the service account. By this, you can see the uploaded file on your Google Drive by your browser.
4. Prepare sample script.
Please set the filename of credentials of service account, the filename of the file you want to upload and the folder ID of folder you shared your folder with the service account to the variables of SERVICE_ACCOUNT, UPLOAD_FILE and FOLDER_ID, respectively.
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
SERVICE_ACCOUNT = '###' # Please set the file of your credentials of service account.
UPLOAD_FILE = 'sampleFilename' # Please set the filename with the path you want to upload.
FOLDER_ID = '###' # Please set the folder ID that you shared your folder with the service account.
FILENAME = 'sampleFilename' # You can set the filename of the uploaded file on Google Drive.
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT, SCOPES)
drive = build('drive', 'v3', credentials=credentials)
metadata = {'name': FILENAME, "parents": [FOLDER_ID]}
file = MediaFileUpload(UPLOAD_FILE, resumable=True)
response = drive.files().create(body=metadata, media_body=file).execute()
fileId = response.get('id')
print(fileId) # You can see the file ID of the uploaded file.
When you run this script, the file is uploaded to the shared folder in your Google Drive.
When you set the mimeType of the file you want to use, please modify file = MediaFileUpload(UPLOAD_FILE, resumable=True) to file = MediaFileUpload(UPLOAD_FILE, mimeType='###', resumable=True).
References:
google-api-python-client
Creating a service account
Upload file data
gdownload.py using Python3
from apiclient.http import MediaIoBaseDownload
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
import io,os
CLIENT_SECRET = 'client_secrets.json'
SCOPES = ['https://www.googleapis.com/auth/admin.datatransfer','https://www.googleapis.com/auth/drive.appfolder','https://www.googleapis.com/auth/drive']
store = file.Storage('tokenWrite.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET, SCOPES)
flags = tools.argparser.parse_args(args=[])
creds = tools.run_flow(flow, store, flags)
DRIVE = build('drive', 'v2', http=creds.authorize(Http()))
files = DRIVE.files().list().execute().get('items', [])
def download_file(filename,file_id):
#request = DRIVE.files().get(fileId=file_id)
request = DRIVE.files().get_media(fileId=file_id)
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request,chunksize=-1)
done = False
while done is False:
status, done = downloader.next_chunk()
print("Download %d%%." % int(status.progress() * 100))
fh.seek(0)
f=open(filename,'wb')
f.write(fh.read())
f.close()
rinput = vars(__builtins__).get('raw_input',input)
fname=rinput('enter file name: ')
for f in files:
if f['title'].encode('utf-8')==fname:
print('downloading...',f['title'])
download_file(f['title'],f['id'])
os._exit(0)
Google Drive API docs are not super great at helping determine best way to authenticate using a service account that I can then upload a .png file to the Drive. My end goal it so upload a .png file, copy a template doc, batch update that doc using text replace, and insert the newly uploaded .png image into that doc.
Sample code below:
from dotenv import load_dotenv
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
def credentials_from_file():
credentials = service_account.Credentials.from_service_account_file(
os.getenv('SERVICE_ACCOUNT_FILE'),
scopes=os.getenv('SCOPES')
)
drive_service = build('drive', 'v3', credentials=credentials)
return drive_service
def google_upload(drive_service, metadata_name, parents, permissions, file_path, mime_type):
file_metadata = {'kind':'drive#file', 'name':metadata_name, 'parents':parents, 'permissions':permissions}
media = MediaFileUpload(file_path, mimetype=mime_type)
file = drive_service.files().create(body=file_metadata, media_body=media, fields='id', supportsAllDrives=True).execute()
print('File ID: %s' % file.get('id'))
Implementation of Code
credentials = credentials_from_file()
drive_service = build('drive', 'v3', credentials=credentials)
metadata_name = custom_variables_png_table_img
parents = ['xxxx']
permissions = [{'kind':'drive#permission', 'emailAddress':os.getenv('EMAIL_ACCOUNT'), 'role':'owner'}]
file_path = custom_variables_png_table_img
mime_type = 'image/png'
google_upload(drive_service, metadata_name, parents, permissions, file_path, mime_type)
EDIT:
Looks like I forgot to actually write was the problem is. It's two fold.
I keep getting 2 errors when trying to run the google_upload() function which looks like an authentication error with the service account.
Error #1: jwt_grant access_token = response_data["access_token"] KeyError: 'access_token'
The above exception was the direct cause of the following exception:
Error #2: google.auth.exceptions.RefreshError: ('No access token in response.', {'id_token': 'xxx'})
Permissions being properly set on the recently uploaded image file.
The code you are using currently seams to be the same as what I have seen before.
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
SCOPES = ['https://www.googleapis.com/auth/drive.readonly']
KEY_FILE_LOCATION = '<REPLACE_WITH_JSON_FILE_PATH_TO_FILE>'
def initialize_drive():
"""Initializes an Google Drive API V3 service object.
Returns:
An authorized Google Drive API V3 service object.
"""
credentials = ServiceAccountCredentials.from_json_keyfile_name(
KEY_FILE_LOCATION, SCOPES)
# Build the service object.
driveService = build('drive', 'v4', credentials=credentials)
return driveService
You haven't mentioned what is wrong with your code however i can make a few guesses.
The thing is that you mention you want to upload an image and the insert it into a document. You need to remember that the Google drive api is just a file storage api it can do more then that store files.
When you upload the file using the service account you need to remember that the service account is not you. So when you are uploading this file to this directory parents = ['xxxx'] where ever that directory is, either on the service accounts drive account or if this directory is one of your persona directories which you have shared with the service account. You may not have permissions to see this file.
By calling permissions create after uploading your file you can grant your own personal account permissions to access the file as well.
As for adding the image to a document. well the only way google can help you with that is if it is a Google doc type document. Then you would need to go though the Google docs api which would then give you access to add things programmaticlly to a document. I haven't used this API much so im not sure if it has the ability to add images to a document.
You should be able to use the google docs api with your service account you will just need to create a docs service using the same creds you already have from google drive.
service = build('docs', 'v1', credentials=creds)
I am trying to create a simple web application that automatically uploads my database & media backup to a designated google drive. I have followed the official document and created a service account credential, gave it the owner role, and extracted a key(json file) from Google cloud platform. I enabled the Google Drive API on my account and wrote this code, but the credentials.valid returns False and my file would not upload to my drive.
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
print(credentials.valid)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name' : 'python.png'}
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
file_back = service.files().get(fileId=file_up['id']).execute()
print(file_back.get('WebContentLink'))
How about this modification?
Modification points:
I think that in your script, service of service = build('drive', 'v3', credentials=credentials) can be used for uploading the file.
In my environment, I could confirm that the file can be uploaded using your script.
From my file would not upload to my drive., I thought that you might misunderstand about the service account. The file uploaded with the service account is created to the Drive of the service account. This Drive is different from your Google Drive of your account. I thought that this might be the reason of my file would not upload to my drive..
If you want to see the file uploaded with the service account at your Google Drive, it is required to share the uploaded file with your Google account. Or, it is required to upload the file to the folder in your Google Drive shared with the service account.
And also, in your script, file_back.get('WebContentLink') is used. In this case, None is always returned because WebContentLink is required to be WebContentLink. And also, in Drive API v3, the default returned values don't include webContentLink. So it is required to set fields.
When above points are reflected to your script, your script becomes as follows.
Modified script:
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name': 'python.png'}
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
# Create a permission. Here, your Google account is shared with the uploaded file.
yourEmailOfGoogleAccount = '###' # <--- Please set your Email address of Google account.
permission = {
'type': 'user',
'role': 'writer',
'emailAddress': yourEmailOfGoogleAccount,
}
service.permissions().create(fileId=file_up['id'], body=permission).execute()
file_back = service.files().get(fileId=file_up['id'], fields='webContentLink').execute() # or fields='*'
print(file_back.get('webContentLink'))
When you run above script, the uploaded file can be seen at "Shared with me" in your Google Drive.
If you want to put the specific folder of your Google Drive, please use the following script. In this case, before you run the script, please share the folder with the email of the service account. Please be careful this.
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name': 'python.png', 'parents': ['###']} # <--- Please set the folder ID shared with the service account.
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
file_back = service.files().get(fileId=file_up['id'], fields='webContentLink').execute() # or fields='*'
print(file_back.get('webContentLink'))
Note:
In the current stage, when the owner of file uploaded with the service account is changed, an error like You can't yet change the owner of this item. (We're working on it.). So I proposed above modified script.
References:
Files: create
Files: get
Permissions: create
I've written the following code to upload an image to my own Google Drive using a service account.
My code is returning successfully, giving me an ID back but there's nothing appearing on my actual Google Drive.
from django.conf import settings
import os
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.service_account import ServiceAccountCredentials
def get_service(api_name, api_version, scope, key_file_location):
credentials = ServiceAccountCredentials.from_json_keyfile_name(key_file_location, scopes=scope)
service = build(api_name, api_version, credentials=credentials)
return service
def setup_upload():
scope = ['https://www.googleapis.com/auth/drive']
key_file_location = os.path.join(os.path.dirname(settings.BASE_DIR), 'common/my-json-file.json')
service = get_service('drive', 'v3', scope, key_file_location)
file_path = os.path.join(os.path.dirname(settings.BASE_DIR), 'common/apple.png')
file_metadata = {'name': 'apple.png'}
media = MediaFileUpload(file_path, mimetype="image/png")
file = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
print(file.get('id')) #this returns an actual ID
setup_upload()
I do get a long ID string back from the last line of setup_upload(). But nothing is appearing on my actual Google Drive. I'm expecting to see the apple.png file pop up in my home directory.
What am I missing here?
Quoting from the tutorial you linked to:
"You can take the service account email address and give it access to a directory on your Google drive. It will then be allowed to upload to that directory, but you wont have access to the files. You will need to complete a second step and give yourself personally permission to access those files by updating or patching the file permissions."
Have you given the service account access to the place where you want to write the file? If you haven't specified where to upload the file to, the service account may just be uploading the file into its own Drive.
In addition to #user2705223's answer, if you want to be able to access the files it uploads, then you must grant yourself access to them through the service account. Check if you had a successful login credentials and authorize the service account with right scope. You can try following this documentation to help you do the authorization to make API requests.