Develop Reference

How to use the files().update method in Google Drive Python API?

I'm using the Google Drive Python API (v3), and I'm having trouble updating a file on Drive.
Here is my code for updating a file. The code runs without any errors, and even returns the correct file ID at the end, but it does not update the file on Drive.
basename = "myfile.txt"
filename = "/path/to/myfile.txt"
file_metadata = {'name': basename}
media = MediaFileUpload(filename, mimetype='text/plain', resumable=True)
file = drive_service.files().update(fileId=file_id, body=file_metadata, media_body=media).execute()
print(file)
The print(file) statement produces the following output:
{'id': <fileid>}
What's curious is that I'm able to create a file without any issues. Here is my code for creating a file, which creates a file successfully.
basename = "myfile.txt"
filename = "/path/to/myfile.txt"
file_metadata = {'name': basename, 'parents': [drive_folder_id]}
media = MediaFileUpload(filename, mimetype='text/plain')
file = drive_service.files().create(body=file_metadata, media_body=media, fields='id').execute()
Why am I able to create a file, but not update a file?
How can I improve my code for updating a file so that it successfully updates the file on Drive?
Summary:
My code for updating a file runs smoothly, but it doesn't do the thing it's intended to do, that is, update the contents of the file. How can I update the contents of a file on Drive using the Google Drive Python API v3?

It seems that the body parameter should point to a file resource object, not a plain dictionary as in your code. See this example from the documentation
try:
# First retrieve the file from the API.
file = service.files().get(fileId=file_id).execute()
# File's new metadata.
file['title'] = new_title
file['description'] = new_description
file['mimeType'] = new_mime_type
# File's new content.
media_body = MediaFileUpload(
new_filename, mimetype=new_mime_type, resumable=True)
# Send the request to the API.
updated_file = service.files().update(
fileId=file_id,
body=file,
newRevision=new_revision,
media_body=media_body).execute()
return updated_file
except errors.HttpError, error:
print 'An error occurred: %s' % error
return None

I think the problem may have been my web browser's caching feature. I was looking at files that I updated and I didn't see any changes, and this may have been caused by caching.
I switched browsers and tested, and it seems to be the case.
The update method works for me now.

Can't rename file on server, then upload onto google drive API via service account

I have read through several articles/docs/questions now and each solves a part of my problem, but I just can't seem to put them all together to work. I am using Flask/Python, the Google Drive API v3 and a Service Account, but want to run the site on Heroku, which is setup already. Since Heroku doesn't have a storage for files though, I need to directly upload the files into Google Drive, which would need to be done sooner or later anyway.
Basically what I want to do is:
People type in their data on a website and upload a file.
That file then gets renamed according to the user's data (needs to support Chinese characters/utf8)
then the renamed file gets uploaded onto Google Drive using a service account
I was able to rename the file with utf-8 characters and save it locally, but don't know how to upload it to Google.I have tried a way earlier, but I don't know whether just a file has been created with no content or the file has been uploaded properly,I do see files and folders when listing the files/folders (that doesn't work now though, as I think my attempt now is exactly the same). My Code:
def upload_service(file, filename):
file_metadata = {
'name': filename,
'parents': 'Upload'
}
mime = mimetypes.guess_type(file.filename)[0]
media = MediaFileUpload(filename, mimetype=mime)
cloudFile = service.files().create(body=file_metadata, media_body=media).execute()
result = service.files().list().execute()
In here, file is the actual file uploaded with the filename being the input the user typed in.
All works fine, but for media = MediaFileUpload(filename, mimetype=mime) I get a FileNotFoundError: [Errno 2] No such file or directory: 'gtmuCfr.jpg' error, stating that the file with the name of filename or file are not there. This goes for the original name of the file and the new name.

Ok, I finally found the issue and was able to fix it (for the most part).
with open(new_name.encode("utf-8"), "wb") as save_file:
file = save_file.write(f.read())
upload_service(file, new_name)
doing it like this solves the issue as file is the int where it is stored (I guess), but since it doesn't have a filename attribute, I need to use the path/new_name inside the function.
The only difference inside the function is:
file_metadata = {
'name': path,
'parents': 'Upload'
}
mime = mimetypes.guess_type(path)[0]
media = MediaFileUpload(path, mimetype=mime)
where I need to use the path/new_name that I sent as a second parameter.
Now the only issue left is with the Upload folder/parent not being selected for the upload, but I can see the files (after giving permission) in my Drive and they have the right content.

Downloading a large number of small files from Google Drive

I have a folder in Google Drive that is shared to public and it has thousands of files in the size range 5-8 MBs each.
I tried the following to download each file at a time:
Get a list of file IDs/names from the folder using getfilelistpy package as follows:
from getfilelistpy import getfilelist
resource = {
"api_key": API_KEY,
"id": FOLDER_ID,
"fields": "files(name,id)",
}
res = getfilelist.GetFileList(resource)
print(res)
Once the file list is obtained I am looping through each file and use wget to download the file as follows:
for i in range(Len(id_list)):
command1 = "wget --no-check-certificate -r 'https://docs.google.com/uc?export=download&id={}' -O '{}'".format(id_list[i], file_name_list[i])
os.system(command1)
After 65-66 files, the file size download is 0 KB. Does Google Drive put a limit on the number of files/file size one can download? How can we over come that?
Any help would be appreciated? Thank you.

In your script, the API key is used. In this case, it is considered that the files in the folder are publicly shared. From your replying, also it was found that the files have been publicly shared. So in this answer, I would like to propose to download the files using the API key.
The modified script is as follows.
Modified script:
import requests
from getfilelistpy import getfilelist
API_KEY = '###' # Please set your API key.
FOLDER_ID = '###' # Please set the folder ID.
resource = {
"api_key": API_KEY,
"id": FOLDER_ID,
"fields": 'nextPageToken, files(id,name,webContentLink,mimeType)',
}
res = getfilelist.GetFileList(resource)
for files in res['fileList']:
for file in files['files']:
if 'google' not in file['mimeType']:
filename = file['name']
print('%s is downloading.' % filename)
r = requests.get(file['webContentLink'], stream=True)
if r.status_code == 200:
with open(filename, 'wb') as f:
f.write(r.content)
Reference:
getfilelistpy

How do I upload multiple files using the Flask test_client?

How can I use the Flask test_client to upload multiple files to one API endpoint?
I'm trying to use the Flask test_client to upload multiple files to a web service that accepts multiple files to combine them into one large file.
My controller looks like this:
#app.route("/combine/file", methods=["POST"])
#flask_login.login_required
def combine_files():
user = flask_login.current_user
combined_file_name = request.form.get("file_name")
# Store file locally
file_infos = []
for file_data in request.files.getlist('file[]'):
# Get the content of the file
file_temp_path="/tmp/{}-request.csv".format(file_id)
file_data.save(file_temp_path)
# Create a namedtuple with information about the file
FileInfo = namedtuple("FileInfo", ["id", "name", "path"])
file_infos.append(
FileInfo(
id=file_id,
name=file_data.filename,
path=file_temp_path
)
)
...
My test code looks like this:
def test_combine_file(get_project_files):
project = get_project_files["project"]
r = web_client.post(
"/combine/file",
content_type='multipart/form-data',
buffered=True,
follow_redirects=True,
data={
"project_id": project.project_id,
"file_name": "API Test Combined File",
"file": [
(open("data/CC-Th0-MolsPerCell.csv", "rb"), "CC-Th0-MolsPerCell.csv"),
(open("data/CC-Th1-MolsPerCell.csv", "rb"), "CC-Th1-MolsPerCell.csv")
]})
response_data = json.loads(r.data)
assert "status" in response_data
assert response_data["status"] == "OK"
However, I can't get the test_client to actually upload both files. With more than one file specified, the file_data is empty when the API code loops. I have tried my own ImmutableDict with two "file" entries, a list of file tuples, a tuple of file tuples, anything I could think of.
What is the API to specify multiple files for upload in the Flask test_client? I can't find this anywhere on the web! :(

The test client takes a list of file objects (as returned by open()), so this is the testing utility I use:
def multi_file_upload(test_client, src_file_paths, dest_folder):
files = []
try:
files = [open(fpath, 'rb') for fpath in src_file_paths]
return test_client.post('/api/upload/', data={
'files': files,
'dest': dest_folder
})
finally:
for fp in files:
fp.close()
I think if you lose your tuples (but keeping the open()s) then your code might work.

You should just send data object with your files named as you want:
test_client.post('/api/upload',
data={'title': 'upload sample',
'file1': (io.BytesIO(b'get something'), 'file1'),
'file2': (io.BytesIO(b'forthright'), 'file2')},
content_type='multipart/form-data')

Another way of doing this- if you want to explicitly name your file uploads here (my use case was for two CSVs, but could be anything) with test_client is like this:
resp = test_client.post(
'/data_upload_api', # flask route
file_upload_one=[open(FILE_PATH, 'rb')],
file_upload_two=[open(FILE_PATH_2, 'rb')]
)
Using this syntax, these files would be accessible as:
request.files['file_upload_one'] # etc.

updating file with Google Drive API with mimeType html causes margin increase

I am trying to download a file using Google Drive (in python), edit some values, saving the file locally and then upload that local file as an update. I am getting the HTML version of the file and uploading using mimeType = "text/html". The edits and uploads work except for the margins or line spacing on headings (h2 & h3) increase slightly each time the script is run.
I have tried putting the content directly into the local file after the download without editing it and the same thing happens (see code below). Has anyone any ideas as to what might be causing this?
KB
# get the google drive api object
service = get_service()
# search for file
params = {}
params["q"] = "title = 'Test File'"
values = service.files().list(**params).execute()
# get file
found_file = values['items'][0]
# download file content
content = download_file(service, found_file, "text/html")
f = open('temp.html', 'wb')
f.write(content)
f.close()
file = service.files().get(fileId=found_file['id']).execute()
# set the new values
media_body = MediaFileUpload("temp.html", mimetype="text/html",
resumable=True)
try:
# update the file
updated_file = service.files().update(fileId=found_file['id'], body=file,
media_body=media_body).execute()
except:
print("error uploading file")