The directory stricture on Google Drive is as follows:
Inside mydrive/BTP/BTP-4
I need to get the folder ID for BTP-4 so that I can transfer a specific file from the folder. How do I do it?
fileList = GoogleDrive(self.driveConn).ListFile({'q': "'root' in parents and trashed=false"}).GetList()
for file in fileList:
if (file['title'] == "BTP-4"):
fileID = file['id']
print(remoteFile, fileID)
return fileID
Will be able to give path like /MyDrive/BTP/BTP-4 and filename as "test.csv" and then directly download the file?
Answer:
Unfortunately, this is not possible.
More Information:
Google Drive supports creating multiple files or folders with the same name in the same location:
As a result of this, in some cases, providing a file path isn't enough to identify a file or folder uniquiely - in this case mydrive/Parent folder/Child folder/Child doc points to two different files, and mydrive/Parent folder/Child folder/Child folder points to five different folders.
You have to either directly search for the folder with its ID, or to get a folder/file's ID you have to search for children recursively through the folders like you are already doing.
Related
I am using the Google Drive's API to create and search files and folders using python 3. Searching a folder is done using the files.list method of the drive API. Each search requires a query q as an input. Some examples of queries are given here. I can successfully search for a folder named hello using its name as given below:
service.files().list(q="name = 'hello'", pageSize=10, fields=files(id, name)).execute()
Now, let us say the ID of the folder is 123, then how can I search for this folder using the ID instead of the name. I have tried replacing q with id = '123' as well as id = 123 but none of it seems to work. The closest query I was able to find here was:
q = "'123' in parents"
But this will return all the files whose parent folder is the folder we are looking for.
Is there any way in which we can directly search folders or files by their ID?
Thank you.
I believe your goal is as follows.
You want to retrieve the folder metadata by the folder ID Drive API with googleapis for python.
In this case, how about the following sample script?
folderId = "###" # Please set the folder ID.
res = service.files().get(fileId=folderId, fields="id, name").execute()
In this case, "Files: get" method is used.
Additional information:
If you want to retrieve the folder list just under the specific folder, you can use the following modified search query.
q = "'123' in parents and mimeType='application/vnd.google-apps.folder'"
In this case, 123 is the top folder ID.
In this modification, the retrieved mimeType is given. By this, only the folder list can be retrieved.
If you want to retrieve the file list except for the folder, you can also use the following search query.
q = "'123' in parents and not mimeType='application/vnd.google-apps.folder'"
If you want to retrieve the folder ID from the folder name, you can use the following sample script. In this case, when there are same folder names of hello, those folders are returned.
res = service.files().list(q="name = 'hello' and mimeType='application/vnd.google-apps.folder'", pageSize=10, fields="files(id, name)").execute()
Note:
In your showing script, please modify fields=files(id, name) of service.files().list(q="name = 'hello'", pageSize=10, fields=files(id, name)).execute() to fields="files(id, name)".
References:
Files: get
Search for files and folders
I'm trying to copy files from a local machine to a specific folder in GDrive using PyDrive. If the target folder does not yet exist, I want to create it. Here is the relevant section of my code:
gfile = drive.CreateFile({'title':'dummy.csv',
'mimeType':'text/csv',
'parent': tgt_folder_id})
gfile.SetContentFile('dummy.csv')
gfile.Upload() # Upload it
I am definitely creating/finding the target folder correctly, and the tgt_folder_id is correct, but PyDrive always writes the file to the root folder of my Google Drive, not the target folder I've specified via the 'parent' parameter.
What am I doing wrong here?
OK, looks like this is how you do it:
gfile = drive.CreateFile({'title':'dummy.csv', 'mimeType':'text/csv',
"parents": [{"kind": "drive#fileLink","id": tgt_folder_id}]})
The "parents" map is used in the Google Drive SDK, which PyDrive is supposed to wrap. But the very few examples I've seen with PyDrive use "parent" and don't seem to work.
Anyway, hope this helps anybody else who hits the same problem.
Ahoj #i-am-nik, to list subfolders you may use slightly altered line:
file_list = drive.ListFile({'q': 'trashed=false', 'maxResults': 10}).GetList()
for file1 in file_list:
print('title: %s, id: %s' % (file1['title'], file1['id']))
This way it will list both folders and subfolders (of course, if you have many files, you may need to change maxResults value or add narrowing query.
I would like to iterate over some files in a folder that has its path in databricks Repos.
How would one do this? I don't seem to be able to access the files in Repos
I have added a picture that shows what folders i would like to access (the dbrks & sql folders)
Thanks :)
Image of the repo folder hierarchy
You can read files from repo folders. The path is /mnt/repos/, this is the top folder when opening the repo window. You can then iterate yourself over these files.
Whenever you find the file you want you can read it with (for example) Spark. Example if you want to read a CSV file.
spark.read.format("csv").load(
path, header=True, inferSchema=True, delimiter=";"
)
If you just want to list files in the repositories, then you can use the list command of Workspace REST API. Using it you can implement recursive listing of files. The actual implementation would different, based on your requirements, like, if you need to generate a list of full paths vs. list with subdirectories, etc. This could be something like this (not tested):
import requests
my_pat = "generated personal access token"
workspace_url = "https://name-of-workspace"
def list_files(base_path: str):
lst = requests.request(method='get',
url=f"{workspace_url}/api/2.0/workspace/list",
headers={"Authentication": f"Bearer {my_pat}",
json={"path": base_path}).json()["objects"]
results = []
for i in lst:
if i["object_type"] == "DIRECTORY" or i["object_type"] == "REPO":
results.extend(list_files(i["path"]))
else:
results.append(i["path"])
return results
all_files = list_files("/Repos/<my-initial-folder")
But if you want to read a content of the files in the repository, then you need to use so-called Arbitrary Files support that is available since DBR 8.4.
I'm trying to delete all files with the extension '.pdf' from a google drive folder.
Everything is fine with the API authentication, I can upload the files. The problem is being the delete.
Here I upload
upload_file = 'Test1.pdf'
gfile = drive.CreateFile({'parents': [{'id': '11SsSKYEATgn_VWzSb-8RjRL-VoIxvamC'}]})
gfile.SetContentFile(upload_file)
gfile.Upload()
Here I try to delete
delfile = drive.CreateFile({'parents': [{'id': '11SsSKYEATgn_VWzSb-8RjRL-VoIxvamC'}]})
filedel = "*.pdf"
delfile.SetContentFile(filedel)
delfile.Delete()
Error:
Traceback (most recent call last):
File "C:/Users/linol/Documents/ProjetoRPA-Python/RPA-TESTE.py", line 40, in <module>
delfile.SetContentFile(filedel)
File "C:\Users\linol\Documents\ProjetoRPA-Python\venv\lib\site-packages\pydrive\files.py", line 175, in SetContentFile
self.content = open(filename, 'rb')
OSError: [Errno 22] Invalid argument: '*.pdf'
I believe your goal and your current situation as follows.
You want to delete the files of PDF file in the specific folder.
You want to achieve this using pydrive for python.
You have already been able to get and put values for Google Drive using Drive API.
In this case, I would like to propose the following flow.
Retrieve file list of PDF file from the specific folder.
Delete the files using the file list.
When above flow is reflected to the script, it becomes as follows.
Sample script:
Please modify ### to your folder ID.
# 1. Retrieve file list of PDF file from the specific folder.
fileList = drive.ListFile({'q': "'###' in parents and mimeType='application/pdf'"}).GetList()
# 2. Delete the files using the file list.
for e in fileList:
drive.CreateFile({'id': e['id']}).Trash()
# drive.CreateFile({'id': e['id']}).Delete() # When you use this, the files are completely deleted. Please be careful this.
This sample script retrieves the files using the mimeType. When you want to retrieve the files using the filename, you can also use fileList = drive.ListFile({'q': "'###' in parents and title contains '.pdf'"}).GetList().
IMPORTANT: In this sample script, when Delete() is used, the files are completely deleted from Google Drive. So at first, I would like to recommend to use Trash() instead of Delete() as a test of script. By this, the files are not deleted and moved to the trash box. By this, I thought that you can test the script.
Note:
It seems that PyDrive uses Drive API v2. Please be careful this.
Reference:
PyDrive
I'm trying to copy files from a local machine to a specific folder in GDrive using PyDrive. If the target folder does not yet exist, I want to create it. Here is the relevant section of my code:
gfile = drive.CreateFile({'title':'dummy.csv',
'mimeType':'text/csv',
'parent': tgt_folder_id})
gfile.SetContentFile('dummy.csv')
gfile.Upload() # Upload it
I am definitely creating/finding the target folder correctly, and the tgt_folder_id is correct, but PyDrive always writes the file to the root folder of my Google Drive, not the target folder I've specified via the 'parent' parameter.
What am I doing wrong here?
OK, looks like this is how you do it:
gfile = drive.CreateFile({'title':'dummy.csv', 'mimeType':'text/csv',
"parents": [{"kind": "drive#fileLink","id": tgt_folder_id}]})
The "parents" map is used in the Google Drive SDK, which PyDrive is supposed to wrap. But the very few examples I've seen with PyDrive use "parent" and don't seem to work.
Anyway, hope this helps anybody else who hits the same problem.
Ahoj #i-am-nik, to list subfolders you may use slightly altered line:
file_list = drive.ListFile({'q': 'trashed=false', 'maxResults': 10}).GetList()
for file1 in file_list:
print('title: %s, id: %s' % (file1['title'], file1['id']))
This way it will list both folders and subfolders (of course, if you have many files, you may need to change maxResults value or add narrowing query.