Develop Reference

How to access files in multiple sub-directories using glob

I have several files stored in a folder called Originals. Originals is stored in a folder called B_Substrate, B_Substrate is stored in a folder called Substrate_Training and Substrate_Training is stored in a folder called Jacob_Images.
I am trying to access the files in the folder Originals using glob but unable to access them.
Can anyone help me understand how to access files in multiple sub directories
This is my code but it doesn't work
import glob
file = '/content/drive/My Drive/Jacob_Images/Substrate_Training/B_Substrate/Originals/*.jpg'
for f in glob.glob(file):
print(f)

Instead of specifying .jpg within the path itself, specify it in glob() function. So, you can try this:
import glob
file = '/content/drive/My Drive/Jacob_Images/Substrate_Training/B_Substrate/Originals'
for f in glob.glob(file + "/*.jpg"):
print(f)
Or you can use this code below borrowing code from this post.
for i in glob.glob('/home/studio-lab-user/sagemaker-studiolab-notebooks/Misc/tsv/**/*.csv', recursive=True):
print(i)
Your code also works. I am certain the issue is with the path you are providing.
Using your way:
Using my way:

How can i have multiple loop to reach list of all files from multiple determined folders?

I have multiple folders. I will read some of them; each folder includes various files (images) that I want access to their paths. I used the below code:
[os.path.join(folder,fname) for fname in os.listdir(folder) for folder in selected_train]
previously I had a list of folders in folders.
but I get the below error:
name 'folder' is not defined
How can I correct it?

You can use os.walk function to navigate through the folder.
import os
subfiles=[x[2] for x in os.walk(".")]
print(subfiles)
And also, if you want to just one list you can do this;
import os
subfiles=[]
for x in os.walk("."):
subfiles.extend(x[2])#x[2] represent the all files in one directory
print(subfiles)

How do I copy certain files from multiple sub directories to one destination using python?

I am new to python and i am trying to write a script that moves .txt files from different sub directories to one new folder.
import os
import shutil
source_directory = os.path.join('mysourcedirectory', 'source')
target_directory = os.path.join('destinationdirectory', 'target')
operation= 'copy'
for src_dir, dirs, files in os.walk(source_directory):
if file.endswith(".txt"):
shutil.copy(os.path.join(target_directory, file))
Would someone be able to tell me where I am going wrong and how I can modify this? I'd also like to be able to modify these files (remove header, rename files to consecutive numbers). Can this still be done in one script?

Import list of folder names in a folder with Python

So I've started down the path again of trying to automate something. My end game is to combine the data within Excel files containing the Clean Up in the file name and combine the data from a tab within these files named LOV. So basically it had to go into a folder with folders which have folders again that have 2 files, one file has the words Clean Up in the naming and is a .xlsx file. Which I need to only read those files and and pull the data from the tab called LOV into one large file. --- So that's my end goal. Which I just started and I am no where near, but now you know the end game.
Currently I'm stuck just getting a list of Folder names in the Master folder so I at least know it's getting there lol.
import os
import glob
import pandas as pd
# assigns directory location to PCC Folder
os.chdir('V:/PCC Clean Up Project 2017/_DCS Data SWAT Project/PCC Files
Complete Ready to Submit/Brake System Parts')
FolderList = glob.glob('')
print(FolderList)
Any help is appreciated, thanks guys!

EDITED
Firstly Its hard to understand your question. But from what I understand you need to iterate over folders and subfolders, you can do that with
for root, dirs, files in os.walk(source): #Give your path in source
for file in filenames:
if file.endswith((".xlxs")): # You can check for any file extension
filename = os.path.join(subdir,file)
dirname = subdir.split(os.path.sep)[-1] # gets the directory name
print(dirname)

If you only want the list of folders in your current directory, you can use os.path. Here is how it works:
import os
directory = "V:/PCC Clean Up Project 2017/_DCS Data SWAT Project/PCC Files
Complete Ready to Submit/Brake System Parts"
childDirectories = next(os.walk(directory))[1]
This will give you a list of all folders in your current directory.
Read more about os.walk here.
You can then go into one of the child directories by using os.chdir:
os.chdir(childDirectories[i])

How to create a folder for each item in a directory?

I'm having trouble making folders that I create go where I want them to go. For each file in a given folder, I want to create a new folder, then put that file in the new folder. My problem is that the new folders I create are being put in the parent directory, not the one I want. My example:
def createFolder():
dir_name = 'C:\\Users\\Adrian\\Entertainment\\Coding\\Test Folder'
files = os.listdir(dir_name)
for i in files:
os.mkdir(i)
Let's say that my files in that directory are Hello.txt and Goodbye.txt. When I run the script, it makes new folders for these files, but puts them one level above, in 'C:\Users\Adrian\Entertainment\Coding.
How do I make it so they are created in the same place as the files, AKA 'C:\Users\Adrian\Entertainment\Coding\Test Folder'?

import os, shutil
for i in files:
os.mkdir(os.path.join(dir_name , i.split(".")[0]))
shutil.copy(os.path.join(dir_name , i), os.path.join(dir_name , i.split(".")[0]))

os.listdir(dir_name) lists only the names of the files, not full paths to the files. To get a path to the file, join it with dir_name:
os.mkdir(os.path.join(dir_name, i))