Develop Reference

How to identify Project_Code, File_Name, File_Format. and create new folder structure based of this

I'm not a good python programmer but a good max scripter.
I have been trying to automate a process of cleaning up max files which are corrupted.
There are total of 88000 files which needs to be cleaned up.
The files I get to clean up are in .zip format, with naming conventions like this.
"Project_Name_File_Name_File_Format.zip"
The automation process of loading the .max file and cleaning up the corruption is done via max script.
What i have been trying to do is Create a folder structure like so:
Project Name--> File Name --> File Format
I have been at it for the last two week, still no good progress on this front.
Here is the basic code I have been trying to get it working at least identifying the file names with file formats. I tried a dictionary method for project names and file formats. Still no luck, I created a list to string and then I landed in a loop of creating lists to strings and strings to lists.
import os
files = os.listdir('path\\') # Set location where all the .zip files are present.
files_zip = [i for i in files if i.endswith('.zip')]
for file_name in files_zip:
print(file_name)
token = os.path.splitext(file_name)[0].split("_")
#print(token)
new_token = token[1:-1]
print(new_token)
new_file_name = "_".join(new_token)
print(new_file_name)
Another set of code i tried to do is here
import os
path = 'path\\'
files = os.listdir(path) # Set location where all the .zip files are present.
# Dictionary Project Keys and Values
project_dic = {'ABC': 'Apple Bucket Cake', 'XYZ': 'Xerox Yacht Zoo'}
# Dictionary for File Formats
file_formats = {'FBX': 'FBX', 'OBJ': 'OBJ', '3ds Max': '3ds Max'}
# Looking for the files which ends with Project Names (prj_lst)
files_txt = [i for i in files if i.endswith('.zip')]
# print(files_txt)
prj_lstToStr = ' '.join([str(elem) for elem in files_txt])
name_set = prj_lstToStr.split('prj_lstToStr')
# print(name_set)
#print("Project_List : " + str(name_set))
res = [ele if len(ele) > 0 else () for ele in [[key for key in project_dic if key in sub] for sub in name_set]]
#print("Project_Matching_Keys : " + str(res))
string_key = ''.join(str(res))
format_list = [ele if len(ele) > 0 else () for ele in [[key for key in file_formats if key in sub] for sub in name_set]]
#print("Format_Matching_Keys : " + str(format_list))
format_key = ''.join(str(format_list))
token = files_txt

I don't know what exactly you're trying to do, so I will delete this answer if it is not correct. From what I understand:
input = ["BC_Warren_Vidic_Head_OBJ.zip",
"ALS_Sand_Mound_pr_ann_0479_c_OBJ.zip",
"GRT_ENV-SPE-GRP-SK-ExplorationChestPMC-A_3dsMax.zip",
"KLS_alpha-GEN_PRO_HedgePotPlanter_Group_01A_2021-03-31_FBX.zip",
"MISC_gho_caucasian-mattE_(wise)_OBJ.zip",
"MISC_W_ATT_SalvoXL_FBX.zip",
"MISC_XA-20_Razorback_JetFighter_3dsMax.zip",
"WLD_ENV-GLO-PRO-Bivouac-TacticalSmartphone-A_3dsMax.zip",
"XYZ_WPN_ATT_MAG_MagpulPMAGMOE_FBX.zip"]
for inp in input:
splitted = inp.split('_')
project_name = splitted[0]
file_name = '_'.join(splitted[1:-1])
file_format = splitted[-1].split('.')[0]
path = f'./{project_name}/{file_name}/{file_format}'
os.makedirs(os.path.dirname(path), exist_ok=True)

Move files into different folders based on a text list Python

I´m new to python and I´ve been trying to simplify a manual task that I do on my daily bases, I have a text file with a list of file names separated in groups by a blank line, like this:
fileName1
fileName2
fileName3
fileName4
fileName5
fileName6
fileName7
fileName8
fileName9
fileName10
fileName11
fileName12
All of this files are in one folder and I want to find each group of files and move them into separate folders the name of the new folders should be the name of the first file of each group.
I´m doing my research and I found how to do each step separately using os and shutil modules but I can´t find a way to join them together and make a beautiful script, any help that I can get from you guys will be awesome, thanks!!

Here's a little script that can do that.
I've made two assumptions:
The file with the list of files is stored in the same directory as source files
There is a blank line after the last file so the script can grab the last group
import os
from shutil import move
from itertools import groupby
#Where the files are originally stored
src_path = "C:\\temp\\src\\"
#Where the group folders will go
dest_path = "C:\\temp\\dest\\"
#Open up the file containing the list of files
with open(src_path + "list_of_files.txt") as txt:
lines = txt.readlines() #Read the file
#Split the contents of the file based on the newline character "\n"
i = (list(g) for _, g in groupby(lines, key='\n'.__ne__))
list_of_groups = [a + b for a, b in zip(i, i)]
#Iterate through each group
for group in list_of_groups:
folder_name = dest_path + group[0].replace("\n","") + "\\"
if not os.path.exists(folder_name):
#Create a folder for each group if it doesn't already exist
os.mkdir(folder_name)
#Move over each file in the group. The last element in the group is a newline character
for file in group:
if file != "\n":
move(src_path + file.replace("\n",""),folder_name + file.replace("\n",""))

When reading a file you can look up characters. Blank spaces have a newline character as represented by \n.
import os
filepath1 = os.getcwd() # current working directory
filepath2 = "path\\to\\where\\you\\want\\dir"
filename = os.path.join(filepath1,"yourfilename.txt")
dirName = []
groupName = "filegroup"
idx = 1
newPath = ""
init = True
file = open(filename, "r")
for line in file:
if init == True: # initial folder
newPath = os.path.join(filepath2,groupName + str(idx))
os.mkdir(newPath)
dirName.append(groupName)
init = False
if line == "\n": # line in file is empty
idx += 1
newName = groupName + str(idx)
dirName.append(newName)
newPath = filepath2 + dirName[idx-1]
os.mkdir(newPath)
else:
os.mkdir(os.path.join(newPath,line.rstrip()))
file.close()

Check if a file exists using a text file

I have a folder (Molecules) with many sdf files (M00001.sdf, M00002.sdf and so on) representing different molecules. I also have a csv where each row represents the a molecule (M00001, M00002 etc).
I'm writing a code in order to get files on Molecules folder if their name is a row on the csv file.
First attempt
import os
path_to_files = '/path_to_folder/Molecules' # path to Molecules folder
for files in os.listdir(path_to_files):
names = os.path.splitext(files)[0] # get the basename (molecule name)
with open('molecules.csv') as ligs: # Open the csv file of molecules names
for hits in ligs:
if names == hits:
print names, hits
else:
print 'File is not here'
However this returns nothing on the command line (literally nothing). What is wrong with this code?

I am not sure that this is the best way (I only know that the following code works for my data) but if your molecule.csv has the standard csv format, i.e. "molecule1,molecule2,molecule3 ...", you can try to rearrange your code in this way:
import os
import csv
path_to_files = '/path_to_folder/Molecules' # path to Molecules folder
for files in os.listdir(path_to_files):
names = os.path.basename(files)
names = names.replace(".sdf","")
with open('molecules.csv','r') as ligs:
content = csv.reader(ligs)
for elem in content:
for hits in elem:
if names == hits:
print names, hits
else:
print 'File is not here'
See csv File Reading and Writing for csv module

I solved the problem with a rather brute approach
import os
import csv
import shutil
path_to_files = None # path to Molecules folder
new_path = None # new folder to save files
os.mkdir(new_path) # create the folder to store the molecules
hits = open('molecules.csv', 'r')
ligands = []
for line in hits:
lig = line.rstrip('\n')
ligands.append(lig)
for files in os.listdir(path_to_files):
molecule_name = os.path.splitext(files)[0]
full_name = '/' + molecule_name + '.sdf'
old_file = path_to_files + full_name
new_file = new_path + full_name
if molecule_name in ligands:
shutil.copy(old_file, new_file)

Multiple Paths Traversed and Displayed Filed type in Maya Menu with Python

I'm new here so bare in mind that and I hope my questions are clearly asked for you lot to help me out. I am trying to alter Brent Tylers Dropbox script so that I will be able to list Python under Python, Mel under Mel and so on(eventually plugins and other files too but not for now)
Ok so my directory is like so:
1.
sf=C:/users/scripts/ a.py + b.mel
pf=C:/users/scripts/Python/c.py
mf=C:/users/scripts/Mel/d.mel
(These are the folders my scripts will be placed in)
Code :
absoluteFiles = []
relativeFiles = []
folders = []
allFiles = []
currentFile = ''
for root, dirs, files in os.walk(sf):
for x in files:
correct = root.replace('\\', '/')
currentFile = (correct + '/' + x)
allFiles.append(currentFile)
if currentFile.endswith('.mel'):
relativeFiles.append(currentFile.replace((mf + '/'), ""))
if currentFile.endswith('.py'):
relativeFiles.append(currentFile.replace((pf + '/'), ""))
relativeFiles.sort()
for relativeFile in relativeFiles:
split = relativeFile.split('/')
fileName = split[-1].split('.')
i=0
while i<(len(split)):
### Create Folders ###
if i==0 and len(split) != 1:
if cmds.menu(split[i] ,ex=1) == 0:
cmds.menuItem(split[i], p=PadraigsTools, bld=1, sm=1, to=1, l=split[i])
if i > 0 and i < (len(split)-1):
if cmds.menu(split[i] ,ex=1) == 0:
cmds.menuItem(split[i], p=split[i-1], bld=1, sm=1, to=1, l=split[i])
### Create .mel Files ###
if fileName[-1] == 'mel':
if i==len(split)-1 and len(split) > 1:
scriptName = split[-1].split('.')
temp1 = 'source ' + '"' + sf + '/' + relativeFile + '"; ' + scriptName[0]
command = '''mel.eval(''' + "'" + temp1 + '''')'''
cmds.menuItem(split[i], p=split[i-1], c=command, l=split[i])
if i==len(split)-1 and len(split) == 1:
scriptName = split[-1].split('.')
temp1 = 'source ' + '"' + sf + '/' + relativeFile + '"; ' + scriptName[0]
command = '''mel.eval(''' + "'" + temp1 + '''')'''
cmds.menuItem(split[i], p=Mel, c=command, l=split[i])
### Create .py Files ###
if fileName[-1] == 'py':
if i==len(split)-1 and len(split) > 1:
command = 'import ' + fileName[0] + '\n' + fileName[0] + '.' + fileName[0]+ '()'
cmds.menuItem(split[i], p=split[i-1], c=command, l=split[i])
if i==len(split)-1 and len(split) == 1:
command = 'import ' + fileName[0] + '\n' + fileName[0] + '.' + fileName[0]+ '()'
cmds.menuItem(split[i], p=Python, c=command, l=split[i])
i+=1
So far I can print out individually (sf, pf, mf) to the corresponding Directory but I cant list out everything at once and the files under sf will not show at all. regarding the folders created it ends up very odd. sometimes i would get a duplicate folder as a submenu and if i use sf it give me C:/.
After days and hours of research trying to mend this script I have found no answer including
from itertools import chain
paths = (mf, sf, pf)
for path, dirs, files in chain.from_iterable(os.walk(path) for path in paths):
::QUESTION::
Is there a way i can put this together sanely so that new folders will show up with their contents on refresh as a submenu and the files will show up and allow me to execute them from their corresponding submenu.
I would appreciate any help possible including down votes haha. And bare in mind I don't want you to hand me the answer on a golden spoon because I wont know what is corrected or needs to be :)
Thanks Greatly
-- Padraig

There's a couple of things you can do to simplify things a bit.
First, it's a good idea to make this as data-driven as possible so you don't have to re-write it if your needs change. This does more or less what you do, but collects the results into a dictionary where the key are the root paths you supplied and the values are lists of relative paths:
def find_files(root, extensions = ('mel', 'py')):
def clean_path(*p):
return "/".join(p).replace('\\', '/')
for root, _, files in os.walk(root):
used = [f for f in files if f.split(".")[-1] in extensions]
for u in used:
yield clean_path(root, u)
def relativize(abs, roots):
low_roots = map (str.lower, roots) # all lower for comparison
for root, low_root in zip(roots,low_roots):
if abs.lower().startswith(low_root):
return root, abs[len(root):]
return ("", abs)
relative_paths = find_files('c:/users/scripts')
root_dict = {}
for item in relative_paths :
folder, file = relativize(item, ('C:/users/scripts/Python/', 'C:/users/scripts/Mel/', 'C:/users/scripts/'))
if not folder in root_dict:
root_dict[folder] = []
root_dict[folder].append(file)
So now you have a dictionary with a bunch of root folders and lists of relative paths (files that were not in any relative path you supplied are keyed to empty string and show up as absolute paths). You can make the menus in a very generic way because they are all in the same format. If you need the entire list, you can get it like this:
results = []
for each_root in root_dict:
for relpath in root_dict[each_root]:
results.append(each_root + relpath)
For creating the actual menus, you want to use a single function and bind it to the filename for each menu item as you make it. This is a slightly tricky topic (more detail here). The easy way to do this is to use a functools.partial object, which bundles a command and a bunch of arguments into an object which looks like a function: you can create a partial and attach it to the command of your menu items so they all call the same function with their individual arguments. Here's a simple example using the variables from above and a menubar; you can see how to adapt it to other kinds of menus pretty easily:
from functools import partial
# call this on every button selection
def test(filepath, ignore):
# maya will send "test(name, False)"; we just ignore the 'False'
print "Here's where I would reload", filepath
example = cmds.window(title = 'example')
menubar = cmds.menuBarLayout()
for name in folder_names:
menuname = name
if menuname:
menuname = menuname.split("/")[-2] # we used trailing slashes
else:
menuname = "root"
cmds.menu(label = menuname)
file_names = root_dict[name]
file_names.sort()
for fn in file_names:
mi = cmds.menuItem(label = fn, command = partial(test, fn))
cmds.setParent(menubar)

Getting paths of each file of a directory into an Array in python

Im trying to put into an array files[] the paths of each file from the Data folder but when I try to go into subfolders I want it to be able to go down to the end of the Data file, for example I can read files in a subfolder of the main folder Data which im trying to get a list of all the paths of each file into an array but it doesn't go deeper it does not access the subfolder of the subfolder of Data without writing a loop. Want I want is a loop which has infinit depth of view of files in the Data folder so I can get all the file paths.
For example this is what I get:
['Data/DataReader.py', 'Data/DataReader - Copy.py', 'Data/Dat/DataReader.py', 'Data/fge/er.txt']
This is what I want but it can still go into deeper folders:
['Data/DataReader.py', 'Data/DataReader - Copy.py', 'Data/Dat/DataReader.py', 'Data/fge/er.txt', 'Data/fge/Folder/dummy.png', 'Data/fge/Folder/AnotherFolder/data.dat']
This is my current path, what would i need to add or change?
import os
from os import walk
files = []
folders = []
for (dirname, dirpath, filename) in walk('Data'):
folders.extend(dirpath)
files.extend(filename)
break
filecount = 0
for i in files:
i = 'Data/' + i
files[filecount] = i
filecount += 1
foldercount = 0
for i in folders:
i = 'Data/' + i
folders[foldercount] = i
foldercount += 1
subfolders = []
subf_files = []
for i in folders:
for (dirname, dirpath, filename) in walk(i):
subfolders.extend(dirpath)
subf_files.extend(filename)
break
subf_files_count = 0
for a in subf_files:
a = i + '/'+a
files = files
files.append(a)
print files
subf_files = []
print files
print folders
Thanks a lot!

Don't understand what are your trying to do, especially why you break your walk after the first element:
import os
files = []
folders = []
for (path, dirnames, filenames) in os.walk('Data'):
folders.extend(os.path.join(path, name) for name in dirnames)
files.extend(os.path.join(path, name) for name in filenames)
print files
print folders