Develop Reference

How to add a fixed number to the integer part of a filename?

Using Python, I need to add 100 to the integer part of some filenames to rename the files. The files look like this: 0000000_6dee7e249cf3.log where 6dee7e249cf3 is a random number. At the end I should have:
0000000_6dee7e249cf3.log should change to 0000100_6dee7e249cf3.log
0000001_12b2bb88d493.log should change to 0000101_12b2bb88d493.log
etc, etc…
I can print the initial files using:
initial: glob('{0:07d}_*[a-z]*'.format(NUM))
but the final files returns an empty list:
final: glob('{0:07d}_*[a-z]*'.format(NUM+100))
Moreover, I cannot not rename initial to final using os.rename because it can not read the list created using the globe function.

I've included your regex search. It looks like glob doesn't handle regex, but re does
import os
import re
#for all files in current directory
for f in os.listdir('./'):
#if the first 7 chars are numbers
if re.search('[0-9]{7}',f):
lead_int = int(f.split('_')[0])
#if the leading integer is less than 100
if lead_int < 100:
# rename this file with leading integer + 100
os.rename(f,'%07d_%s'%(lead_int + 100,f.split('_')[-1]))

Split the file name value using '_' separator and use those two values to reconstruct your file name.
s = name.split('_')
n2 = str(int(s[0]) + 100)
new_name = s[0][:len(s[0]) - len(n2)] + n2 + '_' + s[1]

python glob to match a wider range

Trying to match files on disk that either end with .asm,ASM, or with some 1/2/3 digit extension like - .asm.1/.asm.11
My python code is-
asmFiles = glob.glob('*.asm') + glob.glob('*.ASM') + glob.glob('*.asm.[0-9]') + glob.glob('*.ASM.[0-9]')
How do I match the file '.asm.11' as my code can only match the first three?
Thanks

Here is a solution using Python regex and list comprehension:
import re
files = ['foobar.asm', 'foobar.ASM', 'foobar.asm.1', 'foobar.ASM.11', 'foobarasm.csv']
asm_pattern = '\.(asm|ASM)$|(asm|ASM)\.[1-9]$|\.(asm|ASM)\.[1-9][1-9]$'
asmFiles = [f for f in files if re.search(asm_pattern, f)]
[print(asmFile) for asmFile in asmFiles]
The last element from list files is an edge case I thought about to test the search pattern. It does not appear in the result, as expected.

How to perform a case-insensitive search for files of a given suffix?

I'm looking for the equivalent of find $DIR -iname '*.mp3', and I don't want to do the kooky ['mp3', 'Mp3', MP3', etc] thing. But I can't figure out how to combine the re*.IGNORECASE stuff with the simple endswith() approach. My goal is to not miss a single file, and I'd like to eventually expand this to other media/file types/suffixes.
import os
import re
suffix = ".mp3"
mp3_count = 0
for root, dirs, files in os.walk("/Volumes/audio"):
for file in files:
# if file.endswith(suffix):
if re.findall('mp3', suffix, flags=re.IGNORECASE):
mp3_count += 1
print(mp3_count)
TIA for any feedback

Don't bother with os.walk. Learn to use the easier, awesome pathlib.Path instead. Like so:
from pathlib import Path
suffix = ".mp3"
mp3_count = 0
p = Path('Volumes')/'audio': # note the easy path creation syntax
# OR even:
p = Path()/'Volumes'/'audio':
for subp in p.rglob('*'): # recursively iterate all items matching the glob pattern
# .suffix property refers to .ext extension
ext = subp.suffix
# use the .lower() method to get lowercase version of extension
if ext.lower() == suffix:
mp3_count += 1
print(mp3_count)
"One-liner", if you're into that sort of thing (multiple lines for clarity):
sum(1 for subp in (Path('Volumes')/'audio').rglob('*')
if subp.suffix.lower() == suffix)

You can try this :)
import os
# import re
suffix = "mp3"
mp3_count = 0
for root, dirs, files in os.walk("/Volumes/audio"):
for file in files:
# if file.endswith(suffix):
if file.split('.')[-1].lower() == suffix:
mp3_count += 1
print(mp3_count)
Python's string.split() will separate the string into a list, depending on what parameter is given, and you can access the suffix by [-1], the last element in the list

The regex equivalent of .endswith is the $ sign.
To use your example above, you could do this;
re.findall('mp3$', suffix, flags=re.IGNORECASE):
Though it might be more accurate to do this;
re.findall(r'\.mp3$', suffix, flags=re.IGNORECASE):
which makes sure that the filename ends with .mp3 rather than picking up files such as test.amp3.
This is a pretty good example of a situation that doesn't really require regex - so while you're welcome to learn from these examples, it's worth considering the alternatives provided by other answerers.

Python: moving file to a newly created directory

I've got my script creating a bunch of files (size varies depending on inputs) and I want to be certain files in certain folders based on the filenames.
So far I've got the following but although directories are being created no files are being moved, I'm not sure if the logic in the final for loop makes any sense.
In the below code I'm trying to move all .png files ending in _01 into the sub_frame_0 folder.
Additionally is their someway to increment both the file endings _01 to _02 etc., and the destn folder ie. from sub_frame_0 to sub_frame_1 to sub_frame_2 and so on.
for index, i in enumerate(range(num_sub_frames+10)):
path = os.makedirs('./sub_frame_{}'.format(index))
# Slice layers into sub-frames and add to appropriate directory
list_of_files = glob.glob('*.tif')
for fname in list_of_files:
image_slicer.slice(fname, num_sub_frames) # Slices the .tif frames into .png sub-frames
list_of_sub_frames = glob.glob('*.png')
for i in list_of_sub_frames:
if i == '*_01.png':
shutil.move(os.path.join(os.getcwd(), '*_01.png'), './sub_frame_0/')

As you said, the logic of the final loop does not make sense.
if i == '*_01.ng'
It would evaluate something like 'image_01.png' == '*_01.png' and be always false.
Regexp should be the way to go, but for this simple case you just can slice the number from the file name.
for i in list_of_sub_frames:
frame = int(i[-6:-4]) - 1
shutil.move(os.path.join(os.getcwd(), i), './sub_frame_{}/'.format(frame))
If i = 'image_01.png' then i[-6:-4] would take '01', convert it to integer and then just subtract 1 to follow your schema.

A simple fix would be to check if '*_01.png' is in the file name i and change the shutil.move to include i, the filename. (It's also worth mentioning that iis not a good name for a filepath
list_of_sub_frames = glob.glob('*.png')
for i in list_of_sub_frames:
if '*_01.png' in i:
shutil.move(os.path.join(os.getcwd(), i), './sub_frame_0/')
Additionally is [there some way] to increment both the file endings _01 to _02 etc., and the destn folder ie. from sub_frame_0 to sub_frame_1 to sub_frame_2 and so on.
You could create file names doing something as simple as this:
for i in range(10):
#simple string parsing
file_name = 'sub_frame_'+str(i)
folder_name = 'folder_sub_frame_'+str(i)

Here is a complete example using regular expressions. This also implements the incrementing of file names/destination folders
import os
import glob
import shutil
import re
num_sub_frames = 3
# No need to enumerate range list without start or step
for index in range(num_sub_frames+10):
path = os.makedirs('./sub_frame_{0:02}'.format(index))
# Slice layers into sub-frames and add to appropriate directory
list_of_files = glob.glob('*.tif')
for fname in list_of_files:
image_slicer.slice(fname, num_sub_frames) # Slices the .tif frames into .png sub-frames
list_of_sub_frames = glob.glob('*.png')
for name in list_of_sub_frames:
m = re.search('(?P<fname>.+?)_(?P<num>\d+).png', name)
if m:
num = int(m.group('num'))+1
newname = '{0}_{1:02}.png'.format(m.group('fname'), num)
newpath = os.path.join('./sub_frame_{0:02}/'.format(num), newname)
print m.group() + ' -> ' + newpath
shutil.move(os.path.join(os.getcwd(), m.group()), newpath)

File name matching - middle of the string

I have a directory with files that follow the format: LnLnnnnLnnn.txt
where L = letters and n = numbers. E.g: p2c0789c001.txt
I would like to separate these files based on whether the second number (i.e. 0789) is odd or even.
I've only managed to get this to work if the second number ranges between 0001-0009 using the code:
odd_files = []
for root, dirs, filenames in os.walk('.'):
for filename in fnmatch.filter(filenames, 'p2c000[13579]*.txt'):
odd_files.append(os.path.join(root, filename))
This will return the files: ['./p2c0001c054.txt', './p2c0003c055.txt', './p2c0005c056.txt', './p2c0007c057.txt', './p2c0009c058.txt']
Any suggestion how could I get this to work for any given four digit number?

The easiest solution would be to expand your wildcard to match a wider array of things.
to that end I would probably do something like:
for filename in fnmatch.filter(filenames, '??????[13579]*.txt'):
This will match any characters before your values, it will match any of the odd values in your wildcard class and then it will accept anything to match afterwards.
This is a bit gross because as it is aaaaaaaa3alkjfdhalkjfshglkjzsdhfgs.txt would match and that is super gross. If you know that the data in the directories you are walking is well controlled that might be ok. A better solution might be to specify things a bit more. This could be done with the following expression:
'[a-z][0-0][a-z][0-9][0-9][0-9][13579][a-z][0-9][0-9][0-9].txt'
The fnmatch.filter method using Unix style wildcards. That means you can use the following:
? - match any single character
* - matches anything from nothing to everything
[] - this matches a class of things, use a - for a range and ! for exclusion

Would this do it?
import re
regex = re.compile("[a-z][0-9][a-z]([0-9]{4})[a-z][0-9]{3}.txt")
filter(lambda x: int(regex.match(x).groups()[0]) % 2 == 1, fnmatch)

If it's getting a little hairy, you could always turn that into a generator and code the tests by hand:
def odd_files_generator():
for root, dirs, filenames in os.walk('.'):
for filename in filenames:
if filename[6] in '13579':
yield filename
odd_files = list(odd_files_generator)
If your test is growing exceedingly hard to express tersely, replace the if filename ... line with your explicit test code.

There's no particular magic to constructing this kind of filter. It just
requires carefully constructing the appropriate regular expression and testing
against it. When using complex patterns with a lot of repetitive components,
errors can easily creep in. I like to define helper functions that make the
specification more human-readable and easier to modify later if need be.
import re
import os
# helper functions for legible re construction
LETTER = lambda n='': '({0}{1})'.format('[A-Za-z]', n)
NUM = lambda n='': '({0}{1})'.format('\d', n)
FILENAME = LETTER() + NUM() + LETTER() + NUM('{4}') + LETTER() + NUM('{3}') + '\.txt'
FILENAME_RE = re.compile(FILENAME)
is_odd = lambda n: int(n) % 2 > 0
def odd_nnnn(f):
"""
Determine if the given filename `f` matches our desired LnLnnnnLnnn.txt pattern
with the second group of numbers (nnnn) odd.
"""
m = FILENAME_RE.search(f)
return m is not None and is_odd(m.group(4))
if __name__ == '__main__':
print "Search pattern:", FILENAME
files = ['./p2c0001c054.txt', './p2c0001c055.txt', './p2c0003c055.txt', './p2c0005c056.txt', './p2c0022c056.txt', './p2c0004c056.txt', './p2c0007c057.txt', './p2c0009c058.txt', './p2c8888c056.txt', ]
files = [ os.path.normpath(f) for f in files ]
root = '/users/test/whatever'
odd_paths = [ os.path.join(root, f) for f in files if odd_nnnn(f) ]
print odd_paths
The only real downside to this is that it's a little more verbose, especially compared to a hyper-compact answer like Brad Beattie's.
[Update] It later occurred to me that a more compact way to define the regular expression might be:
FILENAME = "LnL(nnnn)Lnnn\.txt"
FILENAME_PAT = FILENAME.replace('L', r'[A-Za-z]').replace('n', r'\d')
FILENAME_RE = re.compile(FILENAME_PAT)
This more closely follows the original 'LnLnnnLnnn.txt' description. The match expression would have to change from m.group(4) to m.group(1), because just one group is captured this way.