Develop Reference

How do you concatenate elements from a string split?

Really simple question but I am struggling with this...If I have a file name called "first_second_third_fourth.txt" and I want to keep the first few elements of that string when creating a new file, I usually run something like this:
import sys
file = sys.argv[1]
new_name = file.split("_")[0] + "_newfile.txt"
The output would just be first_newfile.txt
But if I want to somehow get a new file called first_second_newfile.txt I cannot come up with a simple solution to write it out in one line.
I tried:
import sys
file = sys.argv[1]
new_name = file.split("_")[0:1] + "_newfile.txt"
But this throws an error because you cannot concatenate a string to a list. And the only way I manage is to split it up but this seems so messy:
import sys
file = sys.argv[1]
new_name = file.split("_")[0] + file.split("_")[1] + "_newfile.txt"
I am just curious if there is a shorter way to grab the two elements in the split list without having that second iteration of file.split("_")[1]?

You can use .join() to create desired file name:
f = "first_second_third_fourth.txt"
'_'.join(f.split('_')[0:2]) + '.txt
Output:
'first_second.txt'

How to insert a directory in the middle of a file path in Python?

I want to insert a directory name in the middle of a given file path, like this:
directory_name = 'new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
file_path1 = some_func(file_path0, directory_name, position=2)
print(file_path1)
>>> 'dir1/dir2/new_dir/dir3/dir4/file.txt'
I looked through the os.path and pathlib packages, but it looks like they don't have a function that allows for inserting in the middle of a file path. I tried:
import sys,os
from os.path import join
path_ = file_path0.split(os.sep)
path_.insert(2, 'new_dir')
print(join(path_))
but this results in the error
"expected str, bytes or os.PathLike object, not list"
Does anyone know standard python functions that allow such inserting in the middle of a file path? Alternatively - how can I turn path_ to something that can be processed by os.path. I am new to pathlib, so maybe I missed something out there
Edit: Following the answers to the question I can suggest the following solutions:
1.) As Zach Favakeh suggests and as written in this answer just correct my code above to join(*path_) by using the 'splat' operator * and everything is solved.
2.) As suggested by buran you can use the pathlib package, in very short it results in:
from pathlib import PurePath
path_list = list(PurePath(file_path0).parts)
path_list.insert(2, 'new_dir')
file_path1 = PurePath('').joinpath(*path_list)
print(file_path1)
>>> 'dir1/dir2/new_dir/dir3/dir4/file.txt'

Take a look at pathlib.PurePath.parts. It will return separate components of the path and you can insert at desired position and construct the new path
>>> from pathlib import PurePath
>>> file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
>>> p = PurePath(file_path0)
>>> p.parts
('dir1', 'dir2', 'dir3', 'dir4', 'file.txt')
>>> spam = list(p.parts)
>>> spam.insert(2, 'new_dir')
>>> new_path = PurePath('').joinpath(*spam)
>>> new_path
PurePosixPath('dir1/dir2/new_dir/dir3/dir4/file.txt')
This will work with path as a str as well as with pathlib.Path objects

Since you want to use join on a list to produce the pathname, you should do the following using the "splat" operator: Python os.path.join() on a list
Edit: You could also take your np array and concatenate its elements into a string using np.array2string, using '/' as your separator parameter:https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.array2string.html
Hope this helps.

Solution using regex. The regex will create groups of the following
[^\/]+ - non-'/' characters(i.e. directory names)
\w+\.\w+ - word characters then '.' then word characters (i.e. file name)
import re
directory_name = 'new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
position = 2
regex = re.compile(r'([^\/]+|\w+\.\w+)')
tokens = re.findall(regex, file_path0)
tokens.insert(position, directory_name)
file_path1 = '/'.join(tokens)
Result:
'dir1/dir2/new_dir/dir3/dir4/file.txt'

Your solution has only one flaw. After inserting the new directory in the path list path_.insert(2, 'new_dir')you need to call os.path.join(*path_) to get the new modified path. The error that you get is because you are passing a list as parameter to the join function, but you have to unpack it.

In my case, I knew the portion of path that would precede the insertion point (i.e., "root"). However, the position of the insertion point was not constant due to the possibility of having varying number of path components in the root path. I used Path.relative_to() to break the full path to yield an insertion point for the new_dir.
from pathlib import Path
directory_name = Path('new_dir')
root = Path('dir1/dir2/')
file_path0 = Path('dir1/dir2/dir3/dir4/file.txt')
# non-root component of path
chld = file_path0.relative_to(root)
file_path1 = root / directory_name / chld
print(file_path1)
Result:
'dir1/dir2/new_dir/dir3/dir4/file.txt'

I made a try with your need:
directory_name = '/new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
before_the_newpath = 'dir1/dir2'
position = file_path0.split(before_the_newpath)
new_directory = before_the_newpath + directory_name + position[1]
Hope it helps.

Change only a part of filename

I have these images in my folder:
area11.tif
area12.tif
area14.tif
area21.tif
area22.tif
area25.tif
How can I change only the last digit so they became ordered and "more incremental" ?
Instead if area14.tif it should be area13.tif and the same thing for area22/area25.
I have a code but it's a bit broken because it delete some files (it's strange, I know...).
EDIT: added (maybe broken..) code
try:
path = (os.path.expanduser('~\\FOLDER\\'))
files = os.listdir(path)
idx = 0
for file in files:
idx =+ 1
i = 'ex_area'
if file.endswith('.tif'):
i = i + str(idx)
os.rename(os.path.join(path, file), os.path.join(path, str(i) + '.tif'))
except OSError as e:
if e.errno != errno.EEXIST:
raise

1) Read the files names in the directory into array (of strings).
2) Iterate over the array of filenames
3) For each filename, slice the string and insert the index
4) Rename
For example:
import os
import glob
[os.rename(n, "{}{}.tif".format(n[:5], i)) for i, n in enumerate(glob.glob("area*"))]

First you get the list of the images pathes with the glob module :
images = glob.glob("/sample/*.tif")
then you just rename all of them with the os module :
for i in range(len(images)): os.rename(images[i], ‘area’+i+’.tif’)

First rename all filename to temp name and then add whatever name you prefer
import glob,os
images = glob.glob("*.tif")
for i in range(len(images)):
os.rename(images[i], 'temp_'+str(i)+'.tif')
tempImages = glob.glob("temp*.tif")
for i in range(len(tempImages)):
os.rename(tempImages[i], 'area'+str(i+1)+'.tif')

Found also this other solution. But there is a small difference in this one, and a better way of do the job in the end (at least for me): create a folder for each area. So simple I didn't think of it before...
BTW, here is the code, commented. I am using this one just because I achieved what I want. Thanks to all who answered, made me learn new things.
path = (os.path.expanduser('~\\FOLDER\\AREA1\\')) #select folder
files = os.listdir(path)
i = 1 #counter
name = 'area' #variable which the file will take as name
for file in files:
if file.endswith('.tif'): #search only for .tif. Can change with any supported format
os.rename(os.path.join(path, file), os.path.join(path, name + str(i)+'.tif')) #name + str(i)+'.tif' will take the name and concatenate to first number in counter. #If you put as name "area1" the str(i) will add another number near the name so, here is the second digit.
i += 1 #do this for every .tif file in the folder
It's a little bit simple, but because I put the files in two separate folders. If you keep the files in the same folder, this will not work properly.
EDIT: now that I see, it's the same as my code above....

Python: moving file to a newly created directory

I've got my script creating a bunch of files (size varies depending on inputs) and I want to be certain files in certain folders based on the filenames.
So far I've got the following but although directories are being created no files are being moved, I'm not sure if the logic in the final for loop makes any sense.
In the below code I'm trying to move all .png files ending in _01 into the sub_frame_0 folder.
Additionally is their someway to increment both the file endings _01 to _02 etc., and the destn folder ie. from sub_frame_0 to sub_frame_1 to sub_frame_2 and so on.
for index, i in enumerate(range(num_sub_frames+10)):
path = os.makedirs('./sub_frame_{}'.format(index))
# Slice layers into sub-frames and add to appropriate directory
list_of_files = glob.glob('*.tif')
for fname in list_of_files:
image_slicer.slice(fname, num_sub_frames) # Slices the .tif frames into .png sub-frames
list_of_sub_frames = glob.glob('*.png')
for i in list_of_sub_frames:
if i == '*_01.png':
shutil.move(os.path.join(os.getcwd(), '*_01.png'), './sub_frame_0/')

As you said, the logic of the final loop does not make sense.
if i == '*_01.ng'
It would evaluate something like 'image_01.png' == '*_01.png' and be always false.
Regexp should be the way to go, but for this simple case you just can slice the number from the file name.
for i in list_of_sub_frames:
frame = int(i[-6:-4]) - 1
shutil.move(os.path.join(os.getcwd(), i), './sub_frame_{}/'.format(frame))
If i = 'image_01.png' then i[-6:-4] would take '01', convert it to integer and then just subtract 1 to follow your schema.

A simple fix would be to check if '*_01.png' is in the file name i and change the shutil.move to include i, the filename. (It's also worth mentioning that iis not a good name for a filepath
list_of_sub_frames = glob.glob('*.png')
for i in list_of_sub_frames:
if '*_01.png' in i:
shutil.move(os.path.join(os.getcwd(), i), './sub_frame_0/')
Additionally is [there some way] to increment both the file endings _01 to _02 etc., and the destn folder ie. from sub_frame_0 to sub_frame_1 to sub_frame_2 and so on.
You could create file names doing something as simple as this:
for i in range(10):
#simple string parsing
file_name = 'sub_frame_'+str(i)
folder_name = 'folder_sub_frame_'+str(i)

Here is a complete example using regular expressions. This also implements the incrementing of file names/destination folders
import os
import glob
import shutil
import re
num_sub_frames = 3
# No need to enumerate range list without start or step
for index in range(num_sub_frames+10):
path = os.makedirs('./sub_frame_{0:02}'.format(index))
# Slice layers into sub-frames and add to appropriate directory
list_of_files = glob.glob('*.tif')
for fname in list_of_files:
image_slicer.slice(fname, num_sub_frames) # Slices the .tif frames into .png sub-frames
list_of_sub_frames = glob.glob('*.png')
for name in list_of_sub_frames:
m = re.search('(?P<fname>.+?)_(?P<num>\d+).png', name)
if m:
num = int(m.group('num'))+1
newname = '{0}_{1:02}.png'.format(m.group('fname'), num)
newpath = os.path.join('./sub_frame_{0:02}/'.format(num), newname)
print m.group() + ' -> ' + newpath
shutil.move(os.path.join(os.getcwd(), m.group()), newpath)

How to read filenames in a folder and access them in an alphabetical and increasing number order?

I would like to ask how to efficiently handle accessing of filenames in a folder in the right order (alphabetical and increasing in number).
For example, I have the following files in a folder: apple1.dat, apple2.dat, apple10.dat, banana1.dat, banana2.dat, banana10.dat. I would like to read the contents of the files such that apple1.dat will be read first and banana10.dat will be read last.
Thanks.
This is what I did so far.
from glob import glob
files=glob('*.dat')
for list in files
# I read the files here in order
But as pointed out, apple10.dat comes before apple2.dat

from glob import glob
import os
files_list = glob(os.path.join(my_folder, '*.dat'))
for a_file in sorted(files_list):
# do whatever with the file
# 'open' or 'with' statements depending on your python version

try this one.
import os
def get_sorted_files(Directory)
filenamelist = []
for root, dirs, files in os.walk(Directory):
for name in files:
fullname = os.path.join(root, name)
filenamelist.append(fullname)
return sorted(filenamelist)

You have to cast the numbers to an int first. Doing it the long way would require breaking the names into the strings and numbers, casting the numbers to an int and sorting. Perhaps someone else has a shorter or more efficient way.
def split_in_two(str_in):
## go from right to left until a letter is found
## assume first letter of name is not a digit
for ctr in range(len(str_in)-1, 0, -1):
if not str_in[ctr].isdigit():
return str_in[:ctr+1], str_in[ctr+1:] ## ctr+1 = first digit
## default for no letters found
return str_in, "0"
files=['apple1.dat', 'apple2.dat', 'apple10.dat', 'apple11.dat',
'banana1.dat', 'banana10.dat', 'banana2.dat']
print sorted(files) ## sorted as you say
sort_numbers = []
for f in files:
## split off '.dat.
no_ending = f[:-4]
str_1, str_2 = split_in_two(no_ending)
sort_numbers.append([str_1, int(str_2), ".dat"])
sort_numbers.sort()
print sort_numbers