Develop Reference

Renaming hundreds of long file names with Python

I want to loop through hundreds of file names and rename them. The issue is that the file names are over 260 characters and contain characters that make os.rename not usable.
The code below runs with regular filenames but not with the files described earlier:
import os
filepath = "path"
path = os.chdir(filepath)
count = 1
for filename in os.listdir(path):
new_file_name = f"{count}.csv"
os.rename(filename,new_file_name)
count += 1
When I run with the long file names, I get:
Traceback (most recent call last):
File "C:\Users\sarah\Desktop\Python\program.py", line 7, in <module>
os.rename(filename,new_file_name)
FileNotFoundError: [WinError 3] The system cannot find the path specified:
Is there a way of looping through these file names and renaming them?

Filenames cannot exceed 255 characters in Windows of Linux.
Linux:
touch 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'
touch: cannot touch ‘zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz’: File name too long
Windows:
c:\Users\cp>echo 'foo' > zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
The filename, directory name, or volume label syntax is incorrect.

[Windows specific]
To overcome the issue with MAX_PATH limitation, replace
os.rename(filename,new_file_name)
with
os.rename("\\\\?\\" + filename, "\\\\?\\" + new_file_name)

why not try linux shell?
c=0
a="a"
t=".txt"
for file in `ls key*`; do
newfile="$a$c$t"
mv $file $newfile
let c=c+1
done

FileNotFoundError: [Errno 2] - Possible work network issue?

I am trying to read content of a file on my work network from my work network. I copy and pasted a code snippet from a google search and modified it to the below. Why might I still be getting [Errno 2] (I have changed some of the path names for this question board)
The file path in my file explorer shows that "> This PC > word common common" and I don't have "This PC" in my path. I tried adding that into the place I would think it goes in the string. That didn't solve it.
I tried making sure I have matching capitalization. That didn't solve it.
I tried renaming the file to have a *.txt on the end. That didn't solve it.
I tried the different variations of // and / and \ with and without the r predecessor and while that did eliminate the first error I was getting. It didn't help this error.
(Looking at the code errors in the right gutter is says my line length is greater than the PEP8 standard. While I doubt that is the root of my problem, if you can throw in the 'right' wrap method for a file path that long that would be helpful.)
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
Full Error Copy:
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py", line 1, in
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
FileNotFoundError: [Errno 2] No such file or directory: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
EDIT
DEBUG EFFORTS
working to figure out how to change directory. Just in case that is the problem. Tested this code bit
import os
path = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
Received this error
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py", line 5, in <module>
os.chdir(path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
My intention for adding the picture below is to show how File Explorer displays the file path for my file
FileExplorerPathScreenShot
EDIT
I think this confirms that my 'OS' doesn't have my file.
from os import path
path.exists("PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246")
def main():
print ("File exists:" + str(path.exists('PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246')))
if __name__== "__main__":
main()
Output
File exists: False
I thought OS was a standard variable for Operating system. Now I'm not sure.
EDIT
Using Cmd in DOS, I confirmed that my path for the z: is correct
EDIT - Success
I ran
import os
print( os.listdir("z:/"))
Confirmed I don't need the monster string of folders.
Confirmed, although explorer doesn't show it, it is a *.txt file
Once I implemented these two items the first code worked fine.
Thank you #Furas

To open and read a file specify the filename in your path:
myfile = open("U:/matrix_neo/word common common/hello world.txt", "rt") # open file
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
The U: is a mapped drive in my network.
I did not find any issue with your change dir example. I used a path on my U: path again and it returned True.
import os
path = "U:/matrix_neo/word common common"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
The check the attributes on the directory that you are trying to read from. Also try to copy the file to a local drive for a test and see if you can read the file and also check if it exists.
This is an alternative to the above and uses your path to make sure that the long file path works:
import os
mypath = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
myfile = 'whatever is your filename.txt'
if not os.path.isdir(mypath):
os.makedirs (mypath)
file_path = os.path.join(mypath, myfile)
print(file_path)
if os.path.exists(file_path) is True:
with open(file_path) as filein:
contents = filein.read()
print(contents)
I tested this code using a long csv file.,Replace the variable myfile with whatever is your file name.

How to deal with OSError: [Errno 20] Not a directory: '.DS_Store'?

I want to copy some pictures from on directory to another directory, and here is my code:
import os.path
import shutil
def copyFile(sourceDir,targetDir):
for files in os.listdir(sourceDir):
sourceFile=os.path.join(sourceDir,files)
if os.path.isfile(sourceFile) and sourceFile.find('.jpg')>0:
shutil.copy(sourceFile,targetDir)
for i in os.listdir('/Users/liuchong/Desktop/LFW/new'):
copyFile(i,'/Users/liuchong/Desktop/LFW/lfw')
But when I run it ,the terminal tells me thatOSError: [Errno 20] Not a directory: '.DS_Store'
I know 'DS_dstore' is a hidden file in Mac, but how can I solve this bug?

It is worth to mention what is the general meaning of the "Not a directory" error (Errno 20). It means that you are trying to operate with a path which sub-path is a real file and not a directory. That is it is a malformed, incorrect path.
Example: file.txt/test.txt where file.txt is an existing true file (not a directory).
Python will return this error for every I/O operation with such a file:
$ python -c 'import os; os.path.getsize("file.txt/test.txt");'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/genericpath.py", line 57, in getsize
return os.stat(filename).st_size
OSError: [Errno 20] Not a directory: 'file.txt/test.txt'
The OS will also return this error:
$ stat file.txt/test.txt
stat: cannot stat 'file.txt/test.txt': Not a directory
Every time you have this error it means that you concatenate file+file somewhere in your program.
This error is Unix OS specific. On Windows you should get "File not found" for such malformed path.

Your logic seems badly broken. You iterate through all the files in the directory, passing each one to copyFile. But inside that function, you again try to iterate through each file in the "directory" passed to the function: except that you're not passing only directories to the function, you're passing each file found in the original directory.
It's not clear what you are trying to do, but I think you need to remove one of those calls to listdir and the associated loop.

When you do os.listdir('/Users/liuchong/Desktop/LFW/lfw'), it returns both directories and files. You have to check the entry for directory before passing it as an argument.
def copyFile(sourceDir,targetDir):
for files in os.listdir(sourceDir):
sourceFile=os.path.join(sourceDir,files)
if os.path.isfile(sourceFile) and sourceFile.find('.jpg')>0:
shutil.copy(sourceFile,targetDir)
for i in os.listdir('/Users/liuchong/Desktop/LFW/lfw'):
if os.path.isdir(i):
i = os.path.join( '/Users/liuchong/Desktop/LFW/lfw', i)
copyFile(i,'/Users/liuchong/Desktop/LFW/lfw')

os.listdir('/Users/liuchong/Desktop/LFW/lfw')
returns a list which contains the names of entries in the directory that is given by the user. List is based on random choices.
To address this issue (I know I am late) here's a python 3 compatible code:-
import time
import os
from shutil import copyfile
def Copyfile(in_path, out_path, quantity = None):
filelist = os.listdir(in_path)
counter = 0
print("Copying to %r directory. Please wait... " % out_path)
start = time.time()
for i in range(quantity):
copyfile(filelist[i], out_path+filelist[i])
counter+=1
sys.stdout.write("\rTotal images copied : %r" % counter)
sys.stdout.flush()
end = time.time()
print("This move operation took %r seconds to run." %(end - start))

Shutil.copy IO Error2 when directory exists

I'm encountering a troublesome problem with my code and I've been unable to figure it out. Basically I am copying files from a local directory on my computer to a Dropbox folder that acts as a project repository for me and some other folks.
I keep hitting an IO Error when executing the shutil.copy line. Errno 2, N osuch file or directory. However the directory and file both exist. When I change the directory to a different/test location (test_dir in my code), the code runs perfectly fine. Any insights would be greatly appreciated.
import os, os.path
import re
import shutil
import sys
#File Location
directory_list = "path where files are located"
#Dropbox base directory:
dropbox = "path to dropbox directory"
test_dir = "path to test directory on my local machine"
sed_files = os.listdir(directory_list)
for i in sed_files:
#print i.split("BBB")[0]
#df
copy_dir = re.sub(r'XXX',r'_',i.split("BBB")[0])
copy_dir = re.sub(r'ZZZ',r'/',copy_dir)
copy_dir = dropbox + copy_dir + "/FIXED/"
if not os.path.exists(copy_dir):
os.makedirs(copy_dir)
shutil.copy(directory_list+i,copy_dir)
#print directory_list+i
#os.rename(copy_dir+i,copy_dir+i.split("BBB")[1])
Traceback error is:
Traceback (most recent call last):
File "copy_SE_files.py", line 25, in <module> shutil.copy(direcotry_list+i,copydir)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 116, in copy copyfile(src,dst)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 82, in copyfile with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: 'C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed'
Problem solved thanks to the keen eyes of stack overflow. Modified the line to read:
shutil.copy(directory_list+i,'\\\\?\\'+os.path.abspath(copy_dir))

You're failing because the combined length of the path is greater than Window's MAX_PATH limit. C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed is 274 characters long, and without going to some trouble, most Windows file manipulation APIs won't work properly with a path longer than MAX_PATH (which is 260, and one of them is reserved for the NUL terminator).
Assuming Python uses the correct APIs, you can make it work with the extended path prefix, \\?\ (and it may require you to use backslashes rather than forward slashes in your path; I'm not clear on that; read the docs).

The first thing that jumped out at me was this line:
shutil.copy(directory_list+i,copy_dir)
Consider changing it to
shutil.copy(os.path.join(directory_list,i),copy_dir)
IOW, use os.path.join() when concatenating file paths.

One work around is:
try:
shutil.copy(src, dest)
except:
try:
shutil.copy(src, "\\\\?\\" + dest)
#If Long Path as per Maximum Path limitation Windows
except:
self.failed_TC=True
print("Failed to move the script "+os.path.basename(src)+" to "+dest)

Removing carriage return characters from a file using python

import os
import sys
files = os.listdir(sys.argv[1])
for file in files:
if file[-4:] == ".png":
os.rename(file, file.replace('\r', ''))
Am using the above code to remove \r from the file name, but some how when I execute I get the following error
Traceback (most recent call last):
File "renameImages.py", line 9, in <module>
os.rename(f, f.replace('\r', ''))
OSError: [Errno 2] No such file or directory
Where am I going wrong?

You didn't tell it the directory of the file, that was declared in argv[1]
try os.rename(sys.argv[1]+"/file",sys.argv[1]+"/"+replace('\r'','')
(or '\\' for Windows).

You can use str.rstrip('\r') method to remove a set of characters from the right side of a string:
>>> s="this is your brain on drugs\r\n\r\n"
>>> s
'this is your brain on drugs\r\n\r\n'
>>> s=s.rstrip('\n\r ')
>>> s
'this is your brain on drugs'
You say in your post that the file name has a \r at the end of it; this would be very unusual. Are you sure that your file name has a \r at the end of the string or you are assuming it does because you used Python to print it? Remember that Python appends an automatic return to a string that you print.
Edit
OK: the file name really has a \r on the end. My first recommendation is to fix the script that is producing such unfriendly file names...
For this scipt, you need to either prepend the directory name on the front or CD into the relevant directory. Since THIS answer has the prepending, here is CD'ing:
try:
os.chdir(sys.argv[1])
except OSError:
print "can't change to ",sys.argv[1]
sys.exit(1)
# proceed with the rest of your script...