I am trying to write a program to categorize into folders a large amount of files according to their respective groups indicated in the file name. I wrote the followin code, but when I run it it gives me a file not found error, even though the file is in the given path. I'd appreciate any help in figuring out what is wrong.
import os
old_dir = '/Users/User/Desktop/MyFolder'
for f in os.listdir(old_dir):
file_name, file_ext = os.path.splitext(f)
file_name.split('-')
split_file_name = file_name.split('-')
new_dir = os.path.join(old_dir,
'-'.join(split_file_name[:3]),
split_file_name[5],
f)
os.rename(os.path.join(old_dir, f), new_dir)
Here's the error:
Traceback (most recent call last):
File "/Users/User/Documents/Sort Files into Folders/Sort Files into Folders.py", line 19, in <module>
os.rename(os.path.join(old_dir, f), new_dir)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/User/Desktop/MyFolder/AHA35-3_30x1_12-31-7d-g1a1-ArmPro.jpg' -> '/Users/User/Desktop/MyFolder/AHA35-3_30x1_12-31/ArmPro/AHA35-3_30x1_12-31-7d-g1a1-ArmPro.jpg
os.rename does not automatically create new directories (recursively), if the new name happens to be a filename in a directory that does not exist.
To create the directories first, you can (in Python 3) use:
os.makedirs(dirname, exist_ok=True)
where dirname can contain subdirectories (existing or not).
Alternatively, use os.renames, that can handle new and intermediate directories. From the documentation:
Recursive directory or file renaming function. Works like rename(), except creation of any intermediate directories needed to make the new pathname good is attempted first
os.rename need path, so it should look like:
os.rename(path+old_name, path+new_name)
Related
I am trying to work on copying files to a different directory based on a specific file name listed in excel. I am using shutil to copy files from one directory to another directory, but it keep showing the FileNotFound.
This is the error message:
Traceback (most recent call last):
File "C:\Python\HellWorld\TestCopyPaste.py", line 20, in <module>
shutil.copytree(i, output_file, dirs_exist_ok=True)
File "C:\Users\Asus\Anaconda3\envs\untitled\lib\shutil.py", line 556, in copytree
with os.scandir(src) as itr:
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'Test.pdf'
I am still new to python, please let me know if there's any part can be enhanced :)
Below are my codes:
import os
import shutil
import pandas as pd
#Set file path
input_file = "C:\\Users\\Asus\\Desktop\\Python\\Input\\"
output_file = "C:\\Users\\Asus\\Desktop\\Python\\Output\\"
#Set new variable for the file path to store the list of files
file_list = os.listdir(input_file)
#search the required file name that need to copy to another location
#Create loop to search the files
condition = pd.read_excel(r'C:\\Users\\Asus\\Desktop\\Python\Condition.xlsx')
for i in file_list:
for filename in condition:
if filename in i:
print(i)
shutil.copytree(i, output_file, dirs_exist_ok=True)
As mentioned in comments one issue is that you aren't joining the filename to the full filepath ("input_file").
I'm not really familiar with shutil but I believe the function you want to use is shutil.copy not shutil.copytree. It looks like copytree copies the directory structure of a specified source directory and you are specifically only looking at a list of files within a top level directory. Another issue is how you are reading the excel file.
Assuming the files are listed in a single column it should be something like:
condition = pd.read_excel("C:\\Users\\Asus\\Desktop\\Python\\Condition.xlsx",index_col=None,header=None)
(I also removed your 'r' prefix to the string in this part)
Then to get the items in the first column: condition[0].tolist()
I also believe the second for loop is unnecessary. You can use the same if statement you already have in a single loop.
The following is my solution, just change he paths to what you want. I changed variable names to make it a little more readable as well.
(assumes all files are listed in a single column in excel with no header. And all file are in the input file directory with no subdirectories)
import os
import shutil
import pandas as pd
#Set file path
input_file_dir = "C:\\Users\\myusername\\py\\input\\"
output_file_dir = "C:\\Users\\myusername\\py\\output\\"
#Set new variable for the file path to store the list of files
file_list_from_dir = os.listdir(input_file_dir)
#search the required file name that need to copy to another location
#Create loop to search the files
file_list_from_excel = pd.read_excel("C:\\Users\\myusername\\py\\Condition.xlsx",index_col=None,header=None)
file_list_from_excel = file_list_from_excel[0].tolist()
for thefileNameinDir in file_list_from_dir:
if thefileNameinDir in file_list_from_excel:
print(f"File matched: {thefileNameinDir}")
tempSourcePath = os.path.join(input_file_dir,thefileNameinDir)
shutil.copy(tempSourcePath, output_file_dir)
Okay, so I'm not sure how to get python to use the path to a file that it found through an os.walk function as the source parameter for the shutil.copy(source, destination) arguments.
My code example is this
for folderName, subfolders, filenames in os.walk('/Users/me/Documents'):
print('The current folder is '+folderName)
for subfolder in subfolders:
print('SUBFOLDER OF '+folderName+": "+subfolder)
for filename in filenames:
print("FILE INSIDE "+folderName+": "+filename)
if filename.endswith('.txt'):
os.chdir(filename)
shutil.copy(filename, '/Users/me/Documents/School/IT 145/Text Files')
print("")
If the file has a .txt extension, I would like python to copy that file to the specified folder.
My error message I get is this
The current folder is /Users/me/Documents/Text Files
FILE INSIDE /Users/me/Documents/Text Files: guest.txt
Traceback (most recent call last):
File "/Users/me/Documents/School/IT 145/Programming Work/os_walk.py", line 16, in <module>
os.chdir(filename)
FileNotFoundError: [Errno 2] No such file or directory: 'guest.txt'
From what I understand, python is going back to the current working directory to do the shutil.copy, but I don't understand why if I pass it the file path that it just found it won't use that as the source path for the file to copy.
This is my first ever python programming class, and really my first brush with python all, so any teaching thoughts would be greatly appreciated. Thank you very much.
Your current code never leaves its original working directory. You can verify this by running:
print(os.getcwd())
Your os.walk() does not change the current directory (CWD).
Your attempt at changing the CWD:
os.chdir(filename)
does not work since filename is a file name and not a directory.
os.chdir(folderName)
would work.
You could use the approach of changing into the directory just to copy the file. But, you can't be doing that it every iteration of your loop, either -- only on the first. Or, better yet, you could change into the directory at the start. In any case, I advice against this approach since its an unnecessary overhead.
Instead, just prefix the filename with its parent directory -- i.e. folderName when you call shutil.copy().
For example:
shutil.copy(os.path.join(folderName, filename), '/Users/me/Documents/School/IT 145/Text Files')
Side note for readability:
Put '/Users/me/Documents' and '/Users/me/Documents/School/IT 145/Text Files' into named variables -- which makes it easier to read, change and/or reuse if needed.
Side note for portability:
Aim to use os.path.join() instead of using / for the directory separator.
I want to iterate through files in a directory and test each of them to see if they meet my selection criteria. Python is finding my file, and then immediately claiming to have not found my file. I am not sure what is going on.
import pandas as pd
for subdir, dirs, files in os.walk(path):
for file in files:
print('file %s located' %file)
in_file=pd.read_csv(file)
The error goes as follows:
runfile('F:/School/Research/WGM/NewProject/PythonScripts/EqiSiteSorter.py', wdir='F:/School/Research/WGM/NewProject/PythonScripts')
file FileName.csv located
Traceback (most recent call last):
FileNotFoundError: [Errno 2] File b'FileName.csv' does not exist: b'FileName.csv'
It successfully prints the message which states that the file was located, the file name is stored as a string in my variables, but apparently it suddenly got lost after that. Help please...
I thinks because of the directories. Inside the read calc try suing os.path.join(dir,filename)
This should work.
There is folder path:
P:\\2018\\Archive\\
There are many zipfiles I want to create programmatically, but am starting with test. I will name this test zip file "CO_007_II.zip" and will attempt to create in above location:
import zipfile as zp
with zp.ZipFile("P:\\2018\\Archive\\CO_007_II.zip",'w') as myzip:
myzip.write(r"P:\2018\CO_007_II")
But I get error!
...
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "C:\Python27\ArcGIS10.2\lib\zipfile.py", line 752, in __init__
self.fp = open(file, modeDict[mode])
IOError: [Errno 2] No such file or directory: 'P:\\2018\\Archive\\CO_007_II.zip'
Is this not method for creating new zipfile? I know file does not exist. Is why I am using 'w' mode, no?
This is documentation:
https://docs.python.org/3/library/zipfile.html
It says:
'w' to truncate and write a new file
Example on documentation page:
with ZipFile('spam.zip', 'w') as myzip:
myzip.write('eggs.txt')
code worked two days ago to create new zip file but did not add folder. Today nothing works! Why not? All paths valid. How do I create new zip file with python and add folders to it?
I also encountered a similar issue and came here looking for answers. Since this was the top hit, I'll add what I discovered.
The answer provided by #metatoaster didn't work for me, when stepping through the code I found that the path returned true to isdir.
In my case, the path length exceeded the Windows max path length (260 chars) which was causing it to fail despite the folder path being valid.
Hope that helps someone else down the line!
The only way this could be reproduced was to create a zipfile in a directory that does NOT exist yet. The only way to be sure (you cannot trust a file manager; only way to verify is to check from within the program itself) is to assign the desired path of the new zip file to a variable (e.g. path), and then call isdir(dirname(path)). For example:
from os.path import isdir
from os.path import dirname
target = "P:\\2018\\Archive\\CO_007_II.zip"
if not isdir(dirname(target)):
print('cannot create zipfile because target does not exists')
else:
# create the zipfile
I had the same issue. It was the long path. I solved by adding this //?/C at the beginning of the path
path = r"//?/C:\Users\Camilo\Proyectos"
I am trying to copy my Automater workflows, I have listed them in a config file, and I would like to loop through the config file and copy the directories. They have spaces in the names and I am having trouble.
It prints the filename correctly etc but the copy fails as there seems to be extra " " around the name with the copy
import os
import shutil
confdir=os.getenv("my_config")
dropbox=os.getenv("dropbox")
conffile = ('services.conf')
conffilename=os.path.join(confdir, conffile)
sourcedir= (r'~/Library/Services/')
destdir=os.path.join(dropbox, "My_backups")
for file_name in open(conffilename):
sourcefile=os.path.join(sourcedir, repr(file_name.strip()))
print sourcefile
destfile=os.path.join(destdir, file_name.strip())
shutil.copytree(sourcefile, destfile)
And the error is
~/Library/Services/'Add PDF Metadata.workflow'
Traceback (most recent call last):
File "Untitled 3.py", line 15, in <module>
shutil.copytree(sourcefile, destfile)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 168, in copytree
names = os.listdir(src)
OSError: [Errno 2] No such file or directory: "~/Library/Services/'Add PDF Metadata.workflow'"
Thanks in advance
I have tried the suggestions below but it still is not working
Why are you using repr() on file_name.strip()? That will surround your filename with single quotes - and those aren't present in the file path. Remove the repr(), and it should work.
shutil.copytree(src, dst) will recursively copy a directory tree (and all the files in it) located at src to a new directory tree at dst. It is not meant to be used with files.
Here, you want to copy single files around, not a complete directory tree, you should just use shutil.copy or shutil.copy2.
If the files may be located in a directory tree that you want to reproduce, then you could use os.makedirs for the path returned by os.path.dirname(destfile) before calling shutil.copy(sourcefile) to actually copy the file to destfile.
However, be aware that calling os.makedirs with a destination that already exists will raise an error so you probably want to try / except.
It didn't like the ~ I put the full path in. I have also just edited one more bit and used sourcedir=os.path.expanduser('~/Library/Services/') to expand the home directory