In my code, I create a directory like so:
try:
os.makedirs(playlist_name)
except OSError as e:
if e.errno != errno.EEXIST:
raise
Which creates a directory in the place where I run my python script.
Then I wish to copy three files from the original directory where the folder is located into the newly created directory, like so
# Copy FFMPEG files into that folder so youtube dl can download the videos as audio tracks
# Tried using os.getcwd() to get full path, same error
shutil.copyfile(os.getcwd() + '\\ffmpeg.exe', os.getcwd() + "\\" + playlist_name)
shutil.copyfile('ffplay.exe', "/" + playlist_name + "/")
shutil.copyfile('ffprobe.exe', "/" + playlist_name + "/")
However, trying to copy those files throws this error:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\ME\\Documents\\python\\DIRECTORY\\PLAYLIST_NAME_HERE'
I have tried various shutil copy methods with the same error.
EDIT: This is running on windows
Per the copyfile docs:
dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path.
You can't use it to do what you do in the shell, naming a source file and a target directory and having it deduce the file should be put in the directory with the file's original name. You have to explicitly name the target file, or it thinks you're trying to copy to the same name as the directory, and unlike replacing a file, you can't replace a directory with a file without explicitly renaming the directory or deleting the whole directory tree first. To fix, just make sure to repeat the file name in both source and destination:
for filename in ('ffmpeg.exe', 'ffplay.exe', 'ffprobe.exe'):
shutil.copyfile(filename, os.path.join(playlist_name, filename))
The problem would be more obvious on a UNIX-like system, because those systems would reject the action with EISDIR, causing a Python IsADirectoryError to be raised, but Windows for some reason chose to use more general error codes associated with permission/access problems (EACCES and related Windows specific error codes) that Python translates to PermissionError (because Windows just isn't telling it the real problem, and it would introduce all sorts of race conditions if Python tried to check if the real problem was trying to use a directory as a file to fix the exception type).
Related
While preparing my dataset for a machine learning model I stumbled upon a bug (at least from my perspective). I was trying to move files around in Windows 10 but my system always told me that my file does not exist (FileNotFoundError: [Errno 2] No such file or directory: '../path/to/file'), but I specifically probed for the file before which returned True.
What I was trying in my IPython console was the following (which can reproduce the bug):
import os
import shutil
path = '../path/to/file' # You can see I use relative paths
out_path = '../path/to/out/'
os.makedirs(out_path) # Make sure paths do exist
_, name = os.path.split(path)
out_path += name
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)
You obviously need a file to move at '../path/to/file'. I tested whether I use the correct path with os.path.exist(path) which returned True.
Do I simply misread the documentation and the usage is different to what I am thinking? I understood it like the first argument is the path to the file to move and the second one is the path for the file after moving it.
Or is there really a bug in my Python environment?
I also tried moving the file with absolute paths returned by os.path.abspath(), but this did not work either.
FYI: I am using Windows 10 and Python 3.8.12.
Edit: I found the mistake I did... After 2.5 days of labeling image data it appears like I had an underscore mistaken for a hyphen which is why my output directory seems to be malformed... :/
You are not doing a relative path to the current directory, but to the parent directory of the current directory with a double dot '../path' and not single dot './path'. So if your files aren't in the parent directory of the current directory, they won't be found.
When you try to append the file name to the directory path "out_path" you do a string append. which just makes a new file with the combined names rather than move the file into the directory. Use the os.path.join() function for this.
Here is code I made that corrects these two points and works on my end.
import os
import shutil
path = './somefile.txt' # CORRECTED: One dot for current directory
out_path = './somedir'
try:
os.makedirs(out_path) # Make sure paths do exist
except Exception as e:
print(e)
_, name = os.path.split(path)
out_path = os.path.join(out_path, name) # CORRECTED: Use appropriate function
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)
I have a strange error:
in the code:
if os.path.isfile(df_file):
os.rename(df_file, df_file + '_backup.parquet')
I am getting an error:
os.rename(df_file, df_file + '_backup.parquet')
FileNotFoundError: [Errno 2] No such file or directory: '/data/5faf97ca101ae0413c383678186bf601.parquet' ->
'/data/5faf97ca101ae0413c383678186bf601.parquet_backup.parquet'
Even though I checked and the file exist!
what could be the problem ?
The source file and directory are obviously exist
The code is called within a flask thread.
'/data/5faf97ca101ae0413c383678186bf601.parquet_backup.parquet' This is an absolute path, which points to the data directory under the root path. I guess you want to use a relative path,'./data/5faf97ca101ae0413c383678186bf601.parquet_backup.parquet'
How do I copy a file into a directory which already contains other files (without replacing them)? I basically need to collect files from different sources and directories (all with heterogenous names and directory structures) and put them all into one folder. I've been able to make the destination directory and copy the initial files so far, but once the directory already exists, it will not copy subsequent files...
'''COPY FILES AND DIRECTORIES FROM src (SOURCE) TO dest (DESTINATION)'''
import errno
import shutil
def copy(src, dest):
try:
shutil.copytree(src, dest)
except OSError as e:
# If the error was caused because the source wasn't a directory
if e.errno == errno.ENOTDIR:
shutil.copy2(src, dest)
else:
print('Directory not copied. Error: %s' % e)
copy("source.directory", "Destination/Directory")
This returns the following error after the destination directory is created and the first file has been copied:
Directory not copied. Error: [Errno 17] File exists: 'Destination/Directory'
You can't use copytree this way. Once it gets an error along some subtree, it bails out of that subtree. After it returns you the error, there's no way to resume where it left off
But notice that the shutil docs have a link to the source code. Much of the standard library is deliberately written to be useful not just as a library, but also as sample code. It's not that complicated—in fact, the only really tricky part is the way it collects up errors so it can fail the way it does, which is the one part you don't want.
So, fork that function into your own code and modify it to handle errors the way you want.
I'm doing a Python course on Udacity. And there is a class called Rename Troubles, which presents the following incomplete code:
import os
def rename_files():
file_list = os.listdir("C:\Users\Nick\Desktop\temp")
print (file_list)
for file_name in file_list:
os.rename(file_name, file_name.translate(None, "0123456789"))
rename_files()
As the instructor explains it, this will return an error because Python is not attempting to rename files in the right folder. He then proceeds to check the "current working directory", and then goes on to specify to Python which directory to rename files in.
This makes no sense to me. We are using the for loop to specifically tell Python that we want to rename the contents of file_list, which we have just pointed to the directory we need, in rename_files(). So why does it not attempt to rename in that folder? Why do we still need to figure out cwd and then change it?? The code looks entirely logical without any of that.
Look closely at what os.listdir() gives you. It returns only a list of names, not full paths.
You'll then proceed to os.rename one of those names, which will be interpreted as a relative path, relative to whatever your current working directory is.
Instead of messing with the current working directory, you can os.path.join() the path that you're searching to the front of both arguments to os.rename().
I think your code needs some formatting help.
The basic issue is that os.listdir() returns names relative to the directory specified (in this case an absolute path). But your script can be running from any directory. Manipulate the file names passed to os.rename() to account for this.
Look into relative and absolute paths, listdir returns names relative to the path (in this case absolute path) provided to listdir. os.rename is then given this relative name and unless the app's current working directory (usually the directory you launched the app from) is the same as provided to listdir this will fail.
There are a couple of alternative ways of handling this, changing the current working directory:
os.chdir("C:\Users\Nick\Desktop\temp")
for file_name in os.listdir(os.getcwd()):
os.rename(file_name, file_name.translate(None, "0123456789"))
Or use absolute paths:
directory = "C:\Users\Nick\Desktop\temp"
for file_name in os.listdir(directory):
old_file_path = os.path.join(directory, file_name)
new_file_path = os.path.join(directory, file_name.translate(None, "0123456789"))
os.rename(old_file_path, new_file_path)
You can get a file list from ANY existing directory - i.e.
os.listdir("C:\Users\Nick\Desktop\temp")
or
os.listdir("C:\Users\Nick\Desktop")
or
os.listdir("C:\Users\Nick")
etc.
The instance of the Python interpreter that you're using to run your code is being executed in a directory that is independent of any directory for which you're trying to get information. So, in order to rename the correct file, you need to specify the full path to that file (or the relative path from wherever you're running your Python interpreter).
I am trying to write code that uses a recursive function to search for files in a directory, and returns the path to the file that matches the search term. However, I keep getting this error when I use "../.." as the path name "PermissionError: [WinError 5] Access is denied: '../..\AppData\Local\Application Data'".
import os
def main():
pathname=input('Please enter path name: ')
filenameinput=input('Please enter file name: ')
def disk_usage(path):
if os.path.isdir(path):
for filename in os.listdir(path):
childpath = os.path.join(path, filename)
if os.path.isdir(childpath):
disk_usage(childpath)
else:
if childpath.endswith(filenameinput):
print(childpath)
#return
disk_usage(pathname)
main()
I should not need to use os.walk() for this. I have it working but it returns several paths ending in the filename I specified and then returns the WinError 5 thing.
You're getting a permission error because Application Data is not a real folder in Windows 7+, it's a "junction" (symlink in Unix-speak) pointing to C:\Program Files. It only exists for backwards compatibility.
You have two options:
You can read the junction with some Windows-specific native code, through win32file. See this SO answer.
You can catch the permission error, and ignore it (maybe print a warning message). This is probably the better option unless you really need to read this folder.