While preparing my dataset for a machine learning model I stumbled upon a bug (at least from my perspective). I was trying to move files around in Windows 10 but my system always told me that my file does not exist (FileNotFoundError: [Errno 2] No such file or directory: '../path/to/file'), but I specifically probed for the file before which returned True.
What I was trying in my IPython console was the following (which can reproduce the bug):
import os
import shutil
path = '../path/to/file' # You can see I use relative paths
out_path = '../path/to/out/'
os.makedirs(out_path) # Make sure paths do exist
_, name = os.path.split(path)
out_path += name
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)
You obviously need a file to move at '../path/to/file'. I tested whether I use the correct path with os.path.exist(path) which returned True.
Do I simply misread the documentation and the usage is different to what I am thinking? I understood it like the first argument is the path to the file to move and the second one is the path for the file after moving it.
Or is there really a bug in my Python environment?
I also tried moving the file with absolute paths returned by os.path.abspath(), but this did not work either.
FYI: I am using Windows 10 and Python 3.8.12.
Edit: I found the mistake I did... After 2.5 days of labeling image data it appears like I had an underscore mistaken for a hyphen which is why my output directory seems to be malformed... :/
You are not doing a relative path to the current directory, but to the parent directory of the current directory with a double dot '../path' and not single dot './path'. So if your files aren't in the parent directory of the current directory, they won't be found.
When you try to append the file name to the directory path "out_path" you do a string append. which just makes a new file with the combined names rather than move the file into the directory. Use the os.path.join() function for this.
Here is code I made that corrects these two points and works on my end.
import os
import shutil
path = './somefile.txt' # CORRECTED: One dot for current directory
out_path = './somedir'
try:
os.makedirs(out_path) # Make sure paths do exist
except Exception as e:
print(e)
_, name = os.path.split(path)
out_path = os.path.join(out_path, name) # CORRECTED: Use appropriate function
# Produces almost the same error FileNotFoundError: [WinError 3] The system cannot find the given directory: path
# os.rename(path, out_path)
shutil.move(path, out_path)
Related
My current working directory is
C:\Users\18327\Desktop
I have the following code:
#Load os module
import os
#this is the abs path of my Temp folder
my_path = "C:/Users/18327/Desktop/New/Temp/"
#my parent folders
start = "C:/Users/18327/Desktop"
relative_path = os.path.relpath(my_path, start)
#relative path is New\Temp
print(relative_path)
#get and print current path of the current machine
current_path = os.getcwd()
print('current path is {}'.format(current_path))
#combine them
temp_path = os.path.join(current_path,
relative_path)
print(temp_path)
#set the path to your current medline folder
os.chdir(temp_path)
The first time I ran it I got the desired path I wanted, which is
New\Temp
current path is C:\Users\18327\Desktop
C:\Users\18327\Desktop\New\Temp
However, when I ran the code again, I got
New\Temp
current path is C:\Users\18327\Desktop\New\Temp
C:\Users\18327\Desktop\New\Temp\New\Temp
*** FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\18327\\Desktop\\New\\Temp\\New\\Temp'
How do I modify the code so that each time I run it I get the result
C:\Users\18327\Desktop\New\Temp
This happens since you're changing the working dir with os.chdir and not changing it back to where it was at the end of your run, while also using relative paths instead of absolute paths.
You could add old_working_dir = os.os.getcwd() at the start of your file and os.chdir(old_working_dir) at the end to return to where you were and have it run again as intended, but that's a bit hacky for my taste.
A much better solution would be to not use chdir at all, and make your script work with absolute paths instead of relative ones if possible.
Also, check out pathlib for many easier ways to deal with file paths.
In my code, I create a directory like so:
try:
os.makedirs(playlist_name)
except OSError as e:
if e.errno != errno.EEXIST:
raise
Which creates a directory in the place where I run my python script.
Then I wish to copy three files from the original directory where the folder is located into the newly created directory, like so
# Copy FFMPEG files into that folder so youtube dl can download the videos as audio tracks
# Tried using os.getcwd() to get full path, same error
shutil.copyfile(os.getcwd() + '\\ffmpeg.exe', os.getcwd() + "\\" + playlist_name)
shutil.copyfile('ffplay.exe', "/" + playlist_name + "/")
shutil.copyfile('ffprobe.exe', "/" + playlist_name + "/")
However, trying to copy those files throws this error:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\ME\\Documents\\python\\DIRECTORY\\PLAYLIST_NAME_HERE'
I have tried various shutil copy methods with the same error.
EDIT: This is running on windows
Per the copyfile docs:
dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path.
You can't use it to do what you do in the shell, naming a source file and a target directory and having it deduce the file should be put in the directory with the file's original name. You have to explicitly name the target file, or it thinks you're trying to copy to the same name as the directory, and unlike replacing a file, you can't replace a directory with a file without explicitly renaming the directory or deleting the whole directory tree first. To fix, just make sure to repeat the file name in both source and destination:
for filename in ('ffmpeg.exe', 'ffplay.exe', 'ffprobe.exe'):
shutil.copyfile(filename, os.path.join(playlist_name, filename))
The problem would be more obvious on a UNIX-like system, because those systems would reject the action with EISDIR, causing a Python IsADirectoryError to be raised, but Windows for some reason chose to use more general error codes associated with permission/access problems (EACCES and related Windows specific error codes) that Python translates to PermissionError (because Windows just isn't telling it the real problem, and it would introduce all sorts of race conditions if Python tried to check if the real problem was trying to use a directory as a file to fix the exception type).
I'm doing a Python course on Udacity. And there is a class called Rename Troubles, which presents the following incomplete code:
import os
def rename_files():
file_list = os.listdir("C:\Users\Nick\Desktop\temp")
print (file_list)
for file_name in file_list:
os.rename(file_name, file_name.translate(None, "0123456789"))
rename_files()
As the instructor explains it, this will return an error because Python is not attempting to rename files in the right folder. He then proceeds to check the "current working directory", and then goes on to specify to Python which directory to rename files in.
This makes no sense to me. We are using the for loop to specifically tell Python that we want to rename the contents of file_list, which we have just pointed to the directory we need, in rename_files(). So why does it not attempt to rename in that folder? Why do we still need to figure out cwd and then change it?? The code looks entirely logical without any of that.
Look closely at what os.listdir() gives you. It returns only a list of names, not full paths.
You'll then proceed to os.rename one of those names, which will be interpreted as a relative path, relative to whatever your current working directory is.
Instead of messing with the current working directory, you can os.path.join() the path that you're searching to the front of both arguments to os.rename().
I think your code needs some formatting help.
The basic issue is that os.listdir() returns names relative to the directory specified (in this case an absolute path). But your script can be running from any directory. Manipulate the file names passed to os.rename() to account for this.
Look into relative and absolute paths, listdir returns names relative to the path (in this case absolute path) provided to listdir. os.rename is then given this relative name and unless the app's current working directory (usually the directory you launched the app from) is the same as provided to listdir this will fail.
There are a couple of alternative ways of handling this, changing the current working directory:
os.chdir("C:\Users\Nick\Desktop\temp")
for file_name in os.listdir(os.getcwd()):
os.rename(file_name, file_name.translate(None, "0123456789"))
Or use absolute paths:
directory = "C:\Users\Nick\Desktop\temp"
for file_name in os.listdir(directory):
old_file_path = os.path.join(directory, file_name)
new_file_path = os.path.join(directory, file_name.translate(None, "0123456789"))
os.rename(old_file_path, new_file_path)
You can get a file list from ANY existing directory - i.e.
os.listdir("C:\Users\Nick\Desktop\temp")
or
os.listdir("C:\Users\Nick\Desktop")
or
os.listdir("C:\Users\Nick")
etc.
The instance of the Python interpreter that you're using to run your code is being executed in a directory that is independent of any directory for which you're trying to get information. So, in order to rename the correct file, you need to specify the full path to that file (or the relative path from wherever you're running your Python interpreter).
I am trying to write code that uses a recursive function to search for files in a directory, and returns the path to the file that matches the search term. However, I keep getting this error when I use "../.." as the path name "PermissionError: [WinError 5] Access is denied: '../..\AppData\Local\Application Data'".
import os
def main():
pathname=input('Please enter path name: ')
filenameinput=input('Please enter file name: ')
def disk_usage(path):
if os.path.isdir(path):
for filename in os.listdir(path):
childpath = os.path.join(path, filename)
if os.path.isdir(childpath):
disk_usage(childpath)
else:
if childpath.endswith(filenameinput):
print(childpath)
#return
disk_usage(pathname)
main()
I should not need to use os.walk() for this. I have it working but it returns several paths ending in the filename I specified and then returns the WinError 5 thing.
You're getting a permission error because Application Data is not a real folder in Windows 7+, it's a "junction" (symlink in Unix-speak) pointing to C:\Program Files. It only exists for backwards compatibility.
You have two options:
You can read the junction with some Windows-specific native code, through win32file. See this SO answer.
You can catch the permission error, and ignore it (maybe print a warning message). This is probably the better option unless you really need to read this folder.
I'm trying to get the list of files in a particular directory and count the number of files in the directory. I always get the following error:
WindowsError: [Error 3] The system cannot find the path specified: '/client_side/*.*'
My code is:
print len([name for name in os.listdir('/client_side/') if os.path.isfile(name)])
I followed the code example given here.
I am running the Python script on Pyscripter and the directory /client_side/ do exists. My python code is in the root folder and has a sub-folder called "client_side". Can someone help me out on this?
This error occurs when you use os.listdir on a path which does not refer to an existing path.
For example:
>>> os.listdir('Some directory does not exist')
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
WindowsError: [Error 3] : 'Some directory does not exist/*.*'
If you want to use os.listdir, you need to either guarantee the existence of the path that you would use, or use os.path.exists to check the existence first.
if os.path.exists('/client_side/'):
do something
else:
do something
Suppose your current working directory is c:\foobar, os.listdir('/client_side/') is equivalent to os.listdir('c:/client_side'), while os.listdir('client_side/') is equivalent to os.listdir('c:/foobar/client_side'). If your client_side directory is not in the root, such error will occur when using os.listdir.
For your 0 ouput problem, let us recall os.listdir(path)
Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory.
and os.path.isfile(path).
Return True if path is an existing regular file. This follows symbolic links, so both islink() and isfile() can be true for the same path.
listdir returns neither the absolute paths nor relative paths, but a list of the name of your files, while isfile requires path. Therefore, all of those names would yield False.
To obtain the path, we can either use os.path.join , concat two strings directly.
print ([name for name in os.listdir(path)
if os.path.isfile(os.path.join(path, name))])
Or
print ([name for name in os.listdir('client_side/')
if os.path.isfile('client_side/' + name)])
I decided to change the code into:
def numOfFiles(path):
return len(next(os.walk(path))[2])
and use the following the call the code:
print numOfFiles("client_side")
Many thanks to everyone who told me how to pass the windows directory correctly in Python and to nrao91 in here for providing the function code.
EDIT: Thank you eryksun for correcting my code!
Two things:
os.listdir() does not do a glob pattern matching, use the glob module for that
probably you do not have a directory called '/client_side/*.*', but maybe one
without the . in the name
The syntax you used works fine, if the directory you look for exists, but there is no directory called '/client_side/.'.
In addition, be careful if using Python 2.x and os.listdir, as the results on windows are different when you use u'/client_side/' and just '/client_side'.
You can do just
os.listdir('client_side')
without slashes.
As I can see a WindowsError, Just wondering if this has something to do with the '/' in windows ! Ideally, on windows, you should have something like os.path.join('C:','client_side')
You want:
print len([name for name in os.listdir('./client_side/') if os.path.isfile(name)])
with a "." before "/client_side/".
The dot means the current path where you are working (i.e. from where you are calling your code), so "./client_side/" represents the path you want, which is specified relatively to your current directory.
If you write only "/client_side/", in unix, the program would look for a folder in the root of the system, instead of the folder that you want.
If you just want to see all the files in the directory where your script is located, you can use os.path.dirname(sys.argv[0]). This will give the path of the directory where your script is.
Then, with fnmatch function you can obtain the list of files in that directory with a name and/or extension specified in the filenamevariable.
import os,sys
from fnmatch import fnmatch
directory = os.path.dirname(sys.argv[0]) #this determines the directory
file_name= "*" #if you want the python files only, change "*" to "*.py"
for path, subdirs, files in os.walk(directory):
for name in files:
if fnmatch(name, file_name):
print (os.path.join(path, name))
I hope this helps.
Checking for existence is subject to a race. Better to handle the error (beg forgiveness instead of ask permission). Plus, in Python 3 you can suppress errors. Use suppress from contextlib:
with suppress(FileNotFoundError):
for name in os.listdir('foo'):
print(name)