I get the following error:
[Errno 20] Not a directory:
What I have tried so far:
for i in range(3):
numOfData = len(dir_2[i])
try:
for j in range(numOfData):
print os.listdir(os.path.join(dir_1[i], dir_2[i][j]))
except OSError:
print "Hello"
Is there a better way to handle this?
You could of course check if the directory exists before running os.listdir() using os.path.isdir().
But that would not protect you against directory permissions, for instance (if it can be a problem). In that case, it would be tempting to use os.access() to see if you can read the directory except that it isn't that reliable
So trying to read and catching the specific OSError exception seems correct (better ask for forgiveness than permission), maybe with a os.path.isdir() first to print a nicer message for the user if the directory doesn't exist at all.
And seeing your comment, I realize that you should put the try/except block within the loop so if there's an error, you can still process the next directory.
I am attempting to convert the following pseudo-code to Python:
If <directory> does not exist:
Create all subdirectories for <directory>
Create a file in <directory>
This sounds simple enough to accomplish with os.makedirs and os.path.isdir:
if not os.path.isdir('/some/path'):
os.makedirs('/some/path')
open('/some/path/test.txt', 'w')
However, upon further inspection there is clearly a race condition present. Consider the following timeline:
the specified directory (/some/path) does not exist
the Python interpreter executes the first line, which evaluates to True
another process creates the directory (/some/path)
makedirs raises an OSError exception since the directory already exists
There are also problems if the directory does initially exist but is removed by another process before the final line is executed.
When it comes to Python, "it's easier to ask for forgiveness than permission." With that in mind, the fragment above could be better written:
try:
os.makedirs('/some/path')
except OSError:
pass
open('/some/path/test.txt', 'w')
This solves the two problems described above but creates a third: os.makedirs raises an OSError exception when one of the following conditions occurs:
the directory already exists
the directory could not be created
This means that there is no way to determine which of the two conditions caused the exception to be raised. In other words, actual failures will be silently ignored, which is not what I want.
How can I work around this problem?
I'll note that all of this is quite a bit more sane in python 3; FileExistsError and PermissionError are separate (subclass of OSError) exceptions that you can catch, and os.makedirs even has a exist_ok kwarg to suppress the former when you're ok with the directory already existing.
If you want to inspect the reason for the OSError, that info is in a tuple in e.args (or optionally e.errno if you just want to look at the error code):
try:
os.makedirs('/etc/python')
except OSError as e:
print e.args
(17, 'File exists')
try:
os.makedirs('/etc/stuff')
except OSError as e:
print e.args
(13, 'Permission denied')
try:
os.makedirs('/etc/stuff')
except OSError as e:
print e.errno
13
So you'll have to do a bit of introspection and handle the two error codes differently in your except block.
I have a script which create a temporary text file and delete after the user close the window.
The problem is that, the temporary text file may or may not be created depending on what the user does.Or sometimes the temporary text file may be deleted before the user exit. There are three possible scenario.
The temporary text file is created with the name of 'tempfilename'.
The temporary text file is created with the name of 'tempfilename' but deleted before the user exit.So, when trying to remove the file it raise OSError
The temporary text file is not created and no variable called 'tempfilename' is created, so it raise NameError
I have tried using this code:
try:
os.remove(str(tempfilename))
except OSError or NameError:
pass
But it seems that it only catch the OSError only. Did i do something wrong?
try:
os.remove(str(tempfilename))
except (OSError, NameError):
pass
tempfilename = None
# ...
if tempfilename is not None and os.path.exists(tempfilename):
os.remove(tempfilename)
It's not good to catch NameError since it will hide other typos in your code (e.g., os.remov(…)).
Also, OSError does not always means that the file did not exist. On Windows, if the file was in use, an exception would be raised (http://docs.python.org/2/library/os.html#os.remove). In that case, you would want to see the exception so you could be aware of the issue and/or handle it another way.
Exception handlers should be kep as narrow as possible to avoid hiding unrelated errors or bugs
if we take a look at a file copy function, we can see there are several exceptions to handle. A good example is here: http://msdn.microsoft.com/en-us/library/9706cfs5.aspx
my question is if i use python shutil copy2, what should I pay attention to cope with various exceptions (source file not found, access not authorized, etc.)?
e.g.
def copy_file (self):
if not os.path.isdir(dest_path):
os.makedirs(dest_path)
shutil.copy2(src_path, dest_path)
what should i do to the above function?
You may just need handle the IOError exception that may be caused due to any permissions or Invalid destination name issue.
try:
shutil.copy(src,dst)
except IOError as e:
print e
The other exceptions mentioned in the MSDN article seems to fall under the same IOError in python. The FileNotFound and DirectoryNotFound are not really applicable as shutil.copy will create the destination if it not already exists. Also, I find that happening of OSError are also remote this in case.
This question already has answers here:
How can I safely create a directory (possibly including intermediate directories)?
(28 answers)
Closed 5 years ago.
Is there a way to get functionality similar to mkdir -p on the shell from within Python. I am looking for a solution other than a system call. I am sure the code is less than 20 lines, and I am wondering if someone has already written it?
For Python ≥ 3.5, use pathlib.Path.mkdir:
import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)
The exist_ok parameter was added in Python 3.5.
For Python ≥ 3.2, os.makedirs has an optional third argument exist_ok that, when True, enables the mkdir -p functionality—unless mode is provided and the existing directory has different permissions than the intended ones; in that case, OSError is raised as previously:
import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)
For even older versions of Python, you can use os.makedirs and ignore the error:
import errno
import os
def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise
In Python >=3.2, that's
os.makedirs(path, exist_ok=True)
In earlier versions, use #tzot's answer.
This is easier than trapping the exception:
import os
if not os.path.exists(...):
os.makedirs(...)
Disclaimer This approach requires two system calls which is more susceptible to race conditions under certain environments/conditions. If you're writing something more sophisticated than a simple throwaway script running in a controlled environment, you're better off going with the accepted answer that requires only one system call.
UPDATE 2012-07-27
I'm tempted to delete this answer, but I think there's value in the comment thread below. As such, I'm converting it to a wiki.
Recently, I found this distutils.dir_util.mkpath:
In [17]: from distutils.dir_util import mkpath
In [18]: mkpath('./foo/bar')
Out[18]: ['foo', 'foo/bar']
With Pathlib from python3 standard library:
Path(mypath).mkdir(parents=True, exist_ok=True)
If parents is true, any missing parents of this path are created as
needed; they are created with the default permissions without taking
mode into account (mimicking the POSIX mkdir -p command).
If exist_ok is false (the default), an FileExistsError is raised if
the target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same
behavior as the POSIX mkdir -p command), but only if the last path
component is not an existing non-directory file.
Changed in version 3.5: The exist_ok parameter was added.
mkdir -p gives you an error if the file already exists:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory `/tmp/foo': File exists
So a refinement to the previous suggestions would be to re-raise the exception if os.path.isdir returns False (when checking for errno.EEXIST).
(Update) See also this highly similar question; I agree with the accepted answer (and caveats) except I would recommend os.path.isdir instead of os.path.exists.
(Update) Per a suggestion in the comments, the full function would look like:
import os
def mkdirp(directory):
if not os.path.isdir(directory):
os.makedirs(directory)
As mentioned in the other solutions, we want to be able to hit the file system once while mimicking the behaviour of mkdir -p. I don't think that this is possible to do, but we should get as close as possible.
Code first, explanation later:
import os
import errno
def mkdir_p(path):
""" 'mkdir -p' in Python """
try:
os.makedirs(path)
except OSError as exc: # Python >2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
As the comments to #tzot's answer indicate there are problems with checking whether you can create a directory before you actually create it: you can't tell whether someone has changed the file system in the meantime. That also fits in with Python's style of asking for forgiveness, not permission.
So the first thing we should do is try to make the directory, then if it goes wrong, work out why.
As Jacob Gabrielson points out, one of the cases we must look for is the case where a file already exists where we are trying to put the directory.
With mkdir -p:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory '/tmp/foo': File exists
The analogous behaviour in Python would be to raise an exception.
So we have to work out if this was the case. Unfortunately, we can't. We get the same error message back from makedirs whether a directory exists (good) or a file exists preventing the creation of the directory (bad).
The only way to work out what happened is to inspect the file system again to see if there is a directory there. If there is, then return silently, otherwise raise the exception.
The only problem is that the file system may be in a different state now than when makedirs was called. eg: a file existed causing makedirs to fail, but now a directory is in its place. That doesn't really matter that much, because the the function will only exit silently without raising an exception when at the time of the last file system call the directory existed.
I think Asa's answer is essentially correct, but you could extend it a little to act more like mkdir -p, either:
import os
def mkdir_path(path):
if not os.access(path, os.F_OK):
os.mkdirs(path)
or
import os
import errno
def mkdir_path(path):
try:
os.mkdirs(path)
except os.error, e:
if e.errno != errno.EEXIST:
raise
These both handle the case where the path already exists silently but let other errors bubble up.
Function declaration;
import os
def mkdir_p(filename):
try:
folder=os.path.dirname(filename)
if not os.path.exists(folder):
os.makedirs(folder)
return True
except:
return False
usage :
filename = "./download/80c16ee665c8/upload/backup/mysql/2014-12-22/adclient_sql_2014-12-22-13-38.sql.gz"
if (mkdir_p(filename):
print "Created dir :%s" % (os.path.dirname(filename))
import os
import tempfile
path = tempfile.mktemp(dir=path)
os.makedirs(path)
os.rmdir(path)
I've had success with the following personally, but my function should probably be called something like 'ensure this directory exists':
def mkdirRecursive(dirpath):
import os
if os.path.isdir(dirpath): return
h,t = os.path.split(dirpath) # head/tail
if not os.path.isdir(h):
mkdirRecursive(h)
os.mkdir(join(h,t))
# end mkdirRecursive
import os
from os.path import join as join_paths
def mk_dir_recursive(dir_path):
if os.path.isdir(dir_path):
return
h, t = os.path.split(dir_path) # head/tail
if not os.path.isdir(h):
mk_dir_recursive(h)
new_path = join_paths(h, t)
if not os.path.isdir(new_path):
os.mkdir(new_path)
based on #Dave C's answer but with a bug fixed where part of the tree already exists