Python pathlib make directories if they don’t exist - python

If I wanted to specify a path to save files to and make directories that don’t exist in that path, is it possible to do this using the pathlib library in one line of code?

Yes, that is Path.mkdir:
pathlib.Path('/tmp/sub1/sub2').mkdir(parents=True, exist_ok=True)
From the docs:
If parents is true, any missing parents of this path are created as
needed; they are created with the default permissions without taking
mode into account (mimicking the POSIX mkdir -p command).
If parents is false (the default), a missing parent raises
FileNotFoundError.
If exist_ok is false (the default), FileExistsError is raised if the
target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same
behavior as the POSIX mkdir -p command), but only if the last path
component is not an existing non-directory file.

This gives additional control for the case that the path is already there:
path = Path.cwd() / 'new' / 'hi' / 'there'
try:
path.mkdir(parents=True, exist_ok=False)
except FileExistsError:
print("Folder is already there")
else:
print("Folder was created")

Adding to Wim's answer. If your path has a file on the end that you do not want made as a directory.
ie.
'/existing_dir/not_existing_dir/another_dir/a_file'
Then you use PurePath.parents. But the nice thing is that because Paths inherit the attributes of Pure Paths, then you can simply do
filepath = '/existing_dir/not_existing_dir/another_dir/a_file'
pathlib.Path(filepath).parents[0].mkdir(parents=True, exist_ok=True)

Related

check for a folder with python

if os.chdir(path+"micro")==True:
os.chdir(path+"micro")
else:
os.mkdir(path+"micro")
os.chdir(path+"micro")
when I use this method it gives me an error and says file exitsts why can't I access that file
If I understand your question correctly, this code is for you:
if os.path.exists(path+"micro"):
os.chdir(path+"micro")
else:
os.mkdir(path+"micro")
os.chdir(path+"micro")
Your code was incorrect because as you can see from the documentation os.chdir does not return any value, but is used to change the current working directory to the given path. Returns None in all the cases. For this reason your code gave an error.
Instead you need the os.path.exists method which return True if path refers to an existing path. Returns False for broken symbolic links. On some platforms, this function may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.
I think what you are trying to do is:
dir = os.path.join(path, "micro")
if not os.path.isdir(dir):
os.mkdir(dir)
os.chdir(dir)
os.chdir(path+"micro") This does not return false if the path doesn't exist. It rather gives an error. Also writing path + "mirco" is not a good practice either. The standard way is using os.path.join(path, "micro") because it handles choosing "/" or "\" according to the os it is running on.
So these are some alternatives that you might try.
Alternative 1:
dir = os.path.join(path, "micro")
if not os.path.exists(dir):
os.mkdir(dir)
os.chdir(dir)
Alternative 2:
dir = os.path.join(path, "micro")
try:
os.chdir(dir)
except:
os.mkdir(dir)
os.chdir(dir)

How to get the temporary path using pytest tmpdir.as_cwd

In a python test function
def test_something(tmpdir):
with tmpdir.as_cwd() as p:
print('here', p)
print(os.getcwd())
I was expecting p and the os.getcwd() would give the same result. But in reality, p points to the directory of the test file whereas os.getcwd() points to the expected temporary file.
Is this expected behavior?
Take a look at the docs of py.path.as_cwd:
return context manager which changes to current dir during the managed "with" context. On __enter__ it returns the old dir.
The behaviour you are observing is thus correct:
def test_something(tmpdir):
print('current directory where you are before changing it:', os.getcwd())
# the current directory will be changed now
with tmpdir.as_cwd() as old_dir:
print('old directory where you were before:', old_dir)
print('current directory where you are now:', os.getcwd())
print('you now returned to the old current dir', os.getcwd())
Just remember that p in your example is not the "new" current dir you are changing to, it's the "old" one you changed from.
From the documentation:
You can use the tmpdir fixture which will provide a temporary
directory unique to the test invocation, created in the base temporary
directory.
Whereas, getcwd stands for Get Current Working directory, and returns the directory from which your python process has been started.

How to create a directory using os.mkdir()

I'm trying to create a directory using os.mkdir() or os.makedirs() as follows:
if not os.path.exists(directory):
os.mkdir(directory)
This code runs fine, but I could see no directory created in the 'directory' path.
If I only write:
os.mkdir(directory)
it gives error message that directory already exists.
Try the following for a little more robust handling -- similar to how mkdir -p works on Linux:
def _mkdir(_dir):
if os.path.isdir(_dir): pass
elif os.path.isfile(_dir):
raise OSError("%s exists as a regular file." % _dir)
else:
parent, directory = os.path.split(_dir)
if parent and not os.path.isdir(parent): _mkdir(parent)
if directory: os.mkdir(_dir)
If you try making a dir over a file, complain, otherwise, just make sure the dir exists.

Python. Unchroot directory

I chrooted directory using following commands:
os.chroot("/mydir")
How to return to directory to previous - before chrooting?
Maybe it is possible to unchroot directory?
SOLUTION:
Thanks to Phihag. I found a solution. Simple example:
import os
os.mkdir('/tmp/new_dir')
dir1 = os.open('.', os.O_RDONLY)
dir2 = os.open('/tmp/new_dir', os.O_RDONLY)
os.getcwd() # we are in 'tmp'
os.chroot('/tmp/new_dir') # chrooting 'new_dir' directory
os.fchdir(dir2)
os.getcwd() # we are in chrooted directory, but path is '/'. It's OK.
os.fchdir(dir1)
os.getcwd() # we came back to not chrooted 'tmp' directory
os.close(dir1)
os.close(dir2)
More info
If you haven't changed your current working directory, you can simply call
os.chroot('../..') # Add '../' as needed
Of course, this requires the CAP_SYS_CHROOT capability (usually only given to root).
If you have changed your working directory, you can still escape, but it's harder:
os.mkdir('tmp')
os.chroot('tmp')
os.chdir('../../') # Add '../' as needed
os.chroot('.')
If chroot changes the current working directory, you can get around that by opening the directory, and using fchdir to go back.
Of course, if you intend to go out of a chroot in the course of a normal program (i.e. not a demonstration or security exploit), you should rethink your program. First of all, do you really need to escape the chroot? Why can't you just copy the required info into it beforehand?
Also, consider using a second process that stays outside of the chroot and answers to the requests of the chrooted one.

mkdir -p functionality in Python [duplicate]

This question already has answers here:
How can I safely create a directory (possibly including intermediate directories)?
(28 answers)
Closed 5 years ago.
Is there a way to get functionality similar to mkdir -p on the shell from within Python. I am looking for a solution other than a system call. I am sure the code is less than 20 lines, and I am wondering if someone has already written it?
For Python ≥ 3.5, use pathlib.Path.mkdir:
import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)
The exist_ok parameter was added in Python 3.5.
For Python ≥ 3.2, os.makedirs has an optional third argument exist_ok that, when True, enables the mkdir -p functionality—unless mode is provided and the existing directory has different permissions than the intended ones; in that case, OSError is raised as previously:
import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)
For even older versions of Python, you can use os.makedirs and ignore the error:
import errno
import os
def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise
In Python >=3.2, that's
os.makedirs(path, exist_ok=True)
In earlier versions, use #tzot's answer.
This is easier than trapping the exception:
import os
if not os.path.exists(...):
os.makedirs(...)
Disclaimer This approach requires two system calls which is more susceptible to race conditions under certain environments/conditions. If you're writing something more sophisticated than a simple throwaway script running in a controlled environment, you're better off going with the accepted answer that requires only one system call.
UPDATE 2012-07-27
I'm tempted to delete this answer, but I think there's value in the comment thread below. As such, I'm converting it to a wiki.
Recently, I found this distutils.dir_util.mkpath:
In [17]: from distutils.dir_util import mkpath
In [18]: mkpath('./foo/bar')
Out[18]: ['foo', 'foo/bar']
With Pathlib from python3 standard library:
Path(mypath).mkdir(parents=True, exist_ok=True)
If parents is true, any missing parents of this path are created as
needed; they are created with the default permissions without taking
mode into account (mimicking the POSIX mkdir -p command).
If exist_ok is false (the default), an FileExistsError is raised if
the target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same
behavior as the POSIX mkdir -p command), but only if the last path
component is not an existing non-directory file.
Changed in version 3.5: The exist_ok parameter was added.
mkdir -p gives you an error if the file already exists:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory `/tmp/foo': File exists
So a refinement to the previous suggestions would be to re-raise the exception if os.path.isdir returns False (when checking for errno.EEXIST).
(Update) See also this highly similar question; I agree with the accepted answer (and caveats) except I would recommend os.path.isdir instead of os.path.exists.
(Update) Per a suggestion in the comments, the full function would look like:
import os
def mkdirp(directory):
if not os.path.isdir(directory):
os.makedirs(directory)
As mentioned in the other solutions, we want to be able to hit the file system once while mimicking the behaviour of mkdir -p. I don't think that this is possible to do, but we should get as close as possible.
Code first, explanation later:
import os
import errno
def mkdir_p(path):
""" 'mkdir -p' in Python """
try:
os.makedirs(path)
except OSError as exc: # Python >2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
As the comments to #tzot's answer indicate there are problems with checking whether you can create a directory before you actually create it: you can't tell whether someone has changed the file system in the meantime. That also fits in with Python's style of asking for forgiveness, not permission.
So the first thing we should do is try to make the directory, then if it goes wrong, work out why.
As Jacob Gabrielson points out, one of the cases we must look for is the case where a file already exists where we are trying to put the directory.
With mkdir -p:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory '/tmp/foo': File exists
The analogous behaviour in Python would be to raise an exception.
So we have to work out if this was the case. Unfortunately, we can't. We get the same error message back from makedirs whether a directory exists (good) or a file exists preventing the creation of the directory (bad).
The only way to work out what happened is to inspect the file system again to see if there is a directory there. If there is, then return silently, otherwise raise the exception.
The only problem is that the file system may be in a different state now than when makedirs was called. eg: a file existed causing makedirs to fail, but now a directory is in its place. That doesn't really matter that much, because the the function will only exit silently without raising an exception when at the time of the last file system call the directory existed.
I think Asa's answer is essentially correct, but you could extend it a little to act more like mkdir -p, either:
import os
def mkdir_path(path):
if not os.access(path, os.F_OK):
os.mkdirs(path)
or
import os
import errno
def mkdir_path(path):
try:
os.mkdirs(path)
except os.error, e:
if e.errno != errno.EEXIST:
raise
These both handle the case where the path already exists silently but let other errors bubble up.
Function declaration;
import os
def mkdir_p(filename):
try:
folder=os.path.dirname(filename)
if not os.path.exists(folder):
os.makedirs(folder)
return True
except:
return False
usage :
filename = "./download/80c16ee665c8/upload/backup/mysql/2014-12-22/adclient_sql_2014-12-22-13-38.sql.gz"
if (mkdir_p(filename):
print "Created dir :%s" % (os.path.dirname(filename))
import os
import tempfile
path = tempfile.mktemp(dir=path)
os.makedirs(path)
os.rmdir(path)
I've had success with the following personally, but my function should probably be called something like 'ensure this directory exists':
def mkdirRecursive(dirpath):
import os
if os.path.isdir(dirpath): return
h,t = os.path.split(dirpath) # head/tail
if not os.path.isdir(h):
mkdirRecursive(h)
os.mkdir(join(h,t))
# end mkdirRecursive
import os
from os.path import join as join_paths
def mk_dir_recursive(dir_path):
if os.path.isdir(dir_path):
return
h, t = os.path.split(dir_path) # head/tail
if not os.path.isdir(h):
mk_dir_recursive(h)
new_path = join_paths(h, t)
if not os.path.isdir(new_path):
os.mkdir(new_path)
based on #Dave C's answer but with a bug fixed where part of the tree already exists

Categories

Resources