Related
How do I check whether a file exists or not, without using the try statement?
If the reason you're checking is so you can do something like if file_exists: open_it(), it's safer to use a try around the attempt to open it. Checking and then opening risks the file being deleted or moved or something between when you check and when you try to open it.
If you're not planning to open the file immediately, you can use os.path.isfile
Return True if path is an existing regular file. This follows symbolic links, so both islink() and isfile() can be true for the same path.
import os.path
os.path.isfile(fname)
if you need to be sure it's a file.
Starting with Python 3.4, the pathlib module offers an object-oriented approach (backported to pathlib2 in Python 2.7):
from pathlib import Path
my_file = Path("/path/to/file")
if my_file.is_file():
# file exists
To check a directory, do:
if my_file.is_dir():
# directory exists
To check whether a Path object exists independently of whether is it a file or directory, use exists():
if my_file.exists():
# path exists
You can also use resolve(strict=True) in a try block:
try:
my_abs_path = my_file.resolve(strict=True)
except FileNotFoundError:
# doesn't exist
else:
# exists
Use os.path.exists to check both files and directories:
import os.path
os.path.exists(file_path)
Use os.path.isfile to check only files (note: follows symbolic links):
os.path.isfile(file_path)
Unlike isfile(), exists() will return True for directories. So depending on if you want only plain files or also directories, you'll use isfile() or exists(). Here is some simple REPL output:
>>> os.path.isfile("/etc/password.txt")
True
>>> os.path.isfile("/etc")
False
>>> os.path.isfile("/does/not/exist")
False
>>> os.path.exists("/etc/password.txt")
True
>>> os.path.exists("/etc")
True
>>> os.path.exists("/does/not/exist")
False
import os
if os.path.isfile(filepath):
print("File exists")
Use os.path.isfile() with os.access():
import os
PATH = './file.txt'
if os.path.isfile(PATH) and os.access(PATH, os.R_OK):
print("File exists and is readable")
else:
print("Either the file is missing or not readable")
import os
os.path.exists(path) # Returns whether the path (directory or file) exists or not
os.path.isfile(path) # Returns whether the file exists or not
Although almost every possible way has been listed in (at least one of) the existing answers (e.g. Python 3.4 specific stuff was added), I'll try to group everything together.
Note: every piece of Python standard library code that I'm going to post, belongs to version 3.5.3.
Problem statement:
Check file (arguable: also folder ("special" file) ?) existence
Don't use try / except / else / finally blocks
Possible solutions:
1. [Python.Docs]: os.path.exists(path)
Also check other function family members like os.path.isfile, os.path.isdir, os.path.lexists for slightly different behaviors:
Return True if path refers to an existing path or an open file descriptor. Returns False for broken symbolic links. On some platforms, this function may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.
All good, but if following the import tree:
os.path - posixpath.py (ntpath.py)
genericpath.py - line ~20+
def exists(path):
"""Test whether a path exists. Returns False for broken symbolic links"""
try:
st = os.stat(path)
except os.error:
return False
return True
it's just a try / except block around [Python.Docs]: os.stat(path, *, dir_fd=None, follow_symlinks=True). So, your code is try / except free, but lower in the framestack there's (at least) one such block. This also applies to other functions (including os.path.isfile).
1.1. [Python.Docs]: pathlib - Path.is_file()
It's a fancier (and more [Wiktionary]: Pythonic) way of handling paths, but
Under the hood, it does exactly the same thing (pathlib.py - line ~1330):
def is_file(self):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
try:
return S_ISREG(self.stat().st_mode)
except OSError as e:
if e.errno not in (ENOENT, ENOTDIR):
raise
# Path doesn't exist or is a broken symlink
# (see https://bitbucket.org/pitrou/pathlib/issue/12/)
return False
2. [Python.Docs]: With Statement Context Managers
Either:
Create one:
class Swallow: # Dummy example
swallowed_exceptions = (FileNotFoundError,)
def __enter__(self):
print("Entering...")
def __exit__(self, exc_type, exc_value, exc_traceback):
print("Exiting:", exc_type, exc_value, exc_traceback)
# Only swallow FileNotFoundError (not e.g. TypeError - if the user passes a wrong argument like None or float or ...)
return exc_type in Swallow.swallowed_exceptions
And its usage - I'll replicate the os.path.isfile behavior (note that this is just for demonstrating purposes, do not attempt to write such code for production):
import os
import stat
def isfile_seaman(path): # Dummy func
result = False
with Swallow():
result = stat.S_ISREG(os.stat(path).st_mode)
return result
Use [Python.Docs]: contextlib.suppress(*exceptions) - which was specifically designed for selectively suppressing exceptions
But, they seem to be wrappers over try / except / else / finally blocks, as [Python.Docs]: Compound statements - The with statement states:
This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.
3. Filesystem traversal functions
Search the results for matching item(s):
[Python.Docs]: os.listdir(path='.') (or [Python.Docs]: os.scandir(path='.') on Python v3.5+, backport: [PyPI]: scandir)
Under the hood, both use:
Nix: [Man7]: OPENDIR(3) / [Man7]: READDIR(3) / [Man7]: CLOSEDIR(3)
Win: [MS.Learn]: FindFirstFileW function (fileapi.h) / [MS.Learn]: FindNextFileW function (fileapi.h) / [MS.Learn]: FindClose function (fileapi.h)
via [GitHub]: python/cpython - (main) cpython/Modules/posixmodule.c
Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type or file attribute information, because os.DirEntry objects expose this information if the operating system provides it when scanning a directory. All os.DirEntry methods may perform a system call, but is_dir() and is_file() usually only require a system call for symbolic links; os.DirEntry.stat() always requires a system call on Unix, but only requires one for symbolic links on Windows.
[Python.Docs]: os.walk(top, topdown=True, onerror=None, followlinks=False)
Uses os.listdir (os.scandir when available)
[Python.Docs]: glob.iglob(pathname, *, root_dir=None, dir_fd=None, recursive=False, include_hidden=False) (or its predecessor: glob.glob)
Doesn't seem a traversing function per se (at least in some cases), but it still uses os.listdir
Since these iterate over folders, (in most of the cases) they are inefficient for our problem (there are exceptions, like non wildcarded globbing - as #ShadowRanger pointed out), so I'm not going to insist on them. Not to mention that in some cases, filename processing might be required.
4. [Python.Docs]: os.access(path, mode, *, dir_fd=None, effective_ids=False, follow_symlinks=True)
Its behavior is close to os.path.exists (actually it's wider, mainly because of the 2nd argument).
User permissions might restrict the file "visibility" as the doc states:
... test if the invoking user has the specified access to path. mode should be F_OK to test the existence of path...
Security considerations:
Using access() to check if a user is authorized to e.g. open a file before actually doing so using open() creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it.
os.access("/tmp", os.F_OK)
Since I also work in C, I use this method as well because under the hood, it calls native APIs (again, via "${PYTHON_SRC_DIR}/Modules/posixmodule.c"), but it also opens a gate for possible user errors, and it's not as Pythonic as other variants. So, don't use it unless you know what you're doing:
Nix: [Man7]: ACCESS(2)
Warning: Using these calls to check if a user is authorized to, for example, open a file before actually doing so using open(2) creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it. For this reason, the use of this system call should be avoided.
Win: [MS.Learn]: GetFileAttributesW function (fileapi.h)
As seen, this approach is highly discouraged (especially on Nix).
Note: calling native APIs is also possible via [Python.Docs]: ctypes - A foreign function library for Python, but in most cases it's more complicated. Before working with CTypes, check [SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) out.
(Win specific): since vcruntime###.dll (msvcr###.dll for older VStudio versions - I'm going to refer to it as UCRT) exports a [MS.Learn]: _access, _waccess function family as well, here's an example (note that the recommended [Python.Docs]: msvcrt - Useful routines from the MS VC++ runtime doesn't export them):
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes as cts, os
>>> cts.CDLL("msvcrt")._waccess(u"C:\\Windows\\Temp", os.F_OK)
0
>>> cts.CDLL("msvcrt")._waccess(u"C:\\Windows\\Temp.notexist", os.F_OK)
-1
Notes:
Although it's not a good practice, I'm using os.F_OK in the call, but that's just for clarity (its value is 0)
I'm using _waccess so that the same code works on Python 3 and Python 2 (in spite of [Wikipedia]: Unicode related differences between them - [SO]: Passing utf-16 string to a Windows function (#CristiFati's answer))
Although this targets a very specific area, it was not mentioned in any of the previous answers
The Linux (Ubuntu ([Wikipedia]: Ubuntu version history) 16 x86_64 (pc064)) counterpart as well:
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes as cts, os
>>> cts.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp", os.F_OK)
0
>>> cts.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp.notexist", os.F_OK)
-1
Notes:
Instead hardcoding libc.so (LibC)'s path ("/lib/x86_64-linux-gnu/libc.so.6") which may (and most likely, will) vary across systems, None (or the empty string) can be passed to CDLL constructor (ctypes.CDLL(None).access(b"/tmp", os.F_OK)). According to [Man7]: DLOPEN(3):
If filename is NULL, then the returned handle is for the main
program. When given to dlsym(3), this handle causes a search for a
symbol in the main program, followed by all shared objects loaded at
program startup, and then all shared objects loaded by dlopen() with
the flag RTLD_GLOBAL.
Main (current) program (python) is linked against LibC, so its symbols (including access) will be loaded
This has to be handled with care, since functions like main, Py_Main and (all the) others are available; calling them could have disastrous effects (on the current program)
This doesn't also apply to Windows (but that's not such a big deal, since UCRT is located in "%SystemRoot%\System32" which is in %PATH% by default). I wanted to take things further and replicate this behavior on Windows (and submit a patch), but as it turns out, [MS.Learn]: GetProcAddress function (libloaderapi.h) only "sees" exported symbols, so unless someone declares the functions in the main executable as __declspec(dllexport) (why on Earth the common person would do that?), the main program is loadable, but it is pretty much unusable
5. 3rd-party modules with filesystem capabilities
Most likely, will rely on one of the ways above (maybe with slight customizations). One example would be (again, Win specific) [GitHub]: mhammond/pywin32 - Python for Windows (pywin32) Extensions, which is a Python wrapper over WinAPIs.
But, since this is more like a workaround, I'm stopping here.
6. SysAdmin approach
I consider this a (lame) workaround (gainarie): use Python as a wrapper to execute shell commands:
Win:
(py35x64_test) [cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q000082831]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\Temp\" > nul 2>&1'))"
0
(py35x64_test) [cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q000082831]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\Temp.notexist\" > nul 2>&1'))"
1
Nix ([Wikipedia]: Unix-like) - Ubuntu:
[cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q000082831]> python3 -c "import os; print(os.system('ls \"/tmp\" > /dev/null 2>&1'))"
0
[cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q000082831]> python3 -c "import os; print(os.system('ls \"/tmp.notexist\" > /dev/null 2>&1'))"
512
Bottom line:
Do use try / except / else / finally blocks, because they can prevent you running into a series of nasty problems
A possible counterexample that I can think of, is performance: such blocks are costly, so try not to place them in code that it's supposed to run hundreds of thousands times per second (but since (in most cases) it involves disk access, it won't be the case)
Python 3.4+ has an object-oriented path module: pathlib. Using this new module, you can check whether a file exists like this:
import pathlib
p = pathlib.Path('path/to/file')
if p.is_file(): # or p.is_dir() to see if it is a directory
# do stuff
You can (and usually should) still use a try/except block when opening files:
try:
with p.open() as f:
# do awesome stuff
except OSError:
print('Well darn.')
The pathlib module has lots of cool stuff in it: convenient globbing, checking file's owner, easier path joining, etc. It's worth checking out. If you're on an older Python (version 2.6 or later), you can still install pathlib with pip:
# installs pathlib2 on older Python versions
# the original third-party module, pathlib, is no longer maintained.
pip install pathlib2
Then import it as follows:
# Older Python versions
import pathlib2 as pathlib
This is the simplest way to check if a file exists. Just because the file existed when you checked doesn't guarantee that it will be there when you need to open it.
import os
fname = "foo.txt"
if os.path.isfile(fname):
print("file does exist at this time")
else:
print("no such file exists at this time")
How do I check whether a file exists, using Python, without using a try statement?
Now available since Python 3.4, import and instantiate a Path object with the file name, and check the is_file method (note that this returns True for symlinks pointing to regular files as well):
>>> from pathlib import Path
>>> Path('/').is_file()
False
>>> Path('/initrd.img').is_file()
True
>>> Path('/doesnotexist').is_file()
False
If you're on Python 2, you can backport the pathlib module from pypi, pathlib2, or otherwise check isfile from the os.path module:
>>> import os
>>> os.path.isfile('/')
False
>>> os.path.isfile('/initrd.img')
True
>>> os.path.isfile('/doesnotexist')
False
Now the above is probably the best pragmatic direct answer here, but there's the possibility of a race condition (depending on what you're trying to accomplish), and the fact that the underlying implementation uses a try, but Python uses try everywhere in its implementation.
Because Python uses try everywhere, there's really no reason to avoid an implementation that uses it.
But the rest of this answer attempts to consider these caveats.
Longer, much more pedantic answer
Available since Python 3.4, use the new Path object in pathlib. Note that .exists is not quite right, because directories are not files (except in the unix sense that everything is a file).
>>> from pathlib import Path
>>> root = Path('/')
>>> root.exists()
True
So we need to use is_file:
>>> root.is_file()
False
Here's the help on is_file:
is_file(self)
Whether this path is a regular file (also True for symlinks pointing
to regular files).
So let's get a file that we know is a file:
>>> import tempfile
>>> file = tempfile.NamedTemporaryFile()
>>> filepathobj = Path(file.name)
>>> filepathobj.is_file()
True
>>> filepathobj.exists()
True
By default, NamedTemporaryFile deletes the file when closed (and will automatically close when no more references exist to it).
>>> del file
>>> filepathobj.exists()
False
>>> filepathobj.is_file()
False
If you dig into the implementation, though, you'll see that is_file uses try:
def is_file(self):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
try:
return S_ISREG(self.stat().st_mode)
except OSError as e:
if e.errno not in (ENOENT, ENOTDIR):
raise
# Path doesn't exist or is a broken symlink
# (see https://bitbucket.org/pitrou/pathlib/issue/12/)
return False
Race Conditions: Why we like try
We like try because it avoids race conditions. With try, you simply attempt to read your file, expecting it to be there, and if not, you catch the exception and perform whatever fallback behavior makes sense.
If you want to check that a file exists before you attempt to read it, and you might be deleting it and then you might be using multiple threads or processes, or another program knows about that file and could delete it - you risk the chance of a race condition if you check it exists, because you are then racing to open it before its condition (its existence) changes.
Race conditions are very hard to debug because there's a very small window in which they can cause your program to fail.
But if this is your motivation, you can get the value of a try statement by using the suppress context manager.
Avoiding race conditions without a try statement: suppress
Python 3.4 gives us the suppress context manager (previously the ignore context manager), which does semantically exactly the same thing in fewer lines, while also (at least superficially) meeting the original ask to avoid a try statement:
from contextlib import suppress
from pathlib import Path
Usage:
>>> with suppress(OSError), Path('doesnotexist').open() as f:
... for line in f:
... print(line)
...
>>>
>>> with suppress(OSError):
... Path('doesnotexist').unlink()
...
>>>
For earlier Pythons, you could roll your own suppress, but without a try will be more verbose than with. I do believe this actually is the only answer that doesn't use try at any level in the Python that can be applied to prior to Python 3.4 because it uses a context manager instead:
class suppress(object):
def __init__(self, *exceptions):
self.exceptions = exceptions
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is not None:
return issubclass(exc_type, self.exceptions)
Perhaps easier with a try:
from contextlib import contextmanager
#contextmanager
def suppress(*exceptions):
try:
yield
except exceptions:
pass
Other options that don't meet the ask for "without try":
isfile
import os
os.path.isfile(path)
from the docs:
os.path.isfile(path)
Return True if path is an existing regular file. This follows symbolic
links, so both islink() and isfile() can be true for the same path.
But if you examine the source of this function, you'll see it actually does use a try statement:
# This follows symbolic links, so both islink() and isdir() can be true
# for the same path on systems that support symlinks
def isfile(path):
"""Test whether a path is a regular file"""
try:
st = os.stat(path)
except os.error:
return False
return stat.S_ISREG(st.st_mode)
>>> OSError is os.error
True
All it's doing is using the given path to see if it can get stats on it, catching OSError and then checking if it's a file if it didn't raise the exception.
If you intend to do something with the file, I would suggest directly attempting it with a try-except to avoid a race condition:
try:
with open(path) as f:
f.read()
except OSError:
pass
os.access
Available for Unix and Windows is os.access, but to use you must pass flags, and it does not differentiate between files and directories. This is more used to test if the real invoking user has access in an elevated privilege environment:
import os
os.access(path, os.F_OK)
It also suffers from the same race condition problems as isfile. From the docs:
Note:
Using access() to check if a user is authorized to e.g. open a file
before actually doing so using open() creates a security hole, because
the user might exploit the short time interval between checking and
opening the file to manipulate it. It’s preferable to use EAFP
techniques. For example:
if os.access("myfile", os.R_OK):
with open("myfile") as fp:
return fp.read()
return "some default data"
is better written as:
try:
fp = open("myfile")
except IOError as e:
if e.errno == errno.EACCES:
return "some default data"
# Not a permission error.
raise
else:
with fp:
return fp.read()
Avoid using os.access. It is a low level function that has more opportunities for user error than the higher level objects and functions discussed above.
Criticism of another answer:
Another answer says this about os.access:
Personally, I prefer this one because under the hood, it calls native APIs (via "${PYTHON_SRC_DIR}/Modules/posixmodule.c"), but it also opens a gate for possible user errors, and it's not as Pythonic as other variants:
This answer says it prefers a non-Pythonic, error-prone method, with no justification. It seems to encourage users to use low-level APIs without understanding them.
It also creates a context manager which, by unconditionally returning True, allows all Exceptions (including KeyboardInterrupt and SystemExit!) to pass silently, which is a good way to hide bugs.
This seems to encourage users to adopt poor practices.
Prefer the try statement. It's considered better style and avoids race conditions.
Don't take my word for it. There's plenty of support for this theory. Here's a couple:
Style: Section "Handling unusual conditions" of these course notes for Software Design (2007)
Avoiding Race Conditions
Use:
import os
#Your path here e.g. "C:\Program Files\text.txt"
#For access purposes: "C:\\Program Files\\text.txt"
if os.path.exists("C:\..."):
print "File found!"
else:
print "File not found!"
Importing os makes it easier to navigate and perform standard actions with your operating system.
For reference, also see How do I check whether a file exists without exceptions?.
If you need high-level operations, use shutil.
Testing for files and folders with os.path.isfile(), os.path.isdir() and os.path.exists()
Assuming that the "path" is a valid path, this table shows what is returned by each function for files and folders:
You can also test if a file is a certain type of file using os.path.splitext() to get the extension (if you don't already know it)
>>> import os
>>> path = "path to a word document"
>>> os.path.isfile(path)
True
>>> os.path.splitext(path)[1] == ".docx" # test if the extension is .docx
True
TL;DR
The answer is: use the pathlib module
Pathlib is probably the most modern and convenient way for almost all of the file operations. For the existence of a file or a folder a single line of code is enough. If file is not exists, it will not throw any exception.
from pathlib import Path
if Path("myfile.txt").exists(): # works for both file and folders
# do your cool stuff...
The pathlib module was introduced in Python 3.4, so you need to have Python 3.4+. This library makes your life much easier while working with files and folders, and it is pretty to use. Here is more documentation about it: pathlib — Object-oriented filesystem paths.
BTW, if you are going to reuse the path, then it is better to assign it to a variable.
So it will become:
from pathlib import Path
p = Path("loc/of/myfile.txt")
if p.exists(): # works for both file and folders
# do stuffs...
#reuse 'p' if needed.
In 2016 the best way is still using os.path.isfile:
>>> os.path.isfile('/path/to/some/file.txt')
Or in Python 3 you can use pathlib:
import pathlib
path = pathlib.Path('/path/to/some/file.txt')
if path.is_file():
...
It doesn't seem like there's a meaningful functional difference between try/except and isfile(), so you should use which one makes sense.
If you want to read a file, if it exists, do
try:
f = open(filepath)
except IOError:
print 'Oh dear.'
But if you just wanted to rename a file if it exists, and therefore don't need to open it, do
if os.path.isfile(filepath):
os.rename(filepath, filepath + '.old')
If you want to write to a file, if it doesn't exist, do
# Python 2
if not os.path.isfile(filepath):
f = open(filepath, 'w')
# Python 3: x opens for exclusive creation, failing if the file already exists
try:
f = open(filepath, 'wx')
except IOError:
print 'file already exists'
If you need file locking, that's a different matter.
You could try this (safer):
try:
# http://effbot.org/zone/python-with-statement.htm
# 'with' is safer to open a file
with open('whatever.txt') as fh:
# Do something with 'fh'
except IOError as e:
print("({})".format(e))
The ouput would be:
([Errno 2] No such file or directory:
'whatever.txt')
Then, depending on the result, your program can just keep running from there or you can code to stop it if you want.
Date: 2017-12-04
Every possible solution has been listed in other answers.
An intuitive and arguable way to check if a file exists is the following:
import os
os.path.isfile('~/file.md') # Returns True if exists, else False
# Additionally, check a directory
os.path.isdir('~/folder') # Returns True if the folder exists, else False
# Check either a directory or a file
os.path.exists('~/file')
I made an exhaustive cheat sheet for your reference:
# os.path methods in exhaustive cheat sheet
{'definition': ['dirname',
'basename',
'abspath',
'relpath',
'commonpath',
'normpath',
'realpath'],
'operation': ['split', 'splitdrive', 'splitext',
'join', 'normcase'],
'compare': ['samefile', 'sameopenfile', 'samestat'],
'condition': ['isdir',
'isfile',
'exists',
'lexists'
'islink',
'isabs',
'ismount',],
'expand': ['expanduser',
'expandvars'],
'stat': ['getatime', 'getctime', 'getmtime',
'getsize']}
Although I always recommend using try and except statements, here are a few possibilities for you (my personal favourite is using os.access):
Try opening the file:
Opening the file will always verify the existence of the file. You can make a function just like so:
def File_Existence(filepath):
f = open(filepath)
return True
If it's False, it will stop execution with an unhanded IOError
or OSError in later versions of Python. To catch the exception,
you have to use a try except clause. Of course, you can always
use a try except` statement like so (thanks to hsandt
for making me think):
def File_Existence(filepath):
try:
f = open(filepath)
except IOError, OSError: # Note OSError is for later versions of Python
return False
return True
Use os.path.exists(path):
This will check the existence of what you specify. However, it checks for files and directories so beware about how you use it.
import os.path
>>> os.path.exists("this/is/a/directory")
True
>>> os.path.exists("this/is/a/file.txt")
True
>>> os.path.exists("not/a/directory")
False
Use os.access(path, mode):
This will check whether you have access to the file. It will check for permissions. Based on the os.py documentation, typing in os.F_OK, it will check the existence of the path. However, using this will create a security hole, as someone can attack your file using the time between checking the permissions and opening the file. You should instead go directly to opening the file instead of checking its permissions. (EAFP vs LBYP). If you're not going to open the file afterwards, and only checking its existence, then you can use this.
Anyway, here:
>>> import os
>>> os.access("/is/a/file.txt", os.F_OK)
True
I should also mention that there are two ways that you will not be able to verify the existence of a file. Either the issue will be permission denied or no such file or directory. If you catch an IOError, set the IOError as e (like my first option), and then type in print(e.args) so that you can hopefully determine your issue. I hope it helps! :)
If the file is for opening you could use one of the following techniques:
with open('somefile', 'xt') as f: # Using the x-flag, Python 3.3 and above
f.write('Hello\n')
if not os.path.exists('somefile'):
with open('somefile', 'wt') as f:
f.write("Hello\n")
else:
print('File already exists!')
Note: This finds either a file or a directory with the given name.
Additionally, os.access():
if os.access("myfile", os.R_OK):
with open("myfile") as fp:
return fp.read()
Being R_OK, W_OK, and X_OK the flags to test for permissions (doc).
if os.path.isfile(path_to_file):
try:
open(path_to_file)
pass
except IOError as e:
print "Unable to open file"
Raising exceptions is considered to be an acceptable, and Pythonic,
approach for flow control in your program. Consider handling missing
files with IOErrors. In this situation, an IOError exception will be
raised if the file exists but the user does not have read permissions.
Source: Using Python: How To Check If A File Exists
If you imported NumPy already for other purposes then there is no need to import other libraries like pathlib, os, paths, etc.
import numpy as np
np.DataSource().exists("path/to/your/file")
This will return true or false based on its existence.
Check file or directory exists
You can follow these three ways:
1. Using isfile()
Note 1: The os.path.isfile used only for files
import os.path
os.path.isfile(filename) # True if file exists
os.path.isfile(dirname) # False if directory exists
2. Using exists
Note 2: The os.path.exists is used for both files and directories
import os.path
os.path.exists(filename) # True if file exists
os.path.exists(dirname) # True if directory exists
3. The pathlib.Path method (included in Python 3+, installable with pip for Python 2)
from pathlib import Path
Path(filename).exists()
You can write Brian's suggestion without the try:.
from contextlib import suppress
with suppress(IOError), open('filename'):
process()
suppress is part of Python 3.4. In older releases you can quickly write your own suppress:
from contextlib import contextmanager
#contextmanager
def suppress(*exceptions):
try:
yield
except exceptions:
pass
I'm the author of a package that's been around for about 10 years, and it has a function that addresses this question directly. Basically, if you are on a non-Windows system, it uses Popen to access find. However, if you are on Windows, it replicates find with an efficient filesystem walker.
The code itself does not use a try block… except in determining the operating system and thus steering you to the "Unix"-style find or the hand-buillt find. Timing tests showed that the try was faster in determining the OS, so I did use one there (but nowhere else).
>>> import pox
>>> pox.find('*python*', type='file', root=pox.homedir(), recurse=False)
['/Users/mmckerns/.python']
And the doc…
>>> print pox.find.__doc__
find(patterns[,root,recurse,type]); Get path to a file or directory
patterns: name or partial name string of items to search for
root: path string of top-level directory to search
recurse: if True, recurse down from root directory
type: item filter; one of {None, file, dir, link, socket, block, char}
verbose: if True, be a little verbose about the search
On some OS, recursion can be specified by recursion depth (an integer).
patterns can be specified with basic pattern matching. Additionally,
multiple patterns can be specified by splitting patterns with a ';'
For example:
>>> find('pox*', root='..')
['/Users/foo/pox/pox', '/Users/foo/pox/scripts/pox_launcher.py']
>>> find('*shutils*;*init*')
['/Users/foo/pox/pox/shutils.py', '/Users/foo/pox/pox/__init__.py']
>>>
The implementation, if you care to look, is here:
https://github.com/uqfoundation/pox/blob/89f90fb308f285ca7a62eabe2c38acb87e89dad9/pox/shutils.py#L190
Adding one more slight variation which isn't exactly reflected in the other answers.
This will handle the case of the file_path being None or empty string.
def file_exists(file_path):
if not file_path:
return False
elif not os.path.isfile(file_path):
return False
else:
return True
Adding a variant based on suggestion from Shahbaz
def file_exists(file_path):
if not file_path:
return False
else:
return os.path.isfile(file_path)
Adding a variant based on suggestion from Peter Wood
def file_exists(file_path):
return file_path and os.path.isfile(file_path):
Here's a one-line Python command for the Linux command line environment. I find this very handy since I'm not such a hot Bash guy.
python -c "import os.path; print os.path.isfile('/path_to/file.xxx')"
You can use the "OS" library of Python:
>>> import os
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.txt")
True
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.tx")
False
How do I check whether a file exists, without using the try statement?
In 2016, this is still arguably the easiest way to check if both a file exists and if it is a file:
import os
os.path.isfile('./file.txt') # Returns True if exists, else False
isfile is actually just a helper method that internally uses os.stat and stat.S_ISREG(mode) underneath. This os.stat is a lower-level method that will provide you with detailed information about files, directories, sockets, buffers, and more. More about os.stat here
Note: However, this approach will not lock the file in any way and therefore your code can become vulnerable to "time of check to time of use" (TOCTTOU) bugs.
So raising exceptions is considered to be an acceptable, and Pythonic, approach for flow control in your program. And one should consider handling missing files with IOErrors, rather than if statements (just an advice).
How do I check whether a file exists or not, without using the try statement?
If the reason you're checking is so you can do something like if file_exists: open_it(), it's safer to use a try around the attempt to open it. Checking and then opening risks the file being deleted or moved or something between when you check and when you try to open it.
If you're not planning to open the file immediately, you can use os.path.isfile
Return True if path is an existing regular file. This follows symbolic links, so both islink() and isfile() can be true for the same path.
import os.path
os.path.isfile(fname)
if you need to be sure it's a file.
Starting with Python 3.4, the pathlib module offers an object-oriented approach (backported to pathlib2 in Python 2.7):
from pathlib import Path
my_file = Path("/path/to/file")
if my_file.is_file():
# file exists
To check a directory, do:
if my_file.is_dir():
# directory exists
To check whether a Path object exists independently of whether is it a file or directory, use exists():
if my_file.exists():
# path exists
You can also use resolve(strict=True) in a try block:
try:
my_abs_path = my_file.resolve(strict=True)
except FileNotFoundError:
# doesn't exist
else:
# exists
Use os.path.exists to check both files and directories:
import os.path
os.path.exists(file_path)
Use os.path.isfile to check only files (note: follows symbolic links):
os.path.isfile(file_path)
Unlike isfile(), exists() will return True for directories. So depending on if you want only plain files or also directories, you'll use isfile() or exists(). Here is some simple REPL output:
>>> os.path.isfile("/etc/password.txt")
True
>>> os.path.isfile("/etc")
False
>>> os.path.isfile("/does/not/exist")
False
>>> os.path.exists("/etc/password.txt")
True
>>> os.path.exists("/etc")
True
>>> os.path.exists("/does/not/exist")
False
import os
if os.path.isfile(filepath):
print("File exists")
Use os.path.isfile() with os.access():
import os
PATH = './file.txt'
if os.path.isfile(PATH) and os.access(PATH, os.R_OK):
print("File exists and is readable")
else:
print("Either the file is missing or not readable")
import os
os.path.exists(path) # Returns whether the path (directory or file) exists or not
os.path.isfile(path) # Returns whether the file exists or not
Although almost every possible way has been listed in (at least one of) the existing answers (e.g. Python 3.4 specific stuff was added), I'll try to group everything together.
Note: every piece of Python standard library code that I'm going to post, belongs to version 3.5.3.
Problem statement:
Check file (arguable: also folder ("special" file) ?) existence
Don't use try / except / else / finally blocks
Possible solutions:
1. [Python.Docs]: os.path.exists(path)
Also check other function family members like os.path.isfile, os.path.isdir, os.path.lexists for slightly different behaviors:
Return True if path refers to an existing path or an open file descriptor. Returns False for broken symbolic links. On some platforms, this function may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.
All good, but if following the import tree:
os.path - posixpath.py (ntpath.py)
genericpath.py - line ~20+
def exists(path):
"""Test whether a path exists. Returns False for broken symbolic links"""
try:
st = os.stat(path)
except os.error:
return False
return True
it's just a try / except block around [Python.Docs]: os.stat(path, *, dir_fd=None, follow_symlinks=True). So, your code is try / except free, but lower in the framestack there's (at least) one such block. This also applies to other functions (including os.path.isfile).
1.1. [Python.Docs]: pathlib - Path.is_file()
It's a fancier (and more [Wiktionary]: Pythonic) way of handling paths, but
Under the hood, it does exactly the same thing (pathlib.py - line ~1330):
def is_file(self):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
try:
return S_ISREG(self.stat().st_mode)
except OSError as e:
if e.errno not in (ENOENT, ENOTDIR):
raise
# Path doesn't exist or is a broken symlink
# (see https://bitbucket.org/pitrou/pathlib/issue/12/)
return False
2. [Python.Docs]: With Statement Context Managers
Either:
Create one:
class Swallow: # Dummy example
swallowed_exceptions = (FileNotFoundError,)
def __enter__(self):
print("Entering...")
def __exit__(self, exc_type, exc_value, exc_traceback):
print("Exiting:", exc_type, exc_value, exc_traceback)
# Only swallow FileNotFoundError (not e.g. TypeError - if the user passes a wrong argument like None or float or ...)
return exc_type in Swallow.swallowed_exceptions
And its usage - I'll replicate the os.path.isfile behavior (note that this is just for demonstrating purposes, do not attempt to write such code for production):
import os
import stat
def isfile_seaman(path): # Dummy func
result = False
with Swallow():
result = stat.S_ISREG(os.stat(path).st_mode)
return result
Use [Python.Docs]: contextlib.suppress(*exceptions) - which was specifically designed for selectively suppressing exceptions
But, they seem to be wrappers over try / except / else / finally blocks, as [Python.Docs]: Compound statements - The with statement states:
This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.
3. Filesystem traversal functions
Search the results for matching item(s):
[Python.Docs]: os.listdir(path='.') (or [Python.Docs]: os.scandir(path='.') on Python v3.5+, backport: [PyPI]: scandir)
Under the hood, both use:
Nix: [Man7]: OPENDIR(3) / [Man7]: READDIR(3) / [Man7]: CLOSEDIR(3)
Win: [MS.Learn]: FindFirstFileW function (fileapi.h) / [MS.Learn]: FindNextFileW function (fileapi.h) / [MS.Learn]: FindClose function (fileapi.h)
via [GitHub]: python/cpython - (main) cpython/Modules/posixmodule.c
Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type or file attribute information, because os.DirEntry objects expose this information if the operating system provides it when scanning a directory. All os.DirEntry methods may perform a system call, but is_dir() and is_file() usually only require a system call for symbolic links; os.DirEntry.stat() always requires a system call on Unix, but only requires one for symbolic links on Windows.
[Python.Docs]: os.walk(top, topdown=True, onerror=None, followlinks=False)
Uses os.listdir (os.scandir when available)
[Python.Docs]: glob.iglob(pathname, *, root_dir=None, dir_fd=None, recursive=False, include_hidden=False) (or its predecessor: glob.glob)
Doesn't seem a traversing function per se (at least in some cases), but it still uses os.listdir
Since these iterate over folders, (in most of the cases) they are inefficient for our problem (there are exceptions, like non wildcarded globbing - as #ShadowRanger pointed out), so I'm not going to insist on them. Not to mention that in some cases, filename processing might be required.
4. [Python.Docs]: os.access(path, mode, *, dir_fd=None, effective_ids=False, follow_symlinks=True)
Its behavior is close to os.path.exists (actually it's wider, mainly because of the 2nd argument).
User permissions might restrict the file "visibility" as the doc states:
... test if the invoking user has the specified access to path. mode should be F_OK to test the existence of path...
Security considerations:
Using access() to check if a user is authorized to e.g. open a file before actually doing so using open() creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it.
os.access("/tmp", os.F_OK)
Since I also work in C, I use this method as well because under the hood, it calls native APIs (again, via "${PYTHON_SRC_DIR}/Modules/posixmodule.c"), but it also opens a gate for possible user errors, and it's not as Pythonic as other variants. So, don't use it unless you know what you're doing:
Nix: [Man7]: ACCESS(2)
Warning: Using these calls to check if a user is authorized to, for example, open a file before actually doing so using open(2) creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it. For this reason, the use of this system call should be avoided.
Win: [MS.Learn]: GetFileAttributesW function (fileapi.h)
As seen, this approach is highly discouraged (especially on Nix).
Note: calling native APIs is also possible via [Python.Docs]: ctypes - A foreign function library for Python, but in most cases it's more complicated. Before working with CTypes, check [SO]: C function called from Python via ctypes returns incorrect value (#CristiFati's answer) out.
(Win specific): since vcruntime###.dll (msvcr###.dll for older VStudio versions - I'm going to refer to it as UCRT) exports a [MS.Learn]: _access, _waccess function family as well, here's an example (note that the recommended [Python.Docs]: msvcrt - Useful routines from the MS VC++ runtime doesn't export them):
Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes as cts, os
>>> cts.CDLL("msvcrt")._waccess(u"C:\\Windows\\Temp", os.F_OK)
0
>>> cts.CDLL("msvcrt")._waccess(u"C:\\Windows\\Temp.notexist", os.F_OK)
-1
Notes:
Although it's not a good practice, I'm using os.F_OK in the call, but that's just for clarity (its value is 0)
I'm using _waccess so that the same code works on Python 3 and Python 2 (in spite of [Wikipedia]: Unicode related differences between them - [SO]: Passing utf-16 string to a Windows function (#CristiFati's answer))
Although this targets a very specific area, it was not mentioned in any of the previous answers
The Linux (Ubuntu ([Wikipedia]: Ubuntu version history) 16 x86_64 (pc064)) counterpart as well:
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes as cts, os
>>> cts.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp", os.F_OK)
0
>>> cts.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp.notexist", os.F_OK)
-1
Notes:
Instead hardcoding libc.so (LibC)'s path ("/lib/x86_64-linux-gnu/libc.so.6") which may (and most likely, will) vary across systems, None (or the empty string) can be passed to CDLL constructor (ctypes.CDLL(None).access(b"/tmp", os.F_OK)). According to [Man7]: DLOPEN(3):
If filename is NULL, then the returned handle is for the main
program. When given to dlsym(3), this handle causes a search for a
symbol in the main program, followed by all shared objects loaded at
program startup, and then all shared objects loaded by dlopen() with
the flag RTLD_GLOBAL.
Main (current) program (python) is linked against LibC, so its symbols (including access) will be loaded
This has to be handled with care, since functions like main, Py_Main and (all the) others are available; calling them could have disastrous effects (on the current program)
This doesn't also apply to Windows (but that's not such a big deal, since UCRT is located in "%SystemRoot%\System32" which is in %PATH% by default). I wanted to take things further and replicate this behavior on Windows (and submit a patch), but as it turns out, [MS.Learn]: GetProcAddress function (libloaderapi.h) only "sees" exported symbols, so unless someone declares the functions in the main executable as __declspec(dllexport) (why on Earth the common person would do that?), the main program is loadable, but it is pretty much unusable
5. 3rd-party modules with filesystem capabilities
Most likely, will rely on one of the ways above (maybe with slight customizations). One example would be (again, Win specific) [GitHub]: mhammond/pywin32 - Python for Windows (pywin32) Extensions, which is a Python wrapper over WinAPIs.
But, since this is more like a workaround, I'm stopping here.
6. SysAdmin approach
I consider this a (lame) workaround (gainarie): use Python as a wrapper to execute shell commands:
Win:
(py35x64_test) [cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q000082831]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\Temp\" > nul 2>&1'))"
0
(py35x64_test) [cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q000082831]> "e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\Temp.notexist\" > nul 2>&1'))"
1
Nix ([Wikipedia]: Unix-like) - Ubuntu:
[cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q000082831]> python3 -c "import os; print(os.system('ls \"/tmp\" > /dev/null 2>&1'))"
0
[cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q000082831]> python3 -c "import os; print(os.system('ls \"/tmp.notexist\" > /dev/null 2>&1'))"
512
Bottom line:
Do use try / except / else / finally blocks, because they can prevent you running into a series of nasty problems
A possible counterexample that I can think of, is performance: such blocks are costly, so try not to place them in code that it's supposed to run hundreds of thousands times per second (but since (in most cases) it involves disk access, it won't be the case)
Python 3.4+ has an object-oriented path module: pathlib. Using this new module, you can check whether a file exists like this:
import pathlib
p = pathlib.Path('path/to/file')
if p.is_file(): # or p.is_dir() to see if it is a directory
# do stuff
You can (and usually should) still use a try/except block when opening files:
try:
with p.open() as f:
# do awesome stuff
except OSError:
print('Well darn.')
The pathlib module has lots of cool stuff in it: convenient globbing, checking file's owner, easier path joining, etc. It's worth checking out. If you're on an older Python (version 2.6 or later), you can still install pathlib with pip:
# installs pathlib2 on older Python versions
# the original third-party module, pathlib, is no longer maintained.
pip install pathlib2
Then import it as follows:
# Older Python versions
import pathlib2 as pathlib
This is the simplest way to check if a file exists. Just because the file existed when you checked doesn't guarantee that it will be there when you need to open it.
import os
fname = "foo.txt"
if os.path.isfile(fname):
print("file does exist at this time")
else:
print("no such file exists at this time")
How do I check whether a file exists, using Python, without using a try statement?
Now available since Python 3.4, import and instantiate a Path object with the file name, and check the is_file method (note that this returns True for symlinks pointing to regular files as well):
>>> from pathlib import Path
>>> Path('/').is_file()
False
>>> Path('/initrd.img').is_file()
True
>>> Path('/doesnotexist').is_file()
False
If you're on Python 2, you can backport the pathlib module from pypi, pathlib2, or otherwise check isfile from the os.path module:
>>> import os
>>> os.path.isfile('/')
False
>>> os.path.isfile('/initrd.img')
True
>>> os.path.isfile('/doesnotexist')
False
Now the above is probably the best pragmatic direct answer here, but there's the possibility of a race condition (depending on what you're trying to accomplish), and the fact that the underlying implementation uses a try, but Python uses try everywhere in its implementation.
Because Python uses try everywhere, there's really no reason to avoid an implementation that uses it.
But the rest of this answer attempts to consider these caveats.
Longer, much more pedantic answer
Available since Python 3.4, use the new Path object in pathlib. Note that .exists is not quite right, because directories are not files (except in the unix sense that everything is a file).
>>> from pathlib import Path
>>> root = Path('/')
>>> root.exists()
True
So we need to use is_file:
>>> root.is_file()
False
Here's the help on is_file:
is_file(self)
Whether this path is a regular file (also True for symlinks pointing
to regular files).
So let's get a file that we know is a file:
>>> import tempfile
>>> file = tempfile.NamedTemporaryFile()
>>> filepathobj = Path(file.name)
>>> filepathobj.is_file()
True
>>> filepathobj.exists()
True
By default, NamedTemporaryFile deletes the file when closed (and will automatically close when no more references exist to it).
>>> del file
>>> filepathobj.exists()
False
>>> filepathobj.is_file()
False
If you dig into the implementation, though, you'll see that is_file uses try:
def is_file(self):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
try:
return S_ISREG(self.stat().st_mode)
except OSError as e:
if e.errno not in (ENOENT, ENOTDIR):
raise
# Path doesn't exist or is a broken symlink
# (see https://bitbucket.org/pitrou/pathlib/issue/12/)
return False
Race Conditions: Why we like try
We like try because it avoids race conditions. With try, you simply attempt to read your file, expecting it to be there, and if not, you catch the exception and perform whatever fallback behavior makes sense.
If you want to check that a file exists before you attempt to read it, and you might be deleting it and then you might be using multiple threads or processes, or another program knows about that file and could delete it - you risk the chance of a race condition if you check it exists, because you are then racing to open it before its condition (its existence) changes.
Race conditions are very hard to debug because there's a very small window in which they can cause your program to fail.
But if this is your motivation, you can get the value of a try statement by using the suppress context manager.
Avoiding race conditions without a try statement: suppress
Python 3.4 gives us the suppress context manager (previously the ignore context manager), which does semantically exactly the same thing in fewer lines, while also (at least superficially) meeting the original ask to avoid a try statement:
from contextlib import suppress
from pathlib import Path
Usage:
>>> with suppress(OSError), Path('doesnotexist').open() as f:
... for line in f:
... print(line)
...
>>>
>>> with suppress(OSError):
... Path('doesnotexist').unlink()
...
>>>
For earlier Pythons, you could roll your own suppress, but without a try will be more verbose than with. I do believe this actually is the only answer that doesn't use try at any level in the Python that can be applied to prior to Python 3.4 because it uses a context manager instead:
class suppress(object):
def __init__(self, *exceptions):
self.exceptions = exceptions
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is not None:
return issubclass(exc_type, self.exceptions)
Perhaps easier with a try:
from contextlib import contextmanager
#contextmanager
def suppress(*exceptions):
try:
yield
except exceptions:
pass
Other options that don't meet the ask for "without try":
isfile
import os
os.path.isfile(path)
from the docs:
os.path.isfile(path)
Return True if path is an existing regular file. This follows symbolic
links, so both islink() and isfile() can be true for the same path.
But if you examine the source of this function, you'll see it actually does use a try statement:
# This follows symbolic links, so both islink() and isdir() can be true
# for the same path on systems that support symlinks
def isfile(path):
"""Test whether a path is a regular file"""
try:
st = os.stat(path)
except os.error:
return False
return stat.S_ISREG(st.st_mode)
>>> OSError is os.error
True
All it's doing is using the given path to see if it can get stats on it, catching OSError and then checking if it's a file if it didn't raise the exception.
If you intend to do something with the file, I would suggest directly attempting it with a try-except to avoid a race condition:
try:
with open(path) as f:
f.read()
except OSError:
pass
os.access
Available for Unix and Windows is os.access, but to use you must pass flags, and it does not differentiate between files and directories. This is more used to test if the real invoking user has access in an elevated privilege environment:
import os
os.access(path, os.F_OK)
It also suffers from the same race condition problems as isfile. From the docs:
Note:
Using access() to check if a user is authorized to e.g. open a file
before actually doing so using open() creates a security hole, because
the user might exploit the short time interval between checking and
opening the file to manipulate it. It’s preferable to use EAFP
techniques. For example:
if os.access("myfile", os.R_OK):
with open("myfile") as fp:
return fp.read()
return "some default data"
is better written as:
try:
fp = open("myfile")
except IOError as e:
if e.errno == errno.EACCES:
return "some default data"
# Not a permission error.
raise
else:
with fp:
return fp.read()
Avoid using os.access. It is a low level function that has more opportunities for user error than the higher level objects and functions discussed above.
Criticism of another answer:
Another answer says this about os.access:
Personally, I prefer this one because under the hood, it calls native APIs (via "${PYTHON_SRC_DIR}/Modules/posixmodule.c"), but it also opens a gate for possible user errors, and it's not as Pythonic as other variants:
This answer says it prefers a non-Pythonic, error-prone method, with no justification. It seems to encourage users to use low-level APIs without understanding them.
It also creates a context manager which, by unconditionally returning True, allows all Exceptions (including KeyboardInterrupt and SystemExit!) to pass silently, which is a good way to hide bugs.
This seems to encourage users to adopt poor practices.
Prefer the try statement. It's considered better style and avoids race conditions.
Don't take my word for it. There's plenty of support for this theory. Here's a couple:
Style: Section "Handling unusual conditions" of these course notes for Software Design (2007)
Avoiding Race Conditions
Use:
import os
#Your path here e.g. "C:\Program Files\text.txt"
#For access purposes: "C:\\Program Files\\text.txt"
if os.path.exists("C:\..."):
print "File found!"
else:
print "File not found!"
Importing os makes it easier to navigate and perform standard actions with your operating system.
For reference, also see How do I check whether a file exists without exceptions?.
If you need high-level operations, use shutil.
Testing for files and folders with os.path.isfile(), os.path.isdir() and os.path.exists()
Assuming that the "path" is a valid path, this table shows what is returned by each function for files and folders:
You can also test if a file is a certain type of file using os.path.splitext() to get the extension (if you don't already know it)
>>> import os
>>> path = "path to a word document"
>>> os.path.isfile(path)
True
>>> os.path.splitext(path)[1] == ".docx" # test if the extension is .docx
True
TL;DR
The answer is: use the pathlib module
Pathlib is probably the most modern and convenient way for almost all of the file operations. For the existence of a file or a folder a single line of code is enough. If file is not exists, it will not throw any exception.
from pathlib import Path
if Path("myfile.txt").exists(): # works for both file and folders
# do your cool stuff...
The pathlib module was introduced in Python 3.4, so you need to have Python 3.4+. This library makes your life much easier while working with files and folders, and it is pretty to use. Here is more documentation about it: pathlib — Object-oriented filesystem paths.
BTW, if you are going to reuse the path, then it is better to assign it to a variable.
So it will become:
from pathlib import Path
p = Path("loc/of/myfile.txt")
if p.exists(): # works for both file and folders
# do stuffs...
#reuse 'p' if needed.
In 2016 the best way is still using os.path.isfile:
>>> os.path.isfile('/path/to/some/file.txt')
Or in Python 3 you can use pathlib:
import pathlib
path = pathlib.Path('/path/to/some/file.txt')
if path.is_file():
...
It doesn't seem like there's a meaningful functional difference between try/except and isfile(), so you should use which one makes sense.
If you want to read a file, if it exists, do
try:
f = open(filepath)
except IOError:
print 'Oh dear.'
But if you just wanted to rename a file if it exists, and therefore don't need to open it, do
if os.path.isfile(filepath):
os.rename(filepath, filepath + '.old')
If you want to write to a file, if it doesn't exist, do
# Python 2
if not os.path.isfile(filepath):
f = open(filepath, 'w')
# Python 3: x opens for exclusive creation, failing if the file already exists
try:
f = open(filepath, 'wx')
except IOError:
print 'file already exists'
If you need file locking, that's a different matter.
You could try this (safer):
try:
# http://effbot.org/zone/python-with-statement.htm
# 'with' is safer to open a file
with open('whatever.txt') as fh:
# Do something with 'fh'
except IOError as e:
print("({})".format(e))
The ouput would be:
([Errno 2] No such file or directory:
'whatever.txt')
Then, depending on the result, your program can just keep running from there or you can code to stop it if you want.
Date: 2017-12-04
Every possible solution has been listed in other answers.
An intuitive and arguable way to check if a file exists is the following:
import os
os.path.isfile('~/file.md') # Returns True if exists, else False
# Additionally, check a directory
os.path.isdir('~/folder') # Returns True if the folder exists, else False
# Check either a directory or a file
os.path.exists('~/file')
I made an exhaustive cheat sheet for your reference:
# os.path methods in exhaustive cheat sheet
{'definition': ['dirname',
'basename',
'abspath',
'relpath',
'commonpath',
'normpath',
'realpath'],
'operation': ['split', 'splitdrive', 'splitext',
'join', 'normcase'],
'compare': ['samefile', 'sameopenfile', 'samestat'],
'condition': ['isdir',
'isfile',
'exists',
'lexists'
'islink',
'isabs',
'ismount',],
'expand': ['expanduser',
'expandvars'],
'stat': ['getatime', 'getctime', 'getmtime',
'getsize']}
Although I always recommend using try and except statements, here are a few possibilities for you (my personal favourite is using os.access):
Try opening the file:
Opening the file will always verify the existence of the file. You can make a function just like so:
def File_Existence(filepath):
f = open(filepath)
return True
If it's False, it will stop execution with an unhanded IOError
or OSError in later versions of Python. To catch the exception,
you have to use a try except clause. Of course, you can always
use a try except` statement like so (thanks to hsandt
for making me think):
def File_Existence(filepath):
try:
f = open(filepath)
except IOError, OSError: # Note OSError is for later versions of Python
return False
return True
Use os.path.exists(path):
This will check the existence of what you specify. However, it checks for files and directories so beware about how you use it.
import os.path
>>> os.path.exists("this/is/a/directory")
True
>>> os.path.exists("this/is/a/file.txt")
True
>>> os.path.exists("not/a/directory")
False
Use os.access(path, mode):
This will check whether you have access to the file. It will check for permissions. Based on the os.py documentation, typing in os.F_OK, it will check the existence of the path. However, using this will create a security hole, as someone can attack your file using the time between checking the permissions and opening the file. You should instead go directly to opening the file instead of checking its permissions. (EAFP vs LBYP). If you're not going to open the file afterwards, and only checking its existence, then you can use this.
Anyway, here:
>>> import os
>>> os.access("/is/a/file.txt", os.F_OK)
True
I should also mention that there are two ways that you will not be able to verify the existence of a file. Either the issue will be permission denied or no such file or directory. If you catch an IOError, set the IOError as e (like my first option), and then type in print(e.args) so that you can hopefully determine your issue. I hope it helps! :)
If the file is for opening you could use one of the following techniques:
with open('somefile', 'xt') as f: # Using the x-flag, Python 3.3 and above
f.write('Hello\n')
if not os.path.exists('somefile'):
with open('somefile', 'wt') as f:
f.write("Hello\n")
else:
print('File already exists!')
Note: This finds either a file or a directory with the given name.
Additionally, os.access():
if os.access("myfile", os.R_OK):
with open("myfile") as fp:
return fp.read()
Being R_OK, W_OK, and X_OK the flags to test for permissions (doc).
if os.path.isfile(path_to_file):
try:
open(path_to_file)
pass
except IOError as e:
print "Unable to open file"
Raising exceptions is considered to be an acceptable, and Pythonic,
approach for flow control in your program. Consider handling missing
files with IOErrors. In this situation, an IOError exception will be
raised if the file exists but the user does not have read permissions.
Source: Using Python: How To Check If A File Exists
If you imported NumPy already for other purposes then there is no need to import other libraries like pathlib, os, paths, etc.
import numpy as np
np.DataSource().exists("path/to/your/file")
This will return true or false based on its existence.
Check file or directory exists
You can follow these three ways:
1. Using isfile()
Note 1: The os.path.isfile used only for files
import os.path
os.path.isfile(filename) # True if file exists
os.path.isfile(dirname) # False if directory exists
2. Using exists
Note 2: The os.path.exists is used for both files and directories
import os.path
os.path.exists(filename) # True if file exists
os.path.exists(dirname) # True if directory exists
3. The pathlib.Path method (included in Python 3+, installable with pip for Python 2)
from pathlib import Path
Path(filename).exists()
You can write Brian's suggestion without the try:.
from contextlib import suppress
with suppress(IOError), open('filename'):
process()
suppress is part of Python 3.4. In older releases you can quickly write your own suppress:
from contextlib import contextmanager
#contextmanager
def suppress(*exceptions):
try:
yield
except exceptions:
pass
I'm the author of a package that's been around for about 10 years, and it has a function that addresses this question directly. Basically, if you are on a non-Windows system, it uses Popen to access find. However, if you are on Windows, it replicates find with an efficient filesystem walker.
The code itself does not use a try block… except in determining the operating system and thus steering you to the "Unix"-style find or the hand-buillt find. Timing tests showed that the try was faster in determining the OS, so I did use one there (but nowhere else).
>>> import pox
>>> pox.find('*python*', type='file', root=pox.homedir(), recurse=False)
['/Users/mmckerns/.python']
And the doc…
>>> print pox.find.__doc__
find(patterns[,root,recurse,type]); Get path to a file or directory
patterns: name or partial name string of items to search for
root: path string of top-level directory to search
recurse: if True, recurse down from root directory
type: item filter; one of {None, file, dir, link, socket, block, char}
verbose: if True, be a little verbose about the search
On some OS, recursion can be specified by recursion depth (an integer).
patterns can be specified with basic pattern matching. Additionally,
multiple patterns can be specified by splitting patterns with a ';'
For example:
>>> find('pox*', root='..')
['/Users/foo/pox/pox', '/Users/foo/pox/scripts/pox_launcher.py']
>>> find('*shutils*;*init*')
['/Users/foo/pox/pox/shutils.py', '/Users/foo/pox/pox/__init__.py']
>>>
The implementation, if you care to look, is here:
https://github.com/uqfoundation/pox/blob/89f90fb308f285ca7a62eabe2c38acb87e89dad9/pox/shutils.py#L190
Adding one more slight variation which isn't exactly reflected in the other answers.
This will handle the case of the file_path being None or empty string.
def file_exists(file_path):
if not file_path:
return False
elif not os.path.isfile(file_path):
return False
else:
return True
Adding a variant based on suggestion from Shahbaz
def file_exists(file_path):
if not file_path:
return False
else:
return os.path.isfile(file_path)
Adding a variant based on suggestion from Peter Wood
def file_exists(file_path):
return file_path and os.path.isfile(file_path):
Here's a one-line Python command for the Linux command line environment. I find this very handy since I'm not such a hot Bash guy.
python -c "import os.path; print os.path.isfile('/path_to/file.xxx')"
You can use the "OS" library of Python:
>>> import os
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.txt")
True
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.tx")
False
How do I check whether a file exists, without using the try statement?
In 2016, this is still arguably the easiest way to check if both a file exists and if it is a file:
import os
os.path.isfile('./file.txt') # Returns True if exists, else False
isfile is actually just a helper method that internally uses os.stat and stat.S_ISREG(mode) underneath. This os.stat is a lower-level method that will provide you with detailed information about files, directories, sockets, buffers, and more. More about os.stat here
Note: However, this approach will not lock the file in any way and therefore your code can become vulnerable to "time of check to time of use" (TOCTTOU) bugs.
So raising exceptions is considered to be an acceptable, and Pythonic, approach for flow control in your program. And one should consider handling missing files with IOErrors, rather than if statements (just an advice).
Let's focus on one dll: C:\Windows\System32\wbem\wmiutils.dll. Why? Because it's the file in which I personally discovered Windows delivers a different dll depending on process architecture.
TLDR; Is there a way to programmatically determine the actual path of the dll that was returned by the file system redirector?
I understand that if launched as a x86 process, I get C:\Windows\SysWOW64\wbem\wmiutils.dll. And, if launched as a x64 process, I get C:\Windows\System32\wbem\wmiutils.dll.
I need to determine which wmiutils.dll I'm actually looking at. The redirector makes system32\wbem\wmiutils.dll look and feel identical but it's not. If I use parent path, I get C:\Windows\System32\wbem even though I may/may not be looking at C:\Windows\SysWOW64\wbem.
Any sweet python magic to make this happen? I can't seem to see anything from other languages I can port. Based on my use case, I've come up with a couple hacks but they're just that. Hoping somebody has found a solution as easy as parent path that actually works in this case.
import ctypes, hashlib
k32 = ctypes.windll.kernel32
oldValue = ctypes.c_long(0)
k32.Wow64DisableWow64FsRedirection(ctypes.byref(oldValue)) # Should open 32-bit
with open(r"C:\Windows\System32\wbem\wmiutil.dll", "rb") as f:
checksum32 = hashlib.md5(f.read()).hexdigest()
k32.Wow64RevertWow64FsRedirection(oldValue) # Should use what Windows thinks you need
with open(r"C:\Windows\System32\wbem\wmiutil.dll", "rb") as f:
checksum64 = hashlib.md5(f.read()).hexdigest()
if (checksum32 != checksum64):
print("You're running 64bit wmiutil dll")
I don't have Windows Python to test this, but it should work according to https://msdn.microsoft.com/en-us/library/windows/desktop/aa365745%28v=vs.85%29.aspx.
I think an easier way would be to just do some test like creating a struct and seeing if it's 8 bytes or 4 bytes. Then you can assume that Windows is using the 64-bit version of DLLs if it's 8 bytes.
On Windows 7 with Python 2.7 how can I detect if a path is a symbolic link?
This does not work os.path.islink(), it says it returns false if false or not supported and the path I'm providing is definitely a symbolic link so I'm assuming it's not supported on windows? What can I do?
The root problem is that you're using too old a version of Python. If you want to stick to 2.x, you will not be able to take advantage of new features added after early 2010.
One of those features is handling NTFS symlinks. That functionality was added in 3.2 in late 2010. (See the 3.2, 3.1, and 2.7 source for details.)
The reason Python didn't handle NTFS symlinks before then is that there was no such thing until late 2009. (IIRC, support was included in the 6.0 kernel, but userland support requires a service pack on Vista/2008; only 7/2008R2 and newer come with it built in. Plus, you need a new-enough MSVCRT to be able to access that userland support, and Python has an explicit policy of not upgrading to new Visual Studio versions within a minor release.)
The reason the code wasn't ported back to 2.x is that there will never be a 2.8, and bug fix releases like 2.7.3 (or 2.7.4) don't get new features, only bug fixes.
This has been reported as issue 13143, and the intended fix is to change the 2.7 docs to clarify that islink always returns False on Windows.
So, if you want to read NTFS symlinks under Windows, either upgrade to Python 3.2+, or you have to use win32api, ctypes, etc. to do it yourself.
Or, as Martijn Pieters suggests, instead of doing it yourself, use a third-party library like jaraco.windows that does it and/or borrow their code.
Or, if you really want, borrow the code from the 3.2 source and build a C extension module around it. If you trace down from ntpath to os to nt (which is actually posixmodule.c), I believe the guts of it are in win32_xstat_impl and win32_xstat_impl_w.
This is what I ended up using to determine if a file or a directory is a link in Windows 7:
from subprocess import check_output, CalledProcessError
import os.path
import ctypes
def isLink(path):
if os.path.exists(path):
if os.path.isdir(path):
FILE_ATTRIBUTE_REPARSE_POINT = 0x0400
attributes = ctypes.windll.kernel32.GetFileAttributesW(unicode(path))
return (attributes & FILE_ATTRIBUTE_REPARSE_POINT) > 0
else:
command = ['dir', path]
try:
with open(os.devnull, 'w') as NULL_FILE:
o0 = check_output(command, stderr=NULL_FILE, shell=True)
except CalledProcessError as e:
print e.output
return False
o1 = [s.strip() for s in o0.split('\n')]
if len(o1) < 6:
return False
else:
return 'SYMLINK' in o1[5]
else:
return False
EDIT: Modified code as per suggestions of Zitrax and Annan
EDIT: Added include statements as per the suggestion of shioko
For directories:
import os, ctypes
def IsSymlink(path):
FILE_ATTRIBUTE_REPARSE_POINT = 0x0400
return os.path.isdir(path) and (ctypes.windll.kernel32.GetFileAttributesW(unicode(path)) & FILE_ATTRIBUTE_REPARSE_POINT):
Source
You can also use the pywin32 module: GetFileAttributes is available in the win32api sub-module and FILE_ATTRIBUTE_REPARSE_POINT in the win32con module. For instance, to test if a given path is a symlink to a directory, the code becomes:
import os
import win32api
import win32con
def is_directory_symlink(path):
return bool(os.path.isdir(path)
and (win32api.GetFileAttributes(path) &
win32con.FILE_ATTRIBUTE_REPARSE_POINT))
If using Python 2 and the path may contain non-ascii characters, GetFileAttributes requires a unicode string. However, simply using unicode(path) will generally fail: you should test if path is a str and, if so, use its decode method.
Just using if file[-4:len(file)] != ".lnk": works for me
I need a way to determine the space remaining on a disk volume using python on linux, Windows and OS X. I'm currently parsing the output of the various system calls (df, dir) to accomplish this - is there a better way?
import ctypes
import os
import platform
import sys
def get_free_space_mb(dirname):
"""Return folder/drive free space (in megabytes)."""
if platform.system() == 'Windows':
free_bytes = ctypes.c_ulonglong(0)
ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(dirname), None, None, ctypes.pointer(free_bytes))
return free_bytes.value / 1024 / 1024
else:
st = os.statvfs(dirname)
return st.f_bavail * st.f_frsize / 1024 / 1024
Note that you must pass a directory name for GetDiskFreeSpaceEx() to work
(statvfs() works on both files and directories). You can get a directory name
from a file with os.path.dirname().
Also see the documentation for os.statvfs() and GetDiskFreeSpaceEx.
Install psutil using pip install psutil. Then you can get the amount of free space in bytes using:
import psutil
print(psutil.disk_usage(".").free)
You could use the wmi module for windows and os.statvfs for unix
for window
import wmi
c = wmi.WMI ()
for d in c.Win32_LogicalDisk():
print( d.Caption, d.FreeSpace, d.Size, d.DriveType)
for unix or linux
from os import statvfs
statvfs(path)
If you're running python3:
Using shutil.disk_usage()with os.path.realpath('/') name-regularization works:
from os import path
from shutil import disk_usage
print([i / 1000000 for i in disk_usage(path.realpath('/'))])
Or
total_bytes, used_bytes, free_bytes = disk_usage(path.realpath('D:\\Users\\phannypack'))
print(total_bytes / 1000000) # for Mb
print(used_bytes / 1000000)
print(free_bytes / 1000000)
giving you the total, used, & free space in MB.
If you dont like to add another dependency you can for windows use ctypes to call the win32 function call directly.
import ctypes
free_bytes = ctypes.c_ulonglong(0)
ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(u'c:\\'), None, None, ctypes.pointer(free_bytes))
if free_bytes.value == 0:
print 'dont panic'
From Python 3.3 you can use shutil.disk_usage("/").free from standard library for both Windows and UNIX :)
A good cross-platform way is using psutil: http://pythonhosted.org/psutil/#disks
(Note that you'll need psutil 0.3.0 or above).
You can use df as a cross-platform way. It is a part of GNU core utilities. These are the core utilities which are expected to exist on every operating system. However, they are not installed on Windows by default (Here, GetGnuWin32 comes in handy).
df is a command-line utility, therefore a wrapper required for scripting purposes.
For example:
from subprocess import PIPE, Popen
def free_volume(filename):
"""Find amount of disk space available to the current user (in bytes)
on the file system containing filename."""
stats = Popen(["df", "-Pk", filename], stdout=PIPE).communicate()[0]
return int(stats.splitlines()[1].split()[3]) * 1024
Below code returns correct value on windows
import win32file
def get_free_space(dirname):
secsPerClus, bytesPerSec, nFreeClus, totClus = win32file.GetDiskFreeSpace(dirname)
return secsPerClus * bytesPerSec * nFreeClus
The os.statvfs() function is a better way to get that information for Unix-like platforms (including OS X). The Python documentation says "Availability: Unix" but it's worth checking whether it works on Windows too in your build of Python (ie. the docs might not be up to date).
Otherwise, you can use the pywin32 library to directly call the GetDiskFreeSpaceEx function.
I Don't know of any cross-platform way to achieve this, but maybe a good workaround for you would be to write a wrapper class that checks the operating system and uses the best method for each.
For Windows, there's the GetDiskFreeSpaceEx method in the win32 extensions.
Most previous answers are correct, I'm using Python 3.10 and shutil.
My use case was Windows and C drive only ( but you should be able to extend this for you Linux and Mac as well (here is the documentation)
Here is the example for Windows:
import shutil
total, used, free = shutil.disk_usage("C:/")
print("Total: %d GiB" % (total // (2**30)))
print("Used: %d GiB" % (used // (2**30)))
print("Free: %d GiB" % (free // (2**30)))