What would be the best way in Python to determine whether a directory is writeable for the user executing the script? Since this will likely involve using the os module I should mention I'm running it under a *nix environment.
Although what Christophe suggested is a more Pythonic solution, the os module does have the os.access function to check access:
os.access('/path/to/folder', os.W_OK) # W_OK is for writing, R_OK for reading, etc.
It may seem strange to suggest this, but a common Python idiom is
It's easier to ask for forgiveness
than for permission
Following that idiom, one might say:
Try writing to the directory in question, and catch the error if you don't have the permission to do so.
My solution using the tempfile module:
import tempfile
import errno
def isWritable(path):
try:
testfile = tempfile.TemporaryFile(dir = path)
testfile.close()
except OSError as e:
if e.errno == errno.EACCES: # 13
return False
e.filename = path
raise
return True
Update:
After testing the code again on Windows I see that there is indeed an issue when using tempfile there, see issue22107: tempfile module misinterprets access denied error on Windows.
In the case of a non-writable directory, the code hangs for several seconds and finally throws an IOError: [Errno 17] No usable temporary file name found. Maybe this is what user2171842 was observing?
Unfortunately the issue is not resolved for now so to handle this, the error needs to be caught as well:
except (OSError, IOError) as e:
if e.errno == errno.EACCES or e.errno == errno.EEXIST: # 13, 17
The delay is of course still present in these cases then.
Stumbled across this thread searching for examples for someone. First result on Google, congrats!
People talk about the Pythonic way of doing it in this thread, but no simple code examples? Here you go, for anyone else who stumbles in:
import sys
filepath = 'C:\\path\\to\\your\\file.txt'
try:
filehandle = open( filepath, 'w' )
except IOError:
sys.exit( 'Unable to write to file ' + filepath )
filehandle.write("I am writing this text to the file\n")
This attempts to open a filehandle for writing, and exits with an error if the file specified cannot be written to: This is far easier to read, and is a much better way of doing it rather than doing prechecks on the file path or the directory, as it avoids race conditions; cases where the file becomes unwriteable between the time you run the precheck, and when you actually attempt to write to the file.
If you only care about the file perms, os.access(path, os.W_OK) should do what you ask for. If you instead want to know whether you can write to the directory, open() a test file for writing (it shouldn't exist beforehand), catch and examine any IOError, and clean up the test file afterwards.
More generally, to avoid TOCTOU attacks (only a problem if your script runs with elevated privileges -- suid or cgi or so), you shouldn't really trust these ahead-of-time tests, but drop privs, do the open(), and expect the IOError.
Check the mode bits:
import os, stat
def isWritable(dirname):
uid = os.geteuid()
gid = os.getegid()
s = os.stat(dirname)
mode = s[stat.ST_MODE]
return (
((s[stat.ST_UID] == uid) and (mode & stat.S_IWUSR)) or
((s[stat.ST_GID] == gid) and (mode & stat.S_IWGRP)) or
(mode & stat.S_IWOTH)
)
Here is something I created based on ChristopheD's answer:
import os
def isWritable(directory):
try:
tmp_prefix = "write_tester";
count = 0
filename = os.path.join(directory, tmp_prefix)
while(os.path.exists(filename)):
filename = "{}.{}".format(os.path.join(directory, tmp_prefix),count)
count = count + 1
f = open(filename,"w")
f.close()
os.remove(filename)
return True
except Exception as e:
#print "{}".format(e)
return False
directory = "c:\\"
if (isWritable(directory)):
print "directory is writable"
else:
print "directory is not writable"
if os.access(path_to_folder, os.W_OK) is not True:
print("Folder not writable")
else :
print("Folder writable")
more info about access can be find it here
I ran into this same need while adding an argument via argparse. The built in type=FileType('w') wouldn't work for me as I was looking for a directory. I ended up writing my own method to solve my problem. Here is the result with argparse snippet.
#! /usr/bin/env python
import os
import argparse
def writable_dir(dir):
if os.access(dir, os.W_OK) and os.path.isdir(dir):
return os.path.abspath(dir)
else:
raise argparse.ArgumentTypeError(dir + " is not writable or does not exist.")
parser = argparse.ArgumentParser()
parser.add_argument("-d","--dir", type=writable_dir, default='/tmp/',
help="Directory to use. Default: /tmp")
opts = parser.parse_args()
That results in the following:
$ python dir-test.py -h
usage: dir-test.py [-h] [-d DIR]
optional arguments:
-h, --help show this help message and exit
-d DIR, --dir DIR Directory to use. Default: /tmp
$ python dir-test.py -d /not/real
usage: dir-test.py [-h] [-d DIR]
dir-test.py: error: argument -d/--dir: /not/real is not writable or does not exist.
$ python dir-test.py -d ~
I went back and added print opts.dir to the end, and everything appears to be functioning as desired.
If you need to check the permission of another user (yes, I realize this contradicts the question, but may come in handy for someone), you can do it through the pwd module, and the directory's mode bits.
Disclaimer - does not work on Windows, as it doesn't use the POSIX permissions model (and the pwd module is not available there), e.g. - solution only for *nix systems.
Note that a directory has to have all the 3 bits set - Read, Write and eXecute.
Ok, R is not an absolute must, but w/o it you cannot list the entries in the directory (so you have to know their names). Execute on the other hand is absolutely needed - w/o it the user cannot read the file's inodes; so even having W, without X files cannot be created or modified. More detailed explanation at this link.
Finally, the modes are available in the stat module, their descriptions are in inode(7) man.
Sample code how to check:
import pwd
import stat
import os
def check_user_dir(user, directory):
dir_stat = os.stat(directory)
user_id, group_id = pwd.getpwnam(user).pw_uid, pwd.getpwnam(user).pw_gid
directory_mode = dir_stat[stat.ST_MODE]
# use directory_mode as mask
if user_id == dir_stat[stat.ST_UID] and stat.S_IRWXU & directory_mode == stat.S_IRWXU: # owner and has RWX
return True
elif group_id == dir_stat[stat.ST_GID] and stat.S_IRWXG & directory_mode == stat.S_IRWXG: # in group & it has RWX
return True
elif stat.S_IRWXO & directory_mode == stat.S_IRWXO: # everyone has RWX
return True
# no permissions
return False
check if a stat object is readable or writable
useful to check if a pipe is readable or writable
import os, stat
def check_access(s, check="r"):
"check if s=os.stat(path) is readable or writable"
u = os.geteuid(); g = os.getegid(); m = s.st_mode
if check == "r":
return (
((s[stat.ST_UID] == u) and (m & stat.S_IRUSR)) or
((s[stat.ST_GID] == g) and (m & stat.S_IRGRP)) or
(m & stat.S_IROTH)
) != 0
if check == "w":
return (
((s[stat.ST_UID] == u) and (m & stat.S_IWUSR)) or
((s[stat.ST_GID] == g) and (m & stat.S_IWGRP)) or
(m & stat.S_IWOTH)
) != 0
s = os.stat(0) # fd 0 == stdin
print(f"fd 0 is readable?", check_access(s, "r"))
s = os.stat(1) # fd 1 == stdout
print(f"fd 1 is writable?", check_access(s, "w"))
os.write(1, b"hello\n")
based on the answer by Joe Koberg
see also: same thing in javascript
probably breaks on windows
Related
The docs of shutil tells me:
Even the higher-level file copying functions (shutil.copy(), shutil.copy2()) can’t copy all file metadata. On POSIX platforms, this means that file owner and group are lost as well as ACLs
How can I keep the file owner and group if I need to copy a file in python?
The process runs on linux as root.
Update: We don't use ACLs. We only need to keep the stuff which is preserved with tools like tar and rsync.
You perhaps could use os.stat to get the guid and uid like in this answer and then reset the uid and guid after coping using os.chown.
I did it this way:
import os
import stat
import shutil
def copyComplete(source, target):
# copy content, stat-info (mode too), timestamps...
shutil.copy2(source, target)
# copy owner and group
st = os.stat(source)
os.chown(target, st.st_uid, st.st_gid)
You can use the subprocess module:
from subprocess import Popen
p = Popen(['cp','-p','--preserve',src,dest])
p.wait()
See Keeping fileowner and permissions after copying file in C on how cp itself does this, then just replicate its logic. Python has all the system calls mentioned in its os module.
I suggest you use the OS and subprocess modules. This will only work in Unix, but it should work well. Use the os.fchown to change file ownership, and subprocess.Popen() to pipe ls -l into a variable to read ownership. For one file, the permissions reader would look like this:
import os
import subprocess
def readfile(filepath):
process = subprocess.Popen(["ls","-l"],stdout=subprocess.PIPE)
output = process.communicate()[0]
output = output.split()
return (output[0],output[1]) #insert into this tuple the indexing for the words you want from ls -l
and for the uid setter (just a wrapper for os function):
def setuid(filepath,uid,gid):
os.chown(filepath,uid,gid)
as #Thom Wiggers 's answer
os.stat get uid,gid
os.chmod set uid,gid
demo code:
import os
st = os.stat(src_path)
os.chown(dst_path, st.st_uid, st.st_gid)
check except:
import os
def copy_owner_group(src, dst):
try:
st = os.stat(src)
except Exception as e:
print 'stat except:', e
else:
try:
os.chown(dst, st.st_uid, st.st_gid)
except Exception as e:
print 'chmod except:', e
I'm trying to add to our existing pre-commit SVN hook so that it will check for and block an increase in file size for files in specific directory/s.
I've written a python script to compare two file sizes, which takes two files as arguments and uses sys.exit(0) or (1) to return the result, this part seems to work fine.
My problem is in calling the python script from the batch file, how to reference the newly committed and previous versions of each file? The existing code is new to me and a mess of %REPOS%, %TXN%s etc and I'm not sure how to go about using them. Is there a simple, standard way of doing this?
It also already contains code to loop through the changed files using svnlook changed, so that part shouldn't be an issue.
Thanks very much
If comparing file sizes is all you need to do, look no further than the svnlook filesize command. The default invocation - svnlook filesize repo path - will give you the size of the HEAD revision of path. To get the size of the path in the incoming commit use svnlook filesize repo path -t argv[2].
Still, here is an example of listing all revisions of a versioned path (except the incoming one, since this is pre-commit hook).
#!/usr/bin/env python
from sys import argv, stderr, exit
from subprocess import check_output
repo = argv[1]
transaction = argv[2]
def path_history(path, limit=5):
path = '/%s' % path
cmd = ('svnlook', 'history', '-l', str(limit), repo, path)
out = check_output(cmd).splitlines()[2:]
for rev, _path in (i.split() for i in out):
if _path == path:
yield rev
def commit_changes():
cmd = ('svnlook', 'changed', repo, '-t', transaction)
out = check_output(cmd).splitlines()
for line in out:
yield line.split()
def filesize(path, rev=None, trans=None):
cmd = ['svnlook', 'filesize', repo, path]
if rev: cmd.extend(('-r', str(rev)))
elif trans: cmd.extend(('-t', str(trans)))
out = check_output(cmd)
return out.rstrip()
def filesize_catwc(path, rev=None, trans=None):
'''A `svnlook filesize` substitute for older versions of svn.
Uses `svnlook cat ... | wc -c` and should be very inefficient
for large files.'''
arg = '-r %s' % rev if rev else '-t %s' % trans
cmd = 'svnlook cat %s %s %s | wc -c' % (arg, repo, path)
out = check_output(cmd, shell=True)
return out.rstrip()
for status, path in commit_changes():
if status in ('A', 'M', 'U'):
# get the last 5 revisions of the added/modified path
revisions = list(path_history(path))
headrev = revisions[0]
oldsize = filesize(path, rev=headrev)
newsize = filesize(path, trans=transaction)
It is probably easier to write a whole pre-commit script in python. According to the subversion handbook, there are three inputs to pre-commit;
Two command line arguments
repository path
commit transaction name
lock-token info on standard input
If you want to know which files have changed, I suggest you use the subprocess.check_output() function to call svnlook changed. For the files which contents have changed, you should call svnlook filesize, to get the size of the file as it is in the last revision in the repository. The size of the equivalent file in the working directory you'd have to query with os.stat(), as shown in the function getsizes().
import subprocess
import sys
import os
repo = sys.argv[1]
commit_name = sys.argv[2]
def getsizes(rname, rfile):
'''Get the size of the file in rfile from the repository rname.
Derive the filename in the working directory from rfile, and use
os.stat to get the filesize. Return the two sizes.
'''
localname = rfile[10:].strip() # 'U trunk/foo/bar.txt' -> 'foo/bar.txt'
reposize = subprocess.check_output(['svnlook', 'filesize', rname, rfile])
reposize = int(reposize)
statinfo = os.stat(localname)
return (reposize, statinfo.st_size)
lines = subprocess.check_output(['svnlook', 'changed', repo]).splitlines()
for line in lines:
if line.startswith('U ') or line.startswith('UU'):
# file contents have changed
reposize, wdsize = getsizes(repo, line)
# do something with the sizes here...
elif line.startswith('_U'):
# properties have changed
pass
This question already has answers here:
Test if executable exists in Python?
(15 answers)
Closed 4 years ago.
How do I check if a program exists from a python script?
Let's say you want to check if wget or curl are available. We'll assume that they should be in path.
It would be the best to see a multiplatform solution but for the moment, Linux is enough.
Hints:
running the command and checking for return code is not always enough as some tools do return non 0 result even when you try --version.
nothing should be visible on screen when checking for the command
Also, I would appreciate a solution that that is more general, like is_tool(name)
shutil.which
Let me recommend an option that has not been discussed yet: a Python implementation of which, specifically shutil.which. It was introduced in Python 3.3 and is cross-platform, supporting Linux, Mac, and Windows. It is also available in Python 2.x via whichcraft. You can also just rip the code for which right out of whichcraft here and insert it into your program.
def is_tool(name):
"""Check whether `name` is on PATH and marked as executable."""
# from whichcraft import which
from shutil import which
return which(name) is not None
distutils.spawn.find_executable
Another option that has already been mentioned is distutils.spawn.find_executable.
find_executable's docstring is as follows:
Tries to find 'executable' in the directories listed in 'path'
So if you pay attention, you'll note that the name of the function is somewhat misleading. Unlike which, find_executable does not actually verify that executable is marked as executable, only that it is on the PATH. So it's entirely possible (however unlikely) that find_executable indicates a program is available when it is not.
For example, suppose you have a file /usr/bin/wget that is not marked executable. Running wget from the shell will result in the following error: bash: /usr/bin/wget: Permission denied. which('wget') is not None will return False, yet find_executable('wget') is not None will return True. You can probably get away with using either function, but this is just something to be aware of with find_executable.
def is_tool(name):
"""Check whether `name` is on PATH."""
from distutils.spawn import find_executable
return find_executable(name) is not None
The easiest way is to try to run the program with the desired parameters, and handle the exception if it doesn't exist:
try:
subprocess.call(["wget", "your", "parameters", "here"])
except FileNotFoundError:
# handle file not found error.
This is a common pattern in Python: EAFP
In Python 2, you had to catch OsError instead, since the more fine-grained exception classes for OS errors did not exist yet:
try:
subprocess.call(["wget", "your", "parameters", "here"])
except OSError as e:
if e.errno == errno.ENOENT:
# handle file not found error.
else:
# Something else went wrong while trying to run `wget`
raise
You could use a subprocess call to the binary needed with :
"which" : *nix
"where" : Win 2003 and later (Xp has an addon)
to get the executable path (supposing it is in the environment path).
import os
import platform
import subprocess
cmd = "where" if platform.system() == "Windows" else "which"
try:
subprocess.call([cmd, your_executable_to_check_here])
except:
print "No executable"
or just use Ned Batchelder's wh.py script, that is a "which" cross platform implementation:
http://nedbatchelder.com/code/utilities/wh_py.html
import subprocess
import os
def is_tool(name):
try:
devnull = open(os.devnull)
subprocess.Popen([name], stdout=devnull, stderr=devnull).communicate()
except OSError as e:
if e.errno == os.errno.ENOENT:
return False
return True
I would probably shell out to which wget or which curl and check that the result ends in the name of the program you are using. The magic of unix :)
Actually, all you need to do is check the return code of which. So... using our trusty subprocess module:
import subprocess
rc = subprocess.call(['which', 'wget'])
if rc == 0:
print('wget installed!')
else:
print('wget missing in path!')
Note that I tested this on windows with cygwin... If you want to figure out how to implement which in pure python, i suggest you check here: http://pypi.python.org/pypi/pycoreutils (oh dear - it seems they don't supply which. Time for a friendly nudge?)
UPDATE: On Windows, you can use where instead of which for a similar effect.
I'd go for:
import distutils.spawn
def is_tool(name):
return distutils.spawn.find_executable(name) is not None
I'd change #sorin's answer as follows, the reason is it would check the name of the program without passing the absolute path of the program
from subprocess import Popen, PIPE
def check_program_exists(name):
p = Popen(['/usr/bin/which', name], stdout=PIPE, stderr=PIPE)
p.communicate()
return p.returncode == 0
import os
import subprocess
def is_tool(prog):
for dir in os.environ['PATH'].split(os.pathsep):
if os.path.exists(os.path.join(dir, prog)):
try:
subprocess.call([os.path.join(dir, prog)],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
except OSError, e:
return False
return True
return False
A slight modification to #SvenMarnach's code that addresses the issue of printing to the standard output stream. If you use the subprocess.check_output() function rather than subprocess.call() then you can handle the string that is normally printed to standard out in your code and still catch exceptions and the exit status code.
If you want to suppress the standard output stream in the terminal, don’t print the std out string that is returned from check_output:
import subprocess
import os
try:
stdout_string = subprocess.check_output(["wget", "--help"], stderr=subprocess.STDOUT)
# print(stdout_string)
except subprocess.CalledProcessError as cpe:
print(cpe.returncode)
print(cpe.output)
except OSError as e:
if e.errno == os.errno.ENOENT:
print(e)
else:
# Something else went wrong while trying to run `wget`
print(e)
The non-zero exit status code and output string are raised in the CalledProcessError as subprocess.CalledProcessError.returncode and subprocess.CalledProcessError.output so you can do whatever you'd like with them.
If you want to print the executable's standard output to the terminal, print the string that is returned:
import subprocess
import os
try:
stdout_string = subprocess.check_output(["wget", "--help"], stderr=subprocess.STDOUT)
print(stdout_string)
except subprocess.CalledProcessError as cpe:
print(cpe.returncode)
print(cpe.output)
except OSError as e:
if e.errno == os.errno.ENOENT:
print(e)
else:
# Something else went wrong while trying to run `wget`
print(e)
print() adds an extra newline to the string. If you want to eliminate that (and write std error to the std err stream instead of the std out stream as shown with the print() statements above), use sys.stdout.write(string) and sys.stderr.write(string) instead of print():
import subprocess
import os
import sys
try:
stdout_string = subprocess.check_output(["bogus"], stderr=subprocess.STDOUT)
sys.stdout.write(stdout_string)
except subprocess.CalledProcessError as cpe:
sys.stderr.write(cpe.returncode)
sys.stderr.write(cpe.output)
except OSError as e:
if e.errno == os.errno.ENOENT:
sys.stderr.write(e.strerror)
else:
# Something else went wrong while trying to run `wget`
sys.stderr.write(e.strerror)
This is probably a bit of a silly excercise for me, but it raises a bunch of interesting questions. I have a directory of logfiles from my chat client, and I want to be notified using notify-osd every time one of them changes.
The script that I wrote basically uses os.popen to run the linux tail command on every one of the files to get the last line, and then check each line against a dictionary of what the lines were the last time it ran. If the line changed, it used pynotify to send me a notification.
This script actually worked perfectly, except for the fact that it used a huge amount of cpu (probably because it was running tail about 16 times every time the loop ran, on files that were mounted over sshfs.)
It seems like something like this would be a great solution, but I don't see how to implement that for more than one file.
Here is the script that I wrote. Pardon my lack of comments and poor style.
Edit: To clarify, this is all linux on a desktop.
Not even looking at your source code, there are two ways you could easily do this more efficiently and handle multiple files.
Don't bother running tail unless you have to. Simply os.stat all of the files and record the last modified time. If the last modified time is different, then raise a notification.
Use pyinotify to call out to Linux's inotify facility; this will have the kernel do option 1 for you and call back to you when any files in your directory change. Then translate the callback into your osd notification.
Now, there might be some trickiness depending on how many notifications you want when there are multiple messages and whether you care about missing a notification for a message.
An approach that preserves the use of tail would be to instead use tail -f. Open all of the files with tail -f and then use the select module to have the OS tell you when there's additional input on one of the file descriptors open for tail -f. Your main loop would call select and then iterate over each of the readable descriptors to generate notifications. (You could probably do this without using tail and just calling readline() when it's readable.)
Other areas of improvement in your script:
Use os.listdir and native Python filtering (say, using list comprehensions) instead of a popen with a bunch of grep filters.
Update the list of buffers to scan periodically instead of only doing it at program boot.
Use subprocess.popen instead of os.popen.
If you're already using the pyinotify module, it's easy to do this in pure Python (i.e. no need to spawn a separate process to tail each file).
Here is an example that is event-driven by inotify, and should use very little cpu. When IN_MODIFY occurs for a given path we read all available data from the file handle and output any complete lines found, buffering the incomplete line until more data is available:
import os
import select
import sys
import pynotify
import pyinotify
class Watcher(pyinotify.ProcessEvent):
def __init__(self, paths):
self._manager = pyinotify.WatchManager()
self._notify = pyinotify.Notifier(self._manager, self)
self._paths = {}
for path in paths:
self._manager.add_watch(path, pyinotify.IN_MODIFY)
fh = open(path, 'rb')
fh.seek(0, os.SEEK_END)
self._paths[os.path.realpath(path)] = [fh, '']
def run(self):
while True:
self._notify.process_events()
if self._notify.check_events():
self._notify.read_events()
def process_default(self, evt):
path = evt.pathname
fh, buf = self._paths[path]
data = fh.read()
lines = data.split('\n')
# output previous incomplete line.
if buf:
lines[0] = buf + lines[0]
# only output the last line if it was complete.
if lines[-1]:
buf = lines[-1]
lines.pop()
# display a notification
notice = pynotify.Notification('%s changed' % path, '\n'.join(lines))
notice.show()
# and output to stdout
for line in lines:
sys.stdout.write(path + ': ' + line + '\n')
sys.stdout.flush()
self._paths[path][1] = buf
pynotify.init('watcher')
paths = sys.argv[1:]
Watcher(paths).run()
Usage:
% python watcher.py [path1 path2 ... pathN]
Simple pure python solution (not the best, but doesn't fork, spits out 4 empty lines after idle period and marks everytime the source of the chunk, if changed):
#!/usr/bin/env python
from __future__ import with_statement
'''
Implement multi-file tail
'''
import os
import sys
import time
def print_file_from(filename, pos):
with open(filename, 'rb') as fh:
fh.seek(pos)
while True:
chunk = fh.read(8192)
if not chunk:
break
sys.stdout.write(chunk)
def _fstat(filename):
st_results = os.stat(filename)
return (st_results[6], st_results[8])
def _print_if_needed(filename, last_stats, no_fn, last_fn):
changed = False
#Find the size of the file and move to the end
tup = _fstat(filename)
# print tup
if last_stats[filename] != tup:
changed = True
if not no_fn and last_fn != filename:
print '\n<%s>' % filename
print_file_from(filename, last_stats[filename][0])
last_stats[filename] = tup
return changed
def multi_tail(filenames, stdout=sys.stdout, interval=1, idle=10, no_fn=False):
S = lambda (st_size, st_mtime): (max(0, st_size - 124), st_mtime)
last_stats = dict((fn, S(_fstat(fn))) for fn in filenames)
last_fn = None
last_print = 0
while 1:
# print last_stats
changed = False
for filename in filenames:
if _print_if_needed(filename, last_stats, no_fn, last_fn):
changed = True
last_fn = filename
if changed:
if idle > 0:
last_print = time.time()
else:
if idle > 0 and last_print is not None:
if time.time() - last_print >= idle:
last_print = None
print '\n' * 4
time.sleep(interval)
if '__main__' == __name__:
from optparse import OptionParser
op = OptionParser()
op.add_option('-F', '--no-fn', help="don't print filename when changes",
default=False, action='store_true')
op.add_option('-i', '--idle', help='idle time, in seconds (0 turns off)',
type='int', default=10)
op.add_option('--interval', help='check interval, in seconds', type='int',
default=1)
opts, args = op.parse_args()
try:
multi_tail(args, interval=opts.interval, idle=opts.idle,
no_fn=opts.no_fn)
except KeyboardInterrupt:
pass
cd is the shell command to change the working directory.
How do I change the current working directory in Python?
You can change the working directory with:
import os
os.chdir(path)
There are two best practices to follow when using this method:
Catch the exception (WindowsError, OSError) on invalid path. If the exception is thrown, do not perform any recursive operations, especially destructive ones. They will operate on the old path and not the new one.
Return to your old directory when you're done. This can be done in an exception-safe manner by wrapping your chdir call in a context manager, like Brian M. Hunt did in his answer.
Changing the current working directory in a subprocess does not change the current working directory in the parent process. This is true of the Python interpreter as well. You cannot use os.chdir() to change the CWD of the calling process.
Here's an example of a context manager to change the working directory. It is simpler than an ActiveState version referred to elsewhere, but this gets the job done.
Context Manager: cd
import os
class cd:
"""Context manager for changing the current working directory"""
def __init__(self, newPath):
self.newPath = os.path.expanduser(newPath)
def __enter__(self):
self.savedPath = os.getcwd()
os.chdir(self.newPath)
def __exit__(self, etype, value, traceback):
os.chdir(self.savedPath)
Or try the more concise equivalent(below), using ContextManager.
Example
import subprocess # just to call an arbitrary command e.g. 'ls'
# enter the directory like this:
with cd("~/Library"):
# we are in ~/Library
subprocess.call("ls")
# outside the context manager we are back wherever we started.
cd() is easy to write using a generator and a decorator.
from contextlib import contextmanager
import os
#contextmanager
def cd(newdir):
prevdir = os.getcwd()
os.chdir(os.path.expanduser(newdir))
try:
yield
finally:
os.chdir(prevdir)
Then, the directory is reverted even after an exception is thrown:
os.chdir('/home')
with cd('/tmp'):
# ...
raise Exception("There's no place like /home.")
# Directory is now back to '/home'.
I would use os.chdir like this:
os.chdir("/path/to/change/to")
By the way, if you need to figure out your current path, use os.getcwd().
More here
If you're using a relatively new version of Python, you can also use a context manager, such as this one:
from __future__ import with_statement
from grizzled.os import working_directory
with working_directory(path_to_directory):
# code in here occurs within the directory
# code here is in the original directory
UPDATE
If you prefer to roll your own:
import os
from contextlib import contextmanager
#contextmanager
def working_directory(directory):
owd = os.getcwd()
try:
os.chdir(directory)
yield directory
finally:
os.chdir(owd)
As already pointed out by others, all the solutions above only change the working directory of the current process. This is lost when you exit back to the Unix shell. If desperate you can change the parent shell directory on Unix with this horrible hack:
def quote_against_shell_expansion(s):
import pipes
return pipes.quote(s)
def put_text_back_into_terminal_input_buffer(text):
# use of this means that it only works in an interactive session
# (and if the user types while it runs they could insert characters between the characters in 'text'!)
import fcntl, termios
for c in text:
fcntl.ioctl(1, termios.TIOCSTI, c)
def change_parent_process_directory(dest):
# the horror
put_text_back_into_terminal_input_buffer("cd "+quote_against_shell_expansion(dest)+"\n")
os.chdir() is the right way.
os.chdir() is the Pythonic version of cd.
import os
abs_path = 'C://a/b/c'
rel_path = './folder'
os.chdir(abs_path)
os.chdir(rel_path)
You can use both with os.chdir(abs_path) or os.chdir(rel_path), there's no need to call os.getcwd() to use a relative path.
Further into direction pointed out by Brian and based on sh (1.0.8+)
from sh import cd, ls
cd('/tmp')
print ls()
If You would like to perform something like "cd.." option, just type:
os.chdir("..")
it is the same as in Windows cmd: cd..
Of course import os is neccessary (e.g type it as 1st line of your code)
The Path objects in path (a third-party package available on PyPI, different from pathlib) offer both a context manager and a chdir method for this purpose:
from path import Path # pip install path
with Path("somewhere"):
...
Path("somewhere").chdir()
If you use spyder and love GUI, you can simply click on the folder button on the upper right corner of your screen and navigate through folders/directories you want as current directory.
After doing so you can go to the file explorer tab of the window in spyder IDE and you can see all the files/folders present there.
to check your current working directory
go to the console of spyder IDE and simply type
pwd
it will print the same path as you have selected before.
Changing the current directory of the script process is trivial. I think the question is actually how to change the current directory of the command window from which a python script is invoked, which is very difficult. A Bat script in Windows or a Bash script in a Bash shell can do this with an ordinary cd command because the shell itself is the interpreter. In both Windows and Linux Python is a program and no program can directly change its parent's environment. However the combination of a simple shell script with a Python script doing most of the hard stuff can achieve the desired result. For example, to make an extended cd command with traversal history for backward/forward/select revisit, I wrote a relatively complex Python script invoked by a simple bat script. The traversal list is stored in a file, with the target directory on the first line. When the python script returns, the bat script reads the first line of the file and makes it the argument to cd. The complete bat script (minus comments for brevity) is:
if _%1 == _. goto cdDone
if _%1 == _? goto help
if /i _%1 NEQ _-H goto doCd
:help
echo d.bat and dSup.py 2016.03.05. Extended chdir.
echo -C = clear traversal list.
echo -B or nothing = backward (to previous dir).
echo -F or - = forward (to next dir).
echo -R = remove current from list and return to previous.
echo -S = select from list.
echo -H, -h, ? = help.
echo . = make window title current directory.
echo Anything else = target directory.
goto done
:doCd
%~dp0dSup.py %1
for /F %%d in ( %~dp0dSupList ) do (
cd %%d
if errorlevel 1 ( %~dp0dSup.py -R )
goto cdDone
)
:cdDone
title %CD%
:done
The python script, dSup.py is:
import sys, os, msvcrt
def indexNoCase ( slist, s ) :
for idx in range( len( slist )) :
if slist[idx].upper() == s.upper() :
return idx
raise ValueError
# .........main process ...................
if len( sys.argv ) < 2 :
cmd = 1 # No argument defaults to -B, the most common operation
elif sys.argv[1][0] == '-':
if len(sys.argv[1]) == 1 :
cmd = 2 # '-' alone defaults to -F, second most common operation.
else :
cmd = 'CBFRS'.find( sys.argv[1][1:2].upper())
else :
cmd = -1
dir = os.path.abspath( sys.argv[1] ) + '\n'
# cmd is -1 = path, 0 = C, 1 = B, 2 = F, 3 = R, 4 = S
fo = open( os.path.dirname( sys.argv[0] ) + '\\dSupList', mode = 'a+t' )
fo.seek( 0 )
dlist = fo.readlines( -1 )
if len( dlist ) == 0 :
dlist.append( os.getcwd() + '\n' ) # Prime new directory list with current.
if cmd == 1 : # B: move backward, i.e. to previous
target = dlist.pop(0)
dlist.append( target )
elif cmd == 2 : # F: move forward, i.e. to next
target = dlist.pop( len( dlist ) - 1 )
dlist.insert( 0, target )
elif cmd == 3 : # R: remove current from list. This forces cd to previous, a
# desireable side-effect
dlist.pop( 0 )
elif cmd == 4 : # S: select from list
# The current directory (dlist[0]) is included essentially as ESC.
for idx in range( len( dlist )) :
print( '(' + str( idx ) + ')', dlist[ idx ][:-1])
while True :
inp = msvcrt.getche()
if inp.isdigit() :
inp = int( inp )
if inp < len( dlist ) :
print( '' ) # Print the newline we didn't get from getche.
break
print( ' is out of range' )
# Select 0 means the current directory and the list is not changed. Otherwise
# the selected directory is moved to the top of the list. This can be done by
# either rotating the whole list until the selection is at the head or pop it
# and insert it to 0. It isn't obvious which would be better for the user but
# since pop-insert is simpler, it is used.
if inp > 0 :
dlist.insert( 0, dlist.pop( inp ))
elif cmd == -1 : # -1: dir is the requested new directory.
# If it is already in the list then remove it before inserting it at the head.
# This takes care of both the common case of it having been recently visited
# and the less common case of user mistakenly requesting current, in which
# case it is already at the head. Deleting and putting it back is a trivial
# inefficiency.
try:
dlist.pop( indexNoCase( dlist, dir ))
except ValueError :
pass
dlist = dlist[:9] # Control list length by removing older dirs (should be
# no more than one).
dlist.insert( 0, dir )
fo.truncate( 0 )
if cmd != 0 : # C: clear the list
fo.writelines( dlist )
fo.close()
exit(0)