This question already has answers here:
Deleting read-only directory in Python
(7 answers)
Closed 3 years ago.
I'm trying to have python delete some directories and I get access errors on them. I think its that the python user account doesn't have rights?
WindowsError: [Error 5] Access is denied: 'path'
is what I get when I run the script.
I've tried
shutil.rmtree
os.remove
os.rmdir
they all return the same error.
We've had issues removing files and directories on Windows, even if we had just copied them, if they were set to 'readonly'. shutil.rmtree() offers you sort of exception handlers to handle this situation. You call it and provide an exception handler like this:
import errno, os, stat, shutil
def handleRemoveReadonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise
shutil.rmtree(filename, ignore_errors=False, onerror=handleRemoveReadonly)
You might want to try that.
I've never used Python, but I would assume it runs as whatever user executes the script.
The scripts have no special user, they just run under the currently logged-in user which executed the script.
Have you tried checking that:
you are trying to delete a valid path? and that
the path has no locked files?
How are you running the script? From an interactive console session? If so, just open up a DOS command window (using cmd) and type 'whoami'. That is who you are running the scripts interactively.
Ok I saw your edits just now...why don't you print the path and check the properties to see if the user account running the scripts has the required privileges?
If whoami does not work on your version of Windows, you may use the environment variables like SET USERNAME and SET DOMAINNAME from your command window.
#ThomasH : another brick to the wall.
On unix systems, you have to ensure that parent directory is writeable too.
Here is another version :
def remove_readonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
# ensure parent directory is writeable too
pardir = os.path.abspath(os.path.join(path, os.path.pardir))
if not os.access(pardir, os.W_OK):
os.chmod(pardir, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO)
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise
If the script is being run as a scheduled task (which seems likely for a cleanup script), it will probably run as SYSTEM. It's (unwise, but) possible to set permissions on directories so that SYSTEM has no access.
Simple solution after searching for hours is to check first if that folder actually exist!
GIT_DIR="C:/Users/...."
if os.path.exists(GIT_DIR):
shutil.rmtree(GIT_DIR)
This did the trick for me.
Related
I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?
For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.
I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)
How can i get the current directory to which I am in? like the use of
os.getcwd()
First, I presume you're not asking about a particular subprocess that exists simply to tell you the current working directory and do nothing else (Apducer's answer). If that were the case you could simply as os.getcwd() and forget the subprocess. You clearly already know that. So you must be dealing with some other (arbitrary?) subprocess.
Second, I presume you understand, via dr1fter's answer, that you have control over the working directory in which the subprocess starts. I suspect that's not enough for you.
Rather, I suspect you're thinking that the subprocess might, according to its own internal logic, have changed its working directory sometime since its launch, that you can't predict where it has ended up, and you want to be able to send some sort of signal to the subprocess at an arbitrary time, to interrogate it about where it's currently working. In general, this is only possible if the process has been specifically programmed with the logic that receives such a signal (through whatever route) and issues such a response. I think that's what SuperStew meant by the comment, "isn't that going to depend on the subprocess?"
I say "in general" because there are platform-specific approaches. For example, see:
windows batch command to determine working directory of a process
How do I print the current working directory of another user in linux?
by default, subprocesses you spawn inherit your PWD. you can however, specify the cwd argument to the subprocess.Popen c'tor to set a different initial PWD.
Unix (Linux, MacOS):
import subprocess
arguments = ['pwd']
directory = subprocess.check_output(arguments)
Windows:
import subprocess
arguments = ['cd']
directory = subprocess.check_output(arguments)
If you want to run in both types of OS, you'll have to check the machine OS:
import os
import subprocess
if os.name == 'nt': # Windows
arguments = ['cd']
else: # other (unix)
arguments = ['pwd']
directory = subprocess.check_output(arguments)
I am trying to delete a collection of folders on my drive. These directories are not empty. I have come up with a solution as follows:
import shutil
import os
path = "main/"
folderList = ['Blah', 'Blah', 'Blah'];
print ("Cleaning Project at %s" % path)
for c in folderList:
strippedPath = (path + c).strip("\n")
print ("Cleaning path " + strippedPath)
if os.path.exists(strippedPath):
try:
shutil.rmtree(strippedPath)
except OSError as why:
pass
print ("Done Cleaning Project")
The problem is that without the try / catch I get a error that says
PermissionError: [WinError 5] Access is denied: 'PathToFileHere'
Pressing the delete key on windows will work fine. Can someone provide me a command that will remove this directory without errors?
First you should avoid to silently swallow an Exception, but at least print or log it. But many thing can happen to a file, they may have Hidden, System or ReadOnly attributes. The current user may not have permissions on files but only on the containing folder. As Python is multi-platform its high-level commands can be less optimized for a particular OS (Windows in your case) than native ones.
You should first try to confirm that in a cmd window, the command rd /s folder correctly remove the folder that shutil.rmtree fails to delete, and if yes ask python so execute it vie the subprocess module :
subprocess.call("rd /s/q " + strippedPath)
I see that if we change the HOME (linux) or USERPROFILE (windows) environmental variable and run a python script, it returns the new value as the user home when I try
os.environ['HOME']
os.exp
Is there any way to find the real user home directory without relying on the environmental variable?
edit:
Here is a way to find userhome in windows by reading in the registry,
http://mail.python.org/pipermail/python-win32/2008-January/006677.html
edit:
One way to find windows home using pywin32,
from win32com.shell import shell,shellcon
home = shell.SHGetFolderPath(0, shellcon.CSIDL_PROFILE, None, 0)
I think os.path.expanduser(path) could be helpful.
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
On Unix, an initial ~ is replaced by the environment variable HOME if it is set; otherwise the current user’s home directory is looked up in the password directory through the built-in module pwd. An initial ~user is looked up directly in the password directory.
On Windows, HOME and USERPROFILE will be used if set, otherwise a combination of HOMEPATH and HOMEDRIVE will be used. An initial ~user is handled by stripping the last directory component from the created user path derived above.
If the expansion fails or if the path does not begin with a tilde, the path is returned unchanged.
So you could just do:
os.path.expanduser('~user')
from pathlib import Path
str(Path.home())
works in Python 3.5 and above. Path.home() returns a Path object providing an API I find very useful.
I think os.path.expanduser(path) is the best answer to your question, but there's an alternative that may be worth mentioning in the Unix world: the pwd package. e.g.
import os, pwd
pwd.getpwuid(os.getuid()).pw_dir
For windows;
import os
homepath = os.path.expanduser(os.getenv('USERPROFILE'))
will give you a handle to current user's home directory and
filepath = os.path.expanduser(os.getenv('USERPROFILE'))+'\\Documents\\myfile.txt'
will give you a handle to below file;
C:\Users\urUserName\Documents\myfile.txt
home_folder = os.getenv('HOME')
This should work on Windows and Mac OS too, works well on Linux.
Really, a change in environment variable indicates that the home must be changed. So every program/script should have the new home in context; also the consequences are up to the person who changed it.
I would still stick with
home = os.getenv('USERPROFILE') or os.getenv('HOME')
what exactly is required?
I realize that this is an old question that has been answered but I thought I would add my two cents. The accepted answer was not working for me. I needed to find the user directory and I wanted it to work with and without sudo. In Linux, my user directory is "/home/someuser" but my root directory is "/root/". However, on my Mac, the user directory is "/Users/someuser". Here is what I ended up doing:
_USERNAME = os.getenv("SUDO_USER") or os.getenv("USER")
_HOME = os.path.expanduser('~'+_USERNAME)
This worked both with and without sudo on Mac and Linux.
get (translated) user folder names on Linux:
from gi.repository import GLib
docs = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOCUMENTS)
desktop = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DESKTOP)
pics = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PICTURES)
videos = GLib.get_user_special_dir(GLib.USER_DIRECTORY_VIDEOS)
music = GLib.get_user_special_dir(GLib.USER_DIRECTORY_MUSIC)
downloads = GLib.get_user_special_dir(GLib.USER_DIRECTORY_DOWNLOAD)
public = GLib.get_user_special_dir(GLib.USER_DIRECTORY_PUBLIC_SHARE)
templates = GLib.get_user_special_dir(GLib.USER_DIRECTORY_TEMPLATES)
print(docs)
print(desktop)
print(pics)
print(videos)
print(music)
print(downloads)
print(public)
print(templates)
On Linux and other UNIXoids you can always take a peek in /etc/passwd. The home directory is the sixth colon-separated field in there. No idea on how to do better than the environment variable on Windows though. There'll be a system call for it, but if it's available from Python, ...
This question already has answers here:
How can I safely create a directory (possibly including intermediate directories)?
(28 answers)
Closed 5 years ago.
Is there a way to get functionality similar to mkdir -p on the shell from within Python. I am looking for a solution other than a system call. I am sure the code is less than 20 lines, and I am wondering if someone has already written it?
For Python ≥ 3.5, use pathlib.Path.mkdir:
import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)
The exist_ok parameter was added in Python 3.5.
For Python ≥ 3.2, os.makedirs has an optional third argument exist_ok that, when True, enables the mkdir -p functionality—unless mode is provided and the existing directory has different permissions than the intended ones; in that case, OSError is raised as previously:
import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)
For even older versions of Python, you can use os.makedirs and ignore the error:
import errno
import os
def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise
In Python >=3.2, that's
os.makedirs(path, exist_ok=True)
In earlier versions, use #tzot's answer.
This is easier than trapping the exception:
import os
if not os.path.exists(...):
os.makedirs(...)
Disclaimer This approach requires two system calls which is more susceptible to race conditions under certain environments/conditions. If you're writing something more sophisticated than a simple throwaway script running in a controlled environment, you're better off going with the accepted answer that requires only one system call.
UPDATE 2012-07-27
I'm tempted to delete this answer, but I think there's value in the comment thread below. As such, I'm converting it to a wiki.
Recently, I found this distutils.dir_util.mkpath:
In [17]: from distutils.dir_util import mkpath
In [18]: mkpath('./foo/bar')
Out[18]: ['foo', 'foo/bar']
With Pathlib from python3 standard library:
Path(mypath).mkdir(parents=True, exist_ok=True)
If parents is true, any missing parents of this path are created as
needed; they are created with the default permissions without taking
mode into account (mimicking the POSIX mkdir -p command).
If exist_ok is false (the default), an FileExistsError is raised if
the target directory already exists.
If exist_ok is true, FileExistsError exceptions will be ignored (same
behavior as the POSIX mkdir -p command), but only if the last path
component is not an existing non-directory file.
Changed in version 3.5: The exist_ok parameter was added.
mkdir -p gives you an error if the file already exists:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory `/tmp/foo': File exists
So a refinement to the previous suggestions would be to re-raise the exception if os.path.isdir returns False (when checking for errno.EEXIST).
(Update) See also this highly similar question; I agree with the accepted answer (and caveats) except I would recommend os.path.isdir instead of os.path.exists.
(Update) Per a suggestion in the comments, the full function would look like:
import os
def mkdirp(directory):
if not os.path.isdir(directory):
os.makedirs(directory)
As mentioned in the other solutions, we want to be able to hit the file system once while mimicking the behaviour of mkdir -p. I don't think that this is possible to do, but we should get as close as possible.
Code first, explanation later:
import os
import errno
def mkdir_p(path):
""" 'mkdir -p' in Python """
try:
os.makedirs(path)
except OSError as exc: # Python >2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
As the comments to #tzot's answer indicate there are problems with checking whether you can create a directory before you actually create it: you can't tell whether someone has changed the file system in the meantime. That also fits in with Python's style of asking for forgiveness, not permission.
So the first thing we should do is try to make the directory, then if it goes wrong, work out why.
As Jacob Gabrielson points out, one of the cases we must look for is the case where a file already exists where we are trying to put the directory.
With mkdir -p:
$ touch /tmp/foo
$ mkdir -p /tmp/foo
mkdir: cannot create directory '/tmp/foo': File exists
The analogous behaviour in Python would be to raise an exception.
So we have to work out if this was the case. Unfortunately, we can't. We get the same error message back from makedirs whether a directory exists (good) or a file exists preventing the creation of the directory (bad).
The only way to work out what happened is to inspect the file system again to see if there is a directory there. If there is, then return silently, otherwise raise the exception.
The only problem is that the file system may be in a different state now than when makedirs was called. eg: a file existed causing makedirs to fail, but now a directory is in its place. That doesn't really matter that much, because the the function will only exit silently without raising an exception when at the time of the last file system call the directory existed.
I think Asa's answer is essentially correct, but you could extend it a little to act more like mkdir -p, either:
import os
def mkdir_path(path):
if not os.access(path, os.F_OK):
os.mkdirs(path)
or
import os
import errno
def mkdir_path(path):
try:
os.mkdirs(path)
except os.error, e:
if e.errno != errno.EEXIST:
raise
These both handle the case where the path already exists silently but let other errors bubble up.
Function declaration;
import os
def mkdir_p(filename):
try:
folder=os.path.dirname(filename)
if not os.path.exists(folder):
os.makedirs(folder)
return True
except:
return False
usage :
filename = "./download/80c16ee665c8/upload/backup/mysql/2014-12-22/adclient_sql_2014-12-22-13-38.sql.gz"
if (mkdir_p(filename):
print "Created dir :%s" % (os.path.dirname(filename))
import os
import tempfile
path = tempfile.mktemp(dir=path)
os.makedirs(path)
os.rmdir(path)
I've had success with the following personally, but my function should probably be called something like 'ensure this directory exists':
def mkdirRecursive(dirpath):
import os
if os.path.isdir(dirpath): return
h,t = os.path.split(dirpath) # head/tail
if not os.path.isdir(h):
mkdirRecursive(h)
os.mkdir(join(h,t))
# end mkdirRecursive
import os
from os.path import join as join_paths
def mk_dir_recursive(dir_path):
if os.path.isdir(dir_path):
return
h, t = os.path.split(dir_path) # head/tail
if not os.path.isdir(h):
mk_dir_recursive(h)
new_path = join_paths(h, t)
if not os.path.isdir(new_path):
os.mkdir(new_path)
based on #Dave C's answer but with a bug fixed where part of the tree already exists