Develop Reference

How to detect cycles in directory traversal

I am using Python on Ubuntu (Linux). I would also like this code to work on modern-ish Windows PCs. I can take or leave Macs, since I have no plan to get one, but I am hoping to make this code as portable as possible.
I have written some code that is supposed to traverse a directory and run a test on all of its subdirectories (and later, some code that will do something with each file, so I need to know how to detect links there too).
I have added a check for symlinks, but I do not know how to protect against hardlinks that could cause infinite recursion. Ideally, I'd like to also protect against duplicate detections in general (root: [A,E], A: [B,C,D], E: [D,F,G], where D is the same file or directory in both A and E).
My current thought is to check if the path from the root directory to the current folder is the same as the path being tested, and if it isn't, skip it as an instance of a cycle. However, I think that would take a lot of extra I/O or it might just retrace the (actually cyclic) path that was just created.
How do I properly detect cycles in my filesystem?
def find(self) -> bool:
if self._remainingFoldersToSearch:
current_folder = self._remainingFoldersToSearch.pop()
if not current_folder.is_symlink():
contents = current_folder.iterdir()
try:
for item in contents:
if item.is_dir():
if item.name == self._indicator:
potentialArchive = [x.name for x in item.iterdir()]
if self._conf in potentialArchive:
self._archives.append(item)
if self._onArchiveReadCallback:
self._onArchiveReadCallback(item)
else:
self._remainingFoldersToSearch.append(item)
self._searched.append(item)
if self._onFolderReadCallback:
self._onFolderReadCallback(item)
except PermissionError:
logging.info("Invalid permissions accessing folder:", exc_info=True)
return True
else:
return False

How to iterate over a list of directories to check if they still exist after a delete attempt?

I am running a script to uninstall a program and to finish the process, I am checking if the applicable directories get deleted as expected. I have the following:
D_PATHS = (
r'C:\ProgramFiles\D1\FolderA',
r'C:\ProgramFiles\D1\FolderB',
r'C:\ProgramFiles\D1\FolderC',
)
for path in D_PATHS:
self.info('Deleting %s', path)
if os.path.exists:
warnings.warn(f'The following directory still exists: {path}')
else:
print(f'Removed all required directories')
When I run the script, it always throws the warning that the directories still exist, even if they don't. What am I doing wrong? Please excuse my very limited knowledge of coding. I know there's probably an easy answer that I am not understanding.

os.path.exists is a function, you need to give it an argument:
...
if os.path.exists(path):
print("Still exists")
...

If you want to print a warning for each directory that exists, then you should set a boolean flag to decide at the end whether to print the "removed all required directories" message.
dirs_exist = False
for path in D_PATHS:
self.info('Deleting %s', path)
if os.path.exists(path):
warnings.warn(f'The following directory still exists: {path}')
dirs_exist = True
if not dirs_exist:
print(f'Removed all required directories')
The alternative, if you only want to warn about the first directory that still exists, is to break from the for loop, in which case you can use an else clause on your for loop for printing the message if no break was encountered:
for path in D_PATHS:
self.info('Deleting %s', path)
if os.path.exists(path):
warnings.warn(f'The following directory still exists: {path}')
break
else:
print(f'Removed all required directories')
Note that despite the messages saying "Deleting", the code that you have shown here is not actually deleting anything. Presumably you have done so earlier in your code.

It may be possible that you dont have the permission so you can not delete them.
1.Generate a python.exe with pyinstaller .
2.Right Click and click the option "run as administrator"

Python throws error when deleting a directory that is open with windows explorer

I'm writing a code in Python, in which I check if a certain folder exists; if it does, I remove it and create a new one (with the same name). The code is as follows:
if os.path.exists(output_folder):
shutil.rmtree(output_folder)
os.makedirs(output_folder)
This code works fine, accept for when I have that specific output_folder open with the windows explorer. When it's open, I get the following error in my code:
WindowsError: [Error 5] Access is denied: [foldername]
Simultaneously, windows explorer switches itself to foldername's parent directory, and throws an error.
Is there a way to make python ignore the error and continue running, or am I asking for something that is impossible due to the system?
I tried using shutil.rmtree(output_folder, ignore_errors=True) but it didn't change anything.

You can use Python's exception handling to catch the error. You would also probably benefit from a short delay before creating the folder again to give Windows Explorer a chance to close:
import shutil
import time
try:
shutil.rmtree(output_folder)
except WindowsError as e:
print("Failed to delete") # Or just pass
time.sleep(0.5)
os.makedirs(output_folder)

Deleting Folders on drive

I am trying to delete a collection of folders on my drive. These directories are not empty. I have come up with a solution as follows:
import shutil
import os
path = "main/"
folderList = ['Blah', 'Blah', 'Blah'];
print ("Cleaning Project at %s" % path)
for c in folderList:
strippedPath = (path + c).strip("\n")
print ("Cleaning path " + strippedPath)
if os.path.exists(strippedPath):
try:
shutil.rmtree(strippedPath)
except OSError as why:
pass
print ("Done Cleaning Project")
The problem is that without the try / catch I get a error that says
PermissionError: [WinError 5] Access is denied: 'PathToFileHere'
Pressing the delete key on windows will work fine. Can someone provide me a command that will remove this directory without errors?

First you should avoid to silently swallow an Exception, but at least print or log it. But many thing can happen to a file, they may have Hidden, System or ReadOnly attributes. The current user may not have permissions on files but only on the containing folder. As Python is multi-platform its high-level commands can be less optimized for a particular OS (Windows in your case) than native ones.
You should first try to confirm that in a cmd window, the command rd /s folder correctly remove the folder that shutil.rmtree fails to delete, and if yes ask python so execute it vie the subprocess module :
subprocess.call("rd /s/q " + strippedPath)

What user do python scripts run as in windows? [duplicate]

This question already has answers here:
Deleting read-only directory in Python
(7 answers)
Closed 3 years ago.
I'm trying to have python delete some directories and I get access errors on them. I think its that the python user account doesn't have rights?
WindowsError: [Error 5] Access is denied: 'path'
is what I get when I run the script.
I've tried
shutil.rmtree
os.remove
os.rmdir
they all return the same error.

We've had issues removing files and directories on Windows, even if we had just copied them, if they were set to 'readonly'. shutil.rmtree() offers you sort of exception handlers to handle this situation. You call it and provide an exception handler like this:
import errno, os, stat, shutil
def handleRemoveReadonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise
shutil.rmtree(filename, ignore_errors=False, onerror=handleRemoveReadonly)
You might want to try that.

I've never used Python, but I would assume it runs as whatever user executes the script.

The scripts have no special user, they just run under the currently logged-in user which executed the script.
Have you tried checking that:
you are trying to delete a valid path? and that
the path has no locked files?

How are you running the script? From an interactive console session? If so, just open up a DOS command window (using cmd) and type 'whoami'. That is who you are running the scripts interactively.
Ok I saw your edits just now...why don't you print the path and check the properties to see if the user account running the scripts has the required privileges?
If whoami does not work on your version of Windows, you may use the environment variables like SET USERNAME and SET DOMAINNAME from your command window.

#ThomasH : another brick to the wall.
On unix systems, you have to ensure that parent directory is writeable too.
Here is another version :
def remove_readonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
# ensure parent directory is writeable too
pardir = os.path.abspath(os.path.join(path, os.path.pardir))
if not os.access(pardir, os.W_OK):
os.chmod(pardir, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO)
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise

If the script is being run as a scheduled task (which seems likely for a cleanup script), it will probably run as SYSTEM. It's (unwise, but) possible to set permissions on directories so that SYSTEM has no access.

Simple solution after searching for hours is to check first if that folder actually exist!
GIT_DIR="C:/Users/...."
if os.path.exists(GIT_DIR):
shutil.rmtree(GIT_DIR)
This did the trick for me.