As a "clean up" after my script's main purpose is complete, a function is called to recursively look through each folder and remove all files that end in a pre-determined set of extensions.
I during my testing, I discovered that some files with a file extension in the list of ones to delete actually throw an error: [Errno 1] Operation not permitted: '/location/of/locked/file.png. Looking at the file itself, it appears to be Locked (on mac).
How would I go about removing the locked attribute (should it exist) from every file/folder using Python, then delete the file if it ends in the extension?
Preferably this can all be done in the same function below, as it takes a long time to traverse the input directory - handling each only once is the way to go.
How does this affect the script's integrity on Windows?
I have taken care of programming it in a way that makes it compatible between the OSs but (to my knowledge) the locked attribute does not exist on Windows the way it does on mac and could cause unknown side-effects.
REMOVE_FILETYPES = ('.png', '.jpg', '.jpeg', '.pdf')
def cleaner(currentPath):
if not os.path.isdir(currentPath):
if currentPath.endswith(REMOVE_FILETYPES) or os.path.basename(currentPath).startswith('.'):
try:
os.remove(currentPath)
print('REMOVED: \"{removed}\"'.format(removed = currentPath))
except BaseException as e:
print('ERROR: Could not remove: \"{failed}\"'.format(failed = str(e)))
finally:
return True
return False
if all([cleaner(os.path.join(currentPath, file)) for file in os.listdir(currentPath)]):
try:
os.rmdir(currentPath)
print('REMOVED: \"{removed}\"'.format(removed = currentPath))
except:
print('ERROR: Could not remove: \"{failed}\"'.format(failed = currentPath))
finally:
return True
return False
cleaner(r'/path/to/parent/dir')
I would really appreciate if somebody could show me how to integrate such functionality into the sub-routine. Cheers.
EDIT: Removed error handling as per request
def cleaner(currentPath):
if sys.platform == 'darwin':
os.system('chflags nouchg {}'.format(currentPath))
if not os.path.isdir(currentPath):
if currentPath.endswith(REMOVE_FILETYPES) or os.path.basename(currentPath).startswith('.'):
try:
os.remove(currentPath)
print('REMOVED: \"{removed}\"'.format(removed=currentPath))
except PermissionError:
if sys.platform == 'darwin':
os.system('chflags nouchg {}'.format(currentPath))
os.remove(currentPath)
if all([cleaner(os.path.join(currentPath, file)) for file in os.listdir(currentPath)]) and not currentPath == SOURCE_DIR:
os.rmdir(currentPath)
print('REMOVED: \"{removed}\"'.format(removed=currentPath))
You can unlock the file with the chflags command:
os.system('chflags nouchg {}'.format(filename))
(There is a function os.chflags, but the flag associated with the locked status is not a regular one, but what the os module documentation calls a "user-defined" flag, as you can see by looking at os.stat(locked_filename).st_flags.)
To solve your problem I'd add the chflags command above to a specific except: for the error you get trying to remove a locked file, along with a platform check:
try:
os.remove(currentPath)
print('REMOVED: \"{removed}\"'.format(removed = currentPath))
except PermissionError:
if sys.platform == 'darwin':
os.system('chflags nouchg {}'.format(currentPath))
os.remove(currentPath)
else:
raise
except BaseException as e:
...
Related
I have developed code that deletes all the files in a folder. I want it to be only able to run concurrently. Meaning, if another user tries to run the script while it is running, the user will get a message saying that the script is already running as to avoid errors in the process.
I tried using threading.Lock(), acquire() and release() but it's not doing any good.
This is my code:
def delete_files(ini_path, days_num):
config = read_config_file(ini_path)
try:
lock.acquire()
log_processes(str(config[1]))
if os.listdir(str(config[0])) == []:
print("Empty list")
else:
print("List loaded")
for file in os.listdir(str(config[0])):
file_path = config[0] + '/' + file
file_datestamp = datetime.fromtimestamp(os.path.getmtime(file_path))
start_of_deletion_date = date.today() - timedelta(days = int(days_num))
if date.today() > file_datestamp.date() and file_datestamp.date() >= start_of_deletion_date:
if not os.path.isdir(file_path):
os.remove(str(file_path))
else:
if not os.path.isdir(file_path):
compress_file(str(file_path))
lock.release()
except Exception as e:
log_processes(config[1], e)
print(e)
When I tried to run it simultaneously, I get this error:
"ERROR: [Errno 2] No such file or directory"
because the file has already been deleted by the first run.
Does anyone have any idea how to solve this?
In my program I use shutil to copy files to several computers in a list. I am wondering what's the best way to give an error and move onto next computer if one of the computers happens to be turned off.
My original code:
def copyfiles(servername):
# copy config to remote server
source = os.listdir("C:/Users/myname/Desktop/PythonUpdate/") # directory where original configs are located
destination = '//' + servername + '/c$/test/' # destination server directory
for files in source:
if files.endswith(".config"):
shutil.copy(files,destination)
os.system('cls' if os.name == 'nt' else 'clear')
array = []
with open("C:/Users/myname/Desktop/PythonUpdate/serverlist.txt", "r") as f:
for servername in f:
copyfiles(servername.strip())
What I am trying to do:
def copyfiles(servername):
# copy config to remote server
source = os.listdir("C:/Users/myname/Desktop/PythonUpdate/") # directory where original configs are located
destination = '//' + servername + '/c$/test/' # destination server directory
for files in source:
if files.endswith(".config"):
try:
shutil.copy(files,destination)
except:
print (" //////////////////////////////////////////")
print (" Cannot connect to " + servername + ".")
print (" //////////////////////////////////////////")
os.system('cls' if os.name == 'nt' else 'clear')
array = []
with open("C:/Users/myname/Desktop/PythonUpdate/serverlist.txt", "r") as f:
for servername in f:
copyfiles(servername.strip())
Does this seem like it is implemented well?
While your idea of using a try-catch block is correct, you should definitely be much more precise about which conditions you consider to be forgivable. For example, you would not want to continue if you somehow got a MemoryError or KeyboardInterrupt.
As a first step, you should trap only OSError, as this is what shutil.copy uses to indicate all sorts of read/write errors (at least
that is what the documentation for shutil.copyfile says). According to this post, your error is most likely to be a WindowsError (which is a subclass of OSError):
try:
shutil.copy(files, destination)
except WindowsError:
print('...')
If there are likely to be other causes for the exception, you can further narrow down the exact cause by testing the error object itself using the extended form of the else clause. For example, WindowsError, has an attribute winerror, that contains the system-level error code for the exception. Likely candidates that you might want to forgive:
ERROR_BAD_NETPATH (53)
ERROR_NETWORK_BUSY (54)
ERROR_DEV_NOT_EXIST (55)
ERROR_ADAP_HDW_ERR (57)
ERROR_UNEXP_NET_ERR (59)
ERROR_NETNAME_DELETED (64)
ERROR_SHARING_PAUSED (70)
ERROR_NET_WRITE_FAULT (88)
Your code could then look something like this:
try:
shutil.copy(files, destination)
except WindowsError as ex:
if ex.winerror in (53, 54, 55, 57, 59, 64, 70, 88):
print('...')
else:
raise
You may also want to check the filename2 attribute of the exception. This will tell you if the file you were expecting caused this exception. This check is not related to the one on winerror and can be done along with or in exclusion to the one shown above:
try:
shutil.copy(files, destination)
except WindowsError as ex:
if ex.filename2.startswith('//' + filename2):
print('...')
else:
raise
In the following piece of code, some_path is a string that corresponds to a path (may be a relative path)
def editable(some_path):
"""Safely check whether a file is editable."""
delete = not os.path.exists(some_path)
try:
with open(some_path, 'ab'):
return True
except:
return False
finally:
# If the file didn't exist before, remove the created version
if delete:
try:
os.remove(some_path)
except:
pass
In open's docs we read:
If the file cannot be opened, IOError is raised
Is IOError the only possible error I can get (UnicodeError comes to mind, or OSError etc) ?
os.remove docs are even more vague:
If path is a directory, OSError is raised
So what if the file is in use, or protected or...
UPDATE: How about shutil.move ? This seems to raise yet another shutil.Error(StandardError) - if I read the source right
In python, I have made a function to make a directory if does not already exist.
def make_directory_if_not_exists(path):
try:
os.makedirs(path)
break
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
On Windows, sometimes I will get the following exception:
WindowsError: [Error 5] Access is denied: 'C:\\...\\my_path'
It seems to happen when the directory is open in the Windows File Browser, but I can't reliably reproduce it. So instead I just made the following workaround.
def make_directory_if_not_exists(path):
while not os.path.isdir(path):
try:
os.makedirs(path)
break
except OSError as exception:
if exception.errno != errno.EEXIST:
raise
except WindowsError:
print "got WindowsError"
pass
What's going on here, i.e. when does Windows mkdir give such an access error? Is there a better solution?
You should use OSError as well as IOError. See this answer, you'll use something like:
def make_directory_if_not_exists(path):
try:
os.makedirs(path)
except (IOError, OSError) as exception:
if exception.errno != errno.EEXIST:
...
A little googling reveals that this error is raised in various different contexts, but most of them have to do with permissions errors. The script may need to be run as administrator, or there may be another program open using one of the directories that you are trying to use.
on your question on a better solution i would use simple and clear three lines code here:
def make_directory_if_not_exists(path):
if not os.path.isdir(path):
os.makedirs(path)
I have a script where a user is prompted to type a filename (of a file that is to be opened), and if the file doesn't exist in the current directory, the user is prompted again. Here is the short version:
file = input("Type filename: ")
...
try:
fileContent = open(filename, "r")
...
except FileNotFoundError:
...
When I tested my script on my MacOS X in Python 3.3x it worked perfectly fine when I type the wrong filename on purpose (it executes the suite under "expect").
However, when I wanted to run my code
on a Windows computer in Python 3.2x, I get an error that says that "FileNotFoundError" is not defined. So, Python 3.2 on Windows thinks "FileNotFoundError" is a variable and the programs quits with an error.
I figured out that Python 3.2 on Windows throws an "IOError" if the input filename is not valid. I tested it on my Linux machine in Python 2.7, and it's also an IOError.
My problem is now, that the code with
except "FileNotFoundError":
won't run on Windows's Python 3.2, but if I change it to
except "IOError":
it won't work on my Mac anymore.
How could I work around it? The only way I can think of is to use just
except, which I usually don't want.
In 3.3, IOError became an alias for OSError, and FileNotFoundError is a subclass of OSError. So you might try
except (OSError, IOError) as e:
...
This will cast a pretty wide net, and you can't assume that the exception is "file not found" without inspecting e.errno, but it may cover your use case.
PEP 3151 discusses the rationale for the change in detail.
This strikes me as better than a simple except:, but I'm not sure if it is the best solution:
error_to_catch = getattr(__builtins__,'FileNotFoundError', IOError)
try:
f = open('.....')
except error_to_catch:
print('!')
So to exactly catch only when a file is not found, I do:
import errno
try:
open(filename, 'r')
except (OSError, IOError) as e: # FileNotFoundError does not exist on Python < 3.3
if getattr(e, 'errno', 0) == errno.ENOENT:
... # file not found
raise
you can catch 2 errors at the same time
except (FileNotFoundError, IOError):
I didn't realize that is what you were asking. I hope there is a more eloquent solution then to manually inspect
try:
error_to_catch = FileNotFoundError
except NameError:
error_to_catch = IOError
except error_to_catch
cwallenpoole does this conditional more eloquently in his answer
(error_to_catch = getattr(__builtins__,'FileNotFoundError', IOError))