I'm trying to write a function that backs up a directory with files of different permission to an archive on Windows XP. I'm using the tarfile module to tar the directory. Currently as soon as the program encounters a file that does not have read permissions, it stops giving the error: IOError: [Errno 13] Permission denied: 'path to file'. I would like it to instead just skip over the files it cannot read rather than end the tar operation. This is the code I am using now:
def compressTar():
"""Build and gzip the tar archive."""
folder = 'C:\\Documents and Settings'
tar = tarfile.open ("C:\\WINDOWS\\Program\\archive.tar.gz", "w:gz")
try:
print "Attempting to build a backup archive"
tar.add(folder)
except:
print "Permission denied attempting to create a backup archive"
print "Building a limited archive conatining files with read permissions."
for root, dirs, files in os.walk(folder):
for f in files:
tar.add(os.path.join(root, f))
for d in dirs:
tar.add(os.path.join(root, d))
You should add more try statements :
for root, dirs, files in os.walk(folder):
for f in files:
try:
tar.add(os.path.join(root, f))
except IOError:
pass
for d in dirs:
try:
tar.add(os.path.join(root, d), recursive=False)
except IOError:
pass
[edit] As Tarfile.add is recursive by default, I've added the recursive=False parameter when adding directories, otherwise you could run into problems.
You will need the same try/except blocks for when you are trying to add the files with read permissions. Right now, if any of the files or sub directories are not readable, then your program will crash.
Another option that isn't so reliant on try blocks is to check the permissions before trying to add the file/folder to your tarball. There is a whole question about how to best do this (and some pitfalls to avoid when using Windows): Python - Test directory permissions
The basic pseudo code would be something like:
if folder has read permissions:
add folder to tarball
else:
for each item in folder:
if item has read permission:
add item to tarball
Just to add to what everyone else has said, there's a native python function to which you can pass the file parameter and the property you're looking for to check for that property: hasattr('/path/to/file.txt', "read") OR hasattr('/path/to/file.txt', "write") and so on
hope this helps anyone else out there
Related
Folder structure:
Folder
/ \
/ \
subfolder1 files
/\
/ \
inner_subfolder1 files
/\
/ \
sub_inner_folder files
/
files
Problem here is files in sub_inner_folder are not encrypted.
def encypt_Files():
for folder, subfolders, files in os.walk('/home/username/Desktop/folder'):
for subfolder in subfolders:
os.chdir(folder)
for files in os.listdir():
if files.endswith('.pdf'):
PdfReaderobj = PyPDF2.PdfFileReader(open(files, 'rb'))
PdfWriterobj = PyPDF2.PdfFileWriter()
if PdfReaderobj.isEncrypted:
break
else:
PdfWriterobj.addPage(PdfReaderobj.getPage(0))
PdfWriterobj.encrypt(sys.argv[1])
resultPdf = open(files.strip('.pdf')+'_encrypted.pdf', 'wb')
PdfWriterobj.write(resultPdf)
resultPdf.close()
probem here is files in sub_inner_folder are not encrypted.
You do os.chdir(folder) where is should be os.chdir(subfolder). Also, you need to change the directory back using os.chdir("..") when you're done with that directory.
If you start on the wrong working directory, you won't be able to chdir() anywhere. So you need a os.chdir("/home/username/Desktop/folder") first.
Also, permission may break out of the loop. Add
except FileNotFoundError:
pass # or whatever
except PermissionError:
pass # or whatever
But: os.walk() already gives you a list of files. You should just need to loop over these. That way you also get rid of os.listdir()
Yet another option which sounds totally reasonable to me:
import glob
for result in glob.iglob('/home/username/Desktop/folder/**/*.pdf'):
print(result)
One problem I see is that you're breaking out of the inner for loop on finding an encrypted file. That should probably be a continue, but your making a new iterator using files suggests you may need to rethink the whole strategy.
And another problem is that you're chdiring to a relative path that may no longer be relative to where you are. I suggest using os.path.join instead.
Oh, and you're chdiring to folder instead of to the presumably intended subfolder.
I suggest you start over. Use the files iterator provided by os.walk, and use os.path.join to list out the full path to each file in the directory structure. Then add your pdf encryption code using the full path to each file, and ditch chdir.
foldername=[] # used to store folder paths
for folders,subfolders,files in os.walk(path):
foldername.append(folders) # storing folder paths in list
for file_path in foldername:
os.chdir(file_path) # changing directory path from stored folder paths
for files in os.listdir():
if files.endswith('.pdf'):
pdfReaderobj=PyPDF2.PdfFileReader(open(files,'rb'))
pdfWriterobj=PyPDF2.PdfFileWriter()
if pdfReaderobj.isEncrypted:
continue
else:
pdfWriterobj.addPage(pdfReaderobj.getPage(0))
pdfWriterobj.encrypt(sys.argv[1])
resultPdf=open(files.strip('.pdf')+'_encrypted.pdf','wb')
pdfWriterobj.write(resultPdf)
resultPdf.close()
I'm writing a python (2.7) script that extracts files from a zip file with an encrypted password. For this, I'm using the ZipFile module to extract files in a different directory. I have followed all the answer whatever is mentioned on here. How to extract all the files from the zip file into a different directory?
I have tried to extract all files into different directories but result is: it is creating directory inside the targeted directory.
try:
with ZipFile(os.path.join(app.config['UPLOAD_FOLDER'], filename)) as zf:
zf.extractall('/Users/dipak.das/desktop/docs/',None,b'12345')
except RuntimeError as e:
print e
I expected the output of the above script should extract all files inside path directories.But my code is creating a directory inside docs directories "/Users/dipak.das/desktop/docs/" and extracting all files.
Assuming that you want the files extracted with no subdirectories...
Totally untested but perhaps try something like
import os, shutil
destdir = '/Users/dipak.das/desktop/docs/'
with ZipFile(os.path.join(app.config['UPLOAD_FOLDER'], filename)) as zf:
for name in zf.namelist():
source = zf.open(name, 'r', b'12345')
destpath = os.path.join(destdir, os.path.basename(name))
target = open(destpath, 'w')
shutil.copyfileobj(source, target)
target.close()
In my code, I create a directory like so:
try:
os.makedirs(playlist_name)
except OSError as e:
if e.errno != errno.EEXIST:
raise
Which creates a directory in the place where I run my python script.
Then I wish to copy three files from the original directory where the folder is located into the newly created directory, like so
# Copy FFMPEG files into that folder so youtube dl can download the videos as audio tracks
# Tried using os.getcwd() to get full path, same error
shutil.copyfile(os.getcwd() + '\\ffmpeg.exe', os.getcwd() + "\\" + playlist_name)
shutil.copyfile('ffplay.exe', "/" + playlist_name + "/")
shutil.copyfile('ffprobe.exe', "/" + playlist_name + "/")
However, trying to copy those files throws this error:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\ME\\Documents\\python\\DIRECTORY\\PLAYLIST_NAME_HERE'
I have tried various shutil copy methods with the same error.
EDIT: This is running on windows
Per the copyfile docs:
dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path.
You can't use it to do what you do in the shell, naming a source file and a target directory and having it deduce the file should be put in the directory with the file's original name. You have to explicitly name the target file, or it thinks you're trying to copy to the same name as the directory, and unlike replacing a file, you can't replace a directory with a file without explicitly renaming the directory or deleting the whole directory tree first. To fix, just make sure to repeat the file name in both source and destination:
for filename in ('ffmpeg.exe', 'ffplay.exe', 'ffprobe.exe'):
shutil.copyfile(filename, os.path.join(playlist_name, filename))
The problem would be more obvious on a UNIX-like system, because those systems would reject the action with EISDIR, causing a Python IsADirectoryError to be raised, but Windows for some reason chose to use more general error codes associated with permission/access problems (EACCES and related Windows specific error codes) that Python translates to PermissionError (because Windows just isn't telling it the real problem, and it would introduce all sorts of race conditions if Python tried to check if the real problem was trying to use a directory as a file to fix the exception type).
So my friend and I have been having a problem with the first practice project of the above chapter of Automate the Boring Stuff with Python. The prompt goes: "Write a program that walks through a folder tree and searches for files with a certain file extension (such as .pdf or .jpg). Copy these files from whatever location they are in to a new folder."
To simplify, we are trying to write a program that copies all of the .jpg files out of My Pictures to another directory. Here's our code:
#! python3
# moveFileType looks in My Puctures and copies .jpg files to my Python folder
import os, shutil
def moveFileType(folder):
for folderName, subfolders, filenames in os.walk(folder):
for subfolder in subfolders:
for filename in filenames:
if filename.endswith('.jpg'):
shutil.copy(folder + filename, '<destination>')
moveFileType('<source>')
We keep getting an error along the lines of "FileNotFoundError: [Errno 2] No such file or directory".
Edit: I added a "\" to the end of my source path (I'm not sure if that is what you meant, #Jacob H), and was able to copy all of the .jpg files in that directory, but received an error when it tried to copy a file within a subfolder of that directory. I added a for loop for subfolder in subfolders and I no longer get any errors, but it doesn't actually look in the subfolders for .jpg files.
There is a more fundamental problem with your code. When you use os.walk() it will already loop through every directory for you, so looping manually through the subfolders is going to produce the same results multiple times.
The other, and more immediate, problem is that os.walk() produces relative file names, so you need to glue them back together. Basically you are omitting the directory name and looking in the current directory for files which os.walk() is finding down in a subdirectory somewhere.
Here's a quick attempt at fixing your code:
def moveFileType(folder):
for folderName, subfolders, filenames in os.walk(folder):
for filename in filenames:
if filename.endswith('.jpg'):
shutil.copy(os.path.join(folderName, filename), '<destination>')
Making the function accept a destination parameter as a second argument, instead of hardcoding <destination>, would make it a lot more useful for the future.
Make sure to type the source file destination address correctly. While i tested your code, i wrote
moveFileType('/home/anum/Pictures')
and i got error;
IOError: [Errno 2] No such file or directory:
and when i wrote
moveFileType('/home/anum/Pictures/')
the code worked perfectly...
Try doing that, hope that will do your work. M using Python 2.7
Herez the re defined code for walking into subfolders and copying ,jpg files from there aswell.
import os, shutil
def moveFileType(folder):
for root, dirs, files in os.walk(folder):
for file in files:
if file.endswith('.jpg'):
image_path=os.path.join(root,file) # get the path location of each jpeg image.
print 'location: ',image_path
shutil.copy(image_path, '/home/anum/Documents/Stackoverflow questions')
moveFileType('/home/anum/Pictures/')
This is part of a program I'm writing. The goal is to extract all the GPX files, say at G:\ (specified with -e G:\ at the command line). It would create an 'Exports' folder and dump all files with matching extensions there, recursively that is. Works great, a friend helped me write it!! Problem: empty directories and subdirectories for dirs that did not contain GPX files.
import argparse, shutil, os
def ignore_list(path, files): # This ignore list is specified in the function below.
ret = []
for fname in files:
fullFileName = os.path.normpath(path) + os.sep + fname
if not os.path.isdir(fullFileName) \
and not fname.endswith('gpx'):
ret.append(fname)
elif os.path.isdir(fullFileName) \ # This isn't doing what it's supposed to.
and len(os.listdir(fullFileName)) == 0:
ret.append(fname)
return ret
def gpxextract(src,dest):
shutil.copytree(src,dest,ignore=ignore_list)
Later in the program we have the call for extractpath():
if args.extractpath:
path = args.extractpath
gpxextract(extractpath, 'Exports')
So the above extraction does work. But the len function call above is designed to prevent the creation of empty dirs and does not. I know the best way is to os.rmdir somehow after the export, and while there's no error, the folders remain.
So how can I successfully prune this Exports folder so that only dirs with GPXs will be in there? :)
If I understand you correctly, you want to delete empty folders? If that is the case, you can do a bottom up delete folder operation -- which will fail for any any folders that are not empty. Something like:
for root, dirs, files in os.walk('G:/', topdown=true):
for dn in dirs:
pth = os.path.join(root, dn)
try:
os.rmdir(pth)
except OSError:
pass