So I create several files in some temp directory dictated by NamedTemporaryFile function.
zf = zipfile.ZipFile( zipPath, mode='w' )
for file in files:
with NamedTemporaryFile(mode='w+b', bufsize=-1, prefix='tmp') as tempFile:
tempPath = tempFile.name
with open(tempPath, 'w') as f:
write stuff to tempPath with contents of the variable 'file'
zf.write(tempPath)
zf.close()
When I use the path of these files to add to a zip file, the temp directories themselves get zipped up.
When I try to unzip, I get a series of temp folders, which eventually contain the files I want.
(i.e. I get the folder Users, which contains my user_id folder, which contains AppData...).
Is there a way to add the files directly, without the folders, so that when I unzip, I get the files directly? Thank you so much!
Try giving the arcname:
from os import path
zf = zipfile.ZipFile( zipPath, mode='w' )
for file in files:
with NamedTemporaryFile(mode='w+b', bufsize=-1, prefix='tmp') as tempFile:
tempPath = tempFile.name
with open(tempPath, 'w') as f:
write stuff to tempPath with contents of the variable 'file'
zf.write(tempPath,arcname=path.basename(tempPath))
zf.close()
Using os.path.basename you can get the file's name from a path. According to zipfile documentation, the default value for arcname is filename without a drive letter and with leading path separators removed.
Try using the arcname parameter to zf.write:
zf.write(tempPath, arcname='Users/mrb0/Documents/things.txt')
Without knowing more about your program, you may find it easier to get your arcname from the file variable in your outermost loop rather than deriving a new name from tempPath.
Related
I'm writing a python (2.7) script that extracts files from a zip file with an encrypted password. For this, I'm using the ZipFile module to extract files in a different directory. I have followed all the answer whatever is mentioned on here. How to extract all the files from the zip file into a different directory?
I have tried to extract all files into different directories but result is: it is creating directory inside the targeted directory.
try:
with ZipFile(os.path.join(app.config['UPLOAD_FOLDER'], filename)) as zf:
zf.extractall('/Users/dipak.das/desktop/docs/',None,b'12345')
except RuntimeError as e:
print e
I expected the output of the above script should extract all files inside path directories.But my code is creating a directory inside docs directories "/Users/dipak.das/desktop/docs/" and extracting all files.
Assuming that you want the files extracted with no subdirectories...
Totally untested but perhaps try something like
import os, shutil
destdir = '/Users/dipak.das/desktop/docs/'
with ZipFile(os.path.join(app.config['UPLOAD_FOLDER'], filename)) as zf:
for name in zf.namelist():
source = zf.open(name, 'r', b'12345')
destpath = os.path.join(destdir, os.path.basename(name))
target = open(destpath, 'w')
shutil.copyfileobj(source, target)
target.close()
I have multiple folders, in a common parent folder, say 'work'. Inside that, I have multiple sub-folders, named 'sub01', 'sub02', etc. All the folders have same files inside, for eg, mean.txt, sd.txt.
I have to add contents of all 'mean.txt' into a single file. I am stuck with, how to open subfolder one by one. Thanks.
getting all files as a list
g = open("new_file", "a+")
for files in list:
f = open(files, 'r')
g.write(f.read())
f.close()
g.close()
I am not getting how to get a list of all files in the subfolder, to make this work
************EDIT*********************
found a solution
os.walk() helped, but had a problem, it was random (it didn't iterate in alphabetical order)
had to use sort to make it in order
import os
p = r"/Users/xxxxx/desktop/bianca_test/" # main_folder
list1 = []
for root, dirs, files in os.walk(p):
if root[-12:] == 'native_space': #this was the sub_folder common in all parent folders
for file in files:
if file == "perfusion_calib_gm_mean.txt":
list1.append(os.path.join(root, file))
list1.sort() # os.walk() iterated folders randomly; this is to overcome that
f = open("gm_mean.txt", 'a+')
for item in list1:
g = open(item, 'r')
f.write(g.read())
print("writing", item)
g.close()
f.close()
Thanks to all who helped.
As i understand it you want to collate all 'mean.txt' files into one file. This should do the job but beware there is no ordering to which file goes where. Note also i'm using StringIO() to buffer all the data since strings are immutable in Python.
import os
from io import StringIO
def main():
buffer = StringIO()
for dirpath, dirnames, filenames in os.walk('.'):
if 'mean.txt' in filenames:
fp = os.path.join(dirpath, 'mean.txt')
with open(fp) as f:
buffer.write(f.read())
all_file_contents = buffer.getvalue()
print(all_file_contents)
if __name__ == '__main__':
main()
Here's a pseudocode to help you get started. Try to google, read and understand the solutions to get better as a programmer:
open mean_combined.txt to write mean.txt contents
open sd_combined.txt to write sd.txt contents
for every subdir inside my_dir:
for every file inside subdir:
if file.name is 'mean.txt':
content = read mean.txt
write content into mean_combined.txt
if file.name is 'sd.txt':
content = read sd.txt
write content into sd_combined.txt
close mean_combined.txt
close sd_combined.txt
You need to look up how to:
open a file to read its contents (hint: use open)
iterate files inside directory (hint: use pathlib)
write a string into a file (hint: read Input and Output)
use context managers for releasing resources (hint: read with statement)
Many files in a folder. I want to zip them all. Every 10 files will be added to a zip file.
import os, glob
import numpy as np
import zipfile
file_folder = "C:\\ABC\\DEF\\"
all_files = glob.glob(file_folder + "/*.*")
several_lists= np.array_split(all_files, 10)
for num, file_names in enumerate(several_lists):
ZipFile = zipfile.ZipFile(file_folder + str(num) + ".zip", "w" )
for f in file_names:
ZipFile.write(f, compress_type=zipfile.ZIP_DEFLATED)
ZipFile.close()
The generated zip files contains also the paths, i.e. every zip file has a folder DEF in a folder ABC. The file themselves are in DEF.
I changed the line to:
ZipFile.write(os.path.basename(f), compress_type=zipfile.ZIP_DEFLATED)
Error pops for:
WindowsError: [Error 2] The system cannot find the file specified:
How to correct it? Thank you.
Btw, is there a big difference in zip and rar file created by Python?
ZipFile.write has a parameter arcname which allows explicitly providing an in-archive filename (by default it's the same as the on-disk path).
So just use zip.write(f, arcname=os.path.basename(f)).
Also for simplicity you could set the compression mode on the zipfile.ZipFile.
edit: and you can use the zipfile as a context manager for more reliability and less lines, and assuming Python 3.6 f-strings are nice:
with zipfile.ZipFile(f'{file_folder}{num}.zip', 'w', compression=zipfile.ZIP_DEFLATED) as zip:
for f in file_names:
zip.write(f, arcname=os.path.basename(f))
I am trying to write a zip file using Python's zipfile module that starts at a certain subfolder but still maintains the tree structure from that subfolder. For example, if I pass "C:\Users\User1\OneDrive\Documents", the zip file will contain everything from Documents onward, with all of Documents' subfolders maintained within Documents. I have the following code:
import zipfile
import os
import datetime
def backup(src, dest):
"""Backup files from src to dest."""
base = os.path.basename(src)
now = datetime.datetime.now()
newFile = f'{base}_{now.month}-{now.day}-{now.year}.zip'
# Set the current working directory.
os.chdir(dest)
if os.path.exists(newFile):
os.unlink(newFile)
newFile = f'{base}_{now.month}-{now.day}-{now.year}_OVERWRITE.zip'
# Write the zipfile and walk the source directory tree.
with zipfile.ZipFile(newFile, 'w') as zip:
for folder, _ , files in os.walk(src):
print(f'Working in folder {os.path.basename(folder)}')
for file in files:
zip.write(os.path.join(folder, file),
arcname=os.path.join(
folder[len(os.path.dirname(folder)) + 1:], file),
compress_type=zipfile.ZIP_DEFLATED)
print(f'\n---------- Backup of {base} to {dest} successful! ----------\n')
I know I have to use the arcname parameter for zipfile.write(), but I can't figure out how to get it to maintain the tree structure of the original directory. The code as it is now writes every subfolder to the first level of the zip file, if that makes sense. I've read several posts suggesting I use os.path.relname() to chop off the root, but I can't seem to figure out how to do it properly. I am also aware that this post looks similar to others on Stack Overflow. I have read those other posts and cannot figure out how to solve this problem.
The arcname parameter will set the exact path within the zip file for the file you are adding. You issue is when you are building the path for arcname you are using the wrong value to get the length of the prefix to remove. Specifically:
arcname=os.path.join(folder[len(os.path.dirname(folder)) + 1:], file)
Should be changed to:
arcname=os.path.join(folder[len(src):], file)
Is there a way to read all the unopened files in a folder only by passing the one specific file name that is present in that folder?I know to read all the files in a directory passing the directory name using os.walk.But in this specific problem I can just pass only one file name.Need your help for this problem.Thank you.
If I understand you correctly, you have a path of a single file, while you want to read all files in the folder it's located in.
You can achieve this easily:
dir_name, file_name = os.path.split(filepath)
for root, dirs, files in os.walk(dir_name):
for file in files:
with open(file) as f:
file_content = f.read()