This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Serving dynamically generated ZIP archives in Django
(Feel free to point me to any potential duplicates if I have missed them)
I have looked at this snippet:
http://djangosnippets.org/snippets/365/
and this answer:
but I wonder how I can tweak them to suit my need: I want multiple files to be zipped and the archive available as a download via a link (or dynamically generated via a view). I am new to Python and Django so I don't know how to go about it.
Thank in advance!
I've posted this on the duplicate question which Willy linked to, but since questions with a bounty cannot be closed as a duplicate, might as well copy it here too:
import os
import zipfile
import StringIO
from django.http import HttpResponse
def getfiles(request):
# Files (local path) to put in the .zip
# FIXME: Change this (get paths from DB etc)
filenames = ["/tmp/file1.txt", "/tmp/file2.txt"]
# Folder name in ZIP archive which contains the above files
# E.g [thearchive.zip]/somefiles/file2.txt
# FIXME: Set this to something better
zip_subdir = "somefiles"
zip_filename = "%s.zip" % zip_subdir
# Open StringIO to grab in-memory ZIP contents
s = StringIO.StringIO()
# The zip compressor
zf = zipfile.ZipFile(s, "w")
for fpath in filenames:
# Calculate path for file in zip
fdir, fname = os.path.split(fpath)
zip_path = os.path.join(zip_subdir, fname)
# Add file, at correct path
zf.write(fpath, zip_path)
# Must close zip for all contents to be written
zf.close()
# Grab ZIP file from in-memory, make response with correct MIME-type
resp = HttpResponse(s.getvalue(), mimetype = "application/x-zip-compressed")
# ..and correct content-disposition
resp['Content-Disposition'] = 'attachment; filename=%s' % zip_filename
return resp
So as I understand your problem is not how to generate dynamically this file, but creating a link for people to download it...
What I suggest is the following:
0) Create a model for your file, if you want to generate it dynamically don't use the FileField, but just the info you need for generating this file:
class ZipStored(models.Model):
zip = FileField(upload_to="/choose/a/path/")
1) Create and store your Zip. This step is important, you create your zip in memory, and then cast it to assign it to the FileField:
function create_my_zip(request, [...]):
[...]
# This is a in-memory file
file_like = StringIO.StringIO()
# Create your zip, do all your stuff
zf = zipfile.ZipFile(file_like, mode='w')
[...]
# Your zip is saved in this "file"
zf.close()
file_like.seek(0)
# To store it we can use a InMemoryUploadedFile
inMemory = InMemoryUploadedFile(file_like, None, "my_zip_%s" % filename, 'application/zip', file_like.len, None)
zip = ZipStored(zip=inMemory)
# Your zip will be stored!
zip.save()
# Notify the user the zip was created or whatever
[...]
2) Create a url, for example get a number matching the id, you can also use a slugfield (this)
url(r'^get_my_zip/(\d+)$', "zippyApp.views.get_zip")
3) Now the view, this view will return the file matching the id passed in the url, you can also use a slug sending the text instead of the id, and make the get filtering by your slugfield.
function get_zip(request, id):
myzip = ZipStored.object.get(pk = id)
filename = myzip.zip.name.split('/')[-1]
# You got the zip! Now, return it!
response = HttpResponse(myzip.file, content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=%s' % filename
Related
I am trying to give the user a "Save as" option when the user clicks the download button in my Django app. When the user clicks the button it kicks-off the following function. The function gets some CSVs from a blob container in Azure and adds them to a zip. That zip should then be offered to download and store in a location of the user's choice.
def create_downloadable_zip():
container_client = az.container_client(container_name=blob_generator.container_name)
blobs = container_client.list_blobs()
zip_file = zipfile.ZipFile(f'{models.AppRun.client_name}.zip', 'w')
for blob in blobs:
if blob.name.endswith(".csv"):
downloaded_blob = container_client.download_blob(blob)
blob_data = downloaded_blob.readall()
zip_file.writestr(blob.name, blob_data)
zip_file.close()
return zip_file
My views.py looks like follow:
def download_file(request):
if request.method == 'POST':
zip = create_downloadable_zip()
response = HttpResponse(zip, content_type='application/zip')
response['Content-Disposition'] = 'attachement;' f'filename={zip}.zip'
return response
#
# else:
# # return a 404 response if this is a POST request
# return HttpResponse(status=404)
return render(request, "download_file.html")
The functionality works, but it returns an empty non-zip file when the "Save as" window pop-ups. However, the actual zip file contains the files is being saved in the root folder of the Django project.
I really don't get why I doesn't return the zip file from memory, but rather directly stores that zip file in root and returns an empty non-zip file with the download functionality.
Someone knows what I am doing wrong?
zipfile is used to open a file, but it is not the actual file, simply a zipfile object as #b-remmelzwaal mentioned. You will need to create a file like object, and return that instead. This can be done using io.BytesIO.
from io import BytesIO
from zipfile import ZipFile
def create_zip():
container_client = az.container_client(container_name=blob_generator.container_name)
blobs = container_client.list_blobs()
buffer = BytesIO()
with ZipFile(buffer, 'w') as zip_file:
for blob in blobs:
if blob.name.endswith(".csv"):
downloaded_blob = container_client.download_blob(blob)
blob_data = downloaded_blob.readall()
zip_file.writestr(blob.name, blob_data)
return buffer.getvalue()
Note we are returning the file like object, not the zip file object. This is because buffer represents the actual file you've created.
You don't have to use a context manager, but I find them very useful.
Also, check your spelling for the line:
# attachment instead attachement
response['Content-Disposition'] = 'attachment;' f'filename={zip}.zip'
BytesIO Documentation
I'm trying to zip several images in a ZIP file.
Although It's a Django app, files are not stored in the same app. I want to fetch them from a url list.
I get the following error: FileNotFoundError [Errno 2] No such file or directory: 'image1.jpg'
def download_image(request):
fotos = ['https://storage.googleapis.com/some/image1.jpg', 'https://storage.googleapis.com/some/image2.jpg']
f = StringIO()
zip = ZipFile(f, 'w')
for foto in fotos:
url = urlopen(foto)
filename = str(foto).split('/')[-1]
zip.write(filename, url.read())
zip.close()
response = HttpResponse(f.getvalue(), content_type="application/zip")
response['Content-Disposition'] = 'attachment; filename=image-test.zip'
return response
I reviewed several posts and finaly followed this one how can I export multiple images using zipfile and urllib2 - django
But I can't get this working. Any clues welcome. Thanks in advance.
Per the zip documentation, write() is used to write an existing file into the filesystem (by path). I think that you are looking for writestr(), which can be passed the actual file contents.
I'm quite new on Django and i'm looking for a way to dwonload a zip file from my django site but i have some issue when i'm running this piece of code:
def download(self):
dirName = settings.DEBUG_FOLDER
name = 'test.zip'
with ZipFile(name, 'w') as zipObj:
# Iterate over all the files in directory
for folderName, subfolders, filenames in os.walk(dirName):
for filename in filenames:
# create complete filepath of file in directory
filePath = os.path.join(folderName, filename)
# Add file to zip
zipObj.write(filePath, basename(filePath))
path_to_file = 'http://' + sys.argv[-1] + '/' + name
resp= {}
# Grab ZIP file from in-memory, make response with correct MIME-type
resp = HttpResponse(content_type='application/zip')
# ..and correct content-disposition
resp['Content-Disposition'] = 'attachment; filename=%s' % smart_str(name)
resp['X-Sendfile'] = smart_str(path_to_file)
return resp
I get:
Exception Value:
<HttpResponse status_code=200, "application/zip"> is not JSON serializable
I tried to change the content_type to octet-stream but it doesn't work
And to use a wrapper as followw:
wrapper = FileWrapper(open('test.zip', 'rb'))
content_type = 'application/zip'
content_disposition = 'attachment; filename=name'
# Grab ZIP file from in-memory, make response with correct MIME-type
resp = HttpResponse(wrapper, content_type=content_type)
# ..and correct content-disposition
resp['Content-Disposition'] = content_disposition
I didn't find useful answer so far but maybe I didn't search well, so if it seems my problem had been already traited, feel free to notify me
Thank you very much for any help
You have to send the zip file as byte
response = HttpResponse(zipObj.read(), content_type="application/zip")
response['Content-Disposition'] = 'attachment; filename=%s' % smart_str(name)
return response
I would do like this:
(Caveat I use wsl so the python function will make use of cmd lines)
In view:
import os
def zipdownfun(request):
""" Please establish in settings.py where media file should be downloaded from.
In my case is media with a series of other folders inside. Media folder is at the same level of project root folder, where settings.py is"""
file_name = os.path.join(MEDIA_URL,'folder_where_your_file_is','file_name.zip')
"""let us put the case that you have zip folder in media folder"""
file_folder_path = os.path.join(MEDIA_URL,'saving_folder')
"""The command line takes as first variable the name of the
future zip file and as second variable the destination folder"""
cmd = f'zip {file_name} {file_folder_path}'
"""With os I open a process in the background so that some magic
happens"""
os.system(cmd)
"""I don't know what you want to do with this, but I placed the
URL of the file in a button for the download, so you will need
the string of the URL to place in href of an <a> element"""
return render(request,'your_html_file.html', {'url':file_name})
The db I have created, will be updated very often. I used a slightly different version of this function with -r clause since I had to zip, each time, a folder. Why I did this? The database I have created has to allow the download of this zipped folder. This folder will be updated daily. So this function basically overwrites the file each time that is downloaded. It will be so fresh of new data each time.
Please refer to this page to understand how to create a button for the download of the generated file.
Take as reference approach 2. The URL variable that you are passing to the Django template should be used at the place of the file (screenshot attached)
I hope it can help!
I'm trying to extract new revisions of Chromium.app from their snapshots, and I can download the file fine, but when it comes to extracting it, ZipFile either extracts the chrome-mac folder within as a file, says that directories don't exist, etc. I am very new to python, so these errors make little sense to me. Here is what I have so far.
import urllib2
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print latestRev
# we have the revision, now we need to download the zip and extract it
latestZip = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev)), '~/Desktop/ChromiumUpdate/%i-update' % (int(latestRev)))
#declare some vars that hold paths n shit
workingDir = '/Users/slehan/Desktop/ChromiumUpdate/'
chromiumZipPath = '%s%i-update.zip' % (workingDir, (int(latestRev)))
chromiumAppPath = 'chrome-mac/' #the path of the chromium executable within the zip file
chromiumAppExtracted = '%s/Chromium.app' % (workingDir) # path of the extracted executable
output = open(chromiumZipPath, 'w') #delete any current file there
output.write(latestZip.read())
output.close()
# we have the .zip now we need to extract the Chromium.app file, it's in ziproot/chrome-mac/Chromium.app
import zipfile, os
zippedFile = open(chromiumZipPath)
zippedChromium = zipfile.ZipFile(zippedFile, 'r')
zippedChromium.extract(chromiumAppPath, workingDir)
#print zippedChromium.namelist()
zippedChromium.close()
#zippedChromium.close()
Any ideas?
It seems you have encountered a bug in Python. This other question details the problem and workarounds. You can elect to use one of those workarounds, or update to Python 2.6.5 or 2.7b2.
One of the workarounds suggests copying the patched zipfile.py module from the fixed Python.
Best of luck!
This seems to be working for me:
import os
import urllib2
import zipfile
from StringIO import StringIO
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print 'getting revision', latestRev
# we have the revision, now we need to download the zip and extract it
locRef='http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev))
latestZip = StringIO(urllib2.urlopen(locRef).read())
# we have the .zip now we need to extract the Chromium.app file, it's in chrome-mac/Chromium.app/
zippedChromium = zipfile.ZipFile(latestZip)
# find all zip members in chrome-mac/Chromium.app
members = [m for m in zippedChromium.namelist() if m.startswith('chrome-mac/Chromium.app/')]
#zippedChromium.extract(chromiumAppPath, workingDir)
target = 'chromium-%s' % latestRev
if os.path.isdir(target):
print 'destination already exists, exiting'
raise SystemExit(1)
os.makedirs(target)
zippedChromium.extractall(target, members)
#zippedChromium.close()
Here's another cut - this is the same technique, but it walks the result to demonstrate that it works.
import os
import urllib2
import zipfile
from StringIO import StringIO
response = urllib2.urlopen('http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/LATEST')
latestRev = response.read()
print 'getting revision', latestRev
# we have the revision, now we need to download the zip and extract it
locRef='http://build.chromium.org/buildbot/snapshots/chromium-rel-mac/%i/chrome-mac.zip' % (int(latestRev))
latestZip = StringIO(urllib2.urlopen(locRef).read())
# we have the .zip now we need to extract the Chromium.app file, it's in chrome-mac/Chromium.app/
zippedChromium = zipfile.ZipFile(latestZip)
# find all zip members in chrome-mac/Chromium.app
members = [m for m in zippedChromium.namelist() if m.startswith('chrome-mac/Chromium.app/')]
#zippedChromium.extract(chromiumAppPath, workingDir)
target = 'chromium-%s' % latestRev
if os.path.isdir(target):
print 'destination already exists, exiting'
raise SystemExit(1)
os.makedirs(target)
zippedChromium.extractall(target, members)
lengths = [
(len(dirnames), len(filenames))
for dirpath, dirnames, filenames in os.walk(target)
]
dirlengths, filelengths = zip(*lengths)
ndirs = sum(dirlengths)
nfiles = sum(filelengths)
print 'extracted %(nfiles)d files in %(ndirs)d dirs' % vars()
#zippedChromium.close()
The output I get when I run it is
> .\getapp.py
getting revision 48479
extracted 537 files in 184 dirs
There is another problem extracting an .app from a zip in Python (which doesn't happen with a typically zip utility). No one else seems to have mentioned this...
The .app can ceases to function post extraction this way, as a result of losing the execution permission bit on the nested binary. You can fix this though, by simply granting that again.
Here's a loose snippet of code that I'm using. Revise this as needed for your purposes (or write a more generic function to handle this situation in a more universal manner):
import os, zipfile
...
ZIP_PATH = APP_PATH + ".zip"
APP_BIN_DIR = os.path.join( APP_PATH, "Contents/MacOS" )
zipfile.ZipFile( ZIP_PATH, 'r' ).extractall( WORK_DIR )
BIN_PATH = os.path.join( APP_BIN_DIR, os.listdir( APP_BIN_DIR )[0] )
os.chmod( BIN_PATH, 0o777 )
My program already knew where to expect the APP_PATH to be found (i.e. within the WORK_DIR). I had to zip it up though, and shoe horn that detail in after the fact. I name my zip like XXXXX.app.zip. I resolve the BIN_PATH here pretty simply without the need to know the name of binary inside the .app, because I know there is only going to be one file in there for my use case. I grant full (777) permissions to it, because I simply delete the .app at the end of my script anyway.
How to serve users a dynamically generated ZIP archive in Django?
I'm making a site, where users can choose any combination of available books and download them as ZIP archive. I'm worried that generating such archives for each request would slow my server down to a crawl. I have also heard that Django doesn't currently have a good solution for serving dynamically generated files.
The solution is as follows.
Use Python module zipfile to create zip archive, but as the file specify StringIO object (ZipFile constructor requires file-like object). Add files you want to compress. Then in your Django application return the content of StringIO object in HttpResponse with mimetype set to application/x-zip-compressed (or at least application/octet-stream). If you want, you can set content-disposition header, but this should not be really required.
But beware, creating zip archives on each request is bad idea and this may kill your server (not counting timeouts if the archives are large). Performance-wise approach is to cache generated output somewhere in filesystem and regenerate it only if source files have changed. Even better idea is to prepare archives in advance (eg. by cron job) and have your web server serving them as usual statics.
Here's a Django view to do this:
import os
import zipfile
import StringIO
from django.http import HttpResponse
def getfiles(request):
# Files (local path) to put in the .zip
# FIXME: Change this (get paths from DB etc)
filenames = ["/tmp/file1.txt", "/tmp/file2.txt"]
# Folder name in ZIP archive which contains the above files
# E.g [thearchive.zip]/somefiles/file2.txt
# FIXME: Set this to something better
zip_subdir = "somefiles"
zip_filename = "%s.zip" % zip_subdir
# Open StringIO to grab in-memory ZIP contents
s = StringIO.StringIO()
# The zip compressor
zf = zipfile.ZipFile(s, "w")
for fpath in filenames:
# Calculate path for file in zip
fdir, fname = os.path.split(fpath)
zip_path = os.path.join(zip_subdir, fname)
# Add file, at correct path
zf.write(fpath, zip_path)
# Must close zip for all contents to be written
zf.close()
# Grab ZIP file from in-memory, make response with correct MIME-type
resp = HttpResponse(s.getvalue(), mimetype = "application/x-zip-compressed")
# ..and correct content-disposition
resp['Content-Disposition'] = 'attachment; filename=%s' % zip_filename
return resp
Many answers here suggest to use a StringIO or BytesIO buffer. However this is not needed as HttpResponse is already a file-like object:
response = HttpResponse(content_type='application/zip')
zip_file = zipfile.ZipFile(response, 'w')
for filename in filenames:
zip_file.write(filename)
response['Content-Disposition'] = 'attachment; filename={}'.format(zipfile_name)
return response
Note that you should not call zip_file.close() as the open "file" is response and we definitely don't want to close it.
I used Django 2.0 and Python 3.6.
import zipfile
import os
from io import BytesIO
def download_zip_file(request):
filelist = ["path/to/file-11.txt", "path/to/file-22.txt"]
byte_data = BytesIO()
zip_file = zipfile.ZipFile(byte_data, "w")
for file in filelist:
filename = os.path.basename(os.path.normpath(file))
zip_file.write(file, filename)
zip_file.close()
response = HttpResponse(byte_data.getvalue(), content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=files.zip'
# Print list files in zip_file
zip_file.printdir()
return response
For python3 i use the io.ByteIO since StringIO is deprecated to achieve this. Hope it helps.
import io
def my_downloadable_zip(request):
zip_io = io.BytesIO()
with zipfile.ZipFile(zip_io, mode='w', compression=zipfile.ZIP_DEFLATED) as backup_zip:
backup_zip.write('file_name_loc_to_zip') # u can also make use of list of filename location
# and do some iteration over it
response = HttpResponse(zip_io.getvalue(), content_type='application/x-zip-compressed')
response['Content-Disposition'] = 'attachment; filename=%s' % 'your_zipfilename' + ".zip"
response['Content-Length'] = zip_io.tell()
return response
Django doesn't directly handle the generation of dynamic content (specifically Zip files). That work would be done by Python's standard library. You can take a look at how to dynamically create a Zip file in Python here.
If you're worried about it slowing down your server you can cache the requests if you expect to have many of the same requests. You can use Django's cache framework to help you with that.
Overall, zipping files can be CPU intensive but Django shouldn't be any slower than another Python web framework.
Shameless plug: you can use django-zipview for the same purpose.
After a pip install django-zipview:
from zipview.views import BaseZipView
from reviews import Review
class CommentsArchiveView(BaseZipView):
"""Download at once all comments for a review."""
def get_files(self):
document_key = self.kwargs.get('document_key')
reviews = Review.objects \
.filter(document__document_key=document_key) \
.exclude(comments__isnull=True)
return [review.comments.file for review in reviews if review.comments.name]
I suggest to use separate model for storing those temp zip files. You can create zip on-fly, save to model with filefield and finally send url to user.
Advantages:
Serving static zip files with django media mechanism (like usual uploads).
Ability to cleanup stale zip files by regular cron script execution (which can use date field from zip file model).
A lot of contributions were made to the topic already, but since I came across this thread when I first researched this problem, I thought I'd add my own two cents.
Integrating your own zip creation is probably not as robust and optimized as web-server-level solutions. At the same time, we're using Nginx and it doesn't come with a module out of the box.
You can, however, compile Nginx with the mod_zip module (see here for a docker image with the latest stable Nginx version, and an alpine base making it smaller than the default Nginx image). This adds the zip stream capabilities.
Then Django just needs to serve a list of files to zip, all done!
It is a little more reusable to use a library for this file list response, and django-zip-stream offers just that.
Sadly it never really worked for me, so I started a fork with fixes and improvements.
You can use it in a few lines:
def download_view(request, name=""):
from django_zip_stream.responses import FolderZipResponse
path = settings.STATIC_ROOT
path = os.path.join(path, name)
return FolderZipResponse(path)
You need a way to have Nginx serve all files that you want to archive, but that's it.
Can't you just write a link to a "zip server" or whatnot? Why does the zip archive itself need to be served from Django? A 90's era CGI script to generate a zip and spit it to stdout is really all that's required here, at least as far as I can see.