Develop Reference

Django: download created zip from function. Zipfile returns empty/not an zip-archive file

I am trying to give the user a "Save as" option when the user clicks the download button in my Django app. When the user clicks the button it kicks-off the following function. The function gets some CSVs from a blob container in Azure and adds them to a zip. That zip should then be offered to download and store in a location of the user's choice.
def create_downloadable_zip():
container_client = az.container_client(container_name=blob_generator.container_name)
blobs = container_client.list_blobs()
zip_file = zipfile.ZipFile(f'{models.AppRun.client_name}.zip', 'w')
for blob in blobs:
if blob.name.endswith(".csv"):
downloaded_blob = container_client.download_blob(blob)
blob_data = downloaded_blob.readall()
zip_file.writestr(blob.name, blob_data)
zip_file.close()
return zip_file
My views.py looks like follow:
def download_file(request):
if request.method == 'POST':
zip = create_downloadable_zip()
response = HttpResponse(zip, content_type='application/zip')
response['Content-Disposition'] = 'attachement;' f'filename={zip}.zip'
return response
#
# else:
# # return a 404 response if this is a POST request
# return HttpResponse(status=404)
return render(request, "download_file.html")
The functionality works, but it returns an empty non-zip file when the "Save as" window pop-ups. However, the actual zip file contains the files is being saved in the root folder of the Django project.
I really don't get why I doesn't return the zip file from memory, but rather directly stores that zip file in root and returns an empty non-zip file with the download functionality.
Someone knows what I am doing wrong?

zipfile is used to open a file, but it is not the actual file, simply a zipfile object as #b-remmelzwaal mentioned. You will need to create a file like object, and return that instead. This can be done using io.BytesIO.
from io import BytesIO
from zipfile import ZipFile
def create_zip():
container_client = az.container_client(container_name=blob_generator.container_name)
blobs = container_client.list_blobs()
buffer = BytesIO()
with ZipFile(buffer, 'w') as zip_file:
for blob in blobs:
if blob.name.endswith(".csv"):
downloaded_blob = container_client.download_blob(blob)
blob_data = downloaded_blob.readall()
zip_file.writestr(blob.name, blob_data)
return buffer.getvalue()
Note we are returning the file like object, not the zip file object. This is because buffer represents the actual file you've created.
You don't have to use a context manager, but I find them very useful.
Also, check your spelling for the line:
# attachment instead attachement
response['Content-Disposition'] = 'attachment;' f'filename={zip}.zip'
BytesIO Documentation

How to save an uploaded image to FastAPI using Python Imaging Library (PIL)?

I am using image compression to reduce the image size. When submitting the post request, I am not getting any error, but can't figure out why the images do not get saved. Here is my code:
#app.post("/post_ads")
async def create_upload_files(title: str = Form(),body: str = Form(),
db: Session = Depends(get_db), files: list[UploadFile] = File(description="Multiple files as UploadFile")):
for file in files:
im = Image.open(file.file)
im = im.convert("RGB")
im_io = BytesIO()
im = im.save(im_io, 'JPEG', quality=50)

PIL.Image.open() takes as fp argumnet the following:
fp – A filename (string), pathlib.Path object or a file object. The
file object must implement file.read(), file.seek(), and
file.tell() methods, and be opened in binary mode.
Using a BytesIO stream, you would need to have something like the below (as shown in client side of this answer):
Image.open(io.BytesIO(file.file.read()))
However, you don't really have to use an in-memory bytes buffer, as you can get the the actual file object using the .file attribute of UploadFile. As per the documentation:
file: A SpooledTemporaryFile (a file-like object).
This is the actual Python file that you can pass directly to other
functions or libraries that expect a "file-like" object.
Example - Saving image to disk:
# ...
from fastapi import HTTPException
from PIL import Image
#app.post("/upload")
def upload(file: UploadFile = File()):
try:
im = Image.open(file.file)
if im.mode in ("RGBA", "P"):
im = im.convert("RGB")
im.save('out.jpg', 'JPEG', quality=50)
except Exception:
raise HTTPException(status_code=500, detail='Something went wrong')
finally:
file.file.close()
im.close()
Example - Saving image to an in-memory bytes buffer (see this answer):
# ...
from fastapi import HTTPException
from PIL import Image
#app.post("/upload")
def upload(file: UploadFile = File()):
try:
im = Image.open(file.file)
if im.mode in ("RGBA", "P"):
im = im.convert("RGB")
buf = io.BytesIO()
im.save(buf, 'JPEG', quality=50)
# to get the entire bytes of the buffer use:
contents = buf.getvalue()
# or, to read from `buf` (which is a file-like object), call this first:
buf.seek(0) # to rewind the cursor to the start of the buffer
except Exception:
raise HTTPException(status_code=500, detail='Something went wrong')
finally:
file.file.close()
buf.close()
im.close()
For more details and code examples on how to upload files/images using FastAPI, please have a look at this answer and this answer. Also, please have a look at this answer for more information on defining your endpoint with def or async def.

I assume you are writing to a BytesIO to get an "in memory" JPEG without slowing yourself down by writing to disk and cluttering your filesystem.
If so, you want:
from PIL import Image
from io import BytesIO
im = Image.open(file.file)
im = im.convert("RGB")
im_io = BytesIO()
# create in-memory JPEG in RAM (not disk)
im.save(im_io, 'JPEG', quality=50)
# get the JPEG image in a variable called JPEG
JPEG = im_io.get_value()

How to get file path from UploadFile in FastAPI?

Basically, I'm trying to create an endpoint to Upload files to Amazon S3.
async def upload_files(filepath: str, upload_file_list: List[UploadFile] = File(...)):
for upload_file in upload_file_list:
abs_file_path = "/manual/path/works" + upload_file.path
# Replace above line to get absolute file path from UploadFile
response = s3_client.upload_file(abs_file_path,bucket_name,
os.path.join(dest_path, upload_file.filename))
Above is my code to upload multiple files to the S3 bucket.
s3_client.upload_file() accepts an absolute file path of the file to upload.
It is working when I manually put the full path.
This, however, didn't work:
response = s3_client.upload_file(upload_file.filename, bucket_name,
os.path.join(dest_path, upload_file.filename))
Is there a way to get this absolute path in FastAPI? Or, any alternative with temp_path without copying or writing the file?
If not, then any alternative with boto3 to upload files to S3 using FastAPI?

UploadFile uses Python's SpooledTemporaryFile, which is a "file stored in memory", and "is destroyed as soon as it is closed". You can either read the file contents (i.e., using contents = file.file.read() or for async read/write have a look at this answer), and then upload these bytes to your server (if it permits), or copy the contents of the uploaded file into a NamedTemporaryFile, as explained here. Unlike SpooledTemporaryFile, a NamedTemporaryFile "is guaranteed to have a visible name in the file system" that "can be used to open the file". That name can be retrieved from the name attribute (i.e., temp.name). Example:
from fastapi import HTTPException
#app.post("/upload")
def upload(file: UploadFile = File(...)):
temp = NamedTemporaryFile(delete=False)
try:
try:
contents = file.file.read()
with temp as f:
f.write(contents);
except Exception:
raise HTTPException(status_code=500, detail='Error on uploading the file')
finally:
file.file.close()
# Here, upload the file to your S3 service using `temp.name`
s3_client.upload_file(temp.name, 'local', 'myfile.txt')
except Exception:
raise HTTPException(status_code=500, detail='Something went wrong')
finally:
#temp.close() # the `with` statement above takes care of closing the file
os.remove(temp.name) # Delete temp file
Update
Additionally, one can access the actual Python file using the .file attribute. As per the documentation:
file: A SpooledTemporaryFile (a file-like object). This is the actual
Python file that you can pass directly to other functions or libraries
that expect a "file-like" object.
Thus, you could also try using upload_fileobj function and passing upload_file.file:
response = s3_client.upload_fileobj(upload_file.file, bucket_name, os.path.join(dest_path, upload_file.filename))
or, passing a file-like object using the ._file attribute of the SpooledTemporaryFile, which returns either an io.BytesIO or io.TextIOWrapper object (depending on whether binary or text mode was specified).
response = s3_client.upload_fileobj(upload_file.file._file, bucket_name, os.path.join(dest_path, upload_file.filename))
Update 2
You could even keep the bytes in an in-memory buffer (i.e., BytesIO), use it to upload the contents to the S3 bucket, and finally close it ("The buffer is discarded when the close() method is called."). Remember to call seek(0) method to reset the cursor back to the beginning of the file after you finish writing to the BytesIO stream.
contents = file.file.read()
temp_file = io.BytesIO()
temp_file.write(contents)
temp_file.seek(0)
s3_client.upload_fileobj(temp_file, bucket_name, os.path.join(dest_path, upload_file.filename))
temp_file.close()

Image processing after upload with Python Bottle

Context
I have made a simple web app for uploading content to a blog. The front sends AJAX requests (using FormData) to the backend which is Bottle running on Python 3.7. Text content is saved to a MySQL database and images are saved to a folder on the server. Everything works fine.
Image processing and PIL/Pillow
Now, I want to enable processing of uploaded images to standardise them (I need them all resized and/or cropped to 700x400px).
I was hoping to use Pillow for this. My problem is creating a PIL Image object from the file object in Bottle. I cannot initialise a valid Image object.
Code
# AJAX sends request to this route
#post('/update')
def update():
# Form data
title = request.forms.get("title")
body = request.forms.get("body")
image = request.forms.get("image")
author = request.forms.get("author")
# Image upload
file = request.files.get("file")
if file:
extension = file.filename.split(".")[-1]
if extension not in ('png', 'jpg', 'jpeg'):
return {"result" : 0, "message": "File Format Error"}
save_path = "my/save/path"
file.save(save_path)
The problem
This all works as expected, but I cannot create a valid Image object with pillow for processing. I even tried reloading the saved image using the save path but this did not work either.
Other attempts
The code below did not work. It caused an internal server error, though I am having trouble setting up more detailed Python debugging.
path = save_path + "/" + file.filename
image_data = open(path, "rb")
image = Image.open(image_data)
When logged manually, the path is a valid relative URL ("../domain-folder/images") and I have checked that I am definitely importing PIL (Pillow) correctly using PIL.PILLOW_VERSION.
I tried adapting this answer:
image = Image.frombytes('RGBA', (128,128), image_data, 'raw')
However, I won’t know the size until I have created the Image object. I also tried using io:
image = Image.open(io.BytesIO(image_data))
This did not work either. In each case, it is only the line trying to initialise the Image object that causes problems.
Summary
The Bottle documentation says the uploaded file is a file-like object, but I am not having much success in creating an Image object that I can process.
How should I go about this? I do not have a preference about processing before or after saving. I am comfortable with the processing, it is initialising the Image object that is causing the problem.
Edit - Solution
I got this to work by adapting the answer from eatmeimadanish. I had to use a io.BytesIO object to save the file from Bottle, then load it with Pillow from there. After processing, it could be saved in the usual way.
obj = io.BytesIO()
file.save(obj) # This saves the file retrieved by Bottle to the BytesIO object
path = save_path + "/" + file.filename
# Image processing
im = Image.open(obj) # Reopen the object with PIL
im = im.resize((700,400))
im.save(path, optimize=True)
I found this from the Pillow documentation about a different function that may also be of use.
PIL.Image.frombuffer(mode, size, data, decoder_name='raw', *args)
Note that this function decodes pixel data only, not entire images.
If you have an entire image file in a string, wrap it in a BytesIO object, and use open() to load it.

Use StringIO instead.
From PIL import Image
try:
import cStringIO as StringIO
except ImportError:
import StringIO
s = StringIO.StringIO()
#save your in memory file to this instead of a regular file
file = request.files.get("file")
if file:
extension = file.filename.split(".")[-1]
if extension not in ('png', 'jpg', 'jpeg'):
return {"result" : 0, "message": "File Format Error"}
file.save(s)
im = Image.open(s)
im.resize((700,400))
im.save(s, 'png', optimize=True)
s64 = base64.b64encode(s.getvalue())

From what I understand, you're trying to resize the image after it has been saved locally (note that you could try to do the resize before it is saved). If this is what you want to achieve here, you can open the image directly using Pillow, it does the job for you (you do not have to open(path, "rb"):
image = Image.open(path)
image.resize((700,400)).save(path)

Python flask ajax save decoded base64 image to server temporarily

I am resizing images client side before sending them to my flask app.
The resized image, which is drawn into a canvas to be resized, is sent via a POST request.
In my app the image is decoded via base64:
def resize_image(item):
content = item.split(';')[1]
image_encoded = content.split(',')[1]
body = base64.decodestring(image_encoded.encode('utf-8'))
return body
The imagedata is stored as type String in the body variable. I can save the data to my local machine and it works:
filename = 'some_image.jpg'
with open(filename, 'wb') as f:
print "written"
f.write(body)
What I need is to upload the resized image to AWS3. On one point I need to read() the image contents, but until the image is saved somewhere as a file it is still a String, so it fails:
file_data = request.values['a']
imagedata = resize_image(file_data)
s3 = boto.connect_s3(app.config['MY_AWS_ID'], app.config['MY_AWS_SECRET'], host='s3.eu-central-1.amazonaws.com')
bucket_name = 'my_bucket'
bucket = s3.get_bucket(bucket_name)
k = Key(bucket)
# fails here
file_contents = imagedata.read()
k.key = "my_images/" + "test.png"
k.set_contents_from_string(file_contents)
Unless there is an other solution, I thought I save the image temporarily to my server (Heroku) and upload it and then delete it, how would this work? Deleting afterwards is important here!

set_contents_from_string takes a string as a parameter, you could probably just pass your image string data directly to it for upload to S3
Solution:
Delete this part:
file_contents = imagedata.read()
Use imagedata directly here:
k.set_contents_from_string(imagedata)

If you need to call .read() on your data, but don't need save file on disk use StringIO:
import StringIO
output = StringIO.StringIO()
output.write('decoded image')
output.seek(0)
output.read()
Out[1]: 'decoded image'