I want to process a Pandas dataframe and send it to download as a CSV without a temp file. The best way to accomplish this I've seen is to use StringIO. Using the code below, a file downloads with the proper name, however the file is completely blank, and no error is shown. Why doesn't the file contain data?
#app.route('/test_download', methods = ['POST'])
def test_download():
buffer = StringIO()
buffer.write('Just some letters.')
buffer.seek(0)
return send_file(
buffer,
as_attachment=True,
download_name='a_file.txt',
mimetype='text/csv'
)
The issue here is that in Python 3 you need to use StringIO with csv.write and send_file requires BytesIO, so you have to do both.
#app.route('/test_download')
def test_download():
row = ['hello', 'world']
proxy = io.StringIO()
writer = csv.writer(proxy)
writer.writerow(row)
# Creating the byteIO object from the StringIO Object
mem = io.BytesIO()
mem.write(proxy.getvalue().encode())
# seeking was necessary. Python 3.5.2, Flask 0.12.2
mem.seek(0)
proxy.close()
return send_file(
mem,
as_attachment=True,
download_name='test.csv',
mimetype='text/csv'
)
Prior to Flask 2.0, download_name was called attachment_filename.
Use BytesIO to write bytes.
from io import BytesIO
from flask import Flask, send_file
app = Flask(__name__)
#app.route('/test_download', methods=['POST'])
def test_download():
# Use BytesIO instead of StringIO here.
buffer = BytesIO()
buffer.write(b'Just some letters.')
# Or you can encode it to bytes.
# buffer.write('Just some letters.'.encode('utf-8'))
buffer.seek(0)
return send_file(
buffer,
as_attachment=True,
download_name='a_file.txt',
mimetype='text/csv'
)
Prior to Flask 2.0, download_name was called attachment_filename.
make_response
To get Flask to download a csv file to the user, we pass a csv string to the make_response function, which returns a
Response object.
Then we add a Header which tells the browser to accept the file as a download.
The Mimetype also must be set to text/csv in order to get the web browser to save it in something other than an html document.
from flask import Flask, make_response
app = Flask(__name__)
#app.route('/test_download', methods=['POST'])
def test_download():
with StringIO() as buffer:
# forming a StringIO object
buffer = StringIO()
buffer.write('Just some letters.')
# forming a Response object with Headers to return from flask
response = make_response(buffer.getvalue())
response.headers['Content-Disposition'] = 'attachment; filename=namaste.csv'
response.mimetype = 'text/csv'
# return the Response object
return response
P.S. It is preferred to use python's built-in csv library to deal with csv files
References
https://matthewmoisen.com/blog/how-to-download-a-csv-file-in-flask/
https://www.geeksforgeeks.org/stringio-module-in-python/
https://docs.python.org/3/library/csv.html
Namaste 🙏
if someone use python 2.7 with Flask and got the error about the module StringIO by importing it. This post can help you to solve your problem.
If you are importing String IO module, you can just change the import syntax by using this : from io import StringIO instead from StringIO import StringIO.
You can Also use from io import BytesIO if you are using image or some others ressource.
Thank you
Related
I am having trouble downloading an html file through the flask send_file.
Basically, to download an html file alone, it works perfectly. by giving the stream to the send_file function as a parameter
However; I need to put this file into a zip along with other unrelated files. There, in the write function, neither the stream nor the string (result_html) work. I need somehow to transform it directly to an html file and put in the zip file
I don't see how I could do this for the moment. I have the data (output) as a dict...
Thank you if you have any pointers
from flask import render_template, send_file
from io import BytesIO
result_html = render_template('myResult.html', **output)
result_stream = BytesIO(str(result_html).encode())
with ZipFile("zipped_result.zip", "w") as zf:
zf.write(result_html)
# zf.write(other_files)
send_file(zf, as_attachment=True, attachment_filename="myfile.zip")
If I understand you correctly, it is sufficient to write the zip file in a stream and add the result of the rendering as a character string to the zip file. The stream can then be transmitted via send_file.
from flask import render_template, send_file
from io import BytesIO
from zipfile import ZipFile
# ...
#app.route('/download')
def download():
output = { 'name': 'Unknown' }
result_html = render_template('result.html', **output)
stream = BytesIO()
with ZipFile(stream, 'w') as zf:
zf.writestr('result.html', result_html)
# ...
stream.seek(0)
return send_file(stream, as_attachment=True, attachment_filename='archive.zip')
I am having an issue trying to download download in-memory ZIP-FILE object using Flask send_file. my zip exists in memory and is full of text documents but when I try with this code
the result I get is: it downloads like it is supposed to but it downloads an empty zip file! it's like it is copying nothing ... I have no idea how to solve this problem.
#app.route('/downloads/', methods=['GET'])
def download():
from flask import send_file
import io
import zipfile
import time
FILEPATH = r"C:\Users\JD\Downloads\trydownload.zip"
fileobj = io.BytesIO()
with zipfile.ZipFile(fileobj, 'w') as zip_file:
zip_info = zipfile.ZipInfo(FILEPATH)
zip_info.date_time = time.localtime(time.time())[:6]
zip_info.compress_type = zipfile.ZIP_DEFLATED
with open(FILEPATH, 'rb') as fd:
zip_file.writestr(zip_info, fd.read())
fileobj.seek(0)
return send_file(fileobj, mimetype='zip', as_attachment=True,
attachment_filename='%s.zip' % os.path.basename(FILEPATH))
I had the exact same issue with the Flask send_file method.
Details:
Flask version 2.0.1
OS: Windows 10
Solution
I figured out a workaround to this i.e. instead of the send_file method, this can be done by returning a Response object with the data. Replace the return statement in your code with the following and this should work.
#app.route('/downloads/', methods=['GET'])
def download():
from flask import Response # Changed line
import io
import zipfile
import time
FILEPATH = r"C:\Users\JD\Downloads\trydownload.zip"
fileobj = io.BytesIO()
with zipfile.ZipFile(fileobj, 'w') as zip_file:
zip_info = zipfile.ZipInfo(FILEPATH)
zip_info.date_time = time.localtime(time.time())[:6]
zip_info.compress_type = zipfile.ZIP_DEFLATED
with open(FILEPATH, 'rb') as fd:
zip_file.writestr(zip_info, fd.read())
fileobj.seek(0)
# Changed line below
return Response(fileobj.getvalue(),
mimetype='application/zip',
headers={'Content-Disposition': 'attachment;filename=your_filename.zip'})
In Flask (Flask-0.10.1 installed via pip) I've tried to handle uploaded files like this
f = flask.request.files[input_name]
stream = f.stream
# then use the stream
But it's confusing that in some cases stream is a BytesIO instance, but also a chance to be a file object.
I've tested in this way
from flask import Flask, request
import cStringIO
app = Flask('test')
#app.route("/", methods=['POST'])
def index():
if request.method == 'POST':
f = request.files['file']
print type(f.stream)
def send_file(client, name):
with open(name, 'rb') as f:
client.post('/', data={'file': (cStringIO.StringIO(f.read()), name)})
if __name__ == "__main__":
with app.test_client() as client:
send_file(client, 'homercat.png')
send_file(client, 'megacat-2.png')
It prints
<type '_io.BytesIO'>
<type 'file'>
The PNG files are from github:
http://octodex.github.com/images/homercat.png
http://octodex.github.com/images/megacat-2.png
I wonder why Flask behaves in such way. And if I want the uploaded data to go to the database, is it ok to call f.stream.read() in both cases?
Files smaller than 1024 * 500 bytes are written to a StringIO object, while files greater than that threshold are written to temporary files.
It's part of Werkzeug's testing framework, but that function isn't part of the online documentation:
def stream_encode_multipart(values, use_tempfile=True, threshold=1024 * 500,
boundary=None, charset='utf-8'):
"""Encode a dict of values (either strings or file descriptors or
:class:`FileStorage` objects.) into a multipart encoded string stored
in a file descriptor.
"""
...
Source
I have the following view code that attempts to "stream" a zipfile to the client for download:
import os
import zipfile
import tempfile
from pyramid.response import FileIter
def zipper(request):
_temp_path = request.registry.settings['_temp']
tmpfile = tempfile.NamedTemporaryFile('w', dir=_temp_path, delete=True)
tmpfile_path = tmpfile.name
## creating zipfile and adding files
z = zipfile.ZipFile(tmpfile_path, "w")
z.write('somefile1.txt')
z.write('somefile2.txt')
z.close()
## renaming the zipfile
new_zip_path = _temp_path + '/somefilegroup.zip'
os.rename(tmpfile_path, new_zip_path)
## re-opening the zipfile with new name
z = zipfile.ZipFile(new_zip_path, 'r')
response = FileIter(z.fp)
return response
However, this is the Response I get in the browser:
Could not convert return value of the view callable function newsite.static.zipper into a response object. The value returned was .
I suppose I am not using FileIter correctly.
UPDATE:
Since updating with Michael Merickel's suggestions, the FileIter function is working correctly. However, still lingering is a MIME type error that appears on the client (browser):
Resource interpreted as Document but transferred with MIME type application/zip: "http://newsite.local:6543/zipper?data=%7B%22ids%22%3A%5B6%2C7%5D%7D"
To better illustrate the issue, I have included a tiny .py and .pt file on Github: https://github.com/thapar/zipper-fix
FileIter is not a response object, just like your error message says. It is an iterable that can be used for the response body, that's it. Also the ZipFile can accept a file object, which is more useful here than a file path. Let's try writing into the tmpfile, then rewinding that file pointer back to the start, and using it to write out without doing any fancy renaming.
import os
import zipfile
import tempfile
from pyramid.response import FileIter
def zipper(request):
_temp_path = request.registry.settings['_temp']
fp = tempfile.NamedTemporaryFile('w+b', dir=_temp_path, delete=True)
## creating zipfile and adding files
z = zipfile.ZipFile(fp, "w")
z.write('somefile1.txt')
z.write('somefile2.txt')
z.close()
# rewind fp back to start of the file
fp.seek(0)
response = request.response
response.content_type = 'application/zip'
response.app_iter = FileIter(fp)
return response
I changed the mode on NamedTemporaryFile to 'w+b' as per the docs to allow the file to be written to and read from.
current Pyramid version has 2 convenience classes for this use case- FileResponse, FileIter. The snippet below will serve a static file. I ran this code - the downloaded file is named "download" like the view name. To change the file name and more set the Content-Disposition header or have a look at the arguments of pyramid.response.Response.
from pyramid.response import FileResponse
#view_config(name="download")
def zipper(request):
path = 'path_to_file'
return FileResponse(path, request) #passing request is required
docs:
http://docs.pylonsproject.org/projects/pyramid/en/latest/api/response.html#
hint: extract the Zip logic from the view if possible
XLRD is installed and tested:
>>> import xlrd
>>> workbook = xlrd.open_workbook('Sample.xls')
When I read the file through html form like below, I'm able to access all the values.
xls_file = request.params['xls_file']
print xls_file.filename, xls_file.type
I'm using Pylons module, request comes from: from pylons import request, tmpl_context as c
My questions:
Is xls_file read through requst.params an object?
How can I read xls_file and make it work with xlrd?
Update:
The xls_file is uploaded on web server, but the xlrd library expects a filename instead of an open file object, How can I make the uploaded file to work with xlrd? (Thanks to Martijn Pieters, I was being unable to formulate the question clearly.)
xlrd does support providing data directly without a filepath, just use the file_contents argument:
xlrd.open_workbook(file_contents=fileobj.read())
From the documentation:
file_contents – A string or an mmap.mmap object or some other behave-alike object. If file_contents is supplied, filename will not be used, except (possibly) in messages.
What I met is not totally the same with the question, but I think maybe it is similar and I can give some hints.
I am using django rest framework's request instead of pylons request.
If I write simple codes like following:
#api_view(['POST'])
#renderer_classes([JSONRenderer])
def upload_files(request):
file_obj = request.FILES['file']
from xlrd import open_workbook
wb = open_workbook(file_contents=file_obj.read())
result = {"code": "0", "message": "success", "data": {}}
return Response(status=200, data=result)
Here We can read using open_workbook(file_contents=file_obj.read()) as mentioned in previous comments.
But if you write code in following way:
from rest_framework.views import APIView
from rest_framework.parsers import MultiPartParser
class FileUploadView(APIView):
parser_classes = (MultiPartParser,)
def put(self, request, filename, format=None):
file_obj = request.FILES.get('file')
from xlrd import open_workbook
wb = open_workbook(file_contents=file_obj.read())
# do some stuff with uploaded file
return Response(status=204)
You must pay attention that using MultiPartParser instead of FileUploadParser, using FileUploadParser will raise some BOF error.
So I am wondering somehow it is also affected by how you write the API.
For me this code works. Python 3
xlrd.open_workbook(file_contents=fileobj.content)
You could try something like...
import xlrd
def newopen(fileobject, modes):
return fileobject
oldopen = __builtins__.open
__builtins__.open = newopen
InputWorkBook = xlrd.open_workbook(fileobject)
__builtins__.open = oldopen
You may have to wrap the fileobject in StringIO if it isn't already a file handle.