How to serializer a file object while using json.dumps ?
I 'm using pytest for testing file upload in django and I 've this function
def test_file_upload(self):
# file_content is a bytest object
request = client.patch(
"/fake-url/",
json.dumps({"file" : file_content}),
content_type="application/json",
)
I 've tried to set the file_content as a bytes object but I 'm getting this error TypeError: Object of type bytes is not JSON serializable I need to send the whole file to my endpoint as json serialized
You can use it mock library for testing file upload;
from mock import MagicMock
from django.core.files import File
mock_image = magicMock(file=File)
mock_image.name="sample.png"
# Another test operations...
def test_file_upload(self):
# file_content is a bytest object
request = client.patch(
"/fake-url/",
{"file" : mock_image},
format="multipart",
)
Detailed another answer; how to unit test file upload in django
Your API endpoint expects a multipart form containing file. Below is the function I use to send a multipart form from a local file for testing. If you already have file bytes, skip the open line and just use file_content in ContentFile.
def send_multipart_form(self, filename):
with open(filename, "rb") as f:
file_data = ContentFile(f.read(), os.path.basename(filename))
res = self.client.put(
self.url,
data=encode_multipart(boundary=BOUNDARY, data={"file": file_data}),
content_type=MULTIPART_CONTENT,
)
return res
Related
I'm trying to let a user download a file from my webpage. The file is in an S3 bucket I'm accessing using smart-open. The issue I'm having is how to combine that with FileReader. Presently I'm getting a TypeError: "expected str, bytes or os.PathLike object, not Reader". The filetypes will be .txt, .pdf and .doc/.docx
My view:
#api_view(['GET'])
def download_attachment(request, pk):
session = boto3.Session(aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY)
obj = Attachment.objects.get(id=pk)
with smart_opener(f's3://______media/public/{str(obj)}',"rb",transport_params={'client':session.client('s3')}) as attachment:
response = FileResponse(open(attachment, 'rb'))
response['Content-Disposition'] = f'attachment; filename={obj.file.name}'
return response
open function expects a filesystem path as the first positional argument to open a file located in your computer's filesystem. That is why you are getting TypeError. I don't know what smart_opener() does but if it returns a file-like object this object must be passed to FileResponse directly.
I need help uploading a file directly from an HTML form to an API. I've seen this being done for remote URLs, but I don't know how to do this for local files? I tried writing this, but its not working:
uploadmedia = request.files['fileupload']
client = Client('thisismykey')
with open(uploadmedia, 'rb') as file:
new_upload = client.uploads('<space-id>').create(file)
The line client.uploads is what is specified in the API docs here. I just need to be able to get the file path.
The comments suggest the following:
# you can use either a file-like object or a path.
# If you use a path, the SDK will open it, create the upload and
# close the file afterwards.
I am assuming that request.files['fileupload'] is a file like object, so I just passed that along.
The above code gives me the following error:
File "D:\Gatsby\submission\flask-tailwindcss-starter\app\__init__.py", line 28, in index
with open(uploadmedia, 'rb') as file:
TypeError: expected str, bytes or os.PathLike object, not FileStorage
I know that in this example, uploadmedia.filename would get me the file's name, but what is the attribute for the path? How do I do that?
The request.files['file'] is an instance of a FileStorage class. refer to api, you cannot use with open(uploadmedia, 'rb') as file: .
try using stream attribute :
uploadmedia = request.files['fileupload']
client = Client('thisismykey')
new_upload = client.uploads('<space-id>').create(uploadmedia.stream)
I have a flask server that grabs binary data for several different files from a database and puts them into a python 'zipfile' object. I want to send the generated zip file with my code using flask's "send_file" method.
I was originally able to send non-zip files successfully by using the BytesIO(bin) as the first argument to send_file, but for some reason I can't do the same thing with my generated zip file. It gives the error:
'ZipFile' does not have the buffer interface.
How do I send this zip file object to the user with Flask?
This is my code:
#app.route("/getcaps",methods=['GET','POST'])
def downloadFiles():
if request.method == 'POST':
mongo = MongoDAO('localhost',27017)
identifier = request.form['CapsuleName']
password = request.form['CapsulePassword']
result = mongo.getCapsuleByIdentifier(identifier,password)
zf = zipfile.ZipFile('capsule.zip','w')
files = result['files']
for individualFile in files:
data = zipfile.ZipInfo(individualFile['fileName'])
data.date_time = time.localtime(time.time())[:6]
data.compress_type = zipfile.ZIP_DEFLATED
zf.writestr(data,individualFile['fileData'])
return send_file(BytesIO(zf), attachment_filename='capsule.zip', as_attachment=True)
return render_template('download.html')
BytesIO() needs to be passed bytes data, but a ZipFile() object is not bytes-data; you actually created a file on your harddisk.
You can create a ZipFile() in memory by using BytesIO() as the base:
memory_file = BytesIO()
with zipfile.ZipFile(memory_file, 'w') as zf:
files = result['files']
for individualFile in files:
data = zipfile.ZipInfo(individualFile['fileName'])
data.date_time = time.localtime(time.time())[:6]
data.compress_type = zipfile.ZIP_DEFLATED
zf.writestr(data, individualFile['fileData'])
memory_file.seek(0)
return send_file(memory_file, attachment_filename='capsule.zip', as_attachment=True)
The with statement ensures that the ZipFile() object is properly closed when you are done adding entries, causing it to write the required trailer to the in-memory file object. The memory_file.seek(0) call is needed to 'rewind' the read-write position of the file object back to the start.
I'm trying to let the user download an xml file that i have generated.
This is my code:
tree.write('output.xml', encoding="utf-16")
# Pathout is the path to the output.xml
xmlFile = open(pathout, 'r')
myfile = FileWrapper(xmlFile.read())
response = HttpResponse(myfile, content_type='application/xml')
response['Content-Disposition'] = 'attachment; filename='+filename
return response
When I try to create my response I get this exception:
'\\'str\\' object has no attribute \\'read\\''
Can't figure out what I'm doing wrong. Any ideas?
Edit:
When I use this code I get no errors but instead the downloaded file is empty
tree.write('output.xml', encoding="utf-16")
xmlFile = open(pathout, 'r')
myfile = FileWrapper(xmlFile)
response = HttpResponse(myfile, content_type='application/xml')
response['Content-Disposition'] = 'attachment; filename='+filename
return response
You are calling xmlFile.read() - which yields a string - and passing the result to FileWrapper() which expects a readable file-like object. You should either just pass xmlFile to FileWrapper, or not use FileWrapper at all and pass the result of xmlFile.read() as your HttpResponse body.
Note that if you are creating the xml dynamically (which seems to be the case according to your snippet's first line), writing it to disk only to read it back a couple lines later is both a waste of time and resources and a potential cause of race conditions. You perhaps want to have a look at https://docs.python.org/2/library/xml.etree.elementtree.html#xml.etree.ElementTree.tostring
You're reading the file and passing the resulting string to FileWrapper, instead of passing the actual file object.
myfile = FileWrapper(xmlFile)
Alternatively from the other answers, I would recommend to completely go around the problem by utilizing the Django template system:
from django.http import HttpResponse
from django.template import Context, loader
def my_view(request):
# View code here...
t = loader.get_template('myapp/myfile.xml')
c = Context({'foo': 'bar'})
response = HttpResponse(t.render(c), content_type="application/xml")
response['Content-Disposition'] = 'attachment; filename=...'
return response
In this way create a myfile.xml template which is used to render the proper xml response without having to deal with writing any files to the filesystem. This is cleaner and faster, given that there is no other need to indeed create the xml and store it permanently.
I have a flask server that grabs binary data for several different files from a database and puts them into a python 'zipfile' object. I want to send the generated zip file with my code using flask's "send_file" method.
I was originally able to send non-zip files successfully by using the BytesIO(bin) as the first argument to send_file, but for some reason I can't do the same thing with my generated zip file. It gives the error:
'ZipFile' does not have the buffer interface.
How do I send this zip file object to the user with Flask?
This is my code:
#app.route("/getcaps",methods=['GET','POST'])
def downloadFiles():
if request.method == 'POST':
mongo = MongoDAO('localhost',27017)
identifier = request.form['CapsuleName']
password = request.form['CapsulePassword']
result = mongo.getCapsuleByIdentifier(identifier,password)
zf = zipfile.ZipFile('capsule.zip','w')
files = result['files']
for individualFile in files:
data = zipfile.ZipInfo(individualFile['fileName'])
data.date_time = time.localtime(time.time())[:6]
data.compress_type = zipfile.ZIP_DEFLATED
zf.writestr(data,individualFile['fileData'])
return send_file(BytesIO(zf), attachment_filename='capsule.zip', as_attachment=True)
return render_template('download.html')
BytesIO() needs to be passed bytes data, but a ZipFile() object is not bytes-data; you actually created a file on your harddisk.
You can create a ZipFile() in memory by using BytesIO() as the base:
memory_file = BytesIO()
with zipfile.ZipFile(memory_file, 'w') as zf:
files = result['files']
for individualFile in files:
data = zipfile.ZipInfo(individualFile['fileName'])
data.date_time = time.localtime(time.time())[:6]
data.compress_type = zipfile.ZIP_DEFLATED
zf.writestr(data, individualFile['fileData'])
memory_file.seek(0)
return send_file(memory_file, attachment_filename='capsule.zip', as_attachment=True)
The with statement ensures that the ZipFile() object is properly closed when you are done adding entries, causing it to write the required trailer to the in-memory file object. The memory_file.seek(0) call is needed to 'rewind' the read-write position of the file object back to the start.