I'd like to download several files with GAE Python code.
My current code is like below
import webapp2, urllib
url1 = 'http://dummy/sample1.jpg'
url2 = 'http://dummy/sample2.jpg'
class DownloadHandler(webapp2.RequestHandler):
def get(self):
#image1
self.response.headers['Content-Type'] = 'application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="' + 'sample1.jpg' + '"'
f = urllib.urlopen(url1)
data = f.read()
self.response.out.write(data)
#image2
self.response.headers['Content-Type'] = 'application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="' + 'sample2.jpg' + '"'
f = urllib.urlopen(url2)
data = f.read()
self.response.out.write(data)
app = webapp2.WSGIApplication([('/.*', DownloadHandler)],
debug=True)
I expected to occur download dialogue twice with this code, but actually occurred once, and only sample2.jpg was downloaded.
How can you handle download dialogue several times?
I'd actually like to realize some other functions adding above as well.
To display progressing message on the browser such as
sample1.jpg was downloaded
sample2.jpg was downloaded
sample3.jpg was downloaded ...
And redirect to the other page after downloading files.
When I wrote a code such as
self.redirect('/otherpage')
after
self.response.out.write(data)
Only redirect had happened and didn't occur download procedure.
Would you give me any ideas to solve it please.
I'm using python2.7
Two things.
You cannot write two files in one response that has a Content-Type of application/octet-stream. To stuff multiple files in in the response, you would have to encode your response with multipart/form-data or multipart/mixed and hope that the client would understand that and parse it and show two download dialogues
Once you've already called self.response.out.write(…), you shouldn't be setting any more headers.
To me it seems that the most foolproof option would be to serve an HTML file that contains something like:
<script>
window.open('/path/to/file/1.jpg');
window.open('/path/to/file/1.jpg');
</script>
… and then handle those paths using different handlers.
Another option would be to zip the two files and serve the zipfile to the client, though it may or may not be preferable in your case.
I reached the goal what I wanted to do.
As user interaction, generating html sources include below
<script type="text/javascript">
window.open("/download?url=http://dummy/sample1.jpg")
window.open("/download?url=http://dummy/sample2.jpg")
</script>
then created new windows are handled with this code.
class DownloadHandler(webapp2.RequestHandler):
def get(self):
url = self.request.get('url')
filename = str(os.path.basename(url))
self.response.headers['Content-Type'] ='application/octet-stream'
self.response.headers['Content-Disposition'] = 'attachment; filename="%s"' % (filename)
data = urllib.urlopen(url).read()
self.response.out.write(data)
app = webapp2.WSGIApplication([('/download', DownloadHandler)], debug=True)
Thank you, Attila.
Related
I have a flask app that was created using a csv file. Using html forms, I give the user the option to input new data, and then I write changes to the csv (using a custom function) and download it. This downloads just fine and saves to my desktop. Is there any way to tweak the code to save it in the same project directory and overwrite the csv that serves the flask app? This way the app might update upon refresh. Thanks!
#app.route('/csv/')
def download_csv():
model_id=request.args['textid']
client_id = session['client_id']
# return response
df=recommender.update_history(client_id, model_id)
df= recommender.get_csv()
resp = make_response(df.to_csv(encoding='iso-8859-1',index=False))
resp.headers["Content-Disposition"] = "attachment; filename=export.csv"
resp.headers["Content-Type"] = "text/csv"
return resp
The comment above is correct. I didn't need the make_response function. Here's what worked:
#app.route('/csv/')
def download_csv():
model_id=request.args['textid']
client_id = session['client_id']
# return response
df=recommender.update_history(client_id, model_id)
df= recommender.get_csv()
path=r'data/file_name.csv'
resp = df.to_csv(path, encoding='iso-8859-1',index=False)
return render_template('index.html')
I am downloading file from Hadoop to Django backend and storing the file using the code below:
import shutil
import requests
url = 'http://112.138.0.12:9870/webhdfs/v1/user/username/1.jpg?op=OPEN&user.name=username'
response = requests.get(url, stream=True)
with open('img.png', 'wb') as out_file:
shutil.copyfileobj(response.raw, out_file)
del response
I don't need to store the file in backend local system since I want to send this file to Angular 5 frontend where user will save this file in their local system. I'm getting the following error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position
0: invalid start byte.
Can someone suggest me what would be the right way to do download large files in a short time?
DJANGO:
views.py:
class DownloadFileView(GenericAPIView):
serializer_class = UserNameSerializer
def get(self, request):
key = request.META.get('HTTP_AUTHORIZATION').split()[1]
user_id = Token.objects.get(key=key).user_id
user_name = User.objects.get(id=user_id).username
response = download_files(user_name)
return Response(response)
def download_files(user_name):
response = requests.get('http://112.138.0.12:9870/webhdfs/v1/user/' + user_name + '/1.jpg?op=OPEN&user.name=username', stream=True)
return response.raw
ANGULAR:
DownloadFile(){
this.userService.DownloadFiles().subscribe((data : any) => {
const blob = new Blob([data], { type: 'application/octet-stream'});
fileUrl = this.sanitizer.bypassSecurityTrustResourceUrl(window.URL.createObjectURL(blob));
}
}
DownloadFiles() {
this.token = localStorage.getItem('userToken')
var reqHeader = new HttpHeaders({ 'Content-Type': 'application/octet-stream', 'Authorization': 'token ' + this.token });
console.log(reqHeader)
return this.http.get(this.rootURL + 'download/', { headers: reqHeader});
}
To begin with your unicode error, it's because:
HttpResponse.init(content='', content_type=None, status=200,
reason=None, charset=None)
Instantiates an HttpResponse
object with the given page content and content type.
content should be an iterator or a string. If it’s an iterator, it
should return strings, and those strings will be joined together to
form the content of the response. If it is not an iterator or a
string, it will be converted to a string when accessed.
I do believe django is having trouble converting the binary data in the file to string. A more common approach when dealing with file downloads is:
response = HttpResponse(content_type="application/jpeg")
response.write(binary_data)
This works because there is a call to make_bytes behind the scenes which handles the binary data correctly.
Having said that, this is not the most efficient way to go about it. Your web app makes a request to a remote server using requests, and then passes that onto the client. Why not get your angular code to fetch the data directly from the end point?
Can't do that because you want authentication you say? Ok, How about checking the authentiation and then sending an HttpResponseDirect like this:
return HttpResponseRedirect('http://112.138.0.12:9870/webhdfs/v1/user/' + user_name + '/1.jpg?op=OPEN&user.name=username')
I'm working on a python script that will:
1) pull GIS metadata from an enterprise database
2) parse the metadata from XML to plain text
3) attach the text files to the corresponding published datasets in Socrata (which are published monthly)
4) The script will also be run monthly, so that any schema changes in the enterprise dataset are reflected in the attached plain text metadata files on Socrata.
I've been able to successfully attach the text metadata files to published Socrata datasets using some code found here. The problem is, each time the script is run, an additional attachment is added. I would like to either delete the existing attachment and add a new one, or overwrite the existing attachment with the contents of the new one.
I've done a fair amount of research on this, and can't seem to find any documentation for managing attachments using the Socrata API. Any suggestions?
I ended up figuring this one out. Had to alter a few lines to empty out the attachments in the attach_file function in the Socrata Python library from:
def attach_file(self, filename):
metadata = self.metadata()
if not metadata.has_key('attachments'):
metadata['attachments'] = []
to:
def attach_file(self, filename):
metadata = self.metadata()
metadata['attachments'] = [] #empty out metadata, regardless of existing metadata
Ended up using the same API and was able to replace attachments using the following code:
def attach_file(self, filename, clear_metadata):
metadata = self.metadata()
if not metadata.has_key('attachments'):
metadata['attachments'] = []
# if the user wants to clear all existing attachments on dataset
if clear_metadata:
metadata['attachments'] = []
response = self.multipart_post('/assets', filename)
if not response.has_key('id'):
print "Error uploading file to assets service: no ID returned: %s" % response
return
attachment = {'blobId': response['id'],
'name': response['nameForOutput'],
'filename': response['nameForOutput']}
metadata['attachments'].append(attachment)
self._request("/views/%s.json" % self.id, 'PUT', {'metadata':metadata})
def multipart_post(self, url, filename, field='file'):
print("Running multipart_post")
authBase64 = base64.encodestring('%s:%s' % (self.username, self.password)).replace('\n', '')
datagen, headers = multipart_encode({field: open(filename, "rb")})
headers['X-App-Token'] = self.app_token
headers['Authorization'] = "Basic %s" % authBase64
print("url=" + url)
request = Request("%s%s" % (self.url, url), datagen, headers)
print(str(Request))
response = urlopen(request).read()
return json.loads(response)
In my application, I'm rendering a PDF file and pass it back as a response. For this purpose I'm using flask_weasyprint's render_pdf, which does exactly this:
def render_pdf(html, stylesheets=None, download_filename=None):
if not hasattr(html, 'write_pdf'):
html = HTML(html)
pdf = html.write_pdf(stylesheets=stylesheets)
response = current_app.response_class(pdf, mimetype='application/pdf')
if download_filename:
response.headers.add('Content-Disposition', 'attachment', filename=download_filename)
return response
I now need to render a template + returning the rendered pdf as a download. Something like
#app.route("/view")
def view() :
resp1 = render_pdf(HTML(string="<p>Render me!</p>"), download_filename = "test.pdf")
resp2 = render_template("test.html")
return resp1, resp2
Is there any way to achieve this? Any workaround?
I am not sure if this is solvable in the backend, you want to send two http responses following one request. Should that be possible? (I really don't know) Shouldn't the client make two responses? (javascript).
An option would be, javascript datablob returned in your render_template call.
Maybe something like this? (untested):
fileData = new Blob([pdf_data_here], { type: 'application/pdf' });
fileUrl = URL.createObjectURL(fileData);
window.location.assign(fileUrl);
Or maybe just use the window.location.assign() function to generate the second request.
Or put the data base64 encoded in a href attribute?
I'm back. :) Again trying to get the gzipped contents of a URL and gunzip them. This time in Python. The #SERVER section of code is the script I'm using to generate the gzipped data. The data is known good, as it works with Java. The #CLIENT section of code is the bit of code I'm using client-side to try and read that data (for eventual JSON parsing). However, somewhere in this transfer, the gzip module forgets how to read the data it created.
#SERVER
outbuf = StringIO.StringIO()
outfile = gzip.GzipFile(fileobj = outbuf, mode = 'wb')
outfile.write(data)
outfile.close()
print "Content-Encoding: gzip\n"
print outbuf.getvalue()
#CLIENT
urlReq = urllib2.Request(url)
urlReq.add_header('Accept-Encoding', '*')
urlConn = urllib2.build_opener().open(urlReq)
urlConnObj = StringIO.StringIO(urlConn.read())
gzin = gzip.GzipFile(fileobj = urlConnObj)
return gzin.read() #IOError: Not a gzipped file.
Other Notes:
outbuf.getvalue() is the same as urlConnObj.getvalue() is the same as urlConn.read()
This StackOverflow question seemed to help me out.
Apparently, it was just wise to by-pass the gzip module entirely, opting for zlib instead. Also, changing "*" to "gzip" in the "Accept-Encoding" header may've helped.
#CLIENT
urlReq = urllib2.Request(url)
urlReq.add_header('Accept-Encoding', 'gzip')
urlConn = urllib2.urlopen(urlReq)
return zlib.decompress(urlConn.read(), 16+zlib.MAX_WBITS)