I am reading user inputs and sending them for processing. After processing, the results are displayed. Along with the results I want a link on webpage to be able to download the data as a csv file. My function to process inputs looks as follows.
#app.route('/process', methods=['POST'])
def process_data():
# create csv_file
return render_template("results.html", data=csv_file)
results.html has following line.
<p> <large>Download simulation data </large></p>
This link is correctly displayed on the webpage. My function to download the data looks as follows.
#app.route('/download/<filename>')
def download(filename):
response = make_response(filename)
response.headers["Content-Disposition"] = "attachment; filename=Simulation.csv"
response.headers["Content-Type"] = "text/csv"
return response
Clicking the download link, I get '414 Request-URI Too Large'.
Is there a better solution for passing data from flask to html to flask again?
I can see that my entire data are appended to url, can I somehow avoid that?
Is it possible to directly pass response while rendering results.html and make it downloadable?
UPDATE
I learnt that putting data in url is a bad idea. Instead I can use dataurl by encoding the data and then use dataurl for csv in href tag of html.
buffer = StringIO()
dataframe.to_csv(buffer, index=False)
buffer.seek(0)
data_str = base64.b64encode(buffer.getvalue().encode('utf8')).decode('ascii')
url = "data:text/csv; base64,{}".format(data_str)
html looks like as follows.
<a download="SSW_Simulation.csv" href="{{ data_url }}">download</a>
However this solution does not work in internet explorer I guess because urls for data are not supported. Should I be saving the csv file somewhere and pass that filename to html? so that when prompted, I can fetch it from the temporary location and download using make_response? I would prefer not to save the file to the disk.
Handling the data in javascript solved the problem as suggested by #Jeronimo. json string was passed to html page.
import json
#app.route('/process', methods=['POST'])
def process_data():
# create csv_file
data = json.dumps(csv_file)
return render_template("results.html", data=data)
Javascript suggested in this answer was added to html and along with download button.
<button onclick="download({{ data }}, 'myfile.csv', 'text/csv')">download data</button>
Related
I am writing a python script that generates a html file out of some images and svg files, which later should be converted into a pdf file.
For the conversion I am using weasyprint, but for some reason it doesnt render the images, only the according alt-text.
Here is a screenshot of the html file rendered in Chrome:
But the generated pdf looks like this:
Here is the code that should generate the pdf:
html = HTML(filename="path/to/file")
html.write_pdf("path/to/output/file")
And here is the according html file:
<img src="C:/Users/jonas/Documents/Dev/PythonLabel/bin/LABEL_SVG.svg" alt="SVG of Label" />
<div class="sn-number">3253343345</div>
<img class="qr-code" src="C:/Users/jonas/Documents/Dev/PythonLabel/bin/qrcode.png" alt="qrcode" />
I dont know what causes the problem, but Firefox isnt able to render the images either, but Chrome and Brave manage to do so. Maybe both problems have a similar reason.
So I managed to get it to work, but its not a pretty solution.
I figured out, that weasyprint was able to get images and SVGs from actual webpages like stackoverflow. So I made a little flask app that would only serve the one html file, and after the PDF was created I just stopped the flask server.
from flask import Flask
from multiprocessing import Process
app = Flask(
__name__, static_folder=path/to/static/folder)
#app.route("/")
def home():
with open(path/to/html, "r") as f:
html = "".join(f.readlines())
return html
def start_server():
app.run()
if __name__ == "__main__":
server = Process(target=start_server)
server.start()
sleep(4)
print("Generating PDF...")
html = HTML(
"http://localhost:5000").write_pdf(path/to/output/file)
server.terminate()
server.join()
I am trying to have my server, in python 3, go grab files from URLs. Specifically, I would like to pass a URL into a function, I would like the function to go grab an audio file(of many varying formats) and save it as an MP3, probably using ffmpeg or ffmpy. If the URL also has a PDF, I would also like to save that, as a PDF. I haven't done much research on the PDF yet, but I have been working on the audio piece and wasn't sure if this was even possible.
I have looked at several questions here, but most notably;
How do I download a file over HTTP using Python?
It's a little old but I tried several methods in there and always get some sort of issue. I have tried using the requests library, urllib, streamripper, and maybe one other.
Is there a way to do this and with a recommended library?
For example, most of the ones I have tried do save something, like the html page, or an empty file called 'file.mp3' in this case.
Streamripper received a try changing user agents error.
I am not sure if this is possible, but I am sure there is something I'm not understanding here, could someone point me in the right direction?
This isn't necessarily the code I'm trying to use, just an example of something I have used that doesn't work.
import requests
url = "http://someurl.com/webcast/something"
r = requests.get(url)
with open('file.mp3', 'wb') as f:
f.write(r.content)
# Retrieve HTTP meta-data
print(r.status_code)
print(r.headers['content-type'])
print(r.encoding)
**Edit
import requests
import ffmpy
import datetime
import os
## THIS SCRIPT CAN BE PASSED A URL AND IF THE URL RETURNS
## HTTP HEADER FOR CONTENT TYPE AUDIO/MPEG, THE FILE WILL
## BE SAVED AS THE CURRENT-DATE-AND-TIME.MP3
##
## THIS SCRIPT CAN BE PASSED A URL AND IF THE URL RETURNS
## HTTP HEADER FOR CONTENT TYPE application/pdf, THE FILE WILL
## BE SAVED AS THE CURRENT-DATE-AND-TIME.PDF
##
## THIS SCRIPT CAN BE PASSED A URL AND IF THE URL RETURNS
## HTTP HEADER FOR CONTENT TYPE other than application/pdf, OR
## audio/mpeg, THE FILE WILL NOT BE SAVED
def BordersPythonDownloader(url):
print('Beginning file download requests')
r = requests.get(url, stream=True)
contype = r.headers['content-type']
if contype == "audio/mpeg":
print("audio file")
filename = '[{}].mp3'.format(str(datetime.datetime.now()))
with open('file.mp3', 'wb+') as f:
f.write(r.content)
ff = ffmpy.FFmpeg(
inputs={'file.mp3': None},
outputs={filename: None}
)
ff.run()
if os.path.exists('file.mp3'):
os.remove('file.mp3')
elif contype == "application/pdf":
print("pdf file")
filename = '[{}].pdf'.format(str(datetime.datetime.now()))
with open(filename, 'wb+') as f:
f.write(r.content)
else:
print("URL DID NOT RETURN AN AUDIO OR PDF FILE, IT RETURNED {}".format(contype))
# INSERT YOUR URL FOR TESTING
# OR CALL THIS SCRIPT FROM ELSEWHERE, PASSING IT THE URL
#DEFINE YOUR URL
#url = 'http://archive.org/download/testmp3testfile/mpthreetest.mp3'
#CALL THE SCRIPT; PASSING IT YOUR URL
#x = BordersPythonDownloader(url)
#ANOTHER EXAMPLE WITH A PDF
#url = 'https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SY/configuration/guide/sy_swcg/etherchannel.pdf'
#x = BordersPythonDownloader(url)
Thanks Richard, this code works and helps me understand this better. Any suggestions for improving the above working example?
I'm trying to use use python to go to this page, https://comtrade.un.org/data/, fill in the form, and "click" the download button. Then get the csv file that is generated.
Anyone have some sample code for automating the download in python?
Thx.
You might be interested in trying out pywinauto. I have not had too much experience, but I do believe it could do the job.
Good luck!
The site you are accessing has an exposed API and you can use that form to generate the API URL and simply call that to return a JSON or CSV response. To get this with Python you can use requests and the core JSON module to parse the data if you want to use the data inside Python:
CSV File
import requests
api_url = 'https://comtrade.un.org/api/get?max=500&type=C&freq=A&px=HS&ps=2017&r=all&p=0&rg=all&cc=TOTAL&fmt=csv'
response = requests.get(api_url)
data = response.content
with open('output.csv', 'wb') as output:
output.write(data)
Note the fmt=csv property in the URL.
Python Dictionary
import requests, json
api_url = 'https://comtrade.un.org/api/get?max=500&type=C&freq=A&px=HS&ps=2017&r=all&p=0&rg=all&cc=TOTAL'
response = requests.get(api_url)
data = json.loads(response.content)
print(data)
Note that the API URL in the example came from submitting the default form and clicking 'View API Call'. Under the generated table.
I am dynamically generating zip file with result after user searches for data. I want to return the zip file as a json response and allow user to download it when download button is clicked.
Below is the code i have used which returns HttpResponse. I want to convert this to a json response. I tried a few approaches and it didn't work. I am not sure how the url should be specified in the template and also the url pattern for the same.
Below is the code I have used in the view:
filename = "{}_{}.txt".format(query_word, doc_id)
zip_dir = "result_for_query_{}.zip".format(query_word)
z = zipfile.ZipFile(zip_dir, 'a')
self.write_to_file(filename, doc, z)
z.close()
response = HttpResponse(z,content_type='application/zip')
response["Content-Disposition"] = "attachment; filename=%s" % zip_dir
return response
And my template:
Download
I am new to programming. So, I am still learning how to write urlpatterns.
I'm currently creating an app thats supposed to take a input in form of a url (here a PDF-file) and recognize this as a PDF and then upload it to a tmp folder i have on a server.
I have absolutely no idea how to proceed with this. I've already made a form which contains a FileField which works perfectly, but when it comes to urls i have no clue.
Thank you for all answers, and sorry about the lacking english skills.
The first 4 bytes of a pdf file are %PDF so you could just download the first 4 bytes from that url and compare them to %PDF. If it matches, then download the whole file.
Example:
import urllib2
url = 'your_url'
req = urllib2.urlopen(url)
first_four_bytes = req.read(4)
if first_four_bytes == '%PDF':
pdf_content = urllib2.urlopen(url).read()
# save to temp folder
else:
# file is not PDF