I'd like to create a small app in Flask which:
1. Allows the user to upload data, in csv
2. Does stuff to the data (in this example, nothing will be done)
3. Plot results
4. Allow the user to download a plot of the results as pdf.
Code so far:
app.py
from flask import Flask, Response, request, render_template
import matplotlib.pyplot as plt
import matplotlib
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.backends.backend_svg import FigureCanvasSVG
import io
import csv
import pandas as pd
app = Flask(__name__)
#app.route("/", methods=['POST', 'GET'])
def index():
return render_template("index.html")
#app.route('/transform', methods=["GET", "POST"])
def transform_view():
""" process data and render plot on the fly.
"""
# GET input csv
f = request.files['input_file']
if not f:
return "Main input file not found!"
# Read uploaded data as a Stream into a dataframe.
stream = io.StringIO(f.stream.read().decode("UTF8"), newline=None)
csv_input = csv.reader(stream)
contents = []
for row in csv_input:
contents.append(row)
d0 = pd.DataFrame(data=contents[1:], columns=contents[0])
# Change to numeric columns
d0[["col1", "col2"]] = d0[["col1", "col2"]].apply(pd.to_numeric)
# Plot
fig, ax = plt.subplots()
bplot = ax.boxplot([d0["col1"], d0["col2"]])
# Print out plot as pdf?
#output = io.BytesIO()
return render_template("index.html")
if __name__ == "__main__":
app.run(debug=True)
index.html
<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body>
<form action="/transform" method="post" enctype="multipart/form-data">
<h3>Upload input file (csv)</h3>
<input type="file" name="input_file" id="input_file">
<br>
<input type="submit" value="Process & Download" name="submit">
</form>
</body>
</html>
sample_input.csv
col1,col2
1,5
2,6
3,7
9,10
6,11
So I can generate the pdf with matplotlib, but I'm not sure how to return this as a pdf to the user (the last step). I'm not married to matplotlib; if there are other plotting libraries which can return plots as pdf, I'm open to that as well.
Not tested (I don't have flask installed on this workstation) but you can save the figure as a pdf by giving the correct extension
fig.savefig('/tmp/some_unique_string.pdf')
and then serve the file with send_from_directory
from flask import send_from_directory
send_from_directory('/tmp', 'some_unique_string.pdf')
Some unsolicited advice:
You can read a csv with pandas.read_csv. This works with a file path or an IO stream and will change numeric columns to an appropriate numeric type automatically.
If you want to switch to a different backend, you need to do so before importing pyplot.
Specifically:
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
Related
I have a system to load image files from html but once I select my file and load it from the front end I need a python api to read that image file I select. How can I connect python to my html?
So, to be specific it would be to load an image file with html and have my python api (I use flask) download it and read it to extract data.enter image description here
The html to load the file would be:
<form action="" method="POST" enctype=multipart/form-data>
<div class="col-md-6">
<input type="file" name="file">
<input class="button button-block button-primary" type="submit" value="Upload">
</div>
</form>
The python API to download and read the file would be:
import os
from flask import Flask, render_template, request
from werkzeug import secure_filename
from PIL import Image
# instance of the Flask object
app = Flask(__name__)
# Carpeta de subida
app.config['UPLOAD_FOLDER'] = './Archivos GTiff'
#app.route("/")
#app.route("imgeo/upload-img.html")
def upload_file():
# render the template "upload-img.html"
return render_template('upload-img.html')
#app.route('imgeo/upload-img.html')
def viewer():
"""Show a map to view the GeoJSON returned from a ImGeo endpoint"""
return render_template('imgeo/upload-img.html')
#app.route("imgeo/upload-img.html", methods=['POST'])
def uploader():
if request.method == 'POST':
# we get the file from the "file" input
f = request.files['archivo']
filename = secure_filename(f.filename)
# Save the file in the directory "tiff files".
f.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
# We return a satisfactory answer
return "<h1>File uploaded successfully</h1>"
if __name__ == '__main__':
# Start the application
app.run(debug=True)
You could try to encode your image in base64 with JavaScript, send the result with a POST request, and then (in Python) decode it. Check out these two threads:
How can I convert an image into Base64 string using JavaScript? (JavaScript)
Convert string in base64 to image and save on filesystem (Python)
Here is a similar question too:
Sending an image from html to python script through CGI
I am trying to render boxplot to html page using python flask. In the code i have imported excel sheet .xls to a data frame. On this dataframe have used pandas boxplot method to get box plot successfully. But when i am trying to render this box plot to html page using flask, i am able to see only box plot grid in html page and not actual box plot.
I have tried to save the box plot image first and then give its image source path in html page. but still unable to render it successfully to html page.
Python code:
from flask import Flask, render_template, send_file, make_response
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from mpl_toolkits.mplot3d import Axes3D
from io import BytesIO
import pandas as pd
plt.style.use('ggplot')
app = Flask(__name__)
#app.route('/do_box_plot/')
def do_box_plot():
df = pd.read_excel('Path\\sfile_name.xls', sheet_name='Sheet1')
df.boxplot(by = 'Programming_Language', column =['Entry_ProgrammingScore'], grid = True)
bytes_image = io.BytesIO()
plt.savefig(bytes_image, format='png')
bytes_image.seek(0)
return send_file(bytes_image, mimetype='image/png')
#app.route('/')
def index():
return render_template("index2.html")
if __name__ == '__main__':
app.run(debug = True)
HTML code:
<!doctype html>
<html>
<body>
<h1>BOX PLOT VISUALIZATIONS FOR TEST SCORES</h1>
<p>BOX PLOTS</p>
<img src='/do_box_plot/' alt="Box Plot" height="442" width="442">
</body>
</html>`enter code here`
Expect to render generated box plot to html page using flask. Could anyone help, thanks.
I know that there are some similar questions, although none of them has helped me to resolve my issue. The ultimate goal is to have an flask app, which takes an excel file, stores it inside of the azure blob storage, which is then used by my python function app to do some further transformations. What I struggle with, is how to encode/decode this file as I have to use
block_blob_service.create_blob_from_bytes() function.
The only thing that came to my mind was to use Pandas library to read this excel and then tobytes() function. This way, I am able to upload my excel to the blob as a CSV file. However I cannot really convert it to its previous form.
This is how it looks like after opening :
9]�0��j�9p/�j�� �`��/wj1=p/�j�� p�^�.wj2=p/�[...]
Trying to decode it with utf-8 gives me some never ending errors saying that 'utf-8' codec can't decode byte[...] I have tried many different encodings but it always ends up with this message at some byte. Excel contains numericals, strings and dates.
So, to the code:
#getting the file
file = request.files['file']
#reading into pandas df
data = pd.read_excel(file)
df_to_records = data.to_records(index=False)
records_to_bytes = df_to_records.tobytes()
block_blob_service = BlockBlobService(account_name='xxx', account_key="xxx")
block_blob_service.create_blob_from_bytes("test","mydata.csv",records_to_bytes)
Thank you for any advice!
Update: The full code working at my side.
import os
from flask import Flask, request, redirect, url_for
from azure.storage.blob import BlockBlobService
import string, random, requests
app = Flask(__name__, instance_relative_config=True)
account = "your_account name" # Azure account name
key = "account key" # Azure Storage account access key
container = "f22" # Container name
blob_service = BlockBlobService(account_name=account, account_key=key)
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
file = request.files['file']
file.seek(0)
filename = "test1.csv" # just use a hardcoded filename for test
blob_service.create_blob_from_stream(container, filename, file)
ref = 'http://'+ account + '.blob.core.windows.net/' + container + '/' + filename
return '''
<!doctype html>
<title>File Link</title>
<h1>Uploaded File Link</h1>
<p>''' + ref + '''</p>
<img src="'''+ ref +'''">
'''
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
'''
if __name__ == '__main__':
app.run(debug=True)
After running:
After select a .csv file and click upload button, check the .csv file in azure portal:
I think you can take a try with method create_blob_from_stream instead of create_blob_from_bytes.
Here is the sample code:
def upload_file():
if request.method == 'POST':
file = request.files['file']
file.seek(0)
try:
blob_service = BlockBlobService(account_name='xxx', account_key="xxx")
blob_service.create_blob_from_stream(container, filename, file)
except Exception:
print 'Exception=' + Exception
pass
I wish to create dynamic plots based on user input on a flask app. However I am getting the following error:
string argument expected, got 'bytes'
If I use io.BytesIO(), I am not getting this error, but I am not getting the plot on test.html
from flask import Flask
from flask import render_template
import matplotlib.pyplot as plt
import io
import base64
app = Flask(__name__)
#app.route('/plot')
def build_plot():
img = io.StringIO()
y = [1,2,3,4,5]
x = [0,2,1,3,4]
plt.plot(x,y)
plt.savefig(img, format='png')
img.seek(0)
plot_url = base64.b64encode(img.getvalue())
return render_template('test.html', plot_url=plot_url)
if __name__ == '__main__':
app.debug = True
app.run()
Test.html
<!DOCTYPE html>
<html>
<title> Plot</title>
<body>
<img src="data:image/png;base64, {{ plot_url }}">
</body>
</html>
Use BytesIO and later decode()
Working example
from flask import Flask
#from flask import render_template
import matplotlib.pyplot as plt
import io
import base64
app = Flask(__name__)
#app.route('/plot')
def build_plot():
img = io.BytesIO()
y = [1,2,3,4,5]
x = [0,2,1,3,4]
plt.plot(x,y)
plt.savefig(img, format='png')
img.seek(0)
plot_url = base64.b64encode(img.getvalue()).decode()
return '<img src="data:image/png;base64,{}">'.format(plot_url)
if __name__ == '__main__':
app.debug = True
app.run()
Context
I have created a Flask application that allows me to:
(1) upload a GeoTIFF file to a specified folder (UPLOAD_FOLDER)
(2) use GDAL to open the uploaded GeoTIFF as a Pandas data frame, and return a JSON containing all the cell values. The app code is below:
import os
import gdal
import numpy as np
import pandas as pd
import json
from flask import Flask, flash, request, redirect, url_for
from werkzeug.utils import secure_filename
UPLOAD_FOLDER = 'PATH_GOES_HERE' #specify path
ALLOWED_EXTENSIONS = set(['tif'])
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/upload', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('get_raster_data',
filename=filename))
return '''
<!doctype html>
<title>Upload raster file</title>
<h1>Upload raster file</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/rasterdata', methods=['GET'])
def get_raster_data():
filename = secure_filename(request.args.get('filename'))
try:
if filename and allowed_file(filename):
f = os.path.join(app.config['UPLOAD_FOLDER'], filename)
rast_data = gdal.Open(f)
rast_array = np.array(rast_data.GetRasterBand(1).ReadAsArray())
return pd.DataFrame(rast_array).to_json()
except IOError:
pass
return "Unable to read file"
The application works properly (i.e. I've tested using a local host and running in debug mode). The application allows me to open a web page with "Choose file" and "upload" buttons. Once I upload the file I am redirected to the '/rasterdata' page which has the expected output.
I have been tasked with creating a Jupyter Notebook that basically only requires users to specify the path to a GeoTIFF that they would like to upload. Once the path is specified the Flask app needs to run and return a data frame of all the GeoTIFF's cell values. From there, the Notebook goes through a few processing steps that require the data frame as the input, but these are not relevant to my question.
Question
How can I upload a file to UPLOAD_FOLDER by simply specifying the path to the GeoTIFF in the Jupyter Notebook? Below is the code from my Jupyter Notebook. I've added comments specifying where I am stuck. I suspect that I will need to modify the Flask app to take in a path name. I could not find any tutorials for this though. All the tutorials I could find give me code that is similar to what I currently have.
url = f'http://localhost:5000/upload'
my_path = r'C:\Users\admievan\Desktop\FlaskGDAL\my_raster.tif'
#Opening the upload page
with urllib.request.urlopen(path) as url:
#THIS IS WHERE I AM STUCK
#I want to pass my_path to the Flask application rather than having to
#manually navigate to the file in the GUI interface that comes up when clicking
#the "Choose file" button
#Reading the data web page as a panadas data frame
#This part works fine if 'my_raster.tif' is already in the UPLOAD_FOLDER
url = f'http://localhost:5000/rasterdata?filename=my_raster.tif'
df = pd.read_json(url, orient='rows')
df
The requests package is your best friend when it comes to dealing with uploads/extractions and API calls.
Whatever your host is for the url is where you would need to pass this through.
Uploading is not too difficult and could look something like this:
import os
import base64
import urllib
import json
import requests
def jupyter_upload(token, filePath, resourceDstPath, jupyterUrl='http://localhost:8888'):
dstPath = urllib.quote(resourceDstPath)
dstUrl = '%s/api/contents/%s' % (jupyterUrl, dstPath)
fileName = filePath[1 + filePath.rfind(os.sep):]
headers = {}
headers['Authorization'] = 'token '+token
with open(filePath, 'r') as myfile:
data=myfile.read()
b64data=base64.encodestring(data)
body = json.dumps({
'content':b64data,
'name': fileName,
'path': resourceDstPath,
'format': 'base64',
'type':'file'
})
return requests.put(dstUrl, data=body, headers=headers, verify=True)`