I have a system to load image files from html but once I select my file and load it from the front end I need a python api to read that image file I select. How can I connect python to my html?
So, to be specific it would be to load an image file with html and have my python api (I use flask) download it and read it to extract data.enter image description here
The html to load the file would be:
<form action="" method="POST" enctype=multipart/form-data>
<div class="col-md-6">
<input type="file" name="file">
<input class="button button-block button-primary" type="submit" value="Upload">
</div>
</form>
The python API to download and read the file would be:
import os
from flask import Flask, render_template, request
from werkzeug import secure_filename
from PIL import Image
# instance of the Flask object
app = Flask(__name__)
# Carpeta de subida
app.config['UPLOAD_FOLDER'] = './Archivos GTiff'
#app.route("/")
#app.route("imgeo/upload-img.html")
def upload_file():
# render the template "upload-img.html"
return render_template('upload-img.html')
#app.route('imgeo/upload-img.html')
def viewer():
"""Show a map to view the GeoJSON returned from a ImGeo endpoint"""
return render_template('imgeo/upload-img.html')
#app.route("imgeo/upload-img.html", methods=['POST'])
def uploader():
if request.method == 'POST':
# we get the file from the "file" input
f = request.files['archivo']
filename = secure_filename(f.filename)
# Save the file in the directory "tiff files".
f.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
# We return a satisfactory answer
return "<h1>File uploaded successfully</h1>"
if __name__ == '__main__':
# Start the application
app.run(debug=True)
You could try to encode your image in base64 with JavaScript, send the result with a POST request, and then (in Python) decode it. Check out these two threads:
How can I convert an image into Base64 string using JavaScript? (JavaScript)
Convert string in base64 to image and save on filesystem (Python)
Here is a similar question too:
Sending an image from html to python script through CGI
Related
I'm using python pandas and flask for some postprocessing tasks (anlaysis and visualization). Until now I uploaded/read *.csv *.xlsx and *.xls via pd.read_csv, pd.read_xlsx. Everything worked quiet fine.
Now I have a *.xml file as datasource and tried according my habit pattern.
So i tried:
<form action="/input" method="POST" enctype="multipart/form-data">
<input class="form-control" type="file" name="file">
<input type="submit" class="btn btn-outline-secondary" name="Preview" value ="Preview Data" > </input>
from flask import Flask, render_template,request, render_template
import pandas as pd
import xml.etree.ElementTree as ET
#app.route("/input", methods=['POST', 'GET'])
def input():
if request.method == 'POST':
if request.form['Preview'] == "Preview Data":
file = request.files['file']
filename = file.filename
if '.xml' in filename:
content = pd.read_xml(file, parser='lxml')
But when I pass a .xml file to the app via the form. I get the error:
File "C:\ProgramData\MiniforgeEnvs\TestEnv\lib\site-packages\pandas\io\xml.py", line 627, in _parse_doc
with preprocess_data(handle_data) as xml_data:
AttributeError: __enter__
I tried check different options:
when I use the inbuild xml.etree package it works fine:
import xml.etree.ElementTree as ET
if '.xml' in filename:
tree = ET.parse(file)
root = tree.getroot()
print(root[1][0][1].attrib)
when I load the .xml direct from the app directory into pd.read_xml() it also works fine:
if '.xml' in filename:
content = pd.read_xml('SampleExport.xml', parser='lxml')
I tried different prasers: "lxml" and "etree"
But at the end when I pass the .xml via the Form/input and using pd.read_xml(file,parser='lxml') I got the error from above.
I just solved my issue even though I'm not quite sure why pd.read_xml() behaves different compared to pd.read_csv() or pd.read_xlsx().
pd.read_xml is not able to read a FileStorage object. The variable passed by request.file[] is a instance of the class: werkzeug.datastructures.FileStorage(stream=None, filename=None, name=None, content_type=None, content_length=None, headers=None).
Via the read function I extracted the file itsself.
filestorage = request.files['file']
file=filestorage.read()
with this passed to pd.read_xml it works fine.
Is there anybody who can explain why _parse_doc() funtion of pd.read_xml() is not able to read FileStotage type?
I am creating a tool where either
A new XLSX file is generated for the user to download
The user can upload an XLSX file they have, I will read the contents of that file, aand use them to generate a new file for the user to download.
I would like to make use of Pandas to read the XLSX file into a dataframe, so I can work with it easily. However, I can't get it working. Can you help me?
Example extract from CGI file:
import pandas as pd
import cgi
from mako.template import Template
from mako.lookup import TemplateLookup
import http.cookies as Cookie
import os
import tempfile
import shutil
import sys
cookie = Cookie.SimpleCookie(os.environ.get("HTTP_COOKIE"))
method = os.environ.get("REQUEST_METHOD", "GET")
templates = TemplateLookup(directories = ['templates'], output_encoding='utf-8')
if method == "GET": # This is for getting the page
template = templates.get_template("my.html")
sys.stdout.flush()
sys.stdout.buffer.write(b"Content-Type: text/html\n\n")
sys.stdout.buffer.write(
template.render())
if method == "POST":
form = cgi.FieldStorage()
print("Content-Type: application/vnd.ms-excel")
print("Content-Disposition: attachment; filename=NewFile.xlsx\n")
output_path = "/tmp/" + next(tempfile._get_candidate_names()) + '.xlsx'
data = *some pandas dataframe previously created*
if "editfile" in form:
myfilename = form['myfile'].filename
with open(myfilename, 'wb') as f:
f.write(form['myfile'].file.read())
data = pd.read_excel(myfilename)
data.to_excel(output_path)
with open(path, "rb") as f:
sys.stdout.flush()
shutil.copyfileobj(f, sys.stdout.buffer)
Example extract from HTML file:
<p>Press the button below to generate a new version of the xlsx file</p>
<form method=post>
<p><input type=submit value='Generate new version of file' name='newfile'>
<div class="wrapper">
</div>
</form>
<br>
<p>Or upload a file.</p>
<p>In this case, a new file will be created using the contents of this file.</p>
<form method="post" enctype="multipart/form-data">
<input id="fileupload" name="myfile" type="file" />
<input value="Upload and create new file" name='editfile' type="submit" />
</form>
This works without the if "editfile" in form: bit so I know something is going wrong when I am trying to access the file that the user has uploaded.
The problem is that whilst a file is created, the created file has a file size of 0 KB and will not open in Excel. Crucially, the file that the user has uploaded can not be found in the location that I have written it out.
You've passed myfilename to pandas; however that file doesn't exist on the server yet. You'll have to save the file somewhere locally first before using it.
The following will download the file to the current directory (same directory as the CGI script). Of course, you're welcome to save it to some more suitable directory, depending on your setup.
form = cgi.FieldStorage()
myfilename = form['myfile'].filename
with open(myfilename, 'wb') as f: # Save the file locally
f.write(form['myfile'].file.read())
data = pd.read_excel(myfilename)
I know that there are some similar questions, although none of them has helped me to resolve my issue. The ultimate goal is to have an flask app, which takes an excel file, stores it inside of the azure blob storage, which is then used by my python function app to do some further transformations. What I struggle with, is how to encode/decode this file as I have to use
block_blob_service.create_blob_from_bytes() function.
The only thing that came to my mind was to use Pandas library to read this excel and then tobytes() function. This way, I am able to upload my excel to the blob as a CSV file. However I cannot really convert it to its previous form.
This is how it looks like after opening :
9]�0��j�9p/�j�� �`��/wj1=p/�j�� p�^�.wj2=p/�[...]
Trying to decode it with utf-8 gives me some never ending errors saying that 'utf-8' codec can't decode byte[...] I have tried many different encodings but it always ends up with this message at some byte. Excel contains numericals, strings and dates.
So, to the code:
#getting the file
file = request.files['file']
#reading into pandas df
data = pd.read_excel(file)
df_to_records = data.to_records(index=False)
records_to_bytes = df_to_records.tobytes()
block_blob_service = BlockBlobService(account_name='xxx', account_key="xxx")
block_blob_service.create_blob_from_bytes("test","mydata.csv",records_to_bytes)
Thank you for any advice!
Update: The full code working at my side.
import os
from flask import Flask, request, redirect, url_for
from azure.storage.blob import BlockBlobService
import string, random, requests
app = Flask(__name__, instance_relative_config=True)
account = "your_account name" # Azure account name
key = "account key" # Azure Storage account access key
container = "f22" # Container name
blob_service = BlockBlobService(account_name=account, account_key=key)
#app.route('/', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
file = request.files['file']
file.seek(0)
filename = "test1.csv" # just use a hardcoded filename for test
blob_service.create_blob_from_stream(container, filename, file)
ref = 'http://'+ account + '.blob.core.windows.net/' + container + '/' + filename
return '''
<!doctype html>
<title>File Link</title>
<h1>Uploaded File Link</h1>
<p>''' + ref + '''</p>
<img src="'''+ ref +'''">
'''
return '''
<!doctype html>
<title>Upload new File</title>
<h1>Upload new File</h1>
<form action="" method=post enctype=multipart/form-data>
<p><input type=file name=file>
<input type=submit value=Upload>
</form>
'''
if __name__ == '__main__':
app.run(debug=True)
After running:
After select a .csv file and click upload button, check the .csv file in azure portal:
I think you can take a try with method create_blob_from_stream instead of create_blob_from_bytes.
Here is the sample code:
def upload_file():
if request.method == 'POST':
file = request.files['file']
file.seek(0)
try:
blob_service = BlockBlobService(account_name='xxx', account_key="xxx")
blob_service.create_blob_from_stream(container, filename, file)
except Exception:
print 'Exception=' + Exception
pass
I am making a data visualization tool that takes input from the user (choosing a file on the computer); processes it in Python with Pandas, Numpy, etc; and displays the data in the browser on a local server.
I am having trouble accessing the data once the file is selected using an HTML input form.
HTML form:
<form action="getfile" method="POST" enctype="multipart/form-data">
Project file path: <input type="file" name="myfile"><br>
<input type="submit" value="Submit">
</form>
Flask routing:
#app.route("/")
def index():
return render_template('index.html')
#app.route('/getfile', methods=['GET','POST'])
def getfile():
if request.method == 'POST':
result = request.form['myfile']
else:
result = request.args.get['myfile']
return result
This returns a "Bad Request The browser (or proxy) sent a request that this server could not understand." error. I have tried a number of different ways of getting the data out of the file and simply printing it to the screen to start, and have received a range of errors including "TypeError: 'FileStorage' object is not callable" and "ImmutableMultiDict' object is not callable". Any pointers on how to approach this task correctly are appreciated.
Try this. I've been working on saving and unzipping files for the last few days. If you have any trouble with this code, let me know :)
I'd suggest saving the file on disk and then reading it. If you don't want to do that, you needn't.
from flask import Flask, render_template, request
from werkzeug import secure_filename
#app.route('/getfile', methods=['GET','POST'])
def getfile():
if request.method == 'POST':
# for secure filenames. Read the documentation.
file = request.files['myfile']
filename = secure_filename(file.filename)
# os.path.join is used so that paths work in every operating system
file.save(os.path.join("wherever","you","want",filename))
# You should use os.path.join here too.
with open("wherever/you/want/filename") as f:
file_content = f.read()
return file_content
else:
result = request.args.get['myfile']
return result
And as zvone suggested in the comments, I too would advise against using GET to upload files.
Uploading files
os.path by Effbot
Edit:-
You don't want to save the file.
Uploaded files are stored in memory or at a temporary location on the filesystem. You can access those files by looking at the files attribute on the request object. Each uploaded file is stored in that dictionary. It behaves just like a standard Python file object, but it also has a save() method that allows you to store that file on the filesystem of the server.
I got this from the Flask documentation. Since it's a Python file you can directly use file.read() on it without file.save().
Also if you need to save it for sometime and then delete it later, you can use os.path.remove to delete the file after saving it. Deleting a file in Python
A input type=file data isn't passed in as the form dictionary of a request object. It is passed in as request.files (files dictionary in the request object).
So simply change:
result = request.form['myfile']
to
result = request.files['myfile']
Context
I have created a Flask application that allows me to:
(1) upload a GeoTIFF file to a specified folder (UPLOAD_FOLDER)
(2) use GDAL to open the uploaded GeoTIFF as a Pandas data frame, and return a JSON containing all the cell values. The app code is below:
import os
import gdal
import numpy as np
import pandas as pd
import json
from flask import Flask, flash, request, redirect, url_for
from werkzeug.utils import secure_filename
UPLOAD_FOLDER = 'PATH_GOES_HERE' #specify path
ALLOWED_EXTENSIONS = set(['tif'])
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
def allowed_file(filename):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS
#app.route('/upload', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
if 'file' not in request.files:
flash('No file part')
return redirect(request.url)
file = request.files['file']
if file.filename == '':
flash('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('get_raster_data',
filename=filename))
return '''
<!doctype html>
<title>Upload raster file</title>
<h1>Upload raster file</h1>
<form method=post enctype=multipart/form-data>
<input type=file name=file>
<input type=submit value=Upload>
</form>
'''
#app.route('/rasterdata', methods=['GET'])
def get_raster_data():
filename = secure_filename(request.args.get('filename'))
try:
if filename and allowed_file(filename):
f = os.path.join(app.config['UPLOAD_FOLDER'], filename)
rast_data = gdal.Open(f)
rast_array = np.array(rast_data.GetRasterBand(1).ReadAsArray())
return pd.DataFrame(rast_array).to_json()
except IOError:
pass
return "Unable to read file"
The application works properly (i.e. I've tested using a local host and running in debug mode). The application allows me to open a web page with "Choose file" and "upload" buttons. Once I upload the file I am redirected to the '/rasterdata' page which has the expected output.
I have been tasked with creating a Jupyter Notebook that basically only requires users to specify the path to a GeoTIFF that they would like to upload. Once the path is specified the Flask app needs to run and return a data frame of all the GeoTIFF's cell values. From there, the Notebook goes through a few processing steps that require the data frame as the input, but these are not relevant to my question.
Question
How can I upload a file to UPLOAD_FOLDER by simply specifying the path to the GeoTIFF in the Jupyter Notebook? Below is the code from my Jupyter Notebook. I've added comments specifying where I am stuck. I suspect that I will need to modify the Flask app to take in a path name. I could not find any tutorials for this though. All the tutorials I could find give me code that is similar to what I currently have.
url = f'http://localhost:5000/upload'
my_path = r'C:\Users\admievan\Desktop\FlaskGDAL\my_raster.tif'
#Opening the upload page
with urllib.request.urlopen(path) as url:
#THIS IS WHERE I AM STUCK
#I want to pass my_path to the Flask application rather than having to
#manually navigate to the file in the GUI interface that comes up when clicking
#the "Choose file" button
#Reading the data web page as a panadas data frame
#This part works fine if 'my_raster.tif' is already in the UPLOAD_FOLDER
url = f'http://localhost:5000/rasterdata?filename=my_raster.tif'
df = pd.read_json(url, orient='rows')
df
The requests package is your best friend when it comes to dealing with uploads/extractions and API calls.
Whatever your host is for the url is where you would need to pass this through.
Uploading is not too difficult and could look something like this:
import os
import base64
import urllib
import json
import requests
def jupyter_upload(token, filePath, resourceDstPath, jupyterUrl='http://localhost:8888'):
dstPath = urllib.quote(resourceDstPath)
dstUrl = '%s/api/contents/%s' % (jupyterUrl, dstPath)
fileName = filePath[1 + filePath.rfind(os.sep):]
headers = {}
headers['Authorization'] = 'token '+token
with open(filePath, 'r') as myfile:
data=myfile.read()
b64data=base64.encodestring(data)
body = json.dumps({
'content':b64data,
'name': fileName,
'path': resourceDstPath,
'format': 'base64',
'type':'file'
})
return requests.put(dstUrl, data=body, headers=headers, verify=True)`