This question already has answers here:
Create and download a CSV file from a Flask view
(3 answers)
Closed 2 years ago.
My goal is to run my data de-identifying script and download my de-identified data from Flask. Right now, I have created a page to upload my file onto my web and I want to execute my de-identifying script using the execute button on my HTML and download the file.
My HTML:
{% block title %}Upload{% endblock %}
{% block main %}
<div class="container">
<div class="row">
<div class="col">
<h1>Upload the file</h1>
<hr>
<form action="/upload-file" method="POST" enctype="multipart/form-data">
<div class="form-group">
<label>Select file</label>
<div class="custom-file">
<input type="file" class="custom-file-input" name="Dataset" id="Dataset">
<label class="custom-file-label" for="Dataset">Select file...</label>
</div>
</div>
<button type="submit" class="btn btn-primary">De-indentify</button>
</form>
<form action="/upload-file" method="GET" enctype="multipart/form-data">
<button type="submit" class="btn btn-primary">Execute</button>
</form>
</div>
</div>
</div>
{% endblock %}
My App Flask route:
app.config["FILE_UPLOADS"] = "app/app/static/csv/uploads"
app.config["ALLOWED_FILE_EXTENSIONS"] = ["csv"]
def allowed_file(filename):
# We only want files with a . in the filename
if not "." in filename:
return False
# Split the extension from the filename
ext = filename.rsplit(".", 1)[1]
# Check if the extension is in ALLOWED_IMAGE_EXTENSIONS
if ext.upper() in app.config["ALLOWED_FILE_EXTENSIONS"]:
return True
else:
return False
#app.route("/upload-file", methods=["GET", "POST"])
def upload_file():
if request.method == "POST":
if request.files:
Dataset = request.files["Dataset"]
if Dataset.filename == "":
print("File must have a filename ")
return redirect(request.url)
if allowed_file(Dataset.filename):
print("That file extension is not allowed")
return redirect(request.url)
else:
filename = secure_filename(Dataset.filename)
Dataset.save(os.path.join(
app.config["FILE_UPLOADS"], filename))
print("Dataset saved")
return redirect(request.url)
return render_template("public/upload_file.html")
The file that I have uploaded:
Housing,Houseprice,Name,Neighbourhood,State
123556,100000,John,Bloomingdale,Washington
111777,250000,Ian,Bloomingdale,Washington
998273,250000,Tom,Spring Valley,California
My de-identifying script:
import pandas as pd
import uuid as u
# generate a pseudo-identifier sequesnce using python random number generator library uudi.
def uudi_generator(length):
uudi_list= list()
i=0
while i < length:
uudi_list.append(u.uuid4())
i+=1
return uudi_list
#import dataset
dataset = pd.read_csv('C:\\mylocation\\housing.csv', index_col=False)
# pseudo identifier
sLength = len(dataset['Housing'])
dataset.insert(0, 'uuid', pd.Series(uudi_generator(sLength), index=dataset.index))
#delete identifiabLe record from dataset
del dataset['Name']
del dataset['Neigbourhood']
Try this:
from flask import Response
csv_text = ""
for row in dataset.values:
csv_text += f"{row[0]},{row[1]},{row[2]},{row[3]}\n"
return Response(csv_text, mimetype='text/csv')
You don't even need to del the unwanted columns. Just don't add them in csv_text. You don't really even need pandas. You could read the csv as a text file, split('\n') and add the uuids when you compile the csv_text
Related
In my flask app I want the user to be able to upload an image to a folder inside the static folder - called wallpaper. Currently I'm receiving a flask.debughelpers.DebugFilesKeyError, however I don't see any error in the keys I'm using
What I've tried
flaskapp.py
#app.route('/changeWallpaper' , methods = ['POST', 'GET'])
def change_home_wallpaper():
UPLOADS_PATH = join(dirname(realpath(__file__)), 'static\\wallpaper')
if request.method == "POST":
wallpaper = request.files['wallpaper']
if wallpaper.filename != '':
image = request.files['wallpaper']
image.save(os.path.join(UPLOADS_PATH, secure_filename(image.filename)))
cur = db2.cursor()
sql = f"UPDATE wallpaper set pic = {wallpaper.filename} where sno = 1"
cur.execute(sql)
db2.commit()
cur.close()
return redirect(url_for('home'))
else:
return redirect(url_for('home'))
loggedin.html
<div class="jumbotron">
<form action="{{url_for('change_home_wallpaper')}}" method="post">
<div class="container">
<input type="file" name="wallpaper"/>
<input type="submit" class="btn btn-primary"/>
</div>
</form>
</div>
Update your HTML
<form action="/path" method="post" enctype="multipart/form-data">
</form>
and put {wallpaper.filename} in quotes.
I am very much struggling with a concept and hope someone can offer a code implementation suggestion. I have the following route embedded within my flask app. After working on this for a few days I think I now see the nature of the issue. I have modified my code based on suggestion below in comments.
In the web app now, the names of the data frame columns populate a drop down menu and the user can choose one of those variables and click "Show". That variable name then prints to the screen just so I know my POST from the form is communicating with my function in the flask app.
What I would like to do is create a temp version of the file uploaded so that it exists as long as the web session is open (or is overwritten if a new file upload occurs). Then, the user chooses a variable from the drop down menu and the mean of that variable is computed (that part is commented out in the code below for now but you can see what I have tried).
If the user reads in a new file, then the same process would occur on that new file.
Thanks for any suggestions or supports.
from flask import Flask, render_template, request
import numpy as np
from scipy.stats import binom
from scipy.optimize import minimize
from scipy.stats import norm
from scipy import optimize
from pyodbc import connect
import pandas as pd
import os
import tempfile
app = Flask(__name__)
#app.route('/dataTools', methods=['POST', 'GET'])
def data_tools_upload():
tempfile_path = tempfile.NamedTemporaryFile().name
if request.files:
df = pd.read_csv(request.files.get('file'))
#tempfile_path = tempfile.NamedTemporaryFile().name
df.to_csv(tempfile_path)
if os.path.exists(tempfile_path):
orig_df = pd.read_csv(tempfile_path)
vars = list(orig_df.columns)
var2use = request.form.get("var2use")
#indx = vars.index(var2use)
#df[vars[indx]].mean()
mean = orig_df[vars[4]].mean()
dims = orig_df.shape
message = 'You have data! There are %s rows and %s columns and the variable %s has mean %s' % (dims[0],dims[1],vars[4],round(mean,3))
table = orig_df.head(10).to_html(classes='data', header = "true")
return render_template('upload.html', tables = [table], message = message, vars = vars, var_name = var2use)
var2use = request.form.get("var2use")
return render_template('upload.html', var_name = var2use, message = "What happened?")
if __name__ == '__main__':
app.run(debug=True)
And then the following HTML bit (upload.html):
{% extends "layout.html" %}
{% block body %}
<h3> Read in a data file </h3>
<br>
<form method=post enctype=multipart/form-data>
<input type=file name=file class = "btn btn-outline-secondary">
<input type=submit value=Upload class = "btn btn-outline-secondary">
</form>
<br>
<form class="form-inline" action = "{{url_for('data_tools_upload')}}" method = "POST">
<select type = "text" name="var2use" class="custom-select mr-sm-1">
{% for var in vars %}
<option value= "{{ var }}" SELECTED>{{ var }}</option>"
{% endfor %}
</select>
<button class = "btn btn-primary"> Show </button>
</form>
<center>
<h1>
{{message}}
{{name}}
</h1>
<br>
<small>
{% for table in tables %}
{{ table|safe }}
{% endfor %}
{% endblock %}
</small>
</center>
you need to save file to work with it (the code works perfectly in my app):
f = request.files['file']
tmp_filename = '/tmp/my_tmp_file.csv'
f.save(tmp_filename)
pd.read_csv(tmp_filename)
Here is the detailed documentation on Flask file uploads:
https://flask.palletsprojects.com/en/1.1.x/patterns/fileuploads/
Full example for your case is below. I think you might have problems with the following three items:
Uploading the file because of form enctype (should be "multipart/form-data")
Uploading because of file size (try my sample)
Processing the form with empty file overwriting yours (see my example for "csv_file_uploaded")
Python (csv_test.py):
from flask import Flask, render_template, request
import pandas as pd
import os
app = Flask(__name__, static_url_path='')
#app.route('/', methods=['GET', 'POST'])
def index():
data = {}
tmp_filename = '/tmp/csv_test.csv'
if request.files:
csv_file_uploaded = request.files.get('file')
if csv_file_uploaded:
f = request.files['file']
f.save(tmp_filename)
if os.path.exists(tmp_filename):
df = pd.read_csv(tmp_filename)
data['columns'] = ', '.join(df.columns)
col_selected = request.form.get('col_selected')
print(col_selected)
data['col_selected'] = col_selected
if col_selected:
mean = df.get(col_selected).mean()
data['mean'] = mean
return render_template('csv_test.tpl', data=data)
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
Flask template (csv_test.tpl):
<html><head><title>CSV TEST</title></head>
<body><h1>CSV TEST</h1>
{% if data %}
<div>Columns of the uploaded file: <strong>{{ data.columns }}</strong></div>
<div>Selected column: <strong>{{ data.col_selected }}</strong></div>
<div>Mean of the selected = <strong>{{ data.mean }}</strong></div>
{% endif %}
<fieldset>
<form enctype="multipart/form-data" action="" method="post">
<h2>Provide some CSV with first column in numbers and/or column name</h2>
<div>File{% if data.columns %} (UPLOADED){% endif %}: <input id="file" type="file" name="file"/></div>
<div>Column: <input id="col_selected" type="text" name="col_selected" value="{{ data.col_selected }}"/></div>
<div><input id="submit" type="submit" value="SUBMIT"/></div>
</form>
</fieldset>
</body></html>
CSV file (123.csv) - use it to test the form:
col1,col2
1,2
2,3
3,4
5,6
I would like to upload a single file or files at a time using get method.
I have already done with the post method and working fine. But for some reason i would like to do the file upload using get method using command line.
The below code is which i have already tried to get the string from get method and i can able to get the varValue as string. But i would like to get the file using get method.
def home(request):
if request.method == 'GET':
varValue = request.GET.get('myfile', '')
print(varValue)`
HTML code:
<form method="GET" enctype="multipart/form-data">
<input type="file" name="myfile" accept="image/*" multiple>
<button type="submit">Upload files</button>
Try This Method
for filename, file in request.FILES.iteritems():
name = request.FILES[filename].name
A dictionary-like object containing all uploaded files. Each key in FILES is the name from the . Each value in FILES is an UploadedFile.
<form method="GET" enctype="multipart/form-data">
{% csrf_token %}
<input type="file" name="myfile">
<button type="submit">Upload File</button>
</form>
Found the solution using the post method itself. By skipping the csrf token we can do the command line script execution method.
Python views.py code below
from django.views.decorators.csrf import csrf_exempt
#csrf_exempt
def home(request):
if request.method == 'POST':
uploaded_file = request.FILES['myfile']
fs = FileSystemStorage()
name = fs.save(uploaded_file.name, uploaded_file)
print(name)
return render(request, "home.html")
HTML code:
<body>
<form method="POST" enctype="multipart/form-data">
<input type="file" name="myfile" accept="image/*" multiple>
<button type="submit">Upload files</button>
</body>
python code for file upload using post method:
import requests
files = {'myfile': open('d:\\20190819-140341-627485.png','rb')}
y = requests.post("http://127.0.0.1:8000",files=files)
So, I am trying to open an .csv file in Python using Flask. I copies the code from the Python library, but I go from one error message to the other and I don't know what I am doing wrong. The latest error code I get on the below code is: TypeError: invalid file:
Any ideas what I am doing wrong?
My Python code/Flash route is as follows:
#app.route("/admin", methods=["GET", "POST"])
#login_required
def admin():
"""Configure Admin Screen"""
# if user reached route via POST (as by submitting a form via POST)
if request.method == "POST":
# load csv file with portfolio data
with open(request.files["portfolios"]) as csvfile:
portfolios = csv.DictReader(csvfile)
# load csv file in dictionary
for row in portfolios:
print(row['first_name'], row['last_name'])
else:
return render_template("admin.html")
My html/Flask code is:
{% extends "layout.html" %}
{% block title %}
Admin
{% endblock %}
{% block main %}
<h2>Admin Console</h2>
<h3> Upload Portfolio Data</h2>
<form action="{{ url_for('admin') }}" method="post" enctype=multipart/form-data>
<fieldset>
<label class="control-label">Select Portfolio Upload File</label>
<input id="input-1" type="file" class="file" name="portfolios">
<h3>Upload Security Lists</h2>
<label class="control-label">Select Security Upload File</label>
<input id="input-1" type="file" class="file" name="securities">
<div class="form-group">
<button class="btn btn-default" type="submit" value = "upload">Upload</button>
</div>
</fieldset>
</form>
{% endblock %}
The file is already open. open takes a string filename and created an open file object, but you don't need to do that because objects in request.files are already open file-like objects.
portfolios = csv.DictReader(request.files['portfolios'])
images.html
<body>
{% if msg %} <h1>{{ msg }}!</h1> {% endif %}
<div class="container">
<h1>Here you can upload a file</h1>
<form id="upload_form" action="/upload" method="POST" enctype="multipart/form-data">
<input type="file" name="file" accept="uploaded_files/*" multiple>
<input type="submit" name="upload" value="Upload">
</form>
<hr>
<!--- <h1>Here you can download a file</h1>
<form id="download_form" action="/download" method="POST">
File Name: <input type="text" name="filename" placeholder="Enter file name"><br><br>
<input type="submit" name="download" value="download">
</form>
<hr>
<h1>Here you can delete a file</h1>
<form id="delete_form" action="/delete" method="POST">
File Name: <input type="text" name="filename" placeholder="Enter file name"><br><br>
<input type="submit" name="delete" value="delete">
</form>
<hr>--->
<h1>Here you can List,Download and delete all the files</h1>
{% if list_names %}
{% for item in list_names %}
<!--- {{ item }}--->
<img src="{{item}}" height="400px"/>
delete
{% endfor %}
{% endif %}
<form id="list_form" action="/list_files" method="POST">
<input type="submit" name="list" value="list">
</form>
<hr>
</div>
</body>
website.py
from flask import Flask, session, redirect, url_for, escape, request, render_template
import os
import base64
import mysql.connector
import base64
app = Flask(__name__)
if __name__ == '__main__':
db = mysql.connector.connect(host="mounika.cb3d7dg2fmod.us-east-1.rds.amazonaws.com",
user="mounika", password="mounika2061993", database="innodb")
cur = db.cursor()
app.secret_key = 'A0Zr98j/3yX R~XHH!jmN]LWX/,?RT'
class ServerError(Exception):pass
#app.route('/login', methods=['GET', 'POST'])
def login():
error = None
try:
if request.method == 'POST':
username_form = request.form['username']
data=cur.execute("select count(1) from users1 where username=%s",[username_form])
print data
if not cur.fetchone()[0]:
raise ServerError('Invalid username')
password_form = request.form['password']
print "SELECT password FROM users1 WHERE username=%s",[username_form]
data1=cur.execute("SELECT password FROM users1 WHERE username=%s",[username_form])
#print data1
for row in cur.fetchall():
if password_form== row[0]:
session['username'] = request.form['username']
return render_template('images.html', error=error)
raise ServerError('Invalid password')
except ServerError as e:
error = str(e)
except ServerError as e:
error = str(e)
return render_template('index.html', error=error)
#app.route('/')
def index():
return render_template('index.html')
#app.route('/upload',methods=['POST'])
def uploaddownload():
username_session = escape(session['username'])
print username_session
get_files=request.files.get('file')
extension=os.path.splitext(get_files.filename)[1]
print extension
file_name = os.path.splitext(get_files.filename)[0]
new_file_name=file_name+extension
#dir_path = os.path.abspath(new_file_name)
#print dir_path
print "hi"
print new_file_name
print get_files
print file_name
app.config['UPLOAD_FOLDER'] = './static/Uploads'
get_files.save(os.path.join(app.config['UPLOAD_FOLDER'], new_file_name))
path=app.config['UPLOAD_FOLDER']+'/'+new_file_name
print "path"
print path
print "file saved succesfully to local upload folder"
extension_list=['.jpg','.jnpg','.jpeg','.png','.GIF','.BMP','.JPG','.JPNG']
if(extension in extension_list):
# print "hiiiiiiiii"
sql ="INSERT INTO images1 (user_name,images) VALUES(%s,%s)"
args=(session['username'],path,)
data= cur.execute(sql,args)
db.commit()
return render_template("images.html")
#app.route('/list_files',methods=['POST'])
def list_Files():
sql1="select images from images1"
cur.execute(sql1)
data= cur.fetchall()
print data
file_name_list=[]
for file in data:
#file_name_list.append(file.generate_url(3600,query_auth=False,force_http=True))
file_name_list.append(file[0].strip('.'))
print file_name_list
#print file.name
return render_template("images.html",list_names=file_name_list)
#app.route('/delete<filename>')
def delete(filename):
print "Hi"
return render_template("index.html")
if __name__ == '__main__':
app.run(debug=True)
The above is my code. In my html I have included anchor tag to delete. item will give the path of the image. so when I click the anchor tage it is searching for the route as '/delete/static/Uploads/14199543_1107156822698054_3500700356588134783_n.jpg'
In my python code I have included the route #app.route('/delete'). I am expecting that the filename is going to capture the path from html. But I guess this isn't working. How can I get item from html which here is the path of the image to the python code so that I could write a delete query including it in the where clause. Or can someone pls explain how to write a delete function that deletes the image when I click the anchor tag.
My code is messy. But the list and login are working fine. Any help regarding delete is appreciated. Thanks in advance