I am trying to use ng-file-upload in my Flask app, but I'm having a hard time send it to Flask main.py to be saved. I could have just done a simple submit using flask's url_for(), but I would like to keep Flask in the backend and AngularJS in the front-end.
Also, I would like to use the drag and drop field feature from this angular plugin instead of an upload file button, but I think that's where my issue is coming from. I don't think flask is recognizing it from request.files. I'm not sure if I'm calling correctly or maybe is just a simple fix and I'm not seeing it.
Flask main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['user_file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
I know the "POST" works because i can see the message from this print statement, print 'unknown went through', after that it fails to recognize the filestorage variable
Here is my angular controller:
myApp.controller('UploadFileCtrl',[
'$scope',
'$http',
'Upload',
function($scope, $http, Upload){
// upload later on form submit or something similar
$scope.submit = function(files) {
if ($scope.files) {
$scope.upload($scope.files);
}
}
// upload on file select or drop
$scope.upload = function (files) {
console.log('it was uploaded');
Upload.upload({
url: '/',
method: 'POST',
data: {file: files}
}).then(function (resp) {
// console.log('Success ' + resp.config.data.file.name + 'uploaded. Response: ' + resp.data);
console.log(resp.data);
}, function (resp) {
console.log('Error status: ' + resp.status);
console.log(resp);
}, function (evt) {
var progressPercentage = parseInt(100.0 * evt.loaded / evt.total);
console.log('progress: ' + progressPercentage + '% ' + evt.config.data.file.name);
});
}
}])
html form:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true"
name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>
I'm still not familiar with Flask, but it's a cool webframework to work with and it will be awesome to integrate it with AngularJS. Your help will be appreciated, thanks in advance!
UPDATE
I actually got it to upload in Flask, I need it to replace filestorage = request.files['user_file'] with filestorage = request.files['file']. Like I said, Flask was not recognizing it by it's provided name in the html, which I don't know why. I failed to provide this line return render_template('index.html') in the main.py above. Since Flask is not longer present in the html, could that be the reason Flask is not recognizing the name in the form?
revised main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
return render_template('index.html')
Also I had to change in the html the ngf-multiple to false, this was sending an array, so Flask was expecting a dict.
revised html:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true" name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>
Related
I have a python project in PythonAnywhere where the user uploads a file, it gets processed, and then it gets downloaded again by the user.
The file gets processed using a RandomForestRegressor model.
So far, it works perfectly. The user uploads and a new output file is generated. What I want to do is that the results of the model be returned in the website.
A message like this:
Accuracy of model on test set: 0.90
mse of model on train set: 0.08
The HTML is like this:
<html>
<body>
{% for comment in comments %}
<div class="row">
{{ comment }}
</div>
{% endfor %}
</body>
</html>
I was trying to add a comment after the proccess_data function is called. I added the render_template with the comment variable but it doesnt work. I even added it at the end of the def index() as a return but it just doesnt work.
output_file = process_data(filename)
comments = 'File processed succesfully'
render_template("main_page.html", comments=comments)
The python script is like this:
comments = []
#app.route('/', methods=["GET","POST"])
def index():
if request.method == "GET":
return render_template("main_page.html")
if request.method == 'POST':
#comments.append(request.form["contents"])
#check if the post request has the file part
if 'input_file' not in request.files:
#TODO
return redirect(request.url)
file = request.files['input_file']
#if the user does not select a file, the browser submits
#empty file without a filename
if file.filename == '':
#TODO
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
output_file = process_data(filename)
comments = 'File processed succesfully'
render_template("main_page.html", comments=comments)
si = io.StringIO()
output_file.to_csv(si,index=False, encoding='UTF8')
#-----Save dataframe to folder-----
filepath = os.path.join(app.config['UPLOAD_FOLDER'], 'results.csv')
output_file.to_csv(filepath)
response = make_response(si.getvalue())
response.headers["Content-Disposition"] = "attachment; filename=results.csv"
response.headers["Content-type"] = "text/csv"
return response
return
I don't work with python but I see you basically want to send two responses in one request and most likely at different times?
HTTP does not work like that. You can't send one response "mid process" and then some other after a while. It's always one request - one response. And a new tcp connection opens and closes immediately each time.
So if you want to notify user about the results of backend process some time before you actually send him the "final data", you have these options:
Divide this flow into multiple individual requests.
Use WebSockets.
So I tried to develop a web proxy using Flask. in the progress I found out that somehow I need to handle Likns so all redirects take place in the web proxy, this is what I write:
import requests
from flask import Flask , request
from requests import get
import re
app = Flask(__name__)
mainURL = ""
myHost = "http://127.0.0.1:5000/"
#app.route("/", methods =["GET", "POST"])
def Home():
#a small form for users to write thier URL inside it
mainpage = '<!DOCTYPE html><html><body><form action="/" method="post"><label for="url">URL:</label><br><input type="text" id="url" name="url" value="https://www.google.com/"><br><input type="submit" value="Open"></form></body></html>'
#check if submit button trigered
if request.method == "POST":
mainURL = request.form.get("url")
response = requests.get(mainURL)
#this is for handling links and pictures. I know its not optimal, thats why im here!
a = response.text.replace("='/",f"='{mainURL}")
a = a.replace('="/',f'="{mainURL}')
a = a.replace("url(/",f"url({mainURL}")
a = a.replace('href="http',f'href="{myHost}http')
a = a.replace("href='http",f"href='{myHost}http")
a = a.replace(r'href="(?!http)',f'href="{myHost}{mainURL}')
a = a.replace(r"href='(?!http)",f"href='{myHost}{mainURL}")
return a
return mainpage
#decroator for the times when a path opened
#app.route('/<path:path>')
def proxy(path):
#this RegEx find the website FQDN from path
if re.match(r"https?://w{3}\.\w*\.\w*/",path):
temp = re.match(r"https?://w{3}\.\w*\.\w*/",path)
mainURL = temp[0]
response = requests.get(path)
#again links and pictures handler. dont judge me, I wrote a function for it but IDK why it didn't works for pictures!
a = response.text.replace("='/",f"='{mainURL}")
a = a.replace('="/',f'="{mainURL}')
a = a.replace("url(/",f"url({mainURL}")
a = a.replace('href="http',f'href="{myHost}http')
a = a.replace("href='http",f"href='{myHost}http")
a = a.replace(r'href="(?!http)',f'href="{myHost}{mainURL}')
a = a.replace(r"href='(?!http)",f"href='{myHost}{mainURL}")
return a
return "URL is Incorrect!"
if __name__ == "__main__":
app.run(host='127.0.0.1',port='5000',debug=True)
This is Output:
opening Google using proxy
so its open a webpage but its so slow because of all those replace and with all of these its still can't load CSS!
so what I want is a optimized way to redirect all links to proxy and handling CSS!
I have simple Flask App that takes a CSV upload, makes some changes, and streams the results back to the user's download folder as CSV.
HTML Form
<form action = {{uploader_page}} method = "POST" enctype = "multipart/form-data">
<label>CSV file</label><br>
<input type = "file" name = "input_file" required></input><br><br>
<!-- some other inputs -->
<div id = "submit_btn_container">
<input id="submit_btn" onclick = "this.form.submit(); this.disabled = true; this.value = 'Processing';" type = "submit"></input>
</div>
</form>
PYTHON
from flask import Flask, request, Response, redirect, flash, render_template
from io import BytesIO
import pandas as pd
#app.route('/uploader', methods = ['POST'])
def uploadFile():
uploaded_file = request.files['input_file']
data_df = pd.read_csv(BytesIO(uploaded_file.read()))
# do stuff
# stream the pandas df as a csv to the user download folder
return Response(data_df.to_csv(index = False),
mimetype = "text/csv",
headers = {"Content-Disposition": "attachment; filename=result.csv"})
This works great and I see the file in my downloads folder.
However, I'd like to display "Download Complete" page after it finishes.
How can I do this? Normally I use return redirect("some_url") to change pages.
Consider using send_file() or send_from_directory() for sending files.
Getting 2 responses from 1 request is not possible, but you can split the problem into chunks with the help of some JS, following this simple diagram (not very UML precise, but that's it):
this diagram refers to a previous and simpler version of the code, which was later updated after the OP asked for flash() to be called
POST to /uploader through a function called from the form by onsubmit, so that besides saving the file you can also have some logic there, like checking the response status
process the file (I did a mockup of your processing through upper())
if the server responds with 201 ("Created") then you can save the file and print "Download Complete" (I used window.document.body.innerHTML because it's only one tag and we can replace all the previous DOM; it shouldn't be used to change complex HTML)
else, if the server responds with other status codes (like 500), POST to /something-went-wrong to get the new - possibly flashed - HTML to be rendered. The POST step is not shown in the diagram.
To test the error page, make some syntax error in the processing inside upload_file(), like data_df = pd.read_csv(BytesIO(uploaded_file.aread()))
In the something-went-wrong response I addeed a CSP header to mitigate a possible malicious attack, because we can't trust the user enough.
Here's the code:
main.py
from flask import (Flask,
request,
redirect,
render_template,
send_file,
url_for,
Response, jsonify, flash, make_response)
from flask_wtf.csrf import CSRFProtect
from io import BytesIO
import pandas as pd
app = Flask(__name__)
app.secret_key = "your_secret"
csrf = CSRFProtect(app)
#app.route('/')
def index():
return render_template("index.html")
#app.route("/uploader", methods=['POST'])
def upload_file():
try:
uploaded_file = request.files['input_file']
data_df = pd.read_csv(BytesIO(uploaded_file.read()))
# do stuff
data_df["foo"] = data_df["foo"].str.upper()
# Stream buffer:
io_buffer = BytesIO()
data_df.to_csv(io_buffer)
io_buffer.seek(0)
except Exception as ex:
print(ex) # and log if needed
# Here I return the exception string just as an example! Not good in real usage.
return jsonify({"message": str(ex)}), 500
else:
return send_file(io_buffer,
download_name="result.csv",
as_attachment=True), 201
#app.route("/something-went-wrong", methods=["POST"])
def something_went_wrong():
flash(request.get_json()["message"])
response = make_response(render_template("something-went-wrong.html"), 200)
response.headers['Content-Security-Policy'] = "default-src 'self'"
return response
The form with the JS handler:
<form id="myForm" enctype="multipart/form-data" onsubmit="return submitHandler()">
<input type="hidden" name="csrfToken" value="{{ csrf_token() }}"/>
<label>CSV file</label><br>
<input type="file" id="inputFile" name="input_file" required/><br><br>
<!-- some other inputs -->
<div id="submitBtnContainer">
<input id="submitBtn" type="submit"/>
</div>
</form>
<script>
function submitHandler() {
const csrf_token = "{{ csrf_token() }}";
let formData = new FormData();
const file = document.getElementById('inputFile').files[0];
formData.append("input_file", file);
fetch("/uploader", {
method: "POST",
body: formData,
headers: {
"X-CSRFToken": csrf_token,
},
})
.then(response => {
if (response.status != 201) {
response.json().then(data => {
fetch("/something-went-wrong", {
method: "POST",
body: JSON.stringify({"message": data["message"]}),
headers: {
"Content-Type": "application/json",
"X-CSRFToken": csrf_token,
},
})
.then(response => response.text())
.then(text => {
window.document.body.innerHTML = text;
})
});
}
else {
return response.blob().then(blob => {
const file = new Blob([blob], { type: 'text/csv' });
const fileURL = URL.createObjectURL(file);
let fileLink = document.createElement('a');
fileLink.href = fileURL;
fileLink.download = "result.csv";
fileLink.click();
window.document.body.innerHTML = "<h1>Download Complete</h1>";
});
}
})
return false;
}
</script>
For completeness, my dummy csv "file.csv":
foo
bar
Here is some changes.
set window.open('') in input onclick event.
HTML Form
<form action ="/uploader" method = "POST" enctype = "multipart/form-data">
<label>CSV file</label><br>
<input type = "file" name = "input_file" required></input><br><br>
<!-- some other inputs -->
<div id = "submit_btn_container">
<input id="submit_btn" onclick = "this.form.submit(); this.disabled = true; this.value = 'Processing'; window.open('your_url');" type = "submit"></input>
</div>
</form>
You need two functions, one to deal with the processing such as uploadFile(), and another in the same app route to return render template.
When the uploadFile() function is completed: completed = True
Then, code another function which tests the global variable if completed: to return render template.
See: How can I use the same route for multiple functions in Flask
Finally, return a variable to the page with Jinja2 and use Javascript to identify if that variable exists to load your 'download completed' page by Javascript.
Python:
from flask import Flask, request, Response, redirect, flash, render_template
from io import BytesIO
import pandas as pd
completed = False
#app.route('/uploader', methods = ['POST'])
def uploadFile():
uploaded_file = request.files['input_file']
data_df = pd.read_csv(BytesIO(uploaded_file.read()))
# do stuff
# When stuff is done
global completed
completed = True
# stream the pandas df as a csv to the user download folder
return Response(data_df.to_csv(index = False),
mimetype = "text/csv",
headers = {"Content-Disposition": "attachment; filename=result.csv"})
How to load a new page: https://www.geeksforgeeks.org/how-can-a-page-be-forced-to-load-another-page-in-javascript/
Javascript conditionals: https://www.w3docs.com/learn-javascript/conditional-operators-if.html
Using Jinja2 to render a variable: https://jinja.palletsprojects.com/en/3.0.x/templates/
Also, you should really wrap your uploadFile() function with try and except to catch upload errors.
I want to change an image when I click on it with an image from another drive. For this I created this python function:
#main.route('/tester/', methods=['GET', 'POST'])
def tester():
if request.method == "GET":
test_img = "D:\TBV_Data\MS_CO_Front_20120403_140154_0001140_006.png"
return send_file(test_img, mimetype='image/png')
I request this data by the following function:
function testFunc() {
$.ajax({
url: '/tester', //server url
type: 'GET', //passing data as post method
async: false,
success: function(data) {
$("#myimage9").attr("src", "data:image/png;base64," + data);
},
});
};
The outcome of the "src" of the image is, unfortunately, a load of weird data:
<img id="myimage9" src="data:image/png;base64,�PNG
IHDR�I!�IDATx���mK�&�}�;��morg��c�V��)C�� B��.�(z�� ��� ��*��B�y2�I��^~��]D�1��ÁDb�9��&�E����o-���OZl��/_���NJ��%�%�т���6�ݴw�~��EE���-�[p�z^3Y����8��#�
I can imagine that I didn't encode the image correctly. Could somebody please tell me what I am doing wrong?
EDIT:
I tried to encode the data to base64 with:
function utoa(str) {
return window.btoa(unescape(encodeURIComponent(str)));
};
This will unfortunately just changes the data to the following, but will not show the image:
<img id="myimage9" src="data:image/png;base64,/VBORw0KGgoAAAANSUhEUgAABRAAAAP9CAIAAAAUSSH9AAEAAElEQVR4/f397m1L/Sb9ff0OO/39bW9yZ/39Y/0dVv39KUP9/SBC/f0u/Sh6A
On the server side, add a filename variable to your route:
import os
from flask import request, send_from_directory
IMAGE_DIR = r"D:\TBV_Data"
#main.route('/tester/<filename>')
def tester(filename):
return send_from_directory(IMAGE_DIR, filename)
This uses the send_from_directory function to make sure that the client cannot download files from outside the designated directory by entering filenames that contain things like \..\ (this is known as "directory traversal attack").
And on the client side:
function testFunc(selector, filename) {
$(selector).attr("src", "/tester/" + encodeURIComponent(filename) );
}
The use of encodeURIComponent() is good style and prevents problems with filenames that contain special characters. The browser will request the image as soon as its src attribute changes, there is no need to do anything via Ajax here.
Preventing a directory traversal attack with send_file() alone is a bit more involved:
filepath = os.path.realpath(os.path.join(IMAGE_DIR, filename))
if os.path.commonprefix([filepath, IMAGE_DIR]) != IMAGE_DIR:
abort(404)
return send_file(filepath)
File paths on Windows are case-insensitive, so IMAGE_DIR should contain the actual case of the path, otherwise this test will fail.
Solved:
function testFunc(args) {
selector = args.data[0];
filename = args.data[1];
var obj = JSON.stringify({"data" : filename});
console.log(obj);
$.ajax({
url: '/tester2/', //server url
type: 'GET', //passing data as post method
data: obj,
async: false,
success: function(data) {
$(selector).attr("src", "/tester2/");
},
});
};
I was assigning the raw data to the src attribute, but I needed to assign the URL of the GET request to the src attribute.
I'm trying to allow the click trigger to do some backend dynamic csv creation, and then return it back to the user as a csv download file. I guess I'm not sure how I should write out the return statement other than just putting return response. I'v come across some other posts saying that I would need to set my url to a hidden iframe?? Not sure what this means though. Any tips?
Ajax looks like this:
$('#download-maxes').on('click', function(){
$.ajax({
type: "POST",
url: "{{request.path}}download/",
dataType: 'json',
async: false,
data: JSON.stringify(workouts),
success: function(workoutData) {
console.log(workoutData);
},
error:function(error){
console.log(error);
}
});
});
And my django view looks like this:
def download(request):
#(... a lot of mongo stuff here & other things defined)
workouts = json.load(request.body)
response = HttpResponse(content_type='text/xlsx')
response['Content-Disposition'] = 'attachment; filename="team_maxes.xlsx"'
writer = csv.writer(response)
writer.writerow(['Name', 'Date', 'Workout', 'Max'])
for member in team_members.all():
for wo in workouts:
wo_data = db.activity_types.find_one({"name": wo["name"]})
best_wo = db.activity.find_one({"u_id": member.user.id, "a_t": str(wo_data["_id"]), "is_last": 1}) or 0
member_name = member.user.first_name + ' ' + member.user.last_name
try:
max_stat = best_wo["y_ts"]
except:
max_stat = 0
try:
date = best_wo["e_d"]
except:
date = ""
workout_name = wo_data["name"]
writer.writerow([member_name, date, workout_name, max_stat])
return response
You don't need to use ajax. Since you are POSTing some json data to your view, just make a form with a hidden text input, and set it's value to the json data. Then make a regular submit button in the form.
When the form gets submitted, and the server responds with Content-Disposition: attachment; filename="team_maxes.xlsx", your browser will automatically trigger a download.
If you decide to go this route, Keep in mind:
You are using a regular html form now, with the POST method, so you must remember to use django's {% csrf_token %} tag inside of it.
You will probably have set the input's value to your json string, right before submitting the form.
Your workouts json gets sent in a form input. So, assuming you named your input "workouts", in your view you would do something like:
workouts = json.loads(request.POST.get('workouts'))
Plus a bunch of error checking, of course.