I have a python project in PythonAnywhere where the user uploads a file, it gets processed, and then it gets downloaded again by the user.
The file gets processed using a RandomForestRegressor model.
So far, it works perfectly. The user uploads and a new output file is generated. What I want to do is that the results of the model be returned in the website.
A message like this:
Accuracy of model on test set: 0.90
mse of model on train set: 0.08
The HTML is like this:
<html>
<body>
{% for comment in comments %}
<div class="row">
{{ comment }}
</div>
{% endfor %}
</body>
</html>
I was trying to add a comment after the proccess_data function is called. I added the render_template with the comment variable but it doesnt work. I even added it at the end of the def index() as a return but it just doesnt work.
output_file = process_data(filename)
comments = 'File processed succesfully'
render_template("main_page.html", comments=comments)
The python script is like this:
comments = []
#app.route('/', methods=["GET","POST"])
def index():
if request.method == "GET":
return render_template("main_page.html")
if request.method == 'POST':
#comments.append(request.form["contents"])
#check if the post request has the file part
if 'input_file' not in request.files:
#TODO
return redirect(request.url)
file = request.files['input_file']
#if the user does not select a file, the browser submits
#empty file without a filename
if file.filename == '':
#TODO
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
output_file = process_data(filename)
comments = 'File processed succesfully'
render_template("main_page.html", comments=comments)
si = io.StringIO()
output_file.to_csv(si,index=False, encoding='UTF8')
#-----Save dataframe to folder-----
filepath = os.path.join(app.config['UPLOAD_FOLDER'], 'results.csv')
output_file.to_csv(filepath)
response = make_response(si.getvalue())
response.headers["Content-Disposition"] = "attachment; filename=results.csv"
response.headers["Content-type"] = "text/csv"
return response
return
I don't work with python but I see you basically want to send two responses in one request and most likely at different times?
HTTP does not work like that. You can't send one response "mid process" and then some other after a while. It's always one request - one response. And a new tcp connection opens and closes immediately each time.
So if you want to notify user about the results of backend process some time before you actually send him the "final data", you have these options:
Divide this flow into multiple individual requests.
Use WebSockets.
Related
The code to send email in Django is written like this in views.py file - it sends the pdf fine, if the file size is large (~3mb), but for smaller files (~1 or 2mb) it does not send the file (it appears as a zero byte file) in the email, and hence we cannot access it.
The code is as belows -
Please let me know if there is a way to avoid such a compression of file and send small files to.
def addbeast(request):
if request.method == 'POST':
form = BeastForm(request.POST, request.FILES)
# print("here")
# print(form.cleaned_data['name'])
if form.is_valid():
# print("here")
# print(form.cleaned_data['media'])
form.save()
# media = Beast.objects.get(name = '')
name = form.cleaned_data['name']
media = form.cleaned_data['media']
media.seek(0, os.SEEK_END)
filesize = media.tell()
print(filesize)
subject = "NDA IS Attached"
message = "NDA Attachment from " + name
sender = "-----#gmail.com"
cc_myself = True
cc_mail = "----!!!!!!#gmail.com"
recipients = ['9999999999#gmail.com']
if cc_myself:
recipients.append(cc_mail)
try:
mail = EmailMessage(subject, message, sender, recipients)
mail.attach(media.name, media.read(), media.content_type)
mail.send()
return HttpResponse("FORM SUBMISSION IS SUCCESSFUL")
except:
return HttpResponse('no success')
else:
form = BeastForm()
return render(request, "post_list.html", {
"form": form
})
Please help with how the code can be made right, on printing the variable media, I get 'filename.pdf'.
Try inserting "media.seek(0)" before the line "mail.attach(media.name..."
Django uses InMemoryUploadedFile if the content length is less than FILE_UPLOAD_MAX_MEMORY_SIZE (2.5MB by default), otherwise it uses TemporaryUploadedFile.
Your "media.seek(0, os.SEEK_END)" puts the file's current position at the end. Looks like the subsequent read command continues from the current position for in-memory file, but starts from the beginning for temporary files.
idea:
take id's from html input
use id's to run sql and return relevant usernames
download the output as a csv on the front end when the "download" button is clicked
html
Enter comma delimited ids <input type="text" id="text1" name="text1"><br><br>
download
python
#app.route("/getPlotCSV", methods=['GET','POST'])
def getPlotCSV():
text1 = request.form['text1']
result = {}
a = []
x = []
a.extend([str(x) for x in text1.split(",")])
format_strings = ','.join(['%s'] * len(a))
cursor = cnxn.cursor()
sql = "SELECT DisplayName FROM dbo.[Users] where id IN ({seq})".format(
seq=','.join(['?'] * len(a)))
cursor.execute(sql,a)
for row, in cursor:
x.append(row)
csv = x
return Response(
csv,
mimetype="text/csv",
headers={"Content-disposition":
"attachment; filename=myplot.csv"})
The sql and input works because i have tested it separately without the csv download and it returns the correct data. The error i get at the moment is "400 Bad Request: The browser (or proxy) sent a request that this server could not understand." KeyError: 'text1'
what am i missing here?
The KeyError is because you haven't actually passed the value of text1 to your getPlotCSV route. A link on an HTML page won't also transfer the data with it. Instead you need to use a form with an action page, like this:
<form action="/getPageCSV" method="post">
Enter comma delimited ids: <input type="text" id="text1" name="text1">
<input type="submit" name="Submit" id="submit">
</form>
This should then pass those values to the url in the form action attribute, in this case your getPageCSV. It doesn't have to be POST, I've just done that as an example.
Then, when your route receives the data:
#app.route('/getPageCSV')
def getPlotCSV():
if request.method == "POST": #in other words, if the form is being submitted
text1 = request.form.get('text1') #use dict.get in case it isn't there
# your code
csv = x
return redirect(url_for('getPlotCSV')) #this time a get method
return Response(
csv,
mimetype="text/csv",
headers={"Content-disposition":
"attachment; filename=myplot.csv"})
The above won't specifically work without you adding in your own way to move the POST process data/csv over to when the user is redirected. You could do it as a request header, store it in the session storage or even put it in the query string, it's up to you, but you have to be able to display the results of your POST process into a GET request when the user is redirected.
I have a website which takes a .txt file as input through an upload button. The backend model processes this text file and output a new .txt file. My website is working perfectly with the UI. But I was trying to send GET/POST request to my file using the curl command:
curl -F 'file=#CNN.txt' http://127.0.0.1:5000/
The output was that my whole html file got printed (as a cat command does) in the terminal.
I want to know how can I get the processed file using the curl command itself? I think to get the file, I need to return some kind of JSON object too. I am completely new to this stuff. Please bare with me.. My appy.py file is:
#app.route('/', methods = ['GET','POST'])
def hello():
if(request.method == 'POST'):
if('file' not in request.files):
return 'NO FILE'
file = request.files['file']
if(file.filename == ''):
print('NO FILES')
return redirect(request.url)
if(file and allowed_file(file.filename)):
uploadedFile = file.filename
file.save(os.path.join(UPLOAD_FOLDER, file.filename))
if(uploadedFile != ''):
neural_code_sum.starter(uploadedFile)
return render_template('index.html', message='success')
return render_template('index.html', message='NOT UPLOADED (ONLY .TXT FILES ALLOWED)')
#app.route('/download')
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
Just add GET above:
#app.route('/download', methods = ['GET'])
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
The first send the file: curl -F 'file=#CNN.txt' http://127.0.0.1:5000/
Then download it: curl http://127.0.0.1:5000/download -o output.txt
That's it! All the best.
#app.route('/download',methods=['**GET'**])
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
Add the method by which you want to send the request in the methods field.
I am trying to create a conferencing app with max 2 speakers using Twilio using Python/Django. However, in the docs I found out that you can do this by having inbound calls. but, my business model doesnt work like that. Is there a way for this to work like:
My Twilio number calls number 1
My Twilio number calls number 2
Twilio brings two channels to a new conference
I've tried this solution:
Twilio how to make two outbound calls and join(conference) them using node js
but it didn't help me much..
Here's my code:
#csrf_exempt
def conference(request):
print("success")
response = VoiceResponse()
dial = Dial()
dial.conference('Rooxm 1234')
response.append(dial)
print(response)
return HttpResponse('')
def call(number):
client = Client(TWILIO_ACCOUNT_SID,TWILIO_AUTH_TOKEN)
call = client.calls.create(
url='https://<blahblah_removed_purposefully>.ngrok.io/conf/',
to='+' + str(number),
from_='<removed_my_twilio_num>'
)
print(call.sid)
def index(request):
if request.method == 'POST':
# create a form instance and populate it with data from the request:
form = CallForm(request.POST)
# check whether it's valid:
if form.is_valid():
#print(dir(form.data.values))
call(form.cleaned_data['inline'])
call(form.cleaned_data['outline'])
return HttpResponseRedirect('/thanks/')
# if a GET (or any other method) we'll create a blank form
else:
form = CallForm()
return render(request, 'CallForm.html', {'form': form})
This gave me an error message during the call which is:
"An application error has occurred. Goodbye"
And I also get this in the debugger:
"Error on line 1 of document : Premature end of file. "
Any idea?
Okay so, I figured this out. The only thing that was needed to make that setup work is I had to modify the response, add the xml string there and then set the content_type of the returned object.
return HttpResponse(str(response),content_type='application/xml')
I am trying to use ng-file-upload in my Flask app, but I'm having a hard time send it to Flask main.py to be saved. I could have just done a simple submit using flask's url_for(), but I would like to keep Flask in the backend and AngularJS in the front-end.
Also, I would like to use the drag and drop field feature from this angular plugin instead of an upload file button, but I think that's where my issue is coming from. I don't think flask is recognizing it from request.files. I'm not sure if I'm calling correctly or maybe is just a simple fix and I'm not seeing it.
Flask main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['user_file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
I know the "POST" works because i can see the message from this print statement, print 'unknown went through', after that it fails to recognize the filestorage variable
Here is my angular controller:
myApp.controller('UploadFileCtrl',[
'$scope',
'$http',
'Upload',
function($scope, $http, Upload){
// upload later on form submit or something similar
$scope.submit = function(files) {
if ($scope.files) {
$scope.upload($scope.files);
}
}
// upload on file select or drop
$scope.upload = function (files) {
console.log('it was uploaded');
Upload.upload({
url: '/',
method: 'POST',
data: {file: files}
}).then(function (resp) {
// console.log('Success ' + resp.config.data.file.name + 'uploaded. Response: ' + resp.data);
console.log(resp.data);
}, function (resp) {
console.log('Error status: ' + resp.status);
console.log(resp);
}, function (evt) {
var progressPercentage = parseInt(100.0 * evt.loaded / evt.total);
console.log('progress: ' + progressPercentage + '% ' + evt.config.data.file.name);
});
}
}])
html form:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true"
name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>
I'm still not familiar with Flask, but it's a cool webframework to work with and it will be awesome to integrate it with AngularJS. Your help will be appreciated, thanks in advance!
UPDATE
I actually got it to upload in Flask, I need it to replace filestorage = request.files['user_file'] with filestorage = request.files['file']. Like I said, Flask was not recognizing it by it's provided name in the html, which I don't know why. I failed to provide this line return render_template('index.html') in the main.py above. Since Flask is not longer present in the html, could that be the reason Flask is not recognizing the name in the form?
revised main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
return render_template('index.html')
Also I had to change in the html the ngf-multiple to false, this was sending an array, so Flask was expecting a dict.
revised html:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true" name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>