Develop Reference

Getting NULL data when uploading file to Sqlite DB [duplicate]

I am writing my first flask application. I am dealing with file uploads, and basically what I want is to read the data/content of the uploaded file without saving it and then print it on the resulting page. Yes, I am assuming that the user uploads a text file always.
Here is the simple upload function i am using:
#app.route('/upload/', methods=['GET', 'POST'])
def upload():
if request.method == 'POST':
file = request.files['file']
if file:
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
a = 'file uploaded'
return render_template('upload.html', data = a)
Right now, I am saving the file, but what I need is that 'a' variable to contain the content/data of the file .. any ideas?

FileStorage contains stream field. This object must extend IO or file object, so it must contain read and other similar methods. FileStorage also extend stream field object attributes, so you can just use file.read() instead file.stream.read(). Also you can use save argument with dst parameter as StringIO or other IO or file object to copy FileStorage.stream to another IO or file object.
See documentation: http://flask.pocoo.org/docs/api/#flask.Request.files and http://werkzeug.pocoo.org/docs/datastructures/#werkzeug.datastructures.FileStorage.

If you want to use standard Flask stuff - there's no way to avoid saving a temporary file if the uploaded file size is > 500kb. If it's smaller than 500kb - it will use "BytesIO", which stores the file content in memory, and if it's more than 500kb - it stores the contents in TemporaryFile() (as stated in the werkzeug documentation). In both cases your script will block until the entirety of uploaded file is received.
The easiest way to work around this that I have found is:
1) Create your own file-like IO class where you do all the processing of the incoming data
2) In your script, override Request class with your own:
class MyRequest( Request ):
def _get_file_stream( self, total_content_length, content_type, filename=None, content_length=None ):
return MyAwesomeIO( filename, 'w' )
3) Replace Flask's request_class with your own:
app.request_class = MyRequest
4) Go have some beer :)

I share my solution (assuming everything is already configured to connect to google bucket in flask)
from google.cloud import storage
#app.route('/upload/', methods=['POST'])
def upload():
if request.method == 'POST':
# FileStorage object wrapper
file = request.files["file"]
if file:
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = app.config['GOOGLE_APPLICATION_CREDENTIALS']
bucket_name = "bucket_name"
storage_client = storage.Client()
bucket = storage_client.bucket(bucket_name)
# Upload file to Google Bucket
blob = bucket.blob(file.filename)
blob.upload_from_string(file.read())
My post
Direct to Google Bucket in flask

I was trying to do the exact same thing, open a text file (a CSV for Pandas actually). Don't want to make a copy of it, just want to open it. The form-WTF has a nice file browser, but then it opens the file and makes a temporary file, which it presents as a memory stream. With a little work under the hood,
form = UploadForm()
if form.validate_on_submit():
filename = secure_filename(form.fileContents.data.filename)
filestream = form.fileContents.data
filestream.seek(0)
ef = pd.read_csv( filestream )
sr = pd.DataFrame(ef)
return render_template('dataframe.html',tables=[sr.to_html(justify='center, classes='table table-bordered table-hover')],titles = [filename], form=form)

I share my solution, using pandas
#app.route('/upload/', methods=['POST'])
def upload():
if request.method == 'POST':
# FileStorage object wrapper
file = request.files["file"]
if file:
df = pd.read_excel(files_excel["file"])

Building on a great answer by #tbicr the simplest form of that boils down to:
for line in request.files.get('file'):
print("Next line: " + line)

in function
def handleUpload():
if 'photo' in request.files:
photo = request.files['photo']
if photo.filename != '':
image = request.files['photo']
image_string = base64.b64encode(image.read())
image_string = image_string.decode('utf-8')
#use this to remove b'...' to get raw string
return render_template('handleUpload.html',filestring = image_string)
return render_template('upload.html')
in html file
<html>
<head>
<title>Simple file upload using Python Flask</title>
</head>
<body>
{% if filestring %}
<h1>Raw image:</h1>
<h1>{{filestring}}</h1>
<img src="data:image/png;base64, {{filestring}}" alt="alternate" />.
{% else %}
<h1></h1>
{% endif %}
</body>

In case we want to dump the in memory file to disk. This code can be used
if isinstanceof(obj,SpooledTemporaryFile):
obj.rollover()

Flask / Python: #after_this_request is not accessed

I trying to delete a file I've created after it was displayed on the webservice.
However with the current placement of the function, it says 'remove_file(response) is not accessed' and when I run the webservice I get an error when outputting the file, as it apparently already has been deleted.
I don't know what to do anymore, I've tried placing the afer_this_request function right after the upload_file declaration, after the return render template, all of it never works. I'm going crazy, so any help would make my year!
#app.route('/', methods=['POST', 'GET'])
def upload_file():
if request.method == 'POST':
myuuid = uuid.uuid4()
# check if the post request has the file part
if 'file' not in request.files:
return render_template('error.html')
file = request.files['file']
# If the user does not select a file, the browser submits an
# empty file without a filename.
if file.filename == '':
return render_template('error.html')
if file and allowed_file(file.filename):
sfilename = secure_filename(file.filename)
if not os.path.exists(app.config['UPLOAD_FOLDER']):
os.mkdir(app.config['UPLOAD_FOLDER'])
output_path = os.path.join(app.config['UPLOAD_FOLDER'], sfilename)
file.save(output_path)
if 'Alpha Miner' in request.form:
Alpha(pro_file, myuuid)
img_url = url_for('static', filename=str(myuuid) + '.gv.svg')
#after_this_request
def remove_file(response):
os.remove(img_url)
return response
titel = "Petri Net for Alpha Miner: " + sfilename
return render_template('upload.html', img_url=img_url, titel = title)

Try this :
file_handle = open(img_url, 'r')
#after_this_request
def remove_file(response):
try:
os.remove(img_url)
file_handle.close()
except Exception as error:
app.logger.error("Error removing or closing downloaded file handle", error)
return response

Download file from server location, using send_from_directory and filename

I'm struggling to understand how to pass an uploaded filename to the next #app.route in flask to retrieve a file saved on the server.
I have been able to create a form to upload the file to the server like this, and have been able to render the file data and the filename to the proceeding view.
#app.route('/file_upload', methods=['GET', 'POST'])
def upload():
form = UploadForm()
if form.validate_on_submit():
f=form.file.data
filename = secure_filename(f.filename)
file_path = os.path.join(UPLOAD_FOLDER, filename)
f.save(file_path)
Input_data_dataframe = pd.read_excel(f)
return render_template('Set_parameters.html',filename=filename,tables=[Input_data_dataframe.to_html(classes=["table table-dark table-hover"], header="true", index=False,)])
return render_template('file_upload.html',form=form)
The file successfully saves at the required server destination, and the filename is also rendered in the next view like so:
I now want this file to be retrieved from the server location for further processing. I understand that I can use the send_from_directory() function to retrieve the file, however I am not sure how to implement it with the correct filename from the previous #app.route('/file_upload')
#app.route('/Set_Parameters/', methods=['GET', 'POST'])
def processing(filename):
retrievedfile= send_from_directory(app.config['UPLOAD_FOLDER'],filename=filename, as_attachment=False)
Input_data_dataframe = pd.read_excel(retrievedfile) #converts uploaded file to a dataframe astype(int)
if request.method == 'POST':
Input_value1= request.form.get['Input_value1'] #get html form data value
Input_value= request.form.get['Input_value2'] #get html form data value
return render_template('Results.html',Input_value1=Input_value1, Input_value2=Input_value2, tables=[Input_data_dataframe.to_html(classes=["table table-dark table-hover"], header="true", index=False,)])
return render_template('Set_parameters.html')
Currently if I was to run as is, I would get this error
TypeError: processing() missing 1 required positional argument: 'filename' as I can understand I haven't passed the filename to the function.. I am just unsure how to do this without hardcoding the filename into the code, and then to use the same file for further processing.
Any help would be greatly appreciated.

Your processing function expects filename parameter from URL. You can see how it should be done here: https://flask.palletsprojects.com/en/1.1.x/quickstart/#variable-rules.
Route to this view should be: /Set_Parameters/<filename>/. It means that action attribute of the from should be, for example: action="/Set_Parameters/This_is_the_file.xlsx".
If you render that form in flask, using flask templates, it should be easy if you have a filename in a render context:
<form action="/Set_Parameters/{{ filename }}">
...
</form>

Error Reading CSV file using Python, Flask

I'm very new to python and flask. I simply wanted to read a CSV file but it's giving me an error of "FileNotFoundError: [Errno 2] No such file or directory: 'Dog-Data.csv'" everytime I try to run run.py
Here are my file order
DD\
static\
(bunch of image files)
templates\
(bunch of template files)
__init__.py
views.py
Dog-Data.csv
views.py
from flask import render_template
from app import app
import csv
#app.route('/')
#app.route('/Home')
def home():
return render_template("Home.html",title='Home')
#app.route('/MakeaMatch')
def makeamatch():
return render_template("MakeaMatch.html",title='Make a Match!')
#app.route('/Game')
def game():
return render_template("Game.html",title='Game')
#app.route('/ListofDogs')
def listofdogs():
return render_template("ListofDogs.html",title='List of Dogs')
#app.route('/About')
def about():
return render_template("About.html", title='About')
#app.route('/Contact')
def contact():
return render_template("Contact.html", title='Contact')
#app.route('/MatchResult')
def matchresult():
class DogBreed:
def __init__(self, br, per, si, lif, hou, cli):
self.breed = br
self.personality = per
self.size = si
self.lifestyle = lif
self.housing = hou
self.climate = cli
self.match = 0
#List that will contain class DogBreed
ListOfBreeds = []
data_file = open('Dog-Data.csv')
csv_file = csv.reader(data_file)
for row in csv_file:
#print (row) #will print the csv file
#print (row[2]) #will print element of that row
dog_breed = DogBreed(row[0],row[1].lower().split(", "),row[2].lower(),row[3].lower(),row[4].lower(),row[5].lower())
ListOfBreeds.append(dog_breed)
data_file.close()
#MORE CODES HERE. OMITTED BECAUSE I DON'T THINK IT'S RELEVANT
return render_template("MatchResult.html",title='Match Result',posts=ListOfBreeds)
The webpage loads and templates shows up fine if I comment out the lines involving CSV. However, it doesn't of course show the result I want it too.
I've also tried putting the Dog-Data.csv into the static folder and used
data_file = open('static/Dog-Data.csv')
but this didn't work either.
Thanks so much for help.

Have you tried to give the full path instead of just a relative path?
Python sometimes takes the working directory as a "home" path, which might or might not be the same as your view.py is situated in.

ok so really simple mistake that took hours for me to solve.
I should be passing it from the root folder which means I should've put
data_file = open('app/Dog-Data.csv')
and now it works. T__T

Check Flask upload if user does not selected file

If user doesn't select a file in form (equals PHP UPLOAD_ERR_NO_FILE
), request.files['file'] also return a FileStorage object, I am using request.files['file'].filename == '' to check, any better way ? I have seen Flask Doc but can't find the answer.
'file' in request.files will not work on user doesn't select file, browser also will submit a 0 length and filename equals '' (empty string) part.
and how do I detect uploaded file was only partially uploaded error (equals PHP UPLOAD_ERR_PARTIAL) ?

Now I am using
if request.files['file'].filename == '':
return 'No selected file'
or using file length check
import os
file = request.files['file']
file.seek(0, os.SEEK_END)
if file.tell() == 0:
return 'No selected file'

Try:
if not request.files.get('file', None):
pass
If you want have most control over the files, you can use http://pythonhosted.org/Flask-Uploads/

import imghdr
def validate_image(stream):
header = stream.read(512)
stream.seek(0)
format = imghdr.what(None, header)
if not format:
return None
return '.'+format
class UploadFiles(Resource):
def post(self):
doc=request.files.get('image')
if validate_image(doc.stream):
print('file validated')