I am trying to develop as a service where someone can send the csv file to my REST API which I dump in the database. My cURL request is reading the data but flask_restful is not able to process it. Can you please tell me what am I doing wrong and how can I fix it?
[EDITED BELOW]
I found after reading the docs that request.files lets you read the files from the POST request from a form. I also found a way to send a csv file through cURL as a form.
class ForBetaAndUpload(Resource):
def post(self, kind='quotes'):
# parser = reqparse.RequestParser()
file = request.files['file']
print(file)
# kind = parser.add_argument('kind').parse_args()['kind']
if kind:
if file and file[-3:]=='csv':
if kind == 'quotes':
try:
df = pd.read_csv(file)
df.to_sql('QUOTES', helper.conx, index=False, if_exists='append')
return jsonify({'message':'%s rows inserted in the databasee table successfully' %(df.shape[0])})
except Exception as e:
return jsonify({'exception': e})
if kind == 'trace':
try:
df = pd.read_csv(todos)
df.iloc[:10].to_sql('TRACE', helper.conx, index=False, if_exists='append')
return jsonify({'message':'%s rows inserted in the databasee table successfully' %(df.shape[0])})
except Exception as e:
return jsonify({'message': e})
else:
return jsonify({'message': 'Please provide a csv file'})
else:
return jsonify({'message':'Please provide the file for to update relevant table in the databse'})
api.add_resource(ForBetaAndUpload, '/upload', endpoint='upload')
if __name__ == "__main__":
app.run(debug=True)
cURL Request:
curl "https://localhost:5000/upload" -X POST -H 'Content-Type: txt/csv' -d trace.csv --insecure
I'm getting the following message:
curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown
protocol
API Error Message
code 400, message Bad HTTP/0.9 request type
('\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03\x08Ú:ü^¢Ù~ö7W\x9fDyy\x16j\x7fõ>½\x82\x90uÎ&3ÿZ\x08êE\x00\x00')
How can I send a csv file to the flask restful_api. Is it right what I am doing or is there any other way to do it?
My solution to read csv from Flask is:
in Flask:
f = request.files['file']
f will be a handler to the file, you can then use csv_reader.
and to send the file:
curl -s "http://localhost:5000" \
-F file=#./myfile.csv \
-X POST \
-H 'enctype:multipart/form-data ; Content-Type:multipart/form-data'
And that should works. Let me know.
Related
I have a website which takes a .txt file as input through an upload button. The backend model processes this text file and output a new .txt file. My website is working perfectly with the UI. But I was trying to send GET/POST request to my file using the curl command:
curl -F 'file=#CNN.txt' http://127.0.0.1:5000/
The output was that my whole html file got printed (as a cat command does) in the terminal.
I want to know how can I get the processed file using the curl command itself? I think to get the file, I need to return some kind of JSON object too. I am completely new to this stuff. Please bare with me.. My appy.py file is:
#app.route('/', methods = ['GET','POST'])
def hello():
if(request.method == 'POST'):
if('file' not in request.files):
return 'NO FILE'
file = request.files['file']
if(file.filename == ''):
print('NO FILES')
return redirect(request.url)
if(file and allowed_file(file.filename)):
uploadedFile = file.filename
file.save(os.path.join(UPLOAD_FOLDER, file.filename))
if(uploadedFile != ''):
neural_code_sum.starter(uploadedFile)
return render_template('index.html', message='success')
return render_template('index.html', message='NOT UPLOADED (ONLY .TXT FILES ALLOWED)')
#app.route('/download')
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
Just add GET above:
#app.route('/download', methods = ['GET'])
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
The first send the file: curl -F 'file=#CNN.txt' http://127.0.0.1:5000/
Then download it: curl http://127.0.0.1:5000/download -o output.txt
That's it! All the best.
#app.route('/download',methods=['**GET'**])
def download_file():
global uploadedFile
doc = os.path.dirname(os.path.realpath(__file__))+'/output.txt'
return send_file(doc,as_attachment=True,cache_timeout=0)
Add the method by which you want to send the request in the methods field.
I'm developing a small program which inserts a data obtained by curl to a MySQL database. Unfortunately, I haven't found any similar about my question, so if you have something on your mind - please point me out there. Here is what I tried so far
import os
import pymysql
from flask import Flask, request
app = Flask(__name__)
db = pymysql.connect("127.0.0.1","meeting_admin","password","rooms")
cur = db.cursor()
#app.route('/book_room', methods = ['POST'])
def book_room():
try:
room = request.form['room']
book_date = request.form['book_date']
booked_by = request.form['booked_by']
cur.execute("INSERT INTO booked_rooms(room, book_date, booked_by)
VALUES ('%c', '%d', '%s')",(room, book_date, booked_by))
db.commit()
cur.close()
return "OK\n", 200
except:
db.rollback()
return "Smth went wrong", 500
db.close()
if __name__ == '__main__':
app.run()
And the curl command is:
curl -H "Content-type: application/json" -X POST /
http://127.0.0.1:5000/book_room -d /
'{room=602,book_date:27-08-2019,booked_by="someone"}'
And what I'm getting is:
File "server_flask.py", line 23, in book_room
room = request.form['room']
File "/usr/local/lib/python3.5/dist-
packages/werkzeug/datastructures.py", line 431, in __getitem__
raise exceptions.BadRequestKeyError(key)
werkzeug.exceptions.HTTPException.wrap.<locals>.newcls: 400 Bad
Request: The browser (or proxy) sent a request that this server could
not understand.
Thanks in advance.
I am a beginner in making API. I have followed the blog https://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask and able to create the Get - Post API method.
I am using Flask to making Rest API. Please see the code below in which I want to take question as input in API and return an answer in JSON format by making my code as an API.
app = Flask(__name__)
#app.route('/match/api/v1', methods = ['POST'])
def my_form_post():
if not request.json or not 'question' in request.json:
abort(400)
input_text_p = request.json['question'] # access input from curl request
reference_data = request.json['data'] # to access data field from the API request
path = 'airtel_faq.xlsx'
question_list, answer_list = read_excel_file(path) # reading some reference data from an excel file
input_text = input_text_p.translate(None, string.punctuation) # remove punctuation
final_answer = find_similarity(input_text, answer_list, question_list)
print "Final Answer is : ", final_answer
values = [
{'id' : 1,
'answer' : final_answer # answer I want in JSON
'done' : False
}
]
return jsonify({'values': values}), 201
if __name__ == '__main__':
app.run(debug = True)
I am trying to pass hash input for data tag in the request to API. I am not aware how to do this. The curl request I am making is giving error:
Failed to decode JSON object: Expecting ',' delimiter or '}': line 1 column 80 (char 79)
The curl request I am making :
curl -i -H "Content-Type: application/json" -X POST -d '{"question":"Hi","cutoff":"0.3", "data":"{"q":"hello whats going on","a":"I am fine","type":"1"}"}' http://localhost:5000/match/api
Please let me know how to do this. Do I have to include anything in my script. Or is there a way to pass the JSON file in API call by giving path to the file.
In order to parse the JSON request, it needs to be valid JSON. Try this request:
curl -i -H "Content-Type: application/json" -X POST -d '{"question": "Hi", "cutoff": "0.3", "data": {"q": "hello whats going on", "a": "I am fine", "type": "1"}}' http://localhost:5000/match/api
I am about to implement a Push Task queue in Flask/Google App Engine.
Essentially I would like to POST to the API and execute the underlying work in the task queue.
The initial entry point is a REST API (flask_restful)
class FTRecordsAPI(Resource):
def post(self):
arguments = self.reqparser.parse_args()
json_records = arguments.get('records')
user = User.query(...).get()
if user:
taskqueue.add(url='/worker/', params={'user': user})
return '', 201
else:
return '', 401
The worker is defined as a view in the url.py:
app.add_url_rule('/worker', 'worker',
view_func=csrf_protect.exempt(TaskView.as_view('taskView')))
And the TaskView is:
from flask.globals import request
class TaskView(MethodView):
def post(self):
user = request.json['user']
return "OK"
Strangely when I debug in the TaskView nowhere in the request object is any trace of the user object I sent to the /worker. However I find in there the records object which was from the previous call ?!
What am I missing please?
Try:
taskqueue.add(url='/worker', params={'user': user}, method="POST")
and
user = request.form.get('user')
As marcadian pointed out, taskqueue uses POST by default, so perhaps you need the request.form to access the POST vars.
Sorry for the thread resurrection, but should you want to use json directly you can do the following for instance:
taskstop = taskqueue.add(url='/stop_std_watcher',
target='worker',
payload=json.dumps({'watcherjson': client.client_watcher}),
method='POST',
headers={'Content-Type': 'application/json'})
the important bits being to add the headers and payload kwargs
On the receiving end, you got then:
#app.route('/stop_std_watcher', methods=['POST'])
def stop_std_watcher():
# curl -i -H "Content-Type: application/json" -X POST -d #stop.json http://localhost:8081/stop_std_watcher
watcherjson = request.json['watcherjson']
I added the curl in comments should you want to test your post route manually, can be useful
My end goal is to have an external website POST data to my site using DRF. The data they send is in XML Format.
python 2.7
djangorestframework 3.3.2
djangorestframework-xml 1.3.0
My issues is whenever the external site posts data, it returns a HTTP 400 error. But I can CURL the data fine. I am not able to find where I can see or dump the data that is in the rest framework POST. I have tried working through the parser_context dictionary being sent to CustomXMLParse Parse. But I always return None, even with the successful CURL
I overrode the default XMLParser so I could retrieve an attribute. I will say I am able to successfully use the API when I CURL.
Successful Curl
curl http:/website/post/ -H "Content-Type: text/xml" -d "<registrationreport format='summary' regid='abc123' instanceid='0'><complete>complete</complete><success>failed</success><totaltime>19</totaltime><score>0</score></registrationreport>"
Views.py
#api_view(['POST'])
#authentication_classes((BasicAuthentication,))
#parser_classes((CustomXMLParser, ))
#permission_classes((IsAuthenticated,))
def coursePostBack(request):
...
#Error from parsers.py STOPS before running anything in View
Parsers.py
class CustomXMLParser(XMLParser):
"""
Custom XML parser.
"""
logger.debug(' coursePostBack | PARSER HIT ')
media_type = 'text/xml'
def parse(self, stream, media_type=None, parser_context=None):
logger.debug(' coursePostBack | PARSER--PARSE HIT')
assert etree, 'XMLParser requires defusedxml to be installed'
parser_context = parser_context or {}
encoding = parser_context.get('encoding', settings.DEFAULT_CHARSET)
parser = etree.DefusedXMLParser(encoding=encoding)
try:
## This is where it fails! How can I SEE the data being sent in POST ##
tree = etree.parse(stream, parser=parser, forbid_dtd=True)
except (etree.ParseError, ValueError) as exc:
logger.error('XML Parse error - %s' % six.text_type(exc))
raise ParseError('XML parse error - %s' % six.text_type(exc))
data = self._xml_convert(tree.getroot())
attribs = tree.getroot().attrib
data['regid'] = attribs['regid']
return data
Log.py
ERROR [training:36] XML Parse error - not well-formed (invalid token): line 1, column 1