Develop Reference

Why is reqparse not understanding the POST request?

From what I've seen online, I can use reqparse from flask_restful to add data to my GET and POST requests.
Here is what I have:
from flask import Flask, request
from flask_restful import Resource, Api, reqparse
import pandas as pd
import ast
app = Flask(__name__)
api = Api(app)
class User(Resource):
def get(self):
return {'data': 'get'}, 200
def post(self):
parser = reqparse.RequestParser(bundle_errors=True)
parser.add_argument('userId', type=str)
args = parser.parse_args()
print(args)
return {'data': 'post'}, 200
class Game(Resource):
pass
api.add_resource(User, '/user')
api.add_resource(Game, '/game')
if __name__ == '__main__':
app.run()
I'm trying to send this POST request (using Postman):
http://127.0.0.1:5000/user?userId=hello
But I always get this error back:
{
"message": "The browser (or proxy) sent a request that this server could not understand."
}
I truly don't know what I'm doing wrong...

I still have not figured out how to fix reqparse. However, I managed to get the result desired another way.
Basically you can change the api.add_resource() to include variables which you can pass into the class path. Like: api.add_resource(User, 'user/<userID>') allows you to send a request like http://127.0.0.1:5000/user/kayer and have the variable userID be kayer.
The other way (to go back to my original question on how to use this type of request: http://127.0.0.1:5000/user?userID=kayer is to implement reuest.args.get('userID') and assign the return of that function to a variable.
Full code:
from flask import Flask, request
from flask_restful import Resource, Api, reqparse
import pandas as pd
import ast
app = Flask(__name__)
api = Api(app)
class User(Resource):
def get(self, userId):
a = request.args.get('name')
return {'data': {'a': userId, 'b': a}}, 200
def post(self):
pass
class Game(Resource):
pass
api.add_resource(User, '/user/<userId>')
api.add_resource(Game, '/game')
if __name__ == '__main__':
app.run()

I think the problem is that reqparse (or flask_restful itself), by default, is not parsing the query string (?userId=hello), when it matches the request URL with the endpoint.
From https://flask-restful.readthedocs.io/en/latest/reqparse.html#argument-locations:
By default, the RequestParser tries to parse values from flask.Request.values, and flask.Request.json.
Use the location argument to add_argument() to specify alternate locations to pull the values from. Any variable on the flask.Request can be used.
As instructed and from the example on that documentation, you can explicitly specify a location keyword param as location="args", where args refers to flask.Request.args, which means the:
The parsed URL parameters (the part in the URL after the question mark).
...which is exactly where userId is set when you call the endpoint.
It seems to work after adding the location="args" parameter:
def post(self):
parser = reqparse.RequestParser(bundle_errors=True)
parser.add_argument("userId", type=str, location="args") # <--------
args = parser.parse_args()
print(args)
return {"data": args}, 200
Postman:
cURL:
$ curl -XPOST http://localhost:5000/user?userId=hello
{
"data": {
"userId": "hello"
}
}
Tested with:
Flask 2.1.2
Flask-RESTful 0.3.9

Is it possible to run custom code before Swagger validations in a python/flask server stub?

I'm using the swagger editor (OpenApi 2) for creating flask apis in python. When you define a model in swagger and use it as a schema for the body of a request, swagger validates the body before handing the control to you in the X_controller.py files.
I want to add some code before that validation happens (for printing logs for debugging purposes). Swagger just prints to stdout errors like the following and they are not useful when you have a lot of fields (I need the key that isn't valid).
https://host/path validation error: False is not of type 'string'
10.255.0.2 - - [20/May/2020:20:20:20 +0000] "POST /path HTTP/1.1" 400 116 "-" "GuzzleHttp/7"
I know tecnically you can remove the validations in swagger and do them manually in your code but I want to keep using this feature, when it works it's awesome.
Any ideas on how to do this or any alternative to be able to log the requests are welcome.

After some time studying the matter this is what I learnt.
First let's take a look at how a python-flask server made with Swagger Editor works.
Swagger Editor generates the server stub through Swagger Codegen using the definition written in Swagger Editor. This server stub returned by codegen uses the framework Connexion on top of flask to handle all the HTTP requests and responses, including the validation against the swagger definition (swagger.yaml).
Connexion is a framework that makes it easy to develop python-flask servers because it has a lot of functionality you'd have to make yourself already built in, like parameter validation. All we need to do is replace (in this case modify) these connexion validators.
There are three validators:
ParameterValidator
RequestBodyValidator
ResponseValidator
They get mapped to flask by default but we can replace them easily in the __main__.py file as we'll see.
Our goal is to replace the default logs and default error response to some custom ones. I'm using a custom Error model and a function called error_response() for preparing error responses, and Loguru for logging the errors (not mandatory, you can keep the original one).
To make the changes needed, looking at the connexion validators code, we can see that most of it can be reused, we only need to modify:
RequestBodyValidator: __call__() and validate_schema()
ParameterValidator: __call__()
So we only need to create two new classes that extend the original ones, and copy and modify those functions.
Be careful when copying and pasting. This code is based on connexion==1.1.15. If your are on a different version you should base your classes on it.
In a new file custom_validators.py we need:
import json
import functools
from flask import Flask
from loguru import logger
from requests import Response
from jsonschema import ValidationError
from connexion.utils import all_json, is_null
from connexion.exceptions import ExtraParameterProblem
from swagger_server.models import Error
from connexion.decorators.validation import ParameterValidator, RequestBodyValidator
app = Flask(__name__)
def error_response(response: Error) -> Response:
return app.response_class(
response=json.dumps(response.to_dict(), default=str),
status=response.status,
mimetype='application/json')
class CustomParameterValidator(ParameterValidator):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __call__(self, function):
"""
:type function: types.FunctionType
:rtype: types.FunctionType
"""
#functools.wraps(function)
def wrapper(request):
if self.strict_validation:
query_errors = self.validate_query_parameter_list(request)
formdata_errors = self.validate_formdata_parameter_list(request)
if formdata_errors or query_errors:
raise ExtraParameterProblem(formdata_errors, query_errors)
for param in self.parameters.get('query', []):
error = self.validate_query_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('path', []):
error = self.validate_path_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('header', []):
error = self.validate_header_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
for param in self.parameters.get('formData', []):
error = self.validate_formdata_parameter(param, request)
if error:
response = error_response(Error(status=400, description=f'Error: {error}'))
return self.api.get_response(response)
return function(request)
return wrapper
class CustomRequestBodyValidator(RequestBodyValidator):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __call__(self, function):
"""
:type function: types.FunctionType
:rtype: types.FunctionType
"""
#functools.wraps(function)
def wrapper(request):
if all_json(self.consumes):
data = request.json
if data is None and len(request.body) > 0 and not self.is_null_value_valid:
# the body has contents that were not parsed as JSON
return error_response(Error(
status=415,
description="Invalid Content-type ({content_type}), JSON data was expected".format(content_type=request.headers.get("Content-Type", ""))
))
error = self.validate_schema(data, request.url)
if error and not self.has_default:
return error
response = function(request)
return response
return wrapper
def validate_schema(self, data, url):
if self.is_null_value_valid and is_null(data):
return None
try:
self.validator.validate(data)
except ValidationError as exception:
description = f'Validation error. Attribute "{exception.validator_value}" return this error: "{exception.message}"'
logger.error(description)
return error_response(Error(
status=400,
description=description
))
return None
Once we have our validators, we have to map them to the flask app (__main__.py) using validator_map:
validator_map = {
'parameter': CustomParameterValidator,
'body': CustomRequestBodyValidator,
'response': ResponseValidator,
}
app = connexion.App(__name__, specification_dir='./swagger/', validator_map=validator_map)
app.app.json_encoder = encoder.JSONEncoder
app.add_api(Path('swagger.yaml'), arguments={'title': 'MyApp'})
If you also need to replace the validator I didn't use in this example, just create a custom child class of ResponseValidator and replace it on the validator_map dictionary in __main__.py.
Connexion docs:
https://connexion.readthedocs.io/en/latest/request.html

Forgive me for repeating an answer first posted at https://stackoverflow.com/a/73051652/1630244
Have you tried the Connexion before_request feature? Here's an example that logs the headers and content before Connexion validates the body:
import connexion
import logging
from flask import request
logger = logging.getLogger(__name__)
conn_app = connexion.FlaskApp(__name__)
#conn_app.app.before_request
def before_request():
for h in request.headers:
logger.debug('header %s', h)
logger.debug('data %s', request.get_data())

Having Trouble Making a RESTful API with Flask-RestX: "No operations defined in spec!" and "404"s

In summary, I have been following the flask restx tutorials to make an api, however none of my endpoints appear on the swagger page ("No operations defined in spec!") and I just get 404 whenever I call them
I created my api mainly following this https://flask-restx.readthedocs.io/en/latest/scaling.html
I'm using python 3.8.3 for reference.
A cut down example of what I'm doing is as follows.
My question in short is, what am I missing?
Currently drawing blank on why this doesn't work.
Directory Structure
project/
- __init__.py
- views/
- __init__.py
- test.py
manage.py
requirements.txt
File Contents
requirements.txt
Flask-RESTX==0.2.0
Flask-Script==2.0.6
manage.py
from flask_script import Manager
from project import app
manager = Manager(app)
if __name__ == '__main__':
manager.run()
project/init.py
from flask import Flask
from project.views import api
app = Flask(__name__)
api.init_app(app)
project/views/init.py
from flask_restx import Api, Namespace, fields
api = Api(
title='TEST API',
version='1.0',
description='Testing Flask-RestX API.'
)
# Namespaces
ns_test = Namespace('test', description='a test namespace')
# Models
custom_greeting_model = ns_test.model('Custom', {
'greeting': fields.String(required=True),
})
# Add namespaces
api.add_namespace(ns_test)
project/views/test.py
from flask_restx import Resource
from project.views import ns_test, custom_greeting_model
custom_greetings = list()
#ns_test.route('/')
class Hello(Resource):
#ns_test.doc('say_hello')
def get(self):
return 'hello', 200
#ns_test.route('/custom')
class Custom(Resource):
#ns_test.doc('custom_hello')
#ns_test.expect(custom_greeting_model)
#ns_test.marshal_with(custom_greeting_model)
def post(self, **kwargs):
custom_greetings.append(greeting)
pos = len(custom_greetings) - 1
return [{'id': pos, 'greeting': greeting}], 200
How I'm Testing & What I Expect
So going to the swagger page, I expect the 2 endpoints defined to be there, but I just see the aforementioned error.
Just using Ipython in a shell, I've tried to following calls using requests and just get back 404s.
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
response = r.get(base_url + 'api/test')
response
response = r.get(base_url + 'api/test/')
response
data = json.dumps({'greeting': 'hi'})
response = r.post(base_url + 'test/custom', data=data)
response
data = json.dumps({'greeting': 'hi'})
response = r.post(base_url + 'test/custom/', data=data)
response

TL;DR
I made a few mistakes in my code and test:
Registering api before declaring the routes.
Making a wierd assumption about how the arguments would be passed to the post method.
Using a model instead of request parser in the expect decorator
Calling the endpoints in my testing with an erroneous api/ prefix.
In Full
I believe it's because I registered the namespace on the api before declaring any routes.
My understanding is when the api is registered on the app, the swagger documentation and routes on the app are setup at that point. Thus any routes defined on the api after this are not recognised. I think this because when I declared the namespace in the views/test.py file (also the model to avoid circular referencing between this file and views/__init__.py), the swagger documentation had the routes defined and my tests worked (after I corrected them).
There were some more mistakes in my app and my tests, which were
Further Mistake 1
In my app, in the views/test.py file, I made a silly assumption that a variable would be made of the expected parameter (that I would just magically have greeting as some non-local variable). Looking at the documentation, I learnt about the RequestParser, and that I needed to declare one like so
from flask_restx import reqparse
# Parser
custom_greeting_parser = reqparse.RequestParser()
custom_greeting_parser.add_argument('greeting', required=True, location='json')
and use this in the expect decorator. I could then retrieve a dictionary of the parameters in my post method. with the below
...
def post(self):
args = custom_greeting_parser.parse_args()
greeting = args['greeting']
...
The **kwargs turned out to be unnecessary.
Further Mistake 2
In my tests, I was calling the endpoint api/test, which was incorrect, it was just test. The corrected test for this endpoint is
Corrected test for test endpoint
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
response = r.get(base_url + 'test')
print(response)
print(json.loads(response.content.decode()))
Further Mistake 3
The test for the other endpoint, the post, I needed to include a header declaring the content type so that the parser would "see" the parameters, because I had specified the location explictily as json. Corrected test below.
Corrected test for test/custom endpoint
import json
import requests as r
base_url = 'http://127.0.0.1:5000/'
data = json.dumps({'greeting': 'hi'})
headers = {'content-type': 'application/json'}
response = r.post(base_url + 'test/custom', data=data, headers=headers)
print(response)
print(json.loads(response.content.decode()))
Corrected Code
For the files with incorrect code.
views/init.py
from flask_restx import Api
from project.views.test import ns_test
api = Api(
title='TEST API',
version='1.0',
description='Testing Flask-RestX API.'
)
# Add namespaces
api.add_namespace(ns_test)
views/test.py
from flask_restx import Resource, Namespace, fields, reqparse
# Namespace
ns_test = Namespace('test', description='a test namespace')
# Models
custom_greeting_model = ns_test.model('Custom', {
'greeting': fields.String(required=True),
'id': fields.Integer(required=True),
})
# Parser
custom_greeting_parser = reqparse.RequestParser()
custom_greeting_parser.add_argument('greeting', required=True, location='json')
custom_greetings = list()
#ns_test.route('/')
class Hello(Resource):
#ns_test.doc('say_hello')
def get(self):
return 'hello', 200
#ns_test.route('/custom')
class Custom(Resource):
#ns_test.doc('custom_hello')
#ns_test.expect(custom_greeting_parser)
#ns_test.marshal_with(custom_greeting_model)
def post(self):
args = custom_greeting_parser.parse_args()
greeting = args['greeting']
custom_greetings.append(greeting)
pos = len(custom_greetings) - 1
return [{'id': pos, 'greeting': greeting}], 200

Catch http-status code in Flask

I lately started using Flask in one of my projects to provide data via a simple route. So far I return a json file containing the data and some other information. When running my Flask app I see the status code of this request in terminal. I would like to return the status code as a part of my final json file. Is it possible to catch the same code I see in terminal?
Some simple might look like this
from flask import Flask
from flask import jsonify
app = Flask(__name__)
#app.route('/test/<int1>/<int2>/')
def test(int1,int2):
int_sum = int1 + int2
return jsonify({"result":int_sum})
if __name__ == '__main__':
app.run(port=8082)
And in terminal I get:

You are who set the response code (by default 200 on success response), you can't catch this value before the response is emited. But if you know the result of your operation you can put it on the final json.
#app.route('/test/<int1>/<int2>/')
def test(int1, int2):
int_sum = int1 + int2
response_data = {
"result": int_sum,
"sucess": True,
"status_code": 200
}
# make sure the status_code on your json and on the return match.
return jsonify(response_data), 200 # <- the status_code displayed code on console
By the way if you access this endpoint from a request library, on the response object you can find the status_code and all the http refered data plus the json you need.
Python requests library example
import requests
req = requests.get('your.domain/test/3/3')
print req.url # your.domain/test/3/3
print req.status_code # 200
print req.json() # {u'result': 6, u'status_code: 200, u'success': True}

You can send HTTP status code as follow:
#app.route('/test')
def test():
status_code = 200
return jsonify({'name': 'Nabin Khadka'}, status_code) # Notice second element of the return tuple(return)
This way you can control what status code to return to the client (typically to web browser.)

Flask test_client removes query string parameters

I am using Flask to create a couple of very simple services. From outside testing (using HTTPie) parameters through querystring are getting to the service.
But if I am using something like.
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service", **data))
I can see the correct URI being created:
http://localhost:5000/api1.0/my-service?param1=somevalue1&param2=somevalue2
when I breakpoint into the service:
request.args
is actually empty.
self.client is created by calling app.test_client() on my configured Flask application.
Anyone has any idea why anything after ? is being thrown away or how to work around it while still using test_client?

I've just found out a workaround.
Make
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service", **data))
into this:
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service"), query_string = data)
This works but seems a bit unintuitive, and debugging there is a place where the provided query string in the URI is thrown away ....
But anyway this works for the moment.

I know this is an old post, but I ran into this too. There's an open issue about this in the flask github repository. It appears this is intended behavior. From a response in the issue thread:
mitsuhiko commented on Jul 24, 2013
That's currently intended behavior. The first parameter to the test client should be a relative url. If it's not, then the parameters are removed as it's treated as if it was url joined with the second. This works:
>>> from flask import Flask, request
>>> app = Flask(__name__)
>>> app.testing = True
>>> #app.route('/')
... def index():
... return request.url
...
>>> c = app.test_client()
>>> c.get('/?foo=bar').data
'http://localhost/?foo=bar'
One way to convert your absolute url into a relative url and keep the query string is to use urlparse:
from urlparse import urlparse
absolute_url = "http://someurl.com/path/to/endpoint?q=test"
parsed = urlparse(absolute_url)
path = parsed[2] or "/"
query = parsed[4]
relative_url = "{}?{}".format(path, query)
print relative_url

If you are trying any other HTTP Method other than GET
response = self.client.patch(url_for("api.my-service"), query_string=data,
data="{}")
data="{}" or data=json.dumps({}) should be there, even if there is no content in the body. Otherwise, it results in BAD Request.

For me the solution was to use the client within with statements:
with app.app_context():
with app.test_request_context():
with app.test_client() as client:
client.get(...)
instead of
client = app.test_client()
client.get(...)
I put the creation of the test client in a fixture, so that it is "automatically" created for each test method:
from my_back_end import MyBackEnd
sut = None
app = None
client = None
#pytest.fixture(autouse=True)
def before_each():
global sut, app, client
sut = MyBackEnd()
app = sut.create_application('frontEndPathMock')
with app.app_context():
with app.test_request_context():
with app.test_client() as client:
yield