I am attempting to create a "authorizer dependecy" for usine in FastAPI that accepts a scope parameter, which then checks a given auth token and if this user has access to a resource specified as a path parameter.
The problem is the way (and in which order, if any) FastAPI seems to evaluate these depedencies.
For this case, I would find it most ideal if the path parameter in the path operation function (resource_id) would be evaluated first and raise an exception and immediately "abort" if the user attempted to access an invalid resource path.
I guess it's not surprising that the path operation decorator dependency is evaluated first, however, so I've also repeated the same path parameter validation in this one (which is also needed for permission checking against the resource).
The thing I am more surprised about, is that both of them are evaluated and included in a list of validation errors. Wouldn't it give more sense if it raised a single exception on the first dependency which not fulfilled (not continue evaluating all of them)?
Since the authorize dependecy not always require a specified resource_id, I've also attempted to make this an optional, but since this dependecy is evaluated first, it still attempts to authroize the user agianst a (possible) invalid resource_id.
Is there any way to instruct fastapi to fail on first "failed" dependecy, control it's order, or any better designs to solve this problem?
from fastapi import FastAPI, Depends, status, Path
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
app = FastAPI()
http_bearer = HTTPBearer(scheme_name="Token")
def authorize(scope: str):
def _check_permission(
resource_id: str, # = Path(None, regex=r"^[0-9a-f-]+$"),
authorization: HTTPAuthorizationCredentials = Depends(http_bearer),
):
# check credentials and permission `scope` on `resource_id`
return True
return _check_permission
#app.get(
"/{resource_id}",
dependencies=[Depends(authorize("read"))],
status_code=status.HTTP_200_OK,
)
def get_resource(
resource_id: str = Path(..., regex=r"^[0-9a-f-]+$"),
):
return f"access granted: {resource_id}"
{
"detail": [
{
"loc": [
"path",
"resource_id"
],
"msg": "string does not match regex \"^[0-9a-f-]+$\"",
"type": "value_error.str.regex",
"ctx": {
"pattern": "^[0-9a-f-]+$"
}
},
{
"loc": [
"path",
"resource_id"
],
"msg": "string does not match regex \"^[0-9a-f-]+$\"",
"type": "value_error.str.regex",
"ctx": {
"pattern": "^[0-9a-f-]+$"
}
}
]
}
Related
I have a GraphQL system associated with a Django app that seems to be working fine, except that it's completely ignoring errors in mutations. That is, if the mutation executes with no errors, everything behaves as expected. But if I raise an exception on the first line of the mutation, I don't get any indication of the error -- nothing in the app logs, and the graphQL response is just a json will null contents, e.g.:
{
"data": {
"exampleMutation": {
"mutationResponseSchema": null
}
}
}
Even if I wrap the django pieces (e.g. trying to get a filterset) in a try: except:, the behavior is the same as if I raise the exception. IOW, an exception being thrown (even if it's handled) seems to trigger an empty response being sent.
I am at a total loss for where these exceptions are going -- it seems that the behavior on encountering an exception is to ignore it and just return a null JSON.
Furthermore, I have an app with the same basic layout but built off an older image of Python (3.8 vs. this one at 3.11, so the django/graphene versions and related dependencies are newer). The old app handled exceptions as usual and would return messages via the endpoints when raised using Django/Graphene classes with the same structure as the one I'm having the problem with.
I don't know if something changed in graphene's error handling, but I can't seem to find a clear answer to that.
For example, if I write the following mutation:
class ExampleMutation(graphene.Mutation):
class Arguments:
fail_here = graphene.String()
some_schema = graphene.Field(SomeSchema)
#authorized
def mutate(root, info, user, **kwargs):
# could also just raise Exception('automatic exception') here and get same behavior.
if kwargs.get('fail_here') == 'yes':
raise Exception('text') # Doesn't seem to matter what exception is raised
else:
django_model = SomeSchemaModel.objects.first()
return ExampleMutation(some_schema=django_model)
The response to e.g.
mutation exampleMutation($failHere: String){
exampleMutation(
failHere: $failHere,
) {
someSchema
{
field1
field2
}
}
}
is valid and behaves as expected if the mutation is called with e.g. {"failHere": "No"}. Ergo, the structure of the graphQL/Django stuff is not the problem.
The problem is that when the endpoint is called with {"failHere": "yes"} (or if I just raise an error on the first line of the mutation), the response is:
{
"data": {
"exampleMutation": {
"someSchema": null
}
}
}
The above might be a bug, I posted to github. But in case this comes up for someone else, this is the workaround that got things working:
In django's settings (with appropriate SCHEMA for your app):
GRAPHENE = {
'SCHEMA': 'core.graphql.index.schema',
'MIDDLEWARE': ['graphene_django.debug.middleware.DjangoDebugMiddleware'],
}
I'm not certain of this, but I found from some searching that perhaps with newer versions of graphene-django, the DjangoDebugMiddleware middleware is required to send these exceptions in the JSON response.
In any case, the real problem is that the GraphQLView handler seems to need the middleware sent not as a list (as the settings shown above require), but instead as double list, which can be achieved by overriding the instantiation like this:
from graphene_django.views import GraphQLView, graphene_settings
class GQLView(GraphQLView):
def __init__(self, *args, **kwargs):
kwargs.update({'middleware':[graphene_settings.MIDDLEWARE]}) #note the extra list level
and then in urls.py you'd have something like:
urlpatterns = [
# Graphql
(r'graphql', GQLView.as_view())
]
I am trying to figure out ways of validating query string parameters for an API created using AWS API gateway and backed by a Python Lambda function. API Gateway can validate the presence of the required query string parameters. However, I could not find a way for additional validations such as determining if the length of a certain parameter is within some limit (e.g. config_id should be minimum 7 characters long). Such validations are possible for the request body using the API Gateway request validation. Refer this link. However, for the query string paramaters only required/not required validation is possible as it does not use any json schema for validation.
Hence, to overcome this issue, I decided to try the webargs module in Python for validating the query string parameters. It is generally used for request validations for APIs created using Python frameworks such as flask or django. I am using the core parser (Refer webargs doc) as follows:
from webargs import fields, validate, core, ValidationError
parser = core.Parser()
params = {"config_id": fields.Str(required=True, validate=lambda p: len(p) >= 7)}
def main(event, context: Dict):
try:
# print(event["queryStringParameters"])
input_params = event.get("queryStringParameters")
print("queryStringParameters: ", str(input_params))
if input_params is None:
input_params = {}
parsed_params = parser.parse(params, input_params)
print("parsedParams: ", str(parsed_params))
except ValidationError as e:
return {
"statusCode": 400,
"headers": {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Credentials": True,
"x-amzn-ErrorType": "ValidationError",
},
"body": str(e),
}
This is how the validation is done in the lambda function. However, only the required validation works correctly. When I pass a config_id of length 5 it does not return any error and proceeds further in the lambda function.
What could be going wrong with this? The parser seems to work, however, the validate function doesn't.
Any help is appreciated as I am new to this. Also, is there a better way of doing validations in lambda functions especially for queryStringParameters? It can be handled by the code, but we can have many parameters and many APIs which makes writing code for all such validations a cumbersome task. The webargs module comes in handy.
webargs Library is mostly used for validating HTTP Requests coming via popular Python frameworks like Flask, Django, Bottle etc. The core Parser that you are trying to use should not be used directly as it does not have the methods like load_json, load_query etc implemented (Source code showing the missing implementation here). There are child class implementations of the core parser for each of the frameworks, but using them on API GW does not make sense.
So it's better to use a simpler json validation library like jsonschema. I've modified your code to use jsonschema instead of webargs as follows -
from jsonschema import validate, ValidationError
schema = {
"type" : "object",
"properties" : {
"queryStringParameters" : {
"type" : "object",
"properties": {
"config_id": {
"type": "string",
"minLength": 7,
}
}
},
},
}
def main(event, context):
try:
validate(instance=event, schema=schema)
except ValidationError as e:
return {
"statusCode": 400,
"headers": {
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Credentials": True,
"x-amzn-ErrorType": "ValidationError",
},
"body": e.message,
}
My python script to patch automatically has this one error in which the parameter InstanceIds is invalid. Where do I state the value for the InstanceIds in the script?
import boto3
ssm = boto3.client('ssm', region_name='us-east-1')
response = ssm.start_automation_execution(
Parameters={
'AutomationAssumeRole': [
'parameters'
]
},
DocumentName='document-name',
Mode='Auto',
TargetParameterName='test',
Targets=[
{
'Key': 'InstanceIds',
'Values': [ 'i-1234567890abcd' ]
}
],
MaxErrors='10'
)
This gives me the error message
Invalid target value for key InstanceIds
What am I doing wrong here?
tl;dr: Make sure 'InstanceId' is listed as an input parameter in your automation document, and then try updating the Target.Key value you're using to 'ParameterValues'.
This may depend somewhat on your implementation, but I was running into the same error as you, and my parameters matched yours, except that I was using TargetParameterName='InstanceId' instead of TargetParameterName='test'. I tried several different values for Target.Key and none of them worked, until I tried this and it worked:
Targets=[
{
'Key': "ParameterValues",
'Values': [
"i-012345abcdeff",
"i-012345abcdefg"
]
}
],
As an aside, I think they could probably stand to somewhat improve the error messages for this API for situations like this.
I am exploring building an API to my application, as part of developer tool i can see the payload as below -
-X POST -H "Content-Type:application/json" -d '{ "action": "DeviceManagementRouter", "method": "addMaintWindow", "data": [{"uid": "/zport/dmd/Devices/Server/Microsoft/Windows/10.10.10.10", "durationDays":"1", "durationHours":"00", "durationMinutes":"00", "enabled":"True", "name":"Test", "repeat":"Never", "startDate":"08/15/2018", "startHours":"09", "startMinutes":"50", "startProductionState":"300" } ], "type": "rpc", "tid": 1}
I see below error -
{"uuid": "a74b6e27-c9af-402a-acd0-bd9c4254736e", "action": "DeviceManagementRouter", "result": {"msg": "TypeError: addMaintWindow() got an unexpected keyword argument 'startDate'", "type": "exception", "success": false}, "tid": 1, "type": "rpc", "method": "addMaintWindow"}
Code in below URL:
https://zenossapiclient.readthedocs.io/en/latest/_modules/zenossapi/routers/devicemanagement.html
Assuming this is your real python code, then if you want to pass multiple params in python, you should use either *args or **kwargs (keyworded arguments). For you, it seems kwargs is more appropriate.
def addMaintWindow(self, **kwargs):
"""
adds a new Maintenance Window
"""
_name = kwargs["name"]
# _name = kwargs.pop("name", default_value) to be fail-safe and
# it's more defensive because you popped off the argument
# so it won't be misused if you pass **kwargs to next function.
facade = self._getFacade()
facade.addMaintWindow(**kwargs)
return DirectResponse.succeed(msg="Maintenance Window %s added successfully." % (_name))
Here is a good answer about how to use it. A more general one is here.
Read the general one first if you are not familiar with them.
This should get you pass the error at this stage but you will need to do the same for your facade.addMaintWindow; if it not owned by you, make sure you pass in a correct number of named arguments.
I'm invoking a Python-based AWS Lambda method from API Gateway in non-proxy mode. How should I properly handle exceptions, so that an appropriate HTTP status code is set along with a JSON body using parts of the exception.
As an example, I have the following handler:
def my_handler(event, context):
try:
s3conn.head_object(Bucket='my_bucket', Key='my_filename')
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == "404":
raise ClientException("Key '{}' not found".format(filename))
# or: return "Key '{}' not found".format(filename) ?
class ClientException(Exception):
pass
Should I throw an exception or return a string? Then how should I configure the Integration Response? Obviously I've RTFM but the FM is FU.
tl;dr
Your Lambda handler must throw an exception if you want a non-200 response.
Catch all exceptions in your handler method. Format the caught exception message into JSON and throw as a custom Exception type.
Use Integration Response to regex your custom Exception found in the errorMessage field of the Lambda response.
API Gateway + AWS Lambda Exception Handling
There's a number of things you need know about Lambda, API Gateway and how they work together.
Lambda Exceptions
When an exception is thrown from your handler/function/method, the exception is serialised into a JSON message. From your example code, on a 404 from S3, your code would throw:
{
"stackTrace": [
[
"/var/task/mycode.py",
118,
"my_handler",
"raise ClientException(\"Key '{}' not found \".format(filename))"
]
],
"errorType": "ClientException",
"errorMessage": "Key 'my_filename' not found"
}
API Gateway Integration Response
Overview
"Integration Responses" map responses from Lambda to HTTP codes. They also allow the message body to be altered as they pass through.
By default, a "200" Integration Response is configured for you, which passes all responses from Lambda back to client as is, including serialised JSON exceptions, as an HTTP 200 (OK) response.
For good messages, you may want to use the "200" Integration Response to map the JSON payload to one of your defined models.
Catching exceptions
For exceptions, you'll want to set an appropriate HTTP status code and probably remove the stacktrace to hide the internals of your code.
For each HTTP Status code you wish to return, you'll need to add an "Integration Response" entry. The Integration Response is configured with a regex match (using java.util.regex.Matcher.matches() not .find()) that matches against the errorMessage field. Once a match has been made, you can then configure a Body Mapping Template, to selectively format a suitable exception body.
As the regex only matches against the errorMessage field from the exception, you will need to ensure that your exception contains enough information to allow different Integration Responses to match and set the error accordingly.
(You can not use .* to match all exceptions, as this seems to match all responses, including non-exceptions!)
Exceptions with meaning
To create exceptions with enough details in their message, error-handling-patterns-in-amazon-api-gateway-and-aws-lambda blog recommends that you create an exception handler in your handler to stuff the details of the exception into a JSON string to be used in the exception message.
My prefered approach is to create a new top method as your handler which deals with responding to API Gateway. This method either returns the required payload or throws an exception with a original exception encoded as a JSON string as the exception message.
def my_handler_core(event, context):
try:
s3conn.head_object(Bucket='my_bucket', Key='my_filename')
...
return something
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == "404":
raise ClientException("Key '{}' not found".format(filename))
def my_handler(event=None, context=None):
try:
token = my_handler_core(event, context)
response = {
"response": token
}
# This is the happy path
return response
except Exception as e:
exception_type = e.__class__.__name__
exception_message = str(e)
api_exception_obj = {
"isError": True,
"type": exception_type,
"message": exception_message
}
# Create a JSON string
api_exception_json = json.dumps(api_exception_obj)
raise LambdaException(api_exception_json)
# Simple exception wrappers
class ClientException(Exception):
pass
class LambdaException(Exception):
pass
On exception, Lambda will now return:
{
"stackTrace": [
[
"/var/task/mycode.py",
42,
"my_handler",
"raise LambdaException(api_exception_json)"
]
],
"errorType": "LambdaException",
"errorMessage": "{\"message\": \"Key 'my_filename' not found\", \"type\": \"ClientException\", \"isError\": true}"
}
Mapping exceptions
Now that you have all the details in the errorMessage, you can start to map status codes and create well formed error payloads. API Gateway parses and unescapes the errorMessage field, so the regex used does not need to deal with escaping.
Example
To catch this ClientException as 400 error and map the payload to a clean error model, you can do the following:
Create new Error model:
{
"type": "object",
"title": "MyErrorModel",
"properties": {
"isError": {
"type": "boolean"
},
"message": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"token",
"isError",
"type"
]
}
Edit "Method Response" and map new model to 400
Add new Integration Response
Set code to 400
Set regex to match "ClientException" types with tolerance for whitespace: .*"type"\s*:\s*"ClientException".*
Add a Body Mapping Template for application/json to map the contents of errorMessage to your model:
#set($inputRoot = $input.path('$'))
#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))
{
"isError" : true,
"message" : "$errorMessageObj.message",
"type": "$errorMessageObj.type"
}