I'm attempting to make a curl request to my python api that is using the AWS package Chalice.
When I try to access the app.current_request.json_body a JSON Parse error is thrown. Cannot figure out why this is happening. My JSON is formatted properly as far as I can tell.
Here is the curl request:
(echo -n '{"data": "test"}') |
curl -H "Content-Type: application/json" -d #- $URL
Here is the python Chalice code:
app = Chalice(app_name='predictor')
#app.route('/', methods=['POST'], content_types=['application/json'])
def index():
try:
body = app.current_request.json_body
except Exception as e:
return {'error': str(e)}
When I invoke the route using the above curl request I get the following error:
{"error": "BadRequestError: Error Parsing JSON"}
Note: When I remove the .json_body from the app.current_request. I no longer get the error.
Any thoughts?
The documentation indeed indicates that the problem is Content-Type:
The default behavior of a view function supports a request body of application/json. When a request is made with a Content-Type of application/json, the app.current_request.json_body attribute is automatically set for you. This value is the parsed JSON body.
You can also configure a view function to support other content types. You can do this by specifying the content_types parameter value to your app.route function. This parameter is a list of acceptable content types.
It suggests that changing the Content-Type might make json_body work, but I didn't manage to have any success with it.
However using app.current_request.raw_body.decode() instead of app.current_request.json_body solves the problem.
Related
I have the following FastAPI application:
from pydantic import BaseModel as Schema
from fastapi import FastAPI
api = FastAPI()
class User(Schema):
firstname: str
lastname: str
age: int | None = None
#api.post('/user')
def user_selection(user: User):
return {'data': f'{user.firstname} {user.lastname} age: {user.age}'}
The main file is called file.py, so I run the uvicorn server like this:
uvicorn file:api --reload
Through another console, I send this request:
curl -X 'POST' -i 'http://127.0.0.1:8000/user' -d '{firstname":"mike", "lastname":"azer"}'
but, I get this error:
HTTP/1.1 422 Unprocessable Entity
date: Sun, 05 Feb 2023 16:01:14 GMT
server: uvicorn
content-length: 88
content-type: application/json
{"detail":[{"loc":["body"],"msg":"value is not a valid dict","type":"type_error.dict"}]}
Why is that?
If, however, I set the Content-Type header to application/json in my request:
curl -X 'POST' 'http://127.0.0.1:8000/user' -H 'Content-Type: application/json' -d '{
"firstname": "aaa",
"lastname": "zzz"
}'
it works just fine.
Why do I need the header? When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?
Q: "Why Content-Type header is required in a JSON POST request?"
A: According to curl's documentation:
POSTing with curl's -d option will make it include a default header
that looks like Content-Type: application/x-www-form-urlencoded.
That is what your typical browser will use for a plain POST.
Many receivers of POST data do not care about or check the
Content-Type header.
If that header is not good enough for you, you should, of course,
replace that and instead provide the correct one. Such as if you POST
JSON to a server and want to more accurately tell the server about
what the content is:
curl -d '{json}' -H 'Content-Type: application/json' https://example.com
Hence, the reason for 'Content-Type: application/json' header being required when sending a POST request containing JSON data is simply because curl by default uses application/x-www-form-urlencoded Content-Type to encode the data that forms the body of the request—this is what a browser typically uses when submitting an HTML form.
If, however, files are also included in the request, then multipart/form-data content type is automatically used by curl—the same applies to using Python requests, as shown in this answer (as for HTML forms, you need to manually specify the enctype, as demonstrated here)—as per curl's documentation:
Posting binary
When reading data to post from a file, -d will strip out carriage return and newlines. Use --data-binary if you
want curl to read and use the given file in binary exactly as given:
curl --data-binary #filename http://example.com/
I would also suggest you to take a look at related answers here, here and here, as well as use the interactive Swagger UI autodocs at /docs for testing the API, which would also automatically generate the curl command for testing an API endpoint.
Q: "When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?"
A: When issuing a GET request, there is no body included, and hence, no body contents to encode and no need to tell the server what kind of data are being send. In GET requests all parameters must appear in the URL (i.e., the query string) or in a header. While the HTTP standard doesn't define a limit for how long URLs or headers can be, there is usually a length limit dependent on both the server and the client (usually between 2KB and 8KB). In a POST request, however, the limit is much higher and more dependent on the server than the client.
What is the best way to convert the below curl post into python request using the requests module:
curl -X POST https://api.google.com/gmail --data-urlencode json='{"user": [{"message":"abc123", "subject":"helloworld"}]}'
I tried using python requests as below, but it didn't work:
payload = {"user": [{"message":"abc123", "subject":"helloworld"}]}
url = https://api.google.com/gmail
requests.post(url, data=json.dumps(payload),auth=(user, password))
Can anybody help.
As the comment mentioned, you should put your url variable string in quotes "" first.
Otherwise, your question is not clear. What errors are being thrown and/or behavior is happening?
New cURL method in Python
I'm developing a Flask API. I want to create an API route that with accept JSON parameters and based on that json to do a search in database.
My code looks like this:
#mod_api.route('/test', methods=['POST'])
def test():
query_params = json.loads(request.data)
json_resp = mongo.db.mydb.find(query_params)
return Response(response=json_util.dumps(json_resp), status=200, mimetype='application/json')
Now when I run the api i go to my route: This example looks like this:
http://0.0.0.0:5002/api/test
I don't know exactly how to send a json parameter. If i do like this:
http://0.0.0.0:5002/api/test?{'var1':'123', 'var2':'456'}
I get an error ValueError("No JSON object could be decoded")
How to send this json parameter?
You likely aren't supplying JSON data. With your browser at http://0.0.0.0:5002, use XHR in the browser console to test out your API.
var xmlhttp = new XMLHttpRequest(); // new HttpRequest instance
xmlhttp.open("POST", "/api/test");
xmlhttp.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
xmlhttp.send(JSON.stringify({'var1':'123', 'var2':'456'}));
You can see the request/response in the Network tab, and the Flask process will show the request happening as well.
First, I want to point out that usually we use request.get_json() to get the json data, request.data contains the incoming request data that flask can't handle.
Test your app with curl should be easy, send the json data this way:
$ curl -H "Content-Type: application/json" -X POST -d '{"var1":"123", "var2":"456"}' http://localhost:5000/api/test
I am interfacing with an API using requests and requests_oauthlib.
I successfully authenticate and access all the GET methods of the API, but get error 500 with POST methods. For example:
r = oauth.post("https://api.timelyapp.com/1.0/1/clients",
data={"client":{"name":"newclient", "color":"c697c0" }},
allow_redirects=False, headers={"Content-Type": "application/json"})
The issue is that I tested the same exact call with curl and it works correctly, here the curl code:
curl -v -X POST -H "Content-Type: application/json" -H "Authorization: Bearer XXXXXXXXXXXX" --data '{"client": { "name": "newclient", "color":"c697c0" }}' "https://api.timelyapp.com/1.0/1/clients"
how can I dig deeper in requests to compare its call with curl?
UPDATE:
Also, noticed that if I do not specify content type:
r = oauth.post("https://api.timelyapp.com/1.0/1/clients",
data={"client":{"name":"newclient", "color":"c697c0" }},
allow_redirects=True)
I get instead a 302 with redirection to the site homepage, where I fetch the content of the page. In any case the new client is not added.
You might want to try this instead:
data=json.dumps(payload)
From python-requests doc:
There are many times that you want to send data that is not
form-encoded. If you pass in a string instead of a dict, that data
will be posted directly.
I'm passing a json in my request data body as follow:-
curl --data "data={"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}" POSTURL
Upon json loads, it throws an error:-
data = request.form
print json.loads(str(data.get('data'))) # throws an error
Upon printing data.get('data'), I get {applist:{ads:{ad:[{adsid:1,view_points:25,view_type:full,comment:Success}]}}} which is incorrect json, since double quotes("") are missing. How do I json load it?
The issue is with your original post request via curl. You are surrounding the post data with double quotes, but also using double quotes in the post body. Easiest fix is to surround the post body with single quotes:
curl --data 'data={"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}' POSTURL
First of all, if you are using Flask, you should use request.json to get the already parsed json. To do so, you need to set the content type in your curl request:
-H "content-type: application/json"
Second, your data is not valid json. Use this one instead:
--data='{"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}'