I have some RESTful API calls in web2py which take in JSON file. I make the calls through the command line using curl like:
curl -H "Content-Type: application/json" --data #mydata.json https://mywebsite/doWork
In my web2py controller I have it setup to read the POST request. When I examine request.vars by returning it, it contains the full JSON data, but request.vars is of type: class 'gluon.storage.Storage'.
I need the data in string format in the controller. However, if I do str(request.vars), web2py appends a Storage tag in front of my data.
#request.restful()
def doWork():
def POST(*args, **vars):
return(request.vars)
return locals()
Assuming the mydata.json file is something simple like:
{
"key": "value"
}
The request to POST will return {"key": "value"} as a gluon.storage.Storage object.
If I change the line return(request.vars) to return(str(request.vars)) the output is now: < Storage {u'key': u'value'}>
How can I convert this request.vars to a string without it appending the Storage tag to the front, and changing the double quotes to single quotes?
If you want to convert vars back to JSON, you can use the Python json library or just use response.json:
return response.json(vars)
Alternatively, you can add the .json extension to the request and simply return vars directly:
return vars
with the following request:
curl -H "Content-Type: application/json" --data #mydata.json https://mywebsite/doWork.json
In that case, web2py will automatically convert the response to JSON.
Related
I have the following FastAPI application:
from pydantic import BaseModel as Schema
from fastapi import FastAPI
api = FastAPI()
class User(Schema):
firstname: str
lastname: str
age: int | None = None
#api.post('/user')
def user_selection(user: User):
return {'data': f'{user.firstname} {user.lastname} age: {user.age}'}
The main file is called file.py, so I run the uvicorn server like this:
uvicorn file:api --reload
Through another console, I send this request:
curl -X 'POST' -i 'http://127.0.0.1:8000/user' -d '{firstname":"mike", "lastname":"azer"}'
but, I get this error:
HTTP/1.1 422 Unprocessable Entity
date: Sun, 05 Feb 2023 16:01:14 GMT
server: uvicorn
content-length: 88
content-type: application/json
{"detail":[{"loc":["body"],"msg":"value is not a valid dict","type":"type_error.dict"}]}
Why is that?
If, however, I set the Content-Type header to application/json in my request:
curl -X 'POST' 'http://127.0.0.1:8000/user' -H 'Content-Type: application/json' -d '{
"firstname": "aaa",
"lastname": "zzz"
}'
it works just fine.
Why do I need the header? When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?
Q: "Why Content-Type header is required in a JSON POST request?"
A: According to curl's documentation:
POSTing with curl's -d option will make it include a default header
that looks like Content-Type: application/x-www-form-urlencoded.
That is what your typical browser will use for a plain POST.
Many receivers of POST data do not care about or check the
Content-Type header.
If that header is not good enough for you, you should, of course,
replace that and instead provide the correct one. Such as if you POST
JSON to a server and want to more accurately tell the server about
what the content is:
curl -d '{json}' -H 'Content-Type: application/json' https://example.com
Hence, the reason for 'Content-Type: application/json' header being required when sending a POST request containing JSON data is simply because curl by default uses application/x-www-form-urlencoded Content-Type to encode the data that forms the body of the request—this is what a browser typically uses when submitting an HTML form.
If, however, files are also included in the request, then multipart/form-data content type is automatically used by curl—the same applies to using Python requests, as shown in this answer (as for HTML forms, you need to manually specify the enctype, as demonstrated here)—as per curl's documentation:
Posting binary
When reading data to post from a file, -d will strip out carriage return and newlines. Use --data-binary if you
want curl to read and use the given file in binary exactly as given:
curl --data-binary #filename http://example.com/
I would also suggest you to take a look at related answers here, here and here, as well as use the interactive Swagger UI autodocs at /docs for testing the API, which would also automatically generate the curl command for testing an API endpoint.
Q: "When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?"
A: When issuing a GET request, there is no body included, and hence, no body contents to encode and no need to tell the server what kind of data are being send. In GET requests all parameters must appear in the URL (i.e., the query string) or in a header. While the HTTP standard doesn't define a limit for how long URLs or headers can be, there is usually a length limit dependent on both the server and the client (usually between 2KB and 8KB). In a POST request, however, the limit is much higher and more dependent on the server than the client.
I'm attempting to make a curl request to my python api that is using the AWS package Chalice.
When I try to access the app.current_request.json_body a JSON Parse error is thrown. Cannot figure out why this is happening. My JSON is formatted properly as far as I can tell.
Here is the curl request:
(echo -n '{"data": "test"}') |
curl -H "Content-Type: application/json" -d #- $URL
Here is the python Chalice code:
app = Chalice(app_name='predictor')
#app.route('/', methods=['POST'], content_types=['application/json'])
def index():
try:
body = app.current_request.json_body
except Exception as e:
return {'error': str(e)}
When I invoke the route using the above curl request I get the following error:
{"error": "BadRequestError: Error Parsing JSON"}
Note: When I remove the .json_body from the app.current_request. I no longer get the error.
Any thoughts?
The documentation indeed indicates that the problem is Content-Type:
The default behavior of a view function supports a request body of application/json. When a request is made with a Content-Type of application/json, the app.current_request.json_body attribute is automatically set for you. This value is the parsed JSON body.
You can also configure a view function to support other content types. You can do this by specifying the content_types parameter value to your app.route function. This parameter is a list of acceptable content types.
It suggests that changing the Content-Type might make json_body work, but I didn't manage to have any success with it.
However using app.current_request.raw_body.decode() instead of app.current_request.json_body solves the problem.
How to send both file and data in a single curl POST?
Tried something like using -F and -d option together, but it didn't help:
curl -X POST -F 'file=/Users/<uid>/model.tar.gz' http://<uri>/api/v1/modelfiles/ -H 'AUTH-TOKEN:<token>' -d '{"model_id": "<uuid>",
"filename": "model.tar.gz",
"framework": "Tensorflow",
"framework_version": "1.8.0",
"meta": {}
}'
Short version: you can't. That's not how HTTP works. You get to send one blob of data in one POST request, but you are trying to send two (the JSON string and the file).
Longer version: this depends on what your service expects. Is the data from the file supposed to be a part of the JSON? Then you need to preprocess the JSON and put the file data inside, so that you're sending -d '{"file": "<your file data here>", "filename": ... }'
Is the file supposed to be a form field called "file", and the JSON data is contents of a field called "json"? Then you can send both using -F file=#/users/uid/model.tar.gz -F 'json={...}'. Curl will take care of inlining them into the blob, same as if you had a browser form with two fields.
(note also the # sign in front of the filename; you need that, otherwise you are sending the string "/users/uid/model.tar.gz")
Is it something else? Maybe you're supposed to send the data first and the JSON second?
I'm developing a Flask API. I want to create an API route that with accept JSON parameters and based on that json to do a search in database.
My code looks like this:
#mod_api.route('/test', methods=['POST'])
def test():
query_params = json.loads(request.data)
json_resp = mongo.db.mydb.find(query_params)
return Response(response=json_util.dumps(json_resp), status=200, mimetype='application/json')
Now when I run the api i go to my route: This example looks like this:
http://0.0.0.0:5002/api/test
I don't know exactly how to send a json parameter. If i do like this:
http://0.0.0.0:5002/api/test?{'var1':'123', 'var2':'456'}
I get an error ValueError("No JSON object could be decoded")
How to send this json parameter?
You likely aren't supplying JSON data. With your browser at http://0.0.0.0:5002, use XHR in the browser console to test out your API.
var xmlhttp = new XMLHttpRequest(); // new HttpRequest instance
xmlhttp.open("POST", "/api/test");
xmlhttp.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
xmlhttp.send(JSON.stringify({'var1':'123', 'var2':'456'}));
You can see the request/response in the Network tab, and the Flask process will show the request happening as well.
First, I want to point out that usually we use request.get_json() to get the json data, request.data contains the incoming request data that flask can't handle.
Test your app with curl should be easy, send the json data this way:
$ curl -H "Content-Type: application/json" -X POST -d '{"var1":"123", "var2":"456"}' http://localhost:5000/api/test
I'm passing a json in my request data body as follow:-
curl --data "data={"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}" POSTURL
Upon json loads, it throws an error:-
data = request.form
print json.loads(str(data.get('data'))) # throws an error
Upon printing data.get('data'), I get {applist:{ads:{ad:[{adsid:1,view_points:25,view_type:full,comment:Success}]}}} which is incorrect json, since double quotes("") are missing. How do I json load it?
The issue is with your original post request via curl. You are surrounding the post data with double quotes, but also using double quotes in the post body. Easiest fix is to surround the post body with single quotes:
curl --data 'data={"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}' POSTURL
First of all, if you are using Flask, you should use request.json to get the already parsed json. To do so, you need to set the content type in your curl request:
-H "content-type: application/json"
Second, your data is not valid json. Use this one instead:
--data='{"applist":{"ads":{"ad":[{"adsid":1,"view_points":25,"view_type":"full","comment":"Success"}]}}}'