I need to POST data to an API endpoint using the python requests library with application/json content-type but am getting errors due to the API key having a space in it.
API Key format:
Authorization: Token d6bf96a81a58bf6e99ad1f819b244242797c0c07
Successful curl request:
curl -k -H "Authorization: Token d6bf96a81a58bf6e99ad1f819b244242797c0c07" "https://myurlhere.com/api/status/" --data "status=Good&value=foobar"
Failed Python:
import requests
url="https://myurlhere.com/api/status/"
headers={"Authorization": "Token d6bf96a81a58bf6e99ad1f819b244242797c0c07","content-type":"application/json}
data="status=Good&value=foobar"
requests.post(url, headers=headers, data=data)
The python request returns a 403, additional debugging shows "Authentication details were not provided."
It appears as if the space is causing issues due to being a json object? Is there any clean way to work around this? I must use the application/json type as I use the same program with a lot more code for other api requests which require the application/json type.
Related
I have the following FastAPI application:
from pydantic import BaseModel as Schema
from fastapi import FastAPI
api = FastAPI()
class User(Schema):
firstname: str
lastname: str
age: int | None = None
#api.post('/user')
def user_selection(user: User):
return {'data': f'{user.firstname} {user.lastname} age: {user.age}'}
The main file is called file.py, so I run the uvicorn server like this:
uvicorn file:api --reload
Through another console, I send this request:
curl -X 'POST' -i 'http://127.0.0.1:8000/user' -d '{firstname":"mike", "lastname":"azer"}'
but, I get this error:
HTTP/1.1 422 Unprocessable Entity
date: Sun, 05 Feb 2023 16:01:14 GMT
server: uvicorn
content-length: 88
content-type: application/json
{"detail":[{"loc":["body"],"msg":"value is not a valid dict","type":"type_error.dict"}]}
Why is that?
If, however, I set the Content-Type header to application/json in my request:
curl -X 'POST' 'http://127.0.0.1:8000/user' -H 'Content-Type: application/json' -d '{
"firstname": "aaa",
"lastname": "zzz"
}'
it works just fine.
Why do I need the header? When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?
Q: "Why Content-Type header is required in a JSON POST request?"
A: According to curl's documentation:
POSTing with curl's -d option will make it include a default header
that looks like Content-Type: application/x-www-form-urlencoded.
That is what your typical browser will use for a plain POST.
Many receivers of POST data do not care about or check the
Content-Type header.
If that header is not good enough for you, you should, of course,
replace that and instead provide the correct one. Such as if you POST
JSON to a server and want to more accurately tell the server about
what the content is:
curl -d '{json}' -H 'Content-Type: application/json' https://example.com
Hence, the reason for 'Content-Type: application/json' header being required when sending a POST request containing JSON data is simply because curl by default uses application/x-www-form-urlencoded Content-Type to encode the data that forms the body of the request—this is what a browser typically uses when submitting an HTML form.
If, however, files are also included in the request, then multipart/form-data content type is automatically used by curl—the same applies to using Python requests, as shown in this answer (as for HTML forms, you need to manually specify the enctype, as demonstrated here)—as per curl's documentation:
Posting binary
When reading data to post from a file, -d will strip out carriage return and newlines. Use --data-binary if you
want curl to read and use the given file in binary exactly as given:
curl --data-binary #filename http://example.com/
I would also suggest you to take a look at related answers here, here and here, as well as use the interactive Swagger UI autodocs at /docs for testing the API, which would also automatically generate the curl command for testing an API endpoint.
Q: "When I do a GET request, I don't have to add a header and it works. What's the difference with the POST request?"
A: When issuing a GET request, there is no body included, and hence, no body contents to encode and no need to tell the server what kind of data are being send. In GET requests all parameters must appear in the URL (i.e., the query string) or in a header. While the HTTP standard doesn't define a limit for how long URLs or headers can be, there is usually a length limit dependent on both the server and the client (usually between 2KB and 8KB). In a POST request, however, the limit is much higher and more dependent on the server than the client.
I have written API calls according to Grafana's documentation (https://grafana.com/docs/grafana/latest/developers/http_api/dashboard/#gets-the-home-dashboard) but I always end up with a response of 401. For example, just to get a basic GET response for testing with the path /api/dashboards/home, I have done the following:
On the Grafana settings, I have added an api key and set it to admin. I have tried calling the api using curl, Insomnia (like Postman) and via Python. (Replaced api key and grafana url values)
curl:
curl -k -H "Authorization: Bearer <apikey>" https://<grafanaurl>/api/dashboards/home
response:
curl: (6) Could not resolve host: GET
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
python:
import json
import requests
server= "https:<grafanaurl>"
url = server + "/api/dashboards/home"
headers = {
"Authorization": "Bearer <apikey>",
"Accept": "application/json",
"Content-Type": "application/json"
}
r = requests.request("GET", url, headers=headers, verify=False)
print(r)
# print(r.json())
response:
<Response [401]>
Insomnia gives the same error message as curl.
Am I doing something wrong? My organization uses LDAP authentication to automatically log us in to Grafana - could this be a reason why this doesn't work? If so, how would I work with this? I want to be able to call Grafana apis from within an application.
I am trying to get the Indeed vacanties of my company via API with python. I am following https://developer.indeed.com/docs/authorization/3-legged-oauth and https://mathiashaentjens.medium.com/how-do-you-extract-data-using-the-indeed-api-and-build-your-own-indeed-campaign-reporting-8127252ef073.
I create Indeed API keys and recevive the Authorization Code. But i couldnt get Access Token. I send the same POST as documents via curl and python requests but i got this error;
{'error_description': 'Your request might include sensitive information passed in the URL query string. Parameters must be passed in the HTTP request body using the application/x-www-form-urlencoded format (See https://datatracker.ietf.org/doc/html/rfc6749#section-4.1.3). For increased security, we recommend that you periodically rotate your application secret at https://secure.indeed.com/account/apikeys.', 'error': 'invalid_request'}
My python code is like;
headers = {'content-type': 'application/x-www-form-urlencoded','accept':'application/json'}
payload = {'code':'XXXX', 'client_id':'XXXX', 'client_secret':'XXXX', 'redirect_uri': 'http://localhost', 'grant_type':'authorization_code'}
response = requests.post('https://apis.indeed.com/oauth/v2/tokens', params=urllib.parse.urlencode(payload), headers=headers)
response.json()
and via command line;
curl -X POST -H "Content-Length: 0" -H "Content-Type: application/x-www-form-urlencoded" -H "Accept: application/json" "https://apis.indeed.com/oauth/v2/tokens?code=XXXX&client_id=XXXX&client_secret=XXXX&redirect_uri=http://localhost&grant_type=authorization_code"
Is there anyone familiar with this error?
thanks for reading - my questions are simple:
What am I doing wrong?
What am I missing?
Do I need to additional access to make use of the report engine?
I cannot seem to obtain balance or transaction data from my pp account using a simple python client despite what appears to be correct use of documented APIs.
I have:
a business account
dev portal access
valid creds for a Sandbox and Live API use
My py client wraps a few API calls and successfully completes the following:
obtain API token using dev portal creds
use API token to make valid HTTP requests to the balances and transactions API resources
API responses indicate 200 OK
However, except for the OAuth token request, all of the responses all say Empty XML data
Pseudo-sample code
# -- pp_auth_header is a global, contains Auth Bearer token
def get_balances():
print("get_balances...")
headers = pp_auth_header
endpoint = "/v1/reporting/balances"
query = "?currency_code=USD&as_of_time=2021-02-22T00:00:00-0700"
r = requests.get(pp_report_eng_url + endpoint + query, headers=pp_auth_header)
print("get_balances - status: {}".format(r.status_code))
if r.status_code == 200:
print("get_balances - r.text: {}".format(r.text))
# -- output of the above is
get_balances...
get_balances - url: https://api-m.sandbox.paypal.com/v1/reporting/balances?currency_code=USD&as_of_time=2021-02-22T00:00:00-0700
get_balances - status: 200
get_balances - r.text: Empty XML data
Generate an acess_token and then test from the command line with curl, as documented:
curl -v -X GET
https://api-m.sandbox.paypal.com/v1/reporting/balances?currency_code=USD&as_of_time=2016-10-15T06:07:00-0700
\
-H "Content-Type: application/json" \
-H "Authorization: Bearer Access-Token"
Ensure you pass the headers described there, including "Content-Type: application/json".
PayPal REST APIs do not return XML.
If this doesn't work, post the output of that curl -v command as well as the scopes returned by the oauth2 token request. Update your question with those details.
I have use curl in linux to post data to asana and its work fine.
curl -H "Authorization: Bearer <mytoken>" https://app.asana.com/api/1.0/tasks/101/stories -d "text=hello world"
but if I use requests library for python the result is 400
response = requests.post("https://app.asana.com/api/1.0/tasks/101/stories", auth=(self.asana.token_uri, ""), params = "text=repo_name")
You need to add the same "Authorization: Bearer" token header to your request in python. Similar to this answer:
https://stackoverflow.com/a/29931730/6080374
The auth argument to requests methods produces a Authorization: Basic header, not a Authorization: Bearer header.
Set a headers dictionary with a manually created Authorization header instead. POST data (the -d command line switch for curl) should be passed as a dictionary to the data argument:
response = requests.post(
"https://app.asana.com/api/1.0/tasks/101/stories",
headers={'Authorization': 'Bearer {}'.format(self.asana.token_uri),
data={'text': 'repo_name'})