Develop Reference

JSON parse error when using python requests for an API

If I run a curl statement in bash I get the result I want with no problem.
curl --header 'Content-Type: application/json' --header 'Authorization: Token ABC123' --data '{"jsonrpc":"2.0","method":"enterprise/getEnterpriseEdges","params":{"with":["certificates","configuration","links","recentLinks","site","licenses","analyticsMode","selfHealing"],"edgeIds":[4723]},"id":34123423}' --request POST 'https://my-portal.velocloud.net/portal/'
When putting it into a python script, I keep getting this result instead of the massive JSON array of data.
{'jsonrpc': '2.0', 'error': {'code': -32603, 'message': 'JSON parse error'}, 'id': 1}
This is a copy of my Python script
import requests
payload_headers = {
"Content-Type": "application/json",
"Authorization": "Token ABC123"
}
payload_data = {
"jsonrpc": "2.0",
"method": "enterprise/getEnterpriseEdges",
"params": {
"with": ["certificates", "configuration", "links", "recentLinks", "site", "licenses", "analyticsMode", "selfHealing"],
"edgeIds": [4723]
},
"id": 34
}
response = requests.post("https://my-portal.velocloud.net/portal/", headers=payload_headers, data=payload_data, verify= False)
json_info = response.json()
print(json_info)
I was hoping to get a JSON output of the data like I get with the curl in bash, and eventually incorporate a loop to run through a list to get multiple JSONs to build out an API.

You need to dump the dict to str:
import json
response = requests.post("https://my-portal.velocloud.net/portal/",
headers=payload_headers,
data=json.dumps(payload_data),
verify=False)
Or just use json parameter:
response = requests.post("https://my-portal.velocloud.net/portal/",
headers=payload_headers,
json=payload_data,
verify=False)

Storing a cURL as a variable in Python

So, my idea at the first point, was to execute a cURL after a given condition in Python.
I tryed to use the os module, using:
os.system(curl -k "http://myhost.com" -d'{"event": { "Error": "Error Found", "Host": "myhostname"}}')
But I had a syntax error.
I know it is hardcoded, and also that there is a module called requests in Python, so..
My doubt is, can I just set this full link as a variable? like
mycurl = curl -k "http://myhost.com" -d'{"event": { "Error": "Error Found", "Host": "myhostname"}}'
then execute:
os.system(mycurl)
If it's not possible, or it's incorrect, then how can I use the module requests in this case? Because I want to set an Authorization Token with this curl, which is:
curl -k "http://myhost.com" -H "Authorization: MYTOKEN" -d'{"event": { "Error": "Error Found", "Host": "myhostname"}}'

Here is how you might do it via requests:
import requests
# ...
if i in str(read_log):
url = "http://myhost.com"
headers = {
"Authorization": "MYTOKEN"
}
payload = {
"event": {
"Error": "Error Found",
"Host": "myhostname"
}
}
response = requests.post(url, headers=headers, json=payload)
# Do something with response...

Like Barmar above said, the argument to os.system needs to be a string. You can escape quotes " in strings with a backslash \. For example:
os.system("curl -k \"http://myhost.com\" -d'{\"event\": { \"Error\": \"Error Found\", \"Host\": \"myhostname\"}}'")

Just to make the os.system approach a bit clearer, you can combine f-strings and docstrings:
URL = "http://myhost.com"
TOKEN = MYTOKEN
MY_HOSTNAME = myhostname
curl_command = f"""curl -k {URL}
-H "Authorization: {TOKEN}"
-d'{"event": { "Error": "Error Found", "Host": {MY_HOSTNAME}}}'
"""
os.system(curl_command)

Error 500 when requesting Datadog logs by Python Requests

I have the following curl command which works fine:
curl -X POST -H 'content-type: application/json' -H "DD-API-KEY: ${api_key}" -H "DD-APPLICATION-KEY: ${app_key}" \
-d '{
"query": "service:my_service",
"time": {
"from": "2019-11-28T00:00:00Z",
"to": "2019-11-28T16:00:00Z"
},
"sort": "asc",
"limit": 1000
}' "https://api.datadoghq.com/api/v1/logs-queries/list" -o output3.json5
Then I convert this requests to Python Requests, and the curl method works but Python returns a 500 error without any details.
import requests
def main():
headers = {
'content-type': 'application/json',
'DD-API-KEY': 'AAA',
'DD-APPLICATION-KEY': 'XXX',
}
data = {
"query": "service:my_service",
"time": {
"from": "now - 1h",
"to": "now"
},
"sort": "asc",
"limit": 50
}
response=requests.post("https://api.datadoghq.com/api/v1/logs-queries/list",headers=headers, data=data)
I tried it outside my Docker guessing that maybe connection was the key, but it doesn't work either.

Point both of those at a service like httpbin to see how they differ.
Requests' data option for POST requests generates form-encoded data by default, while curl passes the JSON string through directly. You can manually encode your payload as a JSON string:
import json
response = requests.post(..., data=json.dumps(data))
# ^^^^^^^^^^
or if you have Requests version 2.4.2 or later you can use the json parameter to have your dict converted to JSON automatically:
response = requests.post(..., json=data)
# ^^^^

Looking to send Python API request using cURL

I am looking to send a POST request using Python to the OANDA API to open an order. They do not have a Python wrapper and use cURL, so I have had to try and convert from cURL to Python. I have done so using https://curl.trillworks.com/ -- but converting this next one does not work.
You can view the OANDA API documentation here, under the first Green POST tab - http://developer.oanda.com/rest-live-v20/order-ep/
Here is what I am working with. This first block specifies the order details. In this case, a market order in the EUR_USD instrument with a quantity of 100 units and a time in force equalling "fill or kill" :
body=$(cat << EOF
{
"order": {
"units": "100",
"instrument": "EUR_USD",
"timeInForce": "FOK",
"type": "MARKET",
"positionFill": "DEFAULT"
}
}
EOF
)
curl \
-X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer SECRET TOKEN" \
-d "$body" \
"https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders"
Converted into Python:
import requests
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer SECRET TOKEN',
}
data = '$body'
response = requests.post('https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders', headers=headers, data=data)
As you can see, I believe there is a formatting error somewhere in the "body=$" portion, but I am not entirely sure. I simply get a 400 error, "invalid values."

If you're sending data in JSON format, you should pass them into json argument instead of data (explanation, method).
import requests
headers = {
# 'Content-Type': 'application/json', # will be set automatically
'Authorization': 'Bearer SECRET TOKEN',
}
body = {
"order": {
"units": "100",
"instrument": "EUR_USD",
"timeInForce": "FOK",
"type": "MARKET",
"positionFill": "DEFAULT"
}
}
response = requests.post('https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders',
headers=headers, json=body)

Using python requests to POST data with file

I have the following working curl
curl -v -i -H "Content-Type:multipart/form-data" -H "X-Authorization: 12345" -F "file0=#/path/to/image.jpg" -d 'jsonData={ "data": { "field1": 1, "field2": 2 } }' -X POST http://example.com/url/path
I am trying to make the exact same request in python and came up with the following
headers = {
'Content-Type': 'multipart/form-data',
'X-Authorization': '12345',
}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post('http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": {
"data": {
"field1": 1,
"field2": 2,
}
}
})
The problem is that I'm getting a different response for curl and for python. Are these requests not equivalent or should I be looking somewhere else?

You can't just pass in a Python dictionary without encoding it to JSON. Your curl post has JSON-encoded data for the jsonData form field, so your Python code needs to provide the same:
import json
headers = {'X-Authorization': '12345'}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post(
'http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": json.dumps({
"data": {
"field1": 1,
"field2": 2,
}
})
})
By passing in a dictionary to data, each value that is an iterable, is seen as a sequence of separate values (so {'data': ['foo', 'bar']} is translated to data=foo&data=bar). You passed in a dictionary for the jsonData key, which is an iterable and thus treated a a sequence of values. Ultimately you only posted jsonData=data, nothing else.
By using a (JSON encoded) single value, you tell requests to pass that value directly, as the value for the jsonData key.
Using the files keyword argument triggers the right Content-Type header as well, it doesn't need setting manually.