I have the following working curl
curl -v -i -H "Content-Type:multipart/form-data" -H "X-Authorization: 12345" -F "file0=#/path/to/image.jpg" -d 'jsonData={ "data": { "field1": 1, "field2": 2 } }' -X POST http://example.com/url/path
I am trying to make the exact same request in python and came up with the following
headers = {
'Content-Type': 'multipart/form-data',
'X-Authorization': '12345',
}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post('http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": {
"data": {
"field1": 1,
"field2": 2,
}
}
})
The problem is that I'm getting a different response for curl and for python. Are these requests not equivalent or should I be looking somewhere else?
You can't just pass in a Python dictionary without encoding it to JSON. Your curl post has JSON-encoded data for the jsonData form field, so your Python code needs to provide the same:
import json
headers = {'X-Authorization': '12345'}
files = {
'file0': ('/path/to/image.jpg',
open('/path/to/image.jpg', 'rb')),
}
file_post = requests.post(
'http://example.com/url/path',
headers=headers,
files=files,
data={
"jsonData": json.dumps({
"data": {
"field1": 1,
"field2": 2,
}
})
})
By passing in a dictionary to data, each value that is an iterable, is seen as a sequence of separate values (so {'data': ['foo', 'bar']} is translated to data=foo&data=bar). You passed in a dictionary for the jsonData key, which is an iterable and thus treated a a sequence of values. Ultimately you only posted jsonData=data, nothing else.
By using a (JSON encoded) single value, you tell requests to pass that value directly, as the value for the jsonData key.
Using the files keyword argument triggers the right Content-Type header as well, it doesn't need setting manually.
Related
If I run a curl statement in bash I get the result I want with no problem.
curl --header 'Content-Type: application/json' --header 'Authorization: Token ABC123' --data '{"jsonrpc":"2.0","method":"enterprise/getEnterpriseEdges","params":{"with":["certificates","configuration","links","recentLinks","site","licenses","analyticsMode","selfHealing"],"edgeIds":[4723]},"id":34123423}' --request POST 'https://my-portal.velocloud.net/portal/'
When putting it into a python script, I keep getting this result instead of the massive JSON array of data.
{'jsonrpc': '2.0', 'error': {'code': -32603, 'message': 'JSON parse error'}, 'id': 1}
This is a copy of my Python script
import requests
payload_headers = {
"Content-Type": "application/json",
"Authorization": "Token ABC123"
}
payload_data = {
"jsonrpc": "2.0",
"method": "enterprise/getEnterpriseEdges",
"params": {
"with": ["certificates", "configuration", "links", "recentLinks", "site", "licenses", "analyticsMode", "selfHealing"],
"edgeIds": [4723]
},
"id": 34
}
response = requests.post("https://my-portal.velocloud.net/portal/", headers=payload_headers, data=payload_data, verify= False)
json_info = response.json()
print(json_info)
I was hoping to get a JSON output of the data like I get with the curl in bash, and eventually incorporate a loop to run through a list to get multiple JSONs to build out an API.
You need to dump the dict to str:
import json
response = requests.post("https://my-portal.velocloud.net/portal/",
headers=payload_headers,
data=json.dumps(payload_data),
verify=False)
Or just use json parameter:
response = requests.post("https://my-portal.velocloud.net/portal/",
headers=payload_headers,
json=payload_data,
verify=False)
I have the following curl command which works fine:
curl -X POST -H 'content-type: application/json' -H "DD-API-KEY: ${api_key}" -H "DD-APPLICATION-KEY: ${app_key}" \
-d '{
"query": "service:my_service",
"time": {
"from": "2019-11-28T00:00:00Z",
"to": "2019-11-28T16:00:00Z"
},
"sort": "asc",
"limit": 1000
}' "https://api.datadoghq.com/api/v1/logs-queries/list" -o output3.json5
Then I convert this requests to Python Requests, and the curl method works but Python returns a 500 error without any details.
import requests
def main():
headers = {
'content-type': 'application/json',
'DD-API-KEY': 'AAA',
'DD-APPLICATION-KEY': 'XXX',
}
data = {
"query": "service:my_service",
"time": {
"from": "now - 1h",
"to": "now"
},
"sort": "asc",
"limit": 50
}
response=requests.post("https://api.datadoghq.com/api/v1/logs-queries/list",headers=headers, data=data)
I tried it outside my Docker guessing that maybe connection was the key, but it doesn't work either.
Point both of those at a service like httpbin to see how they differ.
Requests' data option for POST requests generates form-encoded data by default, while curl passes the JSON string through directly. You can manually encode your payload as a JSON string:
import json
response = requests.post(..., data=json.dumps(data))
# ^^^^^^^^^^
or if you have Requests version 2.4.2 or later you can use the json parameter to have your dict converted to JSON automatically:
response = requests.post(..., json=data)
# ^^^^
I am looking to send a POST request using Python to the OANDA API to open an order. They do not have a Python wrapper and use cURL, so I have had to try and convert from cURL to Python. I have done so using https://curl.trillworks.com/ -- but converting this next one does not work.
You can view the OANDA API documentation here, under the first Green POST tab - http://developer.oanda.com/rest-live-v20/order-ep/
Here is what I am working with. This first block specifies the order details. In this case, a market order in the EUR_USD instrument with a quantity of 100 units and a time in force equalling "fill or kill" :
body=$(cat << EOF
{
"order": {
"units": "100",
"instrument": "EUR_USD",
"timeInForce": "FOK",
"type": "MARKET",
"positionFill": "DEFAULT"
}
}
EOF
)
curl \
-X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer SECRET TOKEN" \
-d "$body" \
"https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders"
Converted into Python:
import requests
headers = {
'Content-Type': 'application/json',
'Authorization': 'Bearer SECRET TOKEN',
}
data = '$body'
response = requests.post('https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders', headers=headers, data=data)
As you can see, I believe there is a formatting error somewhere in the "body=$" portion, but I am not entirely sure. I simply get a 400 error, "invalid values."
If you're sending data in JSON format, you should pass them into json argument instead of data (explanation, method).
import requests
headers = {
# 'Content-Type': 'application/json', # will be set automatically
'Authorization': 'Bearer SECRET TOKEN',
}
body = {
"order": {
"units": "100",
"instrument": "EUR_USD",
"timeInForce": "FOK",
"type": "MARKET",
"positionFill": "DEFAULT"
}
}
response = requests.post('https://api-fxpractice.oanda.com/v3/accounts/{ACCOUNT-NUMBER}/orders',
headers=headers, json=body)
I'm trying to trigger a GitLab build using Python Requests. Normally one can kick off a build with a curl command.
Example curl command:
curl -X POST \
-F token=TOKEN \
-F ref=master \
-F "variables[UPLOAD_TO_S3]=true" \
https://gitlab.example.com/api/v4/projects/9/trigger/pipeline
I can get this working using the sh module but I'd prefer using requests. I've tried variations with the following:
data = {
'token': token,
'ref': master,
'variables[UPLOAD_TO_S3]': str(uploadS3),
}
headers = {'Content-Type': 'application/json'}
result = requests.post(_trigger_url, headers=headers, json=data)
I tried with and without the headers param. I've also tried passing the data dict using params, json, files, and data. All keep coming back with 400 or 404 errors. Any suggestions?
The above answer is incomplete as it does not deal with the part that makes this complex that is passing trough variables.
To pass variables around with requests using json data one needs to pass the data in the following structure
data = {
'token': token,
'ref': master,
'variables': [{"key": "UPLOAD_TO_S3", "value": True}, {"key": "S3_SERVER", "value": "https://mys3.example.com"}],
}
result = requests.post(_trigger_url, json=data)
You shouldn't send the Content-Type: "application/json" header as part of your request, and you should only need form encoding, so just pass your data dict as the data argument.
Here's a request to my Gitlab instance that succeeds and triggers a build of my project's master branch.
rsp = requests.post('https://gitlab.instance/api/v4/projects/PROJECT_ID/trigger/pipeline',
data={'token': '125fdsfdf1ab3631d2423434347643', 'ref': 'master'})
Here's the output if I inspect my rsp object in `ipython:
In [3]: rsp
Out[3]: <Response [201]>
You should also be able to trigger the pipeline by sending a POST and including the token and ref in the URL itself.
rsp = requests.post('https://gitlab.instance/api/v4/projects/PROJECT_ID/trigger/pipeline?token=TOKEN&ref=master')
If I pass the Content-Type: "application/json" header, Gitlab responds with an HTTP/400 error.
The first answer doesn't address the "variables" part. The second answer didn't work for me. Here's what I ended up with working:
variables = {
"UPLOAD_TO_S3": True,
"S3_SERVER": "https://mys3.example.com"
}
data = {
"token": token,
"ref": ref_name,
"variables": variables
}
res = requests.post(pipeline_trigger, json=data)
I am writing a python script which will call a REST POST endpoint but in response I am getting 400 Bad Request where as if I do same request with curl, it returns me 200 OK. Code snippet for python script is below
import httplib,urllib
def printText(txt):
lines = txt.split('\n')
for line in lines:
print line.strip()
httpServ = httplib.HTTPConnection("127.0.0.1", 9100)
httpServ.connect()
params = urllib.urlencode({"externalId": "801411","name": "RD Core","description": "Tenant create","subscriptionType": "MINIMAL","features": {"capture":False,"correspondence": True,"vault": False}})
headers = {"Content-type": "application/json"}
httpServ.request("POST", "/tenants", params, headers)
response = httpServ.getresponse()
print response.status, response.reason
httpServ.close()
and corresponding curl request is
curl -iX POST \
-H 'Content-Type: application/json' \
-d '
{
"externalId": "801411",
"name": "RD Core seed data test",
"description": "Tenant for Core team seed data testing",
"subscriptionType": "MINIMAL",
"features": {
"capture": false,
"correspondence": true,
"vault": false
}
}' http://localhost:9100/tenants/
Now I am not able figure out where is the issue in python script.
Try using requests (install with pip install requests) instead of urllib.
Also, enclose your data as JSON in the request body, don't pass them as URL parameters. You are passing JSON data in your curl example as well.
import requests
data = {
"externalId": "801411",
"name": "RD Core",
"description": "Tenant create",
"subscriptionType": "MINIMAL",
"features": {
"capture": False,
"correspondence": True,
"vault": False
}
}
response = requests.post(
url="http://localhost:9100/tenants/",
json=data
)
print response.status_code, response.reason
EDIT
From https://2.python-requests.org/en/master/user/quickstart/#more-complicated-post-requests:
Note, the json parameter is ignored if either data or files is passed.
Using the json parameter in the request will change the Content-Type
in the header to application/json.
The problem in your code is, you set Content-Type header as application/json and not sending data in json format
import httplib, json
httpServ = httplib.HTTPConnection("127.0.0.1", 9100)
httpServ.connect()
headers = {"Content-type": "application/json"}
data = json.dumps({
"externalId": "801411",
"name": "RD Core",
"description": "Tenant create",
"subscriptionType": "MINIMAL",
"features": {
"capture": False,
"correspondence": True,
"vault": False
}
})
# here raw data is in json format
httpServ.request("POST", "/tenants", data, headers)
response = httpServ.getresponse()
print response.status, response.reason
httpServ.close()