I am posting to an API that seems to insist on receiving XML data as multipart/form-data with the name (file name?) xml. It works in postman but I can't get it to work using Python's requests. This is my Python code (based on https://stackoverflow.com/a/24443309/1011724):
requests.post(callpro_url,
files={'xml':('data.xml',result)},
verify=False).text
where result is a string containing XML. If I try this code I get the response:
xml post field is empty
which is the response this API give if you don't use the multipart/form-data header.
If I generate code from the working postman post I get something like this (slightly redacted):
import requests
url = "https://blablabla.blablab.com/blabla/api.php"
querystring = {"mode":"import","hash":"redacted-hash","xml":"\"xml\""}
payload = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"xml\"\r\n\r\n<?xml version=\"1.0\" ?>\n<importdata>\n --redacted-XML-- \n</importdata>\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--"
headers = {
'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW",
'cache-control': "no-cache",
'postman-token': "8d3ec8ee-784e-3a65-5240-cf1a9534d1c4"
}
response = requests.request("POST", url, data=payload, headers=headers, params=querystring)
print(response.text)
Executing this code in Python, it gives the correct response.
Note that the params=querystring part in the postman code is taken care of in the URL in my code.
I'm confused by the payload in the postman code. It adds things like Content-Disposition and name in the string. I assume that I can put this stuff in the tuple in the files parameter but I'm not sure how to to do it. I've tried files={'xml':('data.xml',result,'form-data')} for example and also files={'xml':('data.csv',result)} and {'xml':('xml',result)}.
Also, the postman code explicitly defines the header as
'content-type': "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW"
whereas the requests documentation say that I should never explicitly define the headers in that way. However the postman code works and my code does not.
Any suggestions?
I don't know if this is worth answering or if I should delete the question but what was needed was to name the file "xml" not xml. so this works:
requests.post(callpro_url,
files={'xml':('"xml"',result)},
verify=False).text
Related
I have an API I am building out that will allow me to upload media to my WordPress website using their new API. I have searched all over the internet and SO for a solution to this problem, but all of the answers seem to be using an older WordPress API so I have not had much success in finding an answer.
I have the following code written out in a function to send a multipart/form-data request to the API:
img = open(image_path, 'rb')
file_name = os.path.basename(image)
print('Compressing and sending data...')
NEW_HEADER = {
'Authorization': f'BEARER {WORDPRESS_ACCESS_TOKEN}',
'Content-Disposition': f'attachment; filename={file_name}',
'Content-Type': 'multipart/form-data'
}
response = requests.post(
url=WORDPRESS_MEDIA_URL,
files={'media': img.read()},
headers=NEW_HEADER
)
print(response.request)
print(response.text)
return response
Which returns me:
{"media":[],"errors":[]}
<Response [200]>
OK
So, it sends something to the endpoint, but it's return an empty array and empty error box. Again, I can't seem to find any solution for the new WP, so forgive me if the Python seems a little hacky and put-together, I've just been putting together what appears to work.
Further Reading Since Original Posting
I pinged the API with Postman and it returns data as expected:
{"media":[{"id":"xxxx","date":"2022-11-03T20:06:54-05:00","parent":0,"link":"https:mywebsite","title":"mypicture","caption":"","description":"",...}
So, Since the post request seems to not be the issue, I tried to debug my request in Postman and my local console to see what was being sent:
{'User-Agent': 'python-requests/2.28.1', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Authorization': 'BEARER QnkXS43KhiqO7Oa!9G!zN2gnz90hHhZEribF2ddZu0h8&5V0p5M69^OwDb9E6H%B', 'Content-Type': 'multipart/form-data', 'Content-Length': '6'}
media=
And I see that the media returns empty from my local console. In Postman, the media is returned as undefined (though I'm not sure if that's just Postman being unable to parse the image in a JSON format in the console/debugger.
This leads me to believe that either the way I am accessing/calling the directory with the image is wrong, or the way I am trying to encode it to the request.
Even Further Reading Since Posting
I have since circumnavigated any concerns about directory issues by copying the photos from an external directory to that of one inside the project folder.
Since then, it appears that the current way I am sending the data returns the same response as if I am sending nothing at all:
response = requests.post(
url=WORDPRESS_MEDIA_URL, # there is no data sent in this request.
headers=NEW_HEADER
)
Which still returns me:
{"media":[],"errors":[]}
<Response [200]>
OK
This leads me to believe that the way I am sending the data is incorrect, as WordPress is receiving my request, but is not getting the information that it needs.
So, doing some further reading and research, I come across requests-toolbelt, which is to help sending multipart/form-data:
data = MultipartEncoder(fields={'media': (file_name, img, 'image/jpeg')})
new_header = {
'Authorization': f'BEARER {WORDPRESS_ACCESS_TOKEN}',
'Content-Type': data.content_type,
'Content-Disposition': 'form-data; name="media"; filename="%s"' % name_img,
}
response = requests.post(WORDPRESS_MEDIA_URL, data=data, headers=new_header)
Now, when I send a request with this data, I get this response:
{"error":"unsupported_mime_type","message":"File type unknown"}
<Response [400]>
Bad Request
So sending the request with the MultipartEncoder returns an error of an unsupported MIME type. So while it isn't a blank 200 response, it gives me reason to think that perhaps I am using the MultipartEncoder wrong, or I am not making the proper adjustments to the picture I am trying to upload.
Any insight into this would be greatly appreciated.
I want to send a 'multipart/form-data' dictionary data without files
headers = {'key':'value', 'Content-Type':'multipart/form-data'}
data = {'api_key':'api_value', ... }
response = request.post('API_URL', headers=headers, data=data)
but API response is:
FileUploadException: the request was rejected because no multipart boundary was found
so I changed 'multipart/form-data' -> 'multipart/form-data; boundary=____sample'
This code also didn't work. It didn't recognize the parameters.
How can I send 'multipart/form-data' requests without files?
I'm not entirely sure how python works with requests. You can try this accepted answers or any which is suitable
python requests - POST Multipart/form-data without filename in HTTP request
I need to download the response of POST-request on https://www.avast.com/hackcheck/ site as JSON file. But my request to the site return an error 400. My code is:
URL = 'https://identityprotection.avast.com/v1/web/query/site-breaches/unauthorized-data'
params = {'emailAddresses':['xxx#x.ru']}
headers = {'Vaar-Version': '0'}
req = requests.post(URL, params, headers)
req
The request on the site works correctly
The request on the site works correctly
So what am I doing wrong?
Ideally you should include all the other parameters in your request header also (not just the Vaar-Version). See screenshot below of the full POST variables:
Note: Even then it may not necessarily work
Add the following in your header:
headers: {
Accept : "application/json",
}
Also, keep in mind where you put equals (=) and where you put semicolons.
I had been using sockets, with Python, for some time ago and I'm trying to understand why this POST which should send some data on fields data1 and data2 do not work.
POST /method.php HTTP/1.1\r\nHost: localhost\r\nContent-Type: multipart/form-data\r\n\r\ndata1=something&data2= otherthing\r\n\r\n
What is the problem with this request?
There are several things wrong with your request:
POST /method.php HTTP/1.1
Host: localhost
Content-Type: multipart/form-data
data1=something&data2= otherthing
First, whenever a body is used within a HTTP request the length of the body must be known. This is typically done by given the length up-front with Content-length in the HTTP header although also chunked encoding might be used if the full length is not known up front. Your request does not do any of these which means the request is an invalid HTTP request.
Additionally you claim a Content-Type of multipart/form-data although your body is not of this type. With multipart/form-data your body would consist of several MIME parts separated by a text boundary and this boundary would need to have been declared in your Content-type header. The correct type for the body you show would be instead application/x-www-form-urlencoded.
Even with application/x-www-form-urlencoded the body is partly wrong. This type of body should be only pairs of key=value concatenated by &, i.e. there should be neither as space after a key as you have after data2= nor there should be new lines added after the end of the data as you have.
When removing all these problems you should probably send the following request:
body = "data1=something&data2=otherthing"
request = ("POST /method.php HTTP/1.1\r\n" + \
"Host: localhost\r\n" + \
"Content-Type: application/x-www-form-urlencoded\r\n" + \
"Content-Length: %d\r\n" + \
"\r\n%s") % (len(body),body)
But once you have send this request the trouble continues since getting the response correctly is complex too. Generally I recommend to not code your own HTTP handling unless you really know what you do but instead use existing libraries. While HTTP might look simple when just looking at a few example requests it is way more complex than it initially looks. And while your code might seem to work against specific servers it might fail with other servers.
It might be easier to use the requests library so your code would look something like this:
import requests
# Data
data = {
'data1':'something',
'data2':'otherthing'
}
# Custom headers
headers = {
'content-type': 'multipart/form-data'
}
# Get response from server
response = requests.post('http://localhost/', data=data, headers=headers)
# If you care about the response
print(response.json())
You can also send files and a whole lot of other stuff
Have you tried using the Requests library instead, example of a post request below
import requests
header = {"Content-Type": "multipart/form-data"}
data1="something"
data2= "otherthing"
session_requests = requests.session()
result = session_requests.post("http://localhost/", data=dict(data1, data2), headers=header)
I am trying to trigger some builds using a shell script by doing the following :
export url='http://test.com';
export job_name='MY_JOB_NAME';
jso="{\"parameter\": [{\"name\":\"BRANCH\",\"value\":\"master\"}, {\"name\":\"GITURL\",\"value\":\"https://github.test.com/test/test.git\"}]}";
curl $url/job/$job_name/build --data-urlencode json="$jso";
This works fine, but when I try to convert it to a python equivalent, it doesn't seem to trigger the URL:
import requests
import json
url='http://test.com/job/MY_JOB_NAME/build'
params={'name':'release_1.5', 'GITURL':'https://github.test.com/test/test.git'}
payload = json.dumps(params)
resp = requests.get(url=url, data=payload)
This executes without any error, but it does not trigger a build on my CI machine.
There are quite a few things you're doing wrong here. The first thing which I hope is apparent is that the JSON data you're sending is completely different.
Beyond that, the primary issue you're having here is that your curl is doing a POST with urlencoded data in the BODY, and your python request is doing a GET with urlencoded data as separate parameters in the url. Change your .get to a .post, and the params= to data= and you should be a whole lot closer to your intended goal.
resp = requests.post(url=url, data={'json':payload})
Also note, I embedded your payload into a key as json, as that's what is happening in your curl. I'm not fully aware of your implementation details, but I hope that this helped put you on the right track.
Add the content-type to the header of the request, i.e.:
headers = {'content-type': 'application/json'}
...
resp = requests.get(url=url, params=payload, headers=headers)