I have the following curl command
curl -X POST "http://localhost:5000/api/v1/image/cad1b12e-c374-4d46-b43b-3ddbe7d683c4" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "file=#tests/test.jpg;type=image/jpeg"
I am trying to construct a pytest function with the same behavior that the command above but I am not getting the exact result, the pytest function is the following
url = "http://localhost:5000/api/v1/image/cad1b12e-c374-4d46-b43b-3ddbe7d683c4"
payload={}
files=[('file',('test.png',open('./tests/test.png','rb'),'image/png'))]
headers = {
'accept': 'application/json',
'Content-Type': 'multipart/form-data'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
Short answer:
Don't specify a Content-Type in the headers dictionary, when providing a files argument to the requests.request method.
Longer answer
I tried to test that functionality to my own Flask upload script.
This worked when removing 'Content-Type': 'multipart/form-data' form the headers dictionary, otherwise the response was 400 bad request (see response.content)
I noted that after making the request, and inspecting response.request.headers I could see that
when specifying that header it was part of the request, as-is:
'Content-Type': 'multipart/form-data'
however when not specifying it requests automatically generates the header with a boundary, and the request succeeded:
'Content-Type': 'multipart/form-data; boundary=1beba22d28ce1e2cdcd76d253f8ef1fe'
Also:
To find the headers sent from curl, either by using the trace method or if you have access to the server, printing request.headers within the upload route, you can see that the curl command automatically appends a boundary to the header, even when the flag -H "Content-Type: multipart/form-data" is provided:
Content-Type: multipart/form-data; boundary=------------------------c263b5b9297c78d3
The same can be observed via the Network tab in your browser dev-tools, when uploading via an HTML form with enctype="multipart/form-data" set as an attribute.
I had always thought that manually adding 'Content-Type:':'multipart/form-data' to the headers dict was required, but it appears when using the the files argument to requests.request method, this is automatically generated with the boundary.
In fact the requests docs make no mention of setting this header yourself.
Related
I am trying to hit an API endpoint using python requests. I have been unable to successfully send the body of the request except when using cURL. Here is the cURL command that succeeds:
curl --location --request POST '<api endpoint url>' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'obj={"login":"<email>","pword":"<password>"}'
Using python requests like this returns an error from the API because of the body of the request:
payload = 'obj={"login":"<email>","pword":"<password>"}'
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
response = requests.post(url, headers=headers, data=payload)
print(response.text)
I also tried requests.request("POST") but got the same results.
Your data is URL encoded as you can see in the curl Content-Type header, so you have to provide the data in an URL encoded format, not JSON.
Use the following instead. requests will set the Content-Type header to application/x-www-form-urlencoded automatically. It will also take care of the URL encoding.
data = {"login": "<email>", "pword": "<password>"}
response = requests.post(url, data=data)
I am trying to make an API call in Python 3.8.
I have implemented the request in curl and it is as follows:
curl --location --request POST 'https://url.mirakl.net/api/orders/1720178-A/documents' --header 'Authorization: 9xxxxxxxx-4xxx-4xxx-8xxx-xxxxxxxxxxx6' --header 'Accept: application/json' --header 'Content-Type: multipart/form-data' --frm 'files=#"1720178-A.pdf"' --form 'order_documents="<body><order_documents><order_document><file_name>1720178-A.pdf</file_name><type_code>CUSTOMER_INVOICE</type_code> </order_document></order_documents></body>";type=application/xml'
It works, but Python code does not:
url = "https://url.mirakl.net/api/orders/1720178-A/documents"
payload = {"order_documents": '<body><order_documents><order_document><file_name>1720178-A.pdf</file_name>'
'<type_code>CUSTOMER_INVOICE</type_code></order_document></order_documents></body>'}
files = [
("files", ('1720178-A.pdf', open('1720178-A.pdf', 'rb')))
]
headers = {
'Authorization': 'xxxxxxxxx-xxx5-4xxx-xxxx-xxxxxxxxx6',
'Accept': 'application/json',
'Content-Type': 'multipart/form-data'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.text)
Returning the following error: {"status":400,"message":"Bad Request"}
The error is obvious in how I'm implementing the request, but I don't know what I'm missing.
I have also tried with json payload with same result:
payload = {"order_documents":[{"file_name":"1720178-A.pdf","type_code":"CUSTOMER_INVOICE"}]}
In my day to day, I use Python for other tasks, but since I'm using it for this one, I want to do it the right way, without relying on running curl from python.
P.S. Looking at the body of the petition I can't see anything wrong.
print(requests.Request("POST", url, headers=headers, files=files, data=payload).prepare().body.decode('US-ASCII', errors='ignore'))
--b54bd40f2809e798c2a04069686c35fc
Content-Disposition: form-data; name="order_documents"
{'file_name': '1720178-A.pdf', 'type_code': 'CUSTOMER_INVOICE'}
--b54bd40f2809e798c2a04069686c35fc
Content-Disposition: form-data; name="files"; filename="1720178-A.pdf"
Content-Type: application/pdf
%PDF-1.7
%
....
....
....
....
%%EOF
--b54bd40f2809e798c2a04069686c35fc--
I have finally found the source of the problem.
To implement the API call (not just one, but several), I downloaded the Postman collection that the retailer makes available to consumers.
What was my failure after checking that the Postman call worked correctly? Using the auto code generation provided by the software. Although it is an implementation for the request library, it sets the content-type header, which causes the error. requests autogenerate it correctly:
{
"User-Agent": "sitename.app",
"Accept-Encoding": "gzip, deflate",
"Accept": "application/json",
"Connection": "keep-alive",
"Authorization": "xxx-xxxx-xxx-xxx",
"Content-Length": "22680",
"Content-Type": "multipart/form-data; boundary=aea8e12ea4fd0c7d9debb64f69ad0718"
}
In the following screenshot, I show how the collection includes the content type and takes it to the implementation even though it makes use of requests:
I had similar problems very recently with multipart/form sending. Here is how I solved it.
I generated the json part as follows:
import os
data = {
'title': (None, "testFiles", 'application/json'),
"short": (None, "testFileShort", 'application/json'),
"ref_name_1": (None, "TestRef2", 'application/json'),
"upload_file_1" = (os.path.basename("test.txt"), open("test.txt", 'rb'), 'application/octet-stream')
}
I am not entirely sure, why is it so fuzzy about the json, or if this is the most elegant way, but it works perfectly for me.
And the request code is the following:
import requests
def POST(self, address, json=None, files=None):
url = self.URL + address
return requests.post(url=url, json=json, files=files)
try:
r = httpBackend.POST(
"/resources/create-files-item",
files=data)
except Exception:
return None
I'm a Python user, beginner level. I'm trying to follow this instruction on Basecamp 3. Documentation: https://github.com/basecamp/bc3-api
I've successfully gone through the authorization step and was able to retrieve the access token (which consists of 3 keys: access_token, expires_in and refresh_token.
Now i'm trying to pull some actual data from Basecamp, and the most basic call is to https://3.basecampapi.com/999999999/projects.json (with 99999999 being my account number, which I have).
The instruction has an example in curl: curl -H "Authorization: Bearer $ACCESS_TOKEN" -H 'User-Agent: MyApp (yourname#example.com)' https://3.basecampapi.com/999999999/projects.json
But I cannot translate this to Python. I tried many methods of passing the keys to the header call but none works. Can anyone help me out?
Code:
url = "3.basecampapi.com/99999999/projects.json"
headers = {'Content-Type': 'application/json',
'User-Agent': 'MyApp (myemail#gmail.com)',
'access_token': 'Access_Token_String',
'expires_in': '1209600',
'refresh_token': 'Refresh_token_string'}
result = requests.post(url, headers=headers)
This is an old question, but posting an answer for anyone who happens to stumble upon this.
url = f'3.basecampapi.com/{PROJECT_ID}/projects.json'
headers = {'User-Agent': 'MyApp (myemail#gmail.com)',
'Content-Type': 'application/json; charset=utf-8',
'Authorization': f'Bearer {ACCESS_TOKEN}'
response = requests.get(url, headers=headers)
Then view the output via response.json()
So here is my curl command, In command line, curl script is working fine
curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" -u user:password URL -d '{"key1":"value1","key2":"value2"}'
Now I have converted my code to python and the code look like this :
import requests
import json
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
}
data = '{"key1":"value1","key2":"value2"}'
requests.post('URL', headers=headers, data=data, auth=('user', 'password'))
When I ran this code, I am not getting any output.
Please let me know what I am doing wrong in here.
At first, the 'url' parameter in requests.post should be an complete url including the 'http://' or 'https://'.
Secondly, if you need to post some data in form of application/json, using Python protogenic dict will be fine. like this
data = {"key1":"value1","key2":"value2"}
you might be looking at something like this?
r = requests.post("http://{}/Command?".format(web_addr), params=dict)
print r.text
In Python 3.6.1, you should get the response 200 message if the following code gets executed, with a existing 'URL':
from requests import post
data = {"key1":"value1","key2":"value2"}
print(post('URL', json=data))
so I have made a request to a server with pythons request library. The code looks like this (it uses an adapter so it needs to match a certain pattern)
def getRequest(self, url, header):
"""
implementation of a get request
"""
conn = requests.get(url, headers=header)
newBody = conn.content
newHeader = conn.headers
newHeader['status'] = conn.status_code
response = {"Headers" : newHeader, "Body" : newBody.decode('utf-8')}
self._huddleErrors.handleResponseError(response)
return response
the header parameter I am parsing in is this
{'Authorization': 'OAuth2 handsOffMyToken', 'Accept': 'application/vnd.huddle.data+json'}
however I am getting an xml response back from the server. After checking fiddler I see the request being sent is:
Accept-Encoding: identity
Accept: */*
Host: api.huddle.dev
Authorization: OAuth2 HandsOffMyToken
Accept: application/vnd.huddle.data+json
Accept-Encoding: gzip, deflate, compress
User-Agent: python-requests/1.2.3 CPython/3.3.2 Windows/2008ServerR2
As we can see there are 2 Accept Headers! The requests library is adding in this Accept:* / * header which is throwing off the server. Does anyone know how I can stop this?
As stated in comments it seems this is a problem with the requests library in 3.3. In requests there are default headers (which can be found in the utils folder). When you don't specify your own headers these default headers are used. However if you specify your own headers instead requests tries to merge the headers together to make sure you have all the headers you need.
The problem shows its self in def request() method in sessions.py. Instead of merging all the headers it puts in its headers and then chucks in yours. For now I have just done the dirty hack of removing the accept header from the default headers found in util