I'm currently struggling with a really simple problem, but cant spot the problem in detail.
I want to upload a file via requests.post, the current approach looks kinda like:
fin = open(f"{path_dtl}{filename}.dtl", "rb")
files = {"file": fin}
headers = CaseInsensitiveDict()
headers["Authorization"] = f"Bearer {auth_token}"
headers["Content-Type"] = "application/octet-stream"
headers["Content-Disposition"] = f"form-data;filename ='{filename}.dtl'"
upload_dtl = requests.post(url, headers=headers, files=files)
The result is a {'error': 'Invalid Media Type/Mime Type'} response. Is there a need to open a .dtl file in different manner? The usage of a request with this configuration via Postman works fine so I guess I missed a silly mistake. Thanks in advance.
UPDATE
Alright... for anyone interested. The cause of the issue was the default key-value "file" in files = {"file": fin} is different for my API. Following workaround fixed my issue.
with open(f"{path_dtl}{filename}.dtl", "rb") as f: files = f.read()
...
upload_dtl = requests.post(url, headers=headers, files=files)
Have you tried specifying a type? You can do so like this:
files = {'file': ('aaa.csv', open('aaa.csv', 'rb'),'text/csv')}
import requests
import csv
url = "https://paneledgesandbox.//API/v3/surveys/id/import-responses"
with open('Cars.csv', 'r') as file:
payload = csv.reader(file)
print(payload)
headers = {
'X-API-TOKEN': 'zzzz',
'Content-Type': 'text/csv',
'Cookie': 'CPSessID=bb5b5cf55ceff2c2b8237810db3ca3a7; XSRF-TOKEN=XSRF_0wG1WMbV3G0ktBb'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)`
I am getting an error while trying to read a csv file in a post call using python.
I am not sure where exactly the issue is considering that using the with command, the file automatically closes.
In
payload = csv.reader(file)
you are not actually reading your csv, but rather returning a generator which is exhausted after the file is closed when you go out of with's scope.
You need to read the data instead
payload = "\n".join([", ".join(line) for line in csv.reader(file)])
I need to submit an image to the site https://zxing.org/w/decode.jspx, and read the result page : https://zxing.org/w/decode.
I tried this, but it does not work :
def decode_qrcode(path):
s = requests.Session()
url = "https://zxing.org/w/decode.jspx"
files = {'file': open(path, 'rb')}
s.post(url, files=files)
return s.get("https://zxing.org/w/decode").text
I know that there are librairies to read QR code, but I did not find any that works for the kind of QR codes that I work with (they might have an error rate not supported).
You have to use the allow_redirects argument when making the POST request
import requests
def decode_qrcode(path):
s = requests.Session()
url = "https://zxing.org/w/decode.jspx"
files = {'file': open(path, 'rb')}
s.post(url, files=files, allow_redirects = True)
return s.get("https://zxing.org/w/decode").text
I have AWS Lambda set up.
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': json.dumps(event)
}
I would like to POST in a PDF file so that I can operate on it in my lambda function.
Here is my POST code
import requests
headers = {
'X-API-KEY':'1234',
'Content-type': 'multipart/form-data'}
files = {
'document': open('my.pdf', 'rb')
}
r = requests.post(url, files=files, headers=headers)
display(r)
display(r.text)
I am getting the error:
<Response [400]>
'{"message": "Could not parse request body into json: Unexpected character (\\\'-\\\' (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value
How can I POST over my PDF and be able to properly send over my PDF and access it in Lambda?
Note:
I am successful if I do this:
payload = '{"key1": "val1","key2": 22,"key3": 15,"key4": "val4"}'
r = requests.post(url = URL, data=payload, headers=HEADERS)
It is just the PDF part which I can't get
I figured it out. Took me a ton of time but I think I got it. Essentially it's all about encoding and decoding as bytes. Didn't have to touch the API Gateway at all.
Request:
HEADERS = {'X-API-KEY': '12345'}
data = '{"body" : "%s"}' % base64.b64encode(open(path, 'rb').read())
r = requests.post(url, data=data, headers=HEADERS)
In lambda
from io import BytesIO
def lambda_handler(event, context):
pdf64 = event["body"]
# Need this line as it does 'b'b'pdfdatacontent'.
pdf64 = pdf64[2:].encode('utf-8')
buffer = BytesIO()
content = base64.b64decode(pdf64)
buffer.write(content)
I found this worked quite well for me:
Request
import requests
file_loc = 'path/to/test.pdf'
data = open(file_loc,'rb').read() #this is a bytes object
r = requests.post(url, data=data)
r.ok #returns True (also a good idea to check r.text
#one-liner
requests.post(url, data=open(file_loc,'rb').read())
Lambda - Python3.8
import io, base64
body = event["body"]
attachment = base64.b64decode(body.encode()) #this is a bytes object
buff = io.BytesIO(attachment) #this is now useable - read/write etc.
#one-liner
buff = io.BytesIO(base64.b64decode(event["body"].encode()))
Not quite sure why, but for me base64 encoding (even with urlsafe) in the original request corrupted the file and it was no longer recognised as a PDF in Lambda, so the OP's answer didn't work for me.
I need to do a API call to upload a file along with a JSON string with details about the file.
I am trying to use the python requests lib to do this:
import requests
info = {
'var1' : 'this',
'var2' : 'that',
}
data = json.dumps({
'token' : auth_token,
'info' : info,
})
headers = {'Content-type': 'multipart/form-data'}
files = {'document': open('file_name.pdf', 'rb')}
r = requests.post(url, files=files, data=data, headers=headers)
This throws the following error:
raise ValueError("Data must not be a string.")
ValueError: Data must not be a string
If I remove the 'files' from the request, it works.
If I remove the 'data' from the request, it works.
If I do not encode data as JSON it works.
For this reason I think the error is to do with sending JSON data and files in the same request.
Any ideas on how to get this working?
See this thread How to send JSON as part of multipart POST-request
Do not set the Content-type header yourself, leave that to pyrequests to generate
def send_request():
payload = {"param_1": "value_1", "param_2": "value_2"}
files = {
'json': (None, json.dumps(payload), 'application/json'),
'file': (os.path.basename(file), open(file, 'rb'), 'application/octet-stream')
}
r = requests.post(url, files=files)
print(r.content)
Don't encode using json.
import requests
info = {
'var1' : 'this',
'var2' : 'that',
}
data = {
'token' : auth_token,
'info' : info,
}
headers = {'Content-type': 'multipart/form-data'}
files = {'document': open('file_name.pdf', 'rb')}
r = requests.post(url, files=files, data=data, headers=headers)
Note that this may not necessarily be what you want, as it will become another form-data section.
I'm don't think you can send both data and files in a multipart encoded file, so you need to make your data a "file" too:
files = {
'data' : data,
'document': open('file_name.pdf', 'rb')
}
r = requests.post(url, files=files, headers=headers)
I have been using requests==2.22.0
For me , the below code worked.
import requests
data = {
'var1': 'this',
'var2': 'that'
}
r = requests.post("http://api.example.com/v1/api/some/",
files={'document': open('doocument.pdf', 'rb')},
data=data,
headers={"Authorization": "Token jfhgfgsdadhfghfgvgjhN"}. #since I had to authenticate for the same
)
print (r.json())
For sending Facebook Messenger API, I changed all the payload dictionary values to be strings. Then, I can pass the payload as data parameter.
import requests
ACCESS_TOKEN = ''
url = 'https://graph.facebook.com/v2.6/me/messages'
payload = {
'access_token' : ACCESS_TOKEN,
'messaging_type' : "UPDATE",
'recipient' : '{"id":"1111111111111"}',
'message' : '{"attachment":{"type":"image", "payload":{"is_reusable":true}}}',
}
files = {'filedata': (file, open(file, 'rb'), 'image/png')}
r = requests.post(url, files=files, data=payload)
1. Sending request
import json
import requests
cover = 'superneat.jpg'
payload = {'title': 'The 100 (2014)', 'episodes': json.dumps(_episodes)}
files = [
('json', ('payload.json', json.dumps(payload), 'application/json')),
('cover', (cover, open(cover, 'rb')))
]
r = requests.post("https://superneatech.com/store/series", files=files)
print(r.text)
2. Receiving request
You will receive the JSON data as a file, get the content and continue...
Reference: View Here
What is more:
files = {
'document': open('file_name.pdf', 'rb')
}
That will only work if your file is at the same directory where your script is.
If you want to append file from different directory you should do:
files = {
'document': open(os.path.join(dir_path, 'file_name.pdf'), 'rb')
}
Where dir_path is a directory with your 'file_name.pdf' file.
But what if you'd like to send multiple PDFs ?
You can simply make a custom function to return a list of files you need (in your case that can be only those with .pdf extension). That also includes files in subdirectories (search for files recursively):
def prepare_pdfs():
return sorted([os.path.join(root, filename) for root, dirnames, filenames in os.walk(dir_path) for filename in filenames if filename.endswith('.pdf')])
Then you can call it:
my_data = prepare_pdfs()
And with simple loop:
for file in my_data:
pdf = open(file, 'rb')
files = {
'document': pdf
}
r = requests.post(url, files=files, ...)