Develop Reference

Http request boundaries getting caught in pdf decoding process

I'm trying to decode a base64 pdf file and send it to another endpoint.
I used a python policy for the decoding part and here's the code
import base64
pdfB64 = flow.getVariable("request.content")
pdfFile = base64.b64decode(pdfB64)
flow.setVariable("pdfFileDecoded",pdfFile)
Now, when I send my http post request which is below
headers :
Accept : */*
boundary : --Boundaryy
--Boundaryy
Content-Disposition: form-data; name="testdu12janvier"; filename="testdu12janvier.pdf"
Content-Type: application/pdf
<< Heres is sensitive data which is basically a base64 encoded pdf file >>
--Boundaryy--
When I send this POST request and trace it in Apigee Edge, I notice that something else is encoded before the pdf file I think its either the boundary or one of the headers. This makes a corrupt pdf file which can't be read.
How do I isolate the pdf file from the request body without removing boundaries? as I'll need to send multiple in near future.

Error when parsing body of an HTTP request using Python Request lib

I have tested some requests inside the Postman app. First, I want to get the body information of an HTTP request inside Python (package requests used). The response appears positive with 200 OK.
response = session.request("POST", url, headers=headers, data=payload, verify ='custom-proxy-ca.crt')
Now I would like to get the body with
body = response.content
Print(body) delivers
b'\x83\x84\x01\x00\xc4\xff\xd4\xe9\xb4\xf6\xde,\x13\xa9\xc0(\xc7_\x8dL\x90\xf0\xb4K\xc4<\xe7\xb1M\x02)\xe0\x80z\xd0\xdf>\xcf\xd7\xd2\xec\x8d\x1e\xe4un\x0c\x83\xa1\x88g\xe7fah\x89\xbe\xca\xa8\x04_\xa2W\xbd\xfe]W\xd1\x06\x1f\xef~ZN\xa6\x0bq\xfa\x18\xc4\x1f\xb3\xf8\xc2\x9dF\xc5\xf0\xe6\x8d\xb6\xc1\xa0\xab\x7f\xfbyM\xe0\x88I\xb4\xd4\x82\xa1%\xd9R7Nt\xa4~<\x8c\x8e\xdb\xe7<xx-.\xab\xa7|16\xcb"\xba\x89\xbc\xe7\xcaF\xd1\xacV-u\xbf\xaa\x04\xf7\xa2\x88\xa1\x1bUI\xdfkI$`\x18:j\x7fU\x02\x0e\xcb\x97\x8em\xc6\x81\xe6\x85\xbe\xa5\xb9vbjQ$}M&n\xe0$A\xe0\xd9\xd2\xc6\x9aA\xf4\x12\x81/1\x0c\xf0(\x0cy\xf5\xaf\xca\x1bQ\x1082\xa1\xb4n4VRR\xbb7\xa5XO\x08\x0c\x13\xf2:\xc0-\x06\xa9\xda\xaeGX\x97B\x81!\x17\x87\xfa\xd1\x1b\xc0\xd0\x89|\xe8E\x0f\rp\xfd\x00\x96\xeaI\xbe\xda\xbb\xe3\x87\xc7\xdb\x9b\xfd\xab\xe8\xc7\xdd\x0cEL-x\xe0\x9bVhY\x0cT\x08\x95S\xa3\xfd\xdc\xe3\x81/1\x9d\x9e\'T\xf6\xe0pl\xd33#0,T}X%\x04\x0e\xd7r\xfd\x10\x0cs\xe90\x05\xe8\xe8\xf8\xea\xfc\xe5\xf8\xe1\xfd\xb9\xea\xe7\xe0\xc0\x9a!\xa1\\M\xa8\x9d\x9f\xe4\xa2\x07_\xae\xd7\x0c\xdd\xb8\xaa\xbf\xe9\xfc\x1a|\x89^\xf59\x81\xe3J\x91\xa4v(\xff7J1\x1ao\x9c\x89\xa1#0\xf4\xaa\xa0\xc7\xbc\xea\x9f\xae\xa6\xe8\xa9-T\xc9#\xd1\x81\x7f\xee\x9a\xbb\xfd\x87\xc3\xe3+|K\xe2\xfdPe\xa0\xaa\x9d\x18\xf0\xcc\xc0\xf10\x80\xca\xb0XuW\x9d\xcc\xc0\xa5\xc8;bP\xdd\x9d\x1aeC\xfd\xf84\xa6\x14yG\xeb\xb5\x01\x03'
Now I try to search a token in the body, but it seems to be encrypted.
If I want to get the result of the JSON parser with
json.loads(body)
it returns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte`.
Okay, it seems that the encoding is done in a different way than expected. But how did the Postman app do the decoding of the body? For example, I can read it there parsed as JSON (see the figure below). What am I doing wrong in Python?
Request

Okay, the problem is solved, but I want to share with you how to deal with this kind of problem.
The initial problem is to call the HTTP POST request with the header parameter Accept-Encoding like
'Accept-Encoding': 'gzip, deflate, br'
This line of code means: Locally can receive data in compressed format.
The server compresses the large file and sends it back to the client during processing. After receiving the IE, the IE performs a local pressure on the file.
The reason for the error is: the program did not extract the file
Solution: delete this line of code and it works

Python send mtom message

How do I send an mtom message in python? I have tried using the requests library but I am getting an error from the server. The workflow is HTTP headers with a SOAP envelope and a binary attachment. I am using an MTOM template file which is structured as follows:
Content-Type: multipart/related;
boundary=boundary1234567890; type="application/xop+xml";
start="<0.urn:uuid:1FACEDB95C3509148F1570480012346#w3.org>"; start-info="text/xml"
Transfer-Encoding: chunked
SOAPAction: "soap-action"
--boundary1234567890
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.urn:uuid:1FACEDB95C3509148F1570480012346#w3.org>
set soap envelope here
--boundary1234567890
Content-Type: application/zip
Content-Transfer-Encoding: binary
Content-ID: <1.urn:uuid:1FACEDB95C3509148F1570480012347#w3.org>
set binary attachment here
I then send the message with the python requests library as follows:
import requests
data = open(template, 'rb').read()
response = requests.post(url, data=data, headers={})
However, I am getting an error from the server saying that my request was not able to be processed. I am probably doing something wrong, but I do not know how to send an MTOM message. Any help would be greatly appreciated.

I think the only real support for MTOM
is currently in Java and C#.You can use Axis2/C which I believe supports MTOM and
put a Python wrapper around it.

How to recreate an image from the binary string downloaded from a REST API

I would like to recreate a .png file sent from a Flask server.
If I GET in a browser the file downloads and I can view it.
If I GET it from python requests I have problems saving the file and I can never view it because the format seems to be broken.
Client:
req = requests.get(URL + "/image/file.png")
with open("recv.png", "wb") as f:
print(req.text)
f.write(req.text.encode())
Server:
#app.route("/image/<string:path>")
def get(path):
return send_from_directory("images", path, mimetype="image/png", as_attachment=True)

You can use a Base64 encoding with multipart http request:-
Content-Type: multipart/form-data;

python 3 tornados file body with headers

I'm tying to write some simple app on python3 and tornado for server, and requests for client, and I'm getting some headers in 'self.request.body', which I can't dispose of. For instance, for file 'blahblahblah', I get:
--cb5f6ba84bdf42d382dfd3204f6307c7\r\nContent-Disposition: form-data; name="file"; filename="1.bin"\r\n\r\nblahblahblah\n\r\n--cb5f6ba84bdf42d382dfd3204f6307c7--\r\n
Files are sent by
f = {'file': open(FILE, 'rb')}
requests.post(URL_UPLOAD, files=f)
and received by
class UploadHandler(tornado.web.RequestHandler):
def post(self, filename):
with open(Dir + filename, 'wb') as f:
f.write(self.request.body)
My full code can be seen here
When I send the file by curl with curl -X POST -d $(cat ./1.bin) http://localhost:8080/upload/1.bin I get the correct file, but without \n.
There must be something I missed. Please can someone help me with that? Thank You.

There are two ways to upload files: simply using the file as the request body (usually, but not necessarily, with the HTTP PUT method), or using a multipart wrapper (usually with the HTTP POST method). If you upload the file from an HTML form, it will usually use the multipart wrapper. Your requests example is using a multipart wrapper and the curl one is not; your server is not expecting the wrapper.
To use a multipart wrapper: in requests, pass files= as you've done here. With curl, see this answer: Using curl to upload POST data with files. On the server, use self.request.files instead of self.request.body: http://www.tornadoweb.org/en/stable/httpserver.html#tornado.httpserver.HTTPRequest.files
To not use the multipart wrapper, use data=open(FILE, 'rb').read() from requests, and keep the other two components the same.
It is possible to support both styles simultaneously on the server: use self.requests.files when self.request.headers['Content-Type'] == 'multipart/form-data' and self.request.body otherwise.