Apologies if this is a repost/stupid question, I have tried searching around but found nothing.
I have a cgi webserver that takes a POST payload that is neither percentage encoded or form data. i.e, I need to stop the request from being percentage encoded, as with:
req = Request('POST', license_url, data={ None: data})
And I also need the request to not include the multiform boundaries introduced by:
req = Request('POST', license_url, file={ None: file_handle})
My request body looks something like:
abc
def
ghi=
And this is exactly what I want to post to the server (Postman achieves this when set to raw payload).
However, sending through the data param looks like:
data=abc%3Adef%3a..
And through the file Param:
--b70cc8bd5ac1411488c719a1d773edde
Content-Disposition: form-data; filename="blahblah"
abc
def
ghi=
--b70cc8bd5ac1411488c719a1d773edde
^^^ This is the closest to what I need but I also want to strip the boundary/content-disposition terms (not on the server).
Many Thanks
You can just post the raw text content like
req = Request('POST', license_url, data=data.encode())
provided the data is a string.
Related
My API works fine and I see a 200 status when I test it using Postman. However I'm trying to access it using a Kivy application but I'm seeing a 400 response from the server after some waiting or quitting the app. By the way when testing with Postman I specify header as Content-Type: application/json and in body I see my parameters
{
"search_text": "Hello",
"num_results": 1
}
being sent as raw data.
My code
def search(self, search_text):
header = {'Content-Type':'application/json'}
req = UrlRequest('http://127.0.0.1:5000/search',req_body={"search_text": search_text,"num_results": 1},on_success=Test.got_json,req_headers=header)
print("Search method called")
#staticmethod
def got_json(req,result):
print(result)
Kivy docs say that you don't have to specify a method as this would send a POST request so I've not specified that here
Edit: The code for the server is kind of irrelevant for my issue here so I've removed it
UrlRequest should be passed a str object as request body. You can serialize the request dictionary as a string object by dumping it. Pass this dumped dictionary as the request body to UrlRequest.
import json
req_body=json.dumps({'search_text': search_text, 'num_results': 1})
req = UrlRequest(
'http://127.0.0.1:5000/search',
req_body=req_body,
on_success=Test.got_json,
req_headers=header)
req_body is a string parameter, might be a bit confusing as req_headers is a dict. You can use:
req_body=json.dumps({"search_text": search_text,"num_results": 1})
There is a POST request which works perfectly when I pass the data as below:
url = 'https://www.nnnow.com/api/product/details'
requests.post(url, data="{\"styleId\":\"BMHSUR2HTS\"}", headers=headers)
But when I use json.dumps() on a dictionary and send the response, I do not get the response (response code 504), using headers={'Content-Type': 'application/json'} . Have also tried json parameter of Post requests.
requests.post(url, data=json.dumps({"styleId":"BMHSUR2HTS"}), headers={'content-type': 'application/json'})
Now, the data returned by json.dumps({"styleId":"BMHSUR2HTS"}) and
"{\"styleId\":\"BMHSUR2HTS\"}" is not the same.
json.dumps({"styleId":"BMHSUR2HTS"}) == "{\"styleId\":\"BMHSUR2HTS\"}" gives False even though a print on both shows a similar string.
How can I get the same format as "{\"styleId\":\"BMHSUR2HTS\"}" from a dictionary {"styleId":"BMHSUR2HTS"} ?
If you print the json.dumps({"styleId":"BMHSUR2HTS"}), you will notice two things:
your output is a string (just try type(json.dumps({"styleId":"BMHSUR2HTS"})));
if you pay attention the output will add a space between the json name and value: {"styleId": "BMHSURT2HTS"}.
Not sure how do you want to handle this, and in your entry code, but there are 2 main options to workaround this issue:
Replace the space on json.dumps output: json.dumps({"styleId":"BMHSUR2HTS"}).replace(': ', ':')
Convert all to json by using eval(): eval(json.dumps({"styleId":"BMHSUR2HTS"})) and eval(YOUR_JSON_STRING)
I hope this helps you.
In my application, I'm rendering a PDF file and pass it back as a response. For this purpose I'm using flask_weasyprint's render_pdf, which does exactly this:
def render_pdf(html, stylesheets=None, download_filename=None):
if not hasattr(html, 'write_pdf'):
html = HTML(html)
pdf = html.write_pdf(stylesheets=stylesheets)
response = current_app.response_class(pdf, mimetype='application/pdf')
if download_filename:
response.headers.add('Content-Disposition', 'attachment', filename=download_filename)
return response
I now need to render a template + returning the rendered pdf as a download. Something like
#app.route("/view")
def view() :
resp1 = render_pdf(HTML(string="<p>Render me!</p>"), download_filename = "test.pdf")
resp2 = render_template("test.html")
return resp1, resp2
Is there any way to achieve this? Any workaround?
I am not sure if this is solvable in the backend, you want to send two http responses following one request. Should that be possible? (I really don't know) Shouldn't the client make two responses? (javascript).
An option would be, javascript datablob returned in your render_template call.
Maybe something like this? (untested):
fileData = new Blob([pdf_data_here], { type: 'application/pdf' });
fileUrl = URL.createObjectURL(fileData);
window.location.assign(fileUrl);
Or maybe just use the window.location.assign() function to generate the second request.
Or put the data base64 encoded in a href attribute?
so I have been writing a simple web server in Python, and right now I'm trying to handle multipart/form-data POST requests. I can already handle application/x-www-form-urlencoded POST requests, but the same code won't work for the multipart. If it looks like I am misunderstanding anything, please call me out, even if it's something minor. Also if you guys have any advice on making my code better please let me know as well :) Thanks!
When the request comes in, I first parse it, and split it into a dictionary of headers and a string for the body of the request. I use those to then construct a FieldStorage form, which I can then treat like a dictionary to pull the data out:
requestInfo = ''
while requestInfo[-4:] != '\r\n\r\n':
requestInfo += conn.recv(1)
requestSplit = requestInfo.split('\r\n')[0].split(' ')
requestType = requestSplit[0]
url = urlparse.urlparse(requestSplit[1])
path = url[2] # Grab Path
if requestType == "POST":
headers, body = parse_post(conn, requestInfo)
print "!!!Request!!! " + requestInfo
print "!!!Body!!! " + body
form = cgi.FieldStorage(headers = headers, fp = StringIO(body), environ = {'REQUEST_METHOD':'POST'}, keep_blank_values=1)
Here's my parse_post method:
def parse_post(conn, headers_string):
headers = {}
headers_list = headers_string.split('\r\n')
for i in range(1,len(headers_list)-2):
header = headers_list[i].split(': ', 1)
headers[header[0]] = header[1]
content_length = int(headers['Content-Length'])
content = conn.recv(content_length)
# Parse Content differently if it's a multipart request??
return headers, content
So for an x-www-form-urlencoded POST request, I can treat FieldStorage form like a dictionary, and if I call, for example:
firstname = args['firstname'].value
print firstname
It will work. However, if I instead send a multipart POST request, it ends up printing nothing.
This is the body of the x-www-form-urlencoded request:
firstname=TEST&lastname=rwar
This is the body of the multipart request:
--070f6a3146974d399d97c85dcf93ed44
Content-Disposition: form-data; name="lastname"; filename="lastname"
rwar
--070f6a3146974d399d97c85dcf93ed44
Content-Disposition: form-data; name="firstname"; filename="firstname"
TEST
--070f6a3146974d399d97c85dcf93ed44--
So here's the question, should I manually parse the body for the data in parse_post if it's a multipart request?
Or is there a method that I need/can use to parse the multipart body?
Or am I doing this wrong completely?
Thanks again, I know it's a long read but I wanted to make sure my question was comprehensive
So I solved my problem, but in a totally hacky way.
Ended up manually parsing the body of the request, here's the code I wrote:
if("multipart/form-data" in headers["Content-Type"]):
data_list = []
content_list = content.split("\r\n\r\n")
for i in range(len(content_list) - 1):
data_list.append("")
data_list[0] += content_list[0].split("name=")[1].split(";")[0].replace('"','') + "="
for i,c in enumerate(content_list[1:-1]):
key = c.split("name=")[1].split(";")[0].replace('"','')
data_list[i+1] += key + "="
value = c.split("\r\n")
data_list[i] += value[0]
data_list[-1] += content_list[-1].split("\r\n")[0]
content = "&".join(data_list)
If anybody can still solve my problem without having to manually parse the body, please let me know!
There's the streaming-form-data project that provides a Python parser to parse data that's multipart/form-data encoded. It's intended to allow parsing data in chunks, but since there's no chunk size enforced, you could just pass your entire input at once and it should do the job. It should be installable via pip install streaming_form_data.
Here's the source code - https://github.com/siddhantgoel/streaming-form-data
Documentation - https://streaming-form-data.readthedocs.io/en/latest/
Disclaimer: I'm the author. Of course, please create an issue in case you run into a bug. :)
I'm trying to make a POST request to a server. However, when making this post, the data gets messed up somewhere along the way.
My Code:
headers = {"Context-Type" : "application/x-www-form-urlencoded"
"Authorization" : "Basic user pass"
values = {"query" : "select", "table" : "testtable"}
data = urllib.urlencode(values)
request = urllib2.request(url, data, headers = headers)
res = urllib2.urlopen(request)
print res.result()
However, I noticed that the "data" is somehow changed. It should (and does when I print it) look something like
query=select&table=testtable
However, when I actually do a post request, this site registers:
<parameter id="

query">select</parameter>
<parameter id="table">testtab</parameter>
So it looks like the data is somehow shifted over 2 spaces. This is indepenedent of where I do the post request. Anyone ever have an error like this?
It sounds like your original data has a carriage return ('\r') and line feed ('\n') in it (the '
' and '
'). So, it might look like your key name is 'query', but you probably have some extra characters hidden in there.
Your site should use urlparse.parse_qs to read the request data. By default, that function will ignore those two characters:
>>> import urlparse
>>> data = '
query=select&table=testtable'
>>> urlparse.parse_qs(data)
'query': ['select'], 'table': ['testtable']}
>>> # make the parsing strict:
... urlparse.parse_qs(data, strict_parsing=True)
ValueError: bad query field: ''
So, those two characters are ignored if you use the default parameters with urlparse.parse_qs. Best solution is to check your input and get rid of those characters.