The Tornado RequestHandler class has add_header(), clear_header(), and set_header() methods. Is there a way to just see the headers that are currently set?
My use case is that I am writing some utility methods to automatically set response headers under certain conditions. But I want to add some error checking in order to not add duplicates of a header that I do not want to have duplicated.
I want to write come code that is more or less like this:
class MyHandler(tornado.web.RequestHandler):
def ensure_json_header(self):
if not self.has_header_with_key('Content-Type'):
self.set_header('Content-Type', 'application/json')
def finish_json(self, data):
self.ensure_json_header()
return self.finish(json.dumps(data))
But of course there is no has_header_with_key() method in Tornado. How can I accomplish this?
EDIT: this turned out to be an X-Y question. The real answer was to just use set_header instead of add_header. I am leaving this up for anyone else who might come along with a similar question.
There's no documented api for listing the headers present in a response.
But there is a self._headers private attribute (an instance of tornado.httputil.HTTPHeaders) which is basically a dict of all headers in the response. You can do this to check a header:
if 'Content-Type' in self._headers:
# do something
As an addendum, if you want to access all headers of a request, you can do self.request.headers.
Edit: I've opened an issue about this on github after seeing your question; let's see what happens.
Tornado will always have the Content-Type header set as it is in the default headers (https://www.tornadoweb.org/en/stable/_modules/tornado/web.html#RequestHandler.clear). So if you want to ensure you have a specific content type set, just call set_header.
If you want to check that the response does not have a header set in your code, you’ll have to first reset the default header, which you can do by implementing set_default_headers and do a clear_header(“Content-Type”) there.
But you could also achieve the same by setting a property on your handler (say override_content_type), set that in code and then do a non conditional set_header before rendering the result.
Related
I'm using Python with the Django Rest framework and am trying to receive webhook events correctly from stripe.
However I constantly get this error:
stripe.error.SignatureVerificationError: No signatures found matching the expected signature for payload
This is the code:
WEBHOOK_SECRET = settings.STRIPE_WEBHOOK_SK
#csrf_exempt
def webhook(request):
sig_header = request.headers.get('Stripe-Signature', None)
payload = request.body
try:
event = stripe.Webhook.construct_event(
payload=payload,
sig_header=sig_header,
secret=WEBHOOK_SECRET
)
except ValueError as e:
raise e
except stripe.error.SignatureVerificationError as e:
raise e
return HttpResponse(status=200)
I have also tried modifying the request body format like so:
payload = request.body.decode('utf-8')
# and also
payload = json.loads(request.body)
And yet no luck.
The error is coming from the verify_header() class method inside the WebhookSignature class.
This is the part of the method where it fails:
if not any(util.secure_compare(expected_sig, s) for s in signatures):
raise error.SignatureVerificationError(
"No signatures found matching the expected signature for payload",
header,
payload,
)
So I printed out exptected_sig and signatures before this line and found that regardless of what format request.body is in, signatures is always there (which is good), but they never match the signature from the header.
Why is this?
When Stripe calculates the signature for the Event it sends you, it uses a specific "payload" representing the entire Event's content. The signature is done on that exact payload and any change to it such as adding a new line, removing a space or changing the order of the properties will change the payload and the corresponding signature.
When you verify the signature, you need to make sure that you pass the exact raw payload that Stripe sent you, otherwise the signature you calculate won't match the Stripe one.
Frameworks can sometimes try to be helpful when receiving a request and they detect JSON and automatically parse it for you. This means that you think you are getting the "raw payload/body" but really you get an alternate version. It has the same content but it doesn't match what Stripe sent you.
This is fairly common with Express in Node.js for example. So, as the developer, you have to explicitly request the exact raw/original payload Stripe sent you. And how to do this can differ based on a variety of factors. There are 2 issues on the stripe-node github with numerous potential fixes here and here.
With Django, the same can happen and you need to make sure that your code requests the raw payload. You seem to use request.body as expected but that's one thing you want to dig into further.
Additionally, another common mistake is using the wrong Webhook secret. If you use the Stripe CLI for example, it creates a new secret for you that is different from the one you see in the Dashboard for this Webhook Endpoint. You need to make sure you use the correct secret based on the environment you're in.
Using the HTTPretty library for Python, I can create mock HTTP responses for my unit tests. When the code I am testing runs, instead of my request reaching the third party, the request is intercepted and my code receives the response I configured.
I then use last_request() and can check the url my code requested, any parameters, etc.
What I would like is to know how can I access not just the last request but also any other requests my code sent before the last one.
This seems to be possible. In the documentation it uses a list called latest_requests. For example here
But that doesn't seem to work for me. I get an AttributeError AttributeError: module 'httpretty' has no attribute 'latest_requests'
Here is some code that illustrates what I am trying to do and where I get AttributeError
import httpretty
import requests
httpretty.enable()
httpretty.register_uri(
method=httpretty.GET,
uri='http://www.firsturl.com',
status=200,
body='First Body'
)
httpretty.enable()
httpretty.register_uri(
method=httpretty.GET,
uri='http://www.secondurl.com',
status=200,
body='secondBody'
)
firstresponse = requests.get('http://www.firsturl.com')
secondresponse = requests.get('http://www.secondurl.com')
print(httpretty.latest_requests[-1].url)
# clean up
httpretty.disable()
httpretty.reset()
Thanks!!
Unfortunately, after reading the docs and attempting to get your code working, I can only describe the documentation as blatantly incorrect. There appear to be three | separate | pull requests from several years ago that claim to make httpretty.latest_requests a real attribute but none of them have merged in for whatever reason.
With all of that said, I managed to get the list of all previous requests by calling
httpretty.HTTPretty.latest_requests
This returns a list of HTTPrettyRequest objects. Seeing as httpretty.last_request() returns an HTTPrettyRequest object, that attribute is probably what you're looking for.
Unfortunately, .url is not defined on that class (but it is defined on the blank request object which doesn't make any sense). If you want to check that the request URL is what you're expecting, you pretty much have to try reconstructing it yourself:
req = httpretty.HTTPretty.latest_requests[-1]
url = req.headers.get('Host', '') + req.path
If you're passing anything in the query string, you'll have to reconstruct that from req.querystring although that's not ordered so you probably don't want to turn that into a string for matching purposes. Also, if all of your requests are going to the same domain, you can leave off the host part and just compare req.path.
I'm trying to implement a batch request method in pyramid. I see in the docs that it's done with
subrequest = Request.blank('/relative/url')
request.invoke_subrequest(subrequest)
I'm just wondering how do you pass along the headers and cookies? Is it already done for you or is it
request.invoke_subrequest(subrequest, cookies=request.cookies, headers=request.headers)
What about parameters for different methods? The docs only have a POST keyword arg.
I feel like the docs are a little vague, or I can't find the correct docs on how to do this. Thanks
I'm just wondering how do you pass along the headers and cookies?
From http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/subrequest.html#subrequest-chapter :
The pyramid.request.Request.invoke_subrequest() API accepts two
arguments: a positional argument request that must be provided, and
use_tweens keyword argument that is optional; it defaults to False.
This tells us that the constructor only wants a Request object, and optionally a value for use_tweens, so no, this
request.invoke_subrequest(subrequest, cookies=request.cookies, headers=request.headers)
will not work.
Then, from http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/subrequest.html
It's a poor idea to use the original request object as an argument to
invoke_subrequest(). You should construct a new request instead as
demonstrated in the above example, using
pyramid.request.Request.blank(). Once you've constructed a request
object, you'll need to massage it to match the view callable you'd
like to be executed during the subrequest. This can be done by
adjusting the subrequest's URL, its headers, its request method, and
other attributes. The documentation for pyramid.request.Request
exposes the methods you should call and attributes you should set on
the request you create to massage it into something that will actually
match the view you'd like to call via a subrequest.
So basically, you need to configure your request before you pass it to invoke_subrequest().
Luckily there is an entire page that documents the Request class. There we can find a whole lot options to configure it, etc.
What about parameters for different methods? The docs only have a POST keyword arg.
Also on the Request class documentation page, there is this
method
Gets and sets the REQUEST_METHOD key in the environment.
And on http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/narr/viewconfig.html I found
request_method This value can be a string (typically "GET", "POST",
"PUT", "DELETE", or "HEAD") representing an HTTP REQUEST_METHOD
I must agree with you that the documentation is a little vague here and there, but I assume you can use it like this
request.method = 'POST'
# Or
request.method = 'GET'
# Etc.
Summary
You'll want to do it like this
subrequest = Request.blank('/relative/url')
# Configure the subrequest object
# set headers and other options you need.
request.invoke_subrequest(subrequest)
Note
I am aware this is not a 100% complete answer with some code that you can copy paste and it'll just work (especially regarding configuring the request object), but I think this answer contains some information that, at the very least, will get you on the right track and I hope it will be of some help to you.
The following code worked for me. It copies all (headers, cookies, query string, post parameters, etc.):
def subrequest(request, path):
subreq = request.copy()
subreq.path_info = path
response = request.invoke_subrequest(subreq)
return response
Somewhat late, but based on the above two answers here is how I did this. I didn't quite like the above answer to just copy everything. Looking at the documentation of the blank() method there is a kw argument and it says
All necessary keys will be added to the environ, but the values you pass in will take precedence. If you pass in base_url then wsgi.url_scheme, HTTP_HOST, and SCRIPT_NAME will be filled in from that value.
Assuming that the view's request contains the right header information and cookies that you need for your subrequest, you can use the following code:
#view_config( ... )
def something(request):
...
kwargs = { "cookies": request.cookies,
"host" : request.host,
}
req = Request.blank("/relative/url", **kwargs)
resp = request.invoke_subrequest(req)
Other header information (e.g. accept, accept_encoding, etc.) are properties of pyramid.request objects, and can be added to the kwargs dictionary like shown in the code snippet above.
The object returned by invoke_subrequest() is a Response object documented here.
I've written a program in Python, which works with two distinct API to get the data from two different services (CKAN and MediaWiki).
In particular, there is a class Resource, which requests the data from the above mentioned services and process it.
At some point I've come to conclusion, that there is a need for tests for my app.
And the problem is that all examples I've found on web and in books do not deal with such cases.
For example, inside Resource class I've got a method:
def load_from_ckan(self):
"""
Get the resource
specified by self.id
from config.ckan_api_url
"""
data = json.dumps({'id': self.id})
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
url = config.ckan_api_url + '/action/resource_show'
r = requests.post(url, timeout=config.ckan_request_timeout, data=data, headers=headers)
assert r.ok, r
resource = json.loads(r.content)
resource = resource["result"]
for key in resource:
setattr(self, key, resource[key])
The load_from_ckan method get the data about resource from the CKAN API and assign it to the object. It is simple, but...
My question is: how to test the methods like this? OR What should I test here?
I thought about the possibility to pickle (save) results to HDD. Then I could load it in the test and compare with the object initialized with load_from_ckan(). But CKAN is community-driven platform and such behavior of such tests will be unpredictable.
If there exist any books on philosophy of automated tests (like what to test, what not to test, how to make tests meaningful etc.), please, give me a link to it.
With any testing, the key question really is - what could go wrong?
In your case, it looks like the three risks are:
The web API in question could stop working. But you check for this already, with assert r.ok.
You, or someone else, could make a mistaken change to the code in future (e.g. mistyping a variable) which breaks it.
The API could change, so that it no longer returns the fields or the format you need.
It feels like you could write a fairly simple test for the latter two, depending on what data from this API you actually rely on: for example, if you're expecting the JSON to have a field called "temperature" which is a floating-point Celsius number, you could write a test which calls your function, then checks that self.temperature is an instance of 'float' and is within a sensible range of values (-30 to 50?). That should leave you confident that both the API and your function are working as designed.
Typically if you want to test against some external service like this you will need to use a mock/dummy object to fake the api of the external service. This must be configurable at run-time either via the method's arguments or the class's constructor or another type of indirection. Another more complex option would be to monkey patch globals during testing, like "import requests; request.post = fake_post", but that can create more problems.
So for example your method could take an argument like so:
def load_from_ckan(self, post=requests.post):
# ...
r = post(url, timeout=config.ckan_request_timeout, data=data,
headers=headers)
# ...
Then during testing your would write your own post function that returned json results you'd see coming back from ckan. For example:
def mock_post(url, timeout=30, data='', headers=None):
# ... probably check input arguments
class DummyResponse:
pass
r = DummyResponse()
r.ok = True
r.content = json.dumps({'result': {'attr1': 1, 'attr2': 2}})
return r
Constructing the result in your test gives you a lot more flexibility than pickling results and returning them because you can fabricate error conditions or focus in on specific formats your code might not expect but you know could exist.
Overall you can see how complicated this could become so I would only start adding this sort of testing if you are experiencing repeated errors or other difficulties. This will just more code you have to maintain.
At this point, you can test that the response from CKAN is properly parsed. So you can pull the JSON from CKAN and ensure that it's returning data with the attributes you're interested in.
class ...
self.response.headers.add_header('Set-Cookie','user_id = %s'% id_and_hash)
self.redirect("/app/login")
class ThanksHandler(webapp2.RequestHandler):
def get(self):
if 'user_id' in self.request.cookies:
self.response.out.write('user_id')
else:
self.response.out.write("wrong") // what i am getting
and i have this cookie (I saw in a manager of cookies)
name: user_id
content: 30|a9bdc98e952c0e787aaf0e5466809eea71635d38446d30a9f71f2d15e99fa701
well, basically the problem is that i can't retrieve the cookie that has been initialized. Any reason for that?
When setting the cookie directly using the "Set-Cookie" HTTP-header you have to also take care of the cookie-attributes (depending on how you want to use the cookie).
In this case you set the cookie for example in the path /app/auth and then redirect to /app/login. If you don't explicit specify the cookie-path the browser assumes /app/auth and therefore the cookie is not sent when requesting /app/login. This behaviour is specified in the RFC 6265 (this is more recent than the one I cited in my comment but now the exact algorithm is included in section 5.1.4).
To set the path (or any other cookie-attribute) you can append a list of semicolon-delimited name=value pairs. In this case you want to set the path to / (it could be something different like /app):
self.response.headers.add_header('Set-Cookie','user_id=%s; Path=/'% id_and_hash)
Of course most libraries/frameworks already provide a wrapper for the "Set-Cookie" header. In the case of "webapp2" you can set the cookie with response.set_cookie:
self.response.set_cookie('user_id', id_and_hash)
It's automatically setting the path to / so you don't have to worry about it (it does escape the values properly too).