I am using the Tornado Web Server (version 4.1) with Python 2.7 to create a REST web application. One of my request handlers (web.RequestHandler) handles batch requests consisting of multiple HTTP requests combined into one HTTP request using the multipart/mixed content type. I currently have the batch request handler able to receive the POST request and parse the multipart/mixed content into individual requests that look like this:
GET /contacts/3 HTTP/1.1
Accept: application/json
My question is, what would be a good way of converting these inner batched requests into requests that Tornado can service from within my request handler? I would like to collect the responses within my batch request handler and, once these requests are all complete, return a single multipart/mixed response containing all the batched responses.
Using an HTTPClient to execute the batched requests feels like overkill. It seems like I should be able to build a request object and inject it into the web.Application for processing--I'm at a loss as to how to do this however. Thanks!
Tornado doesn't have any direct support for this. Going through an HTTP client is probably going to be the simplest solution. However, if you're really interested in avoiding that route, here's a sketch of a solution, which relies on the interfaces defined in the tornado.httputil module.
Define a class that implements the HTTPConnection interface by saving the arguments to write and write_headers into internal buffers.
The Application is an HTTPServerConnectionDelegate. Call its start_request method with an instance of your connection class as both arguments (the first argument doesn't really matter, but it should be unique and since we won't be reusing "connections" that object is fine).
start_request returns an HTTPMessageDelegate. Call its headers_received, data_received (for POST/PUT), and finish methods to make your request. Once you have called finish, the handler will run and make calls back into your connection object.
Related
I find this code and I really don't understand it, how is it possible to send data (not query) with GET request
response = requests.get(
check_all_info_url_2, files=multipart_form_data, timeout=30)
and what is files= argument in the get request.
Since requests.get is just a wrapper function this will just call requests.request. Unless requests.session implementes any checking, it will happily send off a GET request with multipart data in it.
Is this valid? Not to my knowledge, although I'm willing to be proven wrong. No api I have ever written would accept file upload on a GET request. But not every server will even check the method, so perhaps this code is interacting with a badly written server which doesn't reject for wrong method, or perhap's it's even interacting with a worse server which expects file upload with GET. There are lots of broken servers out there ;)
In any case, the reason this works with requests is that it just passes keyword arguments through to the underlying session without performing any kind of validation.
So, I want to write this Python code that wants to assess data in requests I am getting and act accordingly. I have many different endpoints and I'd like a way to access the request data for all requests without creating manually every possible endpoint.
Is there a way to do it with Flask/Bottle? A proxy of sorts
You can register a function to be called before every request with the #app.before_request decorator:
#app.before_request
def handle_every_request():
# ...
If this function returns anything other than None, it'll be used as the response and no views will be called. This would let you create any number of routing options.
Another option is to use the request_started signal, see the Signals documentation:
from flask import request_started
#request_started.connect_via(app)
def handle_every_request(sender, **kwargs):
# ...
The above listens to requests for the specific Flask app instance. Use #request_started.connect if you want to listen to all requests for all apps. Signals just listen, they don't route, however.
In general, both Flask and Bottle are WSGI applications. WSGI supports wrapping such applications in WSGI middleware, letting you inspect every ingoing and outgoing byte of a request-response pair. This gives you access to the data at a lower level. You could use this to rewrite paths being requested however.
Can I send a http response to a specific requestLine outside the function I received the request in it.
As an example I receive the request then pass this request to some functions and I want to send a response to this request, if there is allowed?
If you're going to split the path your program is taking, so it's doing two different things at the same time, then you'll need to utilize control-flow constructs that do this like threads/processes/events/async.
You might want to look at Celery (http://www.celeryproject.org/) and Jobtastic (http://policystat.github.io/jobtastic/).
I want to write a twisted proxy that splits up very large GET request into smaller fixed size ranges and sends it on to another proxy (using the Range: bytes). The other proxy doesn't allow large responses and when the response is to large it returns a 502.
How can I implement a proxy in twisted that on a 502 error it tries splitting the request into smaller allowed chunks. The documentation is hard to follow. I know I need to extend ProxyRequest, but from there I'm a bit stuck.
It doesn't have to be a twisted proxy, but it seems to be easily modified and I managed to at least get it to forward the request unmodified to the proxy by just setting the connectTCP to my proxy (in ProxyRequest.parsed).
Extending ProxyRequest is probably not the easiest way to do this, actually; ProxyRequest pretty strongly assumes that one request = one response, whereas here you want to split up a single request into multiple requests.
Easier would be to simply write a Resource implementation that does what you want, which briefly would be:
in render_GET, construct a URL to make several outgoing requests using Agent
return NOT_DONE_YET
as each response comes in, call request.write on your original incoming requests, and then issue a new request with a Range header
finally when the last response comes in, call request.finish on your original request
You can simply construct a Site object with your Resource, and set isLeaf on your Resource to true so your Resource doesn't have to implement any traversal logic and can just build the URL using request.prePathURL and request.postpath. (request.postpath is sadly undocumented; it's a list of the not-yet-traversed path segments in the request).
I need to call GET, POST, PUT, etc. requests to another URI because of search, but I cannot find a way to do that internally with pyramid. Is there any way to do it at the moment?
Simply use the existing python libraries for calling other webservers.
On python 2.x, use urllib2, for python 3.x, use urllib.request instead. Alternatively, you could install requests.
Do note that calling external sites from your server while serving a request yourself could mean your visitors end up waiting for a 3rd-party web server that stopped responding. Make sure you set decent time outs.
pyramid uses webob which has a client api as of version 1.2
from webob import Request
r = Request.blank("http://google.com")
response = r.send()
generally anything you want to override for the request you would just pass in as a parameter.
from webob import Request
r = Request.blank("http://facebook.com",method="DELETE")
another handy feature is that you can see the request as the http that is passed over the wire
print r
DELETE HTTP/1.0
Host: facebook.com:80
docs
Also check the response status code: response.status_int
I use it for example, to introspect my internal URIs and see whether or not a given relative URI is really served by the framework (example to generate breadcrumbs and make intermediate paths as links only if there are pages behind)