Develop Reference

Python Uvicorn – obtain SSL certificate information

I have a gunicorn + uvicorn + fastApi stack.
(Basically, I am using https://hub.docker.com/r/tiangolo/uvicorn-gunicorn-fastapi docker image).
I've already implemented SSL based authentication by providing appropriate gunicorn configuration options: certfile, keyfile, ca_certs, cert_reqs.
And it works fine: user have to provide a client SSL certificate in order to be able to make an API calls.
What I need to do now is to obtain client certificate data and pass it further (add it to request headers) into my application, since it contains some client credentials.
For example, I've found a way to do it using gunicorn worker by overrding gunicorn.workers.sync.SyncWorker: https://gist.github.com/jmvrbanac/089540b255d6b40ca555c8e7ee484c13.
But is there a way to do the same thing using UvicornWorker? I've tried to look through the UvicornWorker's source code, but didn't find a way to do it.
I went deeper into the Uvicorn source code, and as far as I understand, in order to access the client TLS certificate data, I need to do some tricks with python asyncio library (https://docs.python.org/3/library/asyncio-eventloop.html), possibly with Server (https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Server) class and override some of the UvicornWorker's methods.
I am still not quite sure if it is possible to achieve the desired result though.

I ended up setting the nginx (Openresty) in front of my server and added a script to get a client certificate and put it into header.
Here is a part of my nginx config:
set_by_lua_block $client_cert {
local client_certificate = ngx.var.ssl_client_raw_cert
if (client_certificate ~= nil) then
client_certificate = string.gsub(client_certificate, "\n", "")
ngx.req.set_header("X-CLIENT-ID", client_certificate)
end
return client_certificate
}
It is also possible to extract some specific field from a client certificate (like CN, serial number etc.) directly inside nginx configuration, but I decided to pass the whole certificate further.
My problem is solved without using gunicorn as I originally wanted though, but this is the only good solution I've found so far.

How to PATCH Django Test Server with cURL

In my API, I have the following dynamics:
POST a processing request to the API.
Do some processing in the background.
When the processing is done, a computer will PATCH the API's original request status field with cURL.
These steps do work when I test them with a normal server, i.e., python manage.py runserver. However, when I try to automate tests within Django, I get:
curl: (7) Failed to connect to 127.0.0.1 port 80: Connection refused
Port 80 is what is specified under the django.test.client module with 'SERVER_PORT': '80', so I really don't get why that wouldn't work.

I don't know if this can be considered a full answer yet because I haven't tested it enough yet. But, apparently, Django's TestCase dummy server is somewhat simple for more specific cases. You could instead use LiveServerTestCase for something more robust. (If you want more info, check out this StackOverflow answer.)
Within the LiveServerTestCase class, you will find the self.live_server_url attribute, which will have the address of where the server actually is. The port assigned to it will be somewhat random apparently due to security reasons. Then, finally, somehow, you can manage to pass the info into your PATCH operation.

Using twisted to selectively reverse proxy to different servers

I'm using Twisted (well twistd actually) to serve content like this currently :
twistd -n -o web --path=./foo/
That's fine but I want to send some requests to another server - like this.
When the client requests
localhost/something.html
I want the request to be handled by the twistd server .
But when the client requests
localhost/api/somedata
I want the request to be reverse proxied to another server .
So in summary if the URL contains the string "api" then I want the request reverse proxied elsewhere.
I can see that Twisted has a built in Reverse Proxy but I don't know how to use that so that I can filter requests made in such a way that some requests would get sent off to the alternative server and some wouldn't.

ReverseProxyResource is a resource. You can place it into a resource hierarchy.
root = Resource()
root.putChild("something.html", SomethingHTML())
root.putChild("api", ReverseProxyResource(...))
This is just one example of an arrangement of the resource hierarchy. You can combine ReverseProxyResource with other resources in any of the ways supported by IResource.

How to make very simple http proxy using werkzeug or other python requests framework?

Is it possible to setup a listener on say port 9090 and add a header, like Host: test.host to each request incoming on 9090 and send it on to say 8080?
Thanks
EDIT: I went with a reverse-proxy for now, applying the hostname:port to any request that comes in.

Twisted has an implementation of a reverse proxy that you could modify to suit your needs. You can look at the examples here. If you look at the source code of twisted.web.proxy, you can see that the 'Host:' header is set in ReverseProxyRequest.process, so you could subclass it and set your own header.

Unless you need to tailor the proxied request based on parameters that only your web application can know (for example, you need to authenticate the proxied request with your webapp's custom authentication system), you should use your web server's proxy capabilities.
Example with Apache:
Listen 0.0.0.0:9090
ProxyRequests off
<VirtualHost myhost:9090>
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
ProxyPassReverseCookieDomain localhost myhost
</VirtualHost>
If you have to proxy things in a Flask or Werkzeug application, you can use httplib, creating requests based on the incoming request data and returning the response, either raw or modified (eg for link rewriting). It's doable, I have one such proxy in use where there was no good alternative. If you do that I recommend against using regular expressions to rewrite HTML links. I used PyQuery instead, it's far easier to get it right.

Routes in bottle which proxy to another server

I have a bottle.py application which has a number of routes already built. I would like to create a new get route which, when accessed, passes the request along to another HTTP server and relays the result back.
What is the simplest way to get that done?

In principle, all you need is to install the wsgiproxy module and do this:
import bottle
from wsgiproxy.app import WSGIProxyApp
root = bottle.Bottle()
proxy_app = WSGIProxyApp("http://localhost/")
root.mount(proxy_app,"/proxytest")
Running this app will then proxy all requests under /proxytest to the server running on localhost:80. In practice, I found this didn't work without taking extra steps to remove hop-by-hop headers. I took the code in this gist and stripped it down to make a simple app that successfully proxies the request.