Develop Reference

Python & HTTPX: How does httpx client's connection pooling work?

Consider this function that makes a simple GET request to an API endpoint:
import httpx
def check_status_without_session(url : str) -> int:
response = httpx.get(url)
return response.status_code
Running this function will open a new TCP connection every time the function check_status_without_session is called. Now, this section of HTTPX documentation recommends using the Client API while making multiple requests to the same URL. The following function does that:
import httpx
def check_status_with_session(url: str) -> int:
with httpx.Client() as client:
response = client.get(url)
return response.status_code
According to the docs using Client will ensure that:
... a Client instance uses HTTP connection pooling. This means that when you make several requests to the same host, the Client will reuse the underlying TCP connection, instead of recreating one for every single request.
My question is, in the second case, I have wrapped the Client context manager in a function. If I call check_status_with_session multiple times with the same URL, wouldn't that just create a new pool of connections each time the function is called? This implies it's not actually reusing the connections. As the function stack gets destroyed after the execution of the function, the Client object should be destroyed as well, right? Is there any advantage in doing it like this or is there a better way?

Is there any advantage in doing it like this or is there a better way?
No, there is no advantage using httpx.Client in the way you've shown. In fact the httpx.<method> API, e.g. httpx.get, does exactly the same thing!
The "pool" is a feature of the transport manager held by Client, which is HTTPTransport by default. The transport is created at Client initialisation time and stored as the instance property self._transport.
Creating a new Client instance means a new HTTPTransport instance, and transport instances have their own TCP connection pool. By creating a new Client instance each time and using it only once, you get no benefit over using e.g. httpx.get directly.
And that might be OK! Connection pooling is an optimisation over creating a new TCP connection for each request. Your application may not need that optimisation, it may be performant enough already for your needs.
If you are making many requests to the same endpoint in a tight loop, iterating within the context of the loop may net you some throughput gains, e.g.
with httpx.Client(base_url="https://example.com") as client:
results = [client.get(f"/api/resource/{idx}") for idx in range(100)]
For such I/O-heavy workloads you may do even better by executing results in parallel, e.g. using httpx.AsyncClient.

Closing client connection to kubernetes API server in python client

I am using kubernetes-client library in python and looking at the various examples, it appears we don't need to explicitly close the client connection to the API server. Does the client connection gets terminated automatically or are the examples missing the call to close the connection? I also found the docs page for the APIs (AppsV1 for example) and the examples shown there use context manager for the calls so the connection gets disconnected automatically there but I still have questions for the scripts that don't use the context manager approach.

Kubernetes's API is HTTP-based, so you can often get away without explicitly closing a connection. If you have a short script, things should get cleaned up automatically at the end of the script and it's okay to not explicitly close things.
The specific documentation page you link to shows a safe way to do it:
with kubernetes.client.ApiClient(configuration) as api_client:
api_instance = kubernetes.client.AppsV1Api(api_client)
api_instance.create_namespaced_controller_revision(...)
The per-API-version client object is stateless if you pass in an ApiClient to its constructor, so it's safe to create these objects as needed.
The ApiClient class includes an explicit close method, so you could also do this (less safely) without the context-manager syntax:
api_client = kubernetes.client.ApiClient(configuration)
apps_client = kubernetes.client.AppsV1Api(api_client)
...
api_client.close()
The library client front-page README suggests a path that doesn't explicitly create an ApiClient. Looking at one of the generated models' code, if you don't pass an ApiClient option explicitly, a new one will be created for each API-version client object; that includes a connection pool as well. That can leak local memory and cause extra connections to the cluster, but this might not matter to you for small scripts.

grpc client python: How to create grpc client connection pool for better throughput?

Our usecase is to make a large number of requests. Each request return 1 MB of data. Right now, on client side, we create a single GRPC channel and the run the following function in a loop
content_grpc_channel = grpc.insecure_channel(content_netloc)
test_stub = test_pb2_grpc.ContentServiceInternalStub(
content_grpc_channel)
def get_feature_data_future(feature_id, span_context=()):
req_feature = test_pb2.GetFeatureRequest()
req_feature.feature_id = feature_id
resp_feature_future = test_stub.GetFeature.future(
req_feature, metadata=span_context)
return resp_feature_future
My question is in python how I can create grpc client connection pool for better throughput?
In golang I see this https://godoc.org/google.golang.org/api/option#WithGRPCConnectionPool but I have a hard time to find the doc in python.
Is there such a utility in python to create grpc connection pool? Or should I create multiple grpc channels and manage those myself? I assume each channel will have different tcp connection, correct?

gRPC uses HTTP/2 and can multiplex many requests on one connection and gRPC client connections should be re-used for the lifetime of the client app.
The Golang link you mentioned, says that WithGRPCConnectionPool would be used to balance the requests. You might search for load balancing if it is what you need but remember that load balancing only makes sense if you have multiple gRPC server instances.
If you are searching for a connection pool inspired by what is done when working with databases, I would say you don't need to worry about it as the opening connection overhead doesn't exist when working with gRPC

Why are redis pub and sub considered different clients when only one connection is opened?

How come that even when only one instance of Redis connection created, every time I call publish or subscribe on that instance, it counts it like another client. So when I connect to redis using python
import redis
redis_server = redis.Redis()
it does not recognize it as new client. Only when I call one of these
redis_server.publish("channel", message)
redis_server.subscribe("channel")
I can see that there are 2 clients connected. Are the pub/sub clients treated seperately in redis? Why not registering connected client when the new connection is open?

By default redis-py gives you get a connection pool with only a maximum number of connections. On the first command you issue a real connection will be made and you'll see it appear in the CLIENT LIST on the server.
Whenever any client library for Redis issues a subscribe command, that entire connection is occupied by this, so redis-py is probably creating a separate connection dedicated to this.
This should explain why you see no clients connected, then 2. It's not necessarily 1 connection for every command issued as the connections in the pool will be reused.

Python - Multiple client servers for scaling

For my current setup, I have a single client server using Tornado, a standalone database server and another standalone server for my website.
I'm looking at having a second client server process running on the same system (to take advantage of its multiple cores) and I would like some advice in locating which server my "clients" have connected to. Each client can have multiple connections (instances).
I've already looked at using memcached to hold a list of user identifiers and link them to which server(s) they are connected to, but that doesn't seem like it would scale very well (eg having six digits of connected users).
I see the same issue with database lookups.
I have already optimized my server as much as possible, without going into micro-optimization and I personally frown upon that.
Current server methodology:
On connect:
Accept connection, rate limit for max connections per IP.
Append client instance to a list named "clientList".
On data from client:
Rate limit for max messages per second.
Append data to a client work queue.
If client has a thread dedicated toward its work queue:
return, its work will be chewed by the current thread
otherwise create a new thread for this users work queue, start it.
TLDR:
How do I efficiently store which servers a client has connected to in order to forward messages to that user.