We are writing an application that leverage gRPC. We implemented a bunch of Unary RPCs that clients can regularly call, getting the expected response.
Also, we have a streaming (from server to client) dedicated to continuously streams some information that are shared across all the clients (let's say some hardware local sensors information – such as room temperatures).
The Python StatusStreaming function now returns the streaming iterator and it loops constantly to stream data to the clients.
The problem is that it returns a new iterator for every client; being it updated every 100ms, that makes our system not able to serve more then a dozen clients before reaching full load. Also, it doesn't make sense to use a separate cycle for every clients, as we need to send the same exact content to all the clients.
Is it actually possible to broadcast the same streaming content to many clients without having different connection/channel for each of them?
Some background
We use gRPC-web to implement the RPCs on browsers clients, so bidi streaming is not possible (I don't have an idea how to use it anyhow, but it wouldn't be availble).
Versions uses are those tied to grpc-web (grpc version grpc # d8772cf and protobuf at 3.7.0).
Our server, is limited in resources, as we are on a small embedded server (1 core, 1gb ram).
Here is the streamer
# Inside a method of the service class here
def _Status(self):
system_controller = self._system_status()
return protos.Stream(temperature_controller)
# Real Python streaming method
def Status(self, request, context):
def streaming_iterator():
while(context.is_active()):
time.sleep(0.5)
yield self._Status()
return streaming_iterator()
A new connection is created (and also the dedicated streaming channel) for every client and so a new while cycle spawns generating data to send. This behaviour isn't needed, because the same information can be actually shared across all the clients in broadcasting fashion.
Related
For my current setup, I have a single client server using Tornado, a standalone database server and another standalone server for my website.
I'm looking at having a second client server process running on the same system (to take advantage of its multiple cores) and I would like some advice in locating which server my "clients" have connected to. Each client can have multiple connections (instances).
I've already looked at using memcached to hold a list of user identifiers and link them to which server(s) they are connected to, but that doesn't seem like it would scale very well (eg having six digits of connected users).
I see the same issue with database lookups.
I have already optimized my server as much as possible, without going into micro-optimization and I personally frown upon that.
Current server methodology:
On connect:
Accept connection, rate limit for max connections per IP.
Append client instance to a list named "clientList".
On data from client:
Rate limit for max messages per second.
Append data to a client work queue.
If client has a thread dedicated toward its work queue:
return, its work will be chewed by the current thread
otherwise create a new thread for this users work queue, start it.
TLDR:
How do I efficiently store which servers a client has connected to in order to forward messages to that user.
Short version of my question:
How do I design a single Python script that can listen and respond to inputs received via HTTP or a serial port, and also initiate communications via these channels on its own? My problem is that I don't understand how to design a single script that both (i) uses a web framework to listen on some port for HTTP inputs, and (ii) also does other work that's independent of incoming HTTP requests.
Long version:
I want to use Python to design a system that does the following:
Listens to a serial port for occasional reports. Specifically, I have a network of JeeNode sensors (wireless Arduino-compatible modules) that talk to a central JeeLink, which connects to my computer via USB and talks to my Python script via pySerial.
Listens to a web URL for occasional inputs. Specifically, users send commands to the system via SMS to a Twilio number. Twilio intercepts the SMS messages and posts them to a URL I designate, and I use the Bottle micro web-framework to listen for new HTTP requests.
Responds to both types (serial and HTTP) of inputs. For example, if a user texts the command "Sleep", I want to (i) tell the sensors to go to sleep via the serial port -> JeeLink (which will then forward the command onto the remotes); and (ii) reply to the sender -- and maybe other users -- that the command has been received and is being executed.
Occasionally initiates its own communications to users (via HTTP -> Twilio -> SMS) or remote sensors (via serial -> JeeLink) without any precipitating input event. Two examples: (1) I want to report out to users or remote sensors every N minutes even if I haven't received any new inputs. (2) I want to tell users remotes have actually entered Sleep mode. Because the remotes are battery-powered, they spend most of the time in an inaccessible low-power mode. They can only receive new commands from the JeeLink when they initiate a wireless "check-in" every 5 min. So while technically remotes go to sleep (or wake up, etc.) in response to a user command, commands and responses are effectively independent.
My problem is that all of usage examples of web frameworks I've seen seem to assume that all precipitating events occur via HTTP requests. I can create a Bottle object, and use decorators to bind code to that object that get executed whenever it sees an HTTP request that matches some specified URL path. But I don't know how to do that while simultaneously doing other work that's independent of HTTP events, for example, listening to the serial port.
After struggling a lot, the potential solutions I'm considering now are:
Splitting the functionality into separate scripts. A.py listens for text messages via HTTP and writes the relevant information to some database; B.py continuously reads the database for new records and reacts accordingly, as well as listening to the serial monitor and doing other work. This seems like it would work fine, but it feels inelegant, and I suspect there's a simpler solution I'm unaware of.
Maybe the answer is related to Python decorators? I use various decorators to specify the URL paths that, when a matching HTTP request comes in, execute the code bound to the decorator. So I'm guessing that maybe there's a way to specify some other kind of decorator that, rather than listening for HTTP requests, gets executed when my "main" Python code tells it to? But I don't know enough about decorators to know if this is true.
It seems like you are trying to write an asynchronous application to manage your network of nodes via HTTP. You want to respond to incoming communications on multiple channels as they occur, you want to initiate communications on a schedule, on multiple channels, and you want those two forms of communication to interact. All of these communications are with an outside world that is slow, so it behooves you not to block if you don't need to.
It will probably be easiest to maintain your system if you organize your code into several Python modules, split by their area of concern - serial interface code, HTTP interface code, common processing code-paths, etc. Weave those components together in a central control module, which imports your libraries, and knows how to start and stop cleanly. Then you can test the serial interface independent of the web interface, and potentially reuse some of those Python modules in other projects.
I'm working on a really basic "image streaming" server as a school subject, and I've done most of the work but I'm still stuck on the separation between data and control related sockets:
My structure is : TCPServer (my server, used as control socket) contains a dataSocket (only used to send images and initialized within my TCPServer object, when I receive a certain query)
When I'm sending data (images) through my dataSocket, I still need to see if the client sent a PAUSE or STOP request, but if I use python's self.request.recv(1024) the server awaits a response instead of continuing to send data (which is quite logical).
What should I do to prevent this behavior ? Should I launch my recv(1024) on a separate thread and run it at each loop (and check if I get any relevant data in between two iterations) ?
Twisted should do the trick! It handles asynchronous sockets in Python
I wrote an audio broadcasting server with Python/Twisted. It works fine, but the usage of memory grows too fast! I think that's because some user's network might not be good enough to download the audio in time.
My audio server broadcast audio data to different listener's client, if some of them can't download the audio in time, that means, my server keep the audio data until listeners received. And what's more, my audio server is a broadcasting server, it receive audio data, and send them to different clients, I though Twisted copy those data in different buffer, even they are same audio piece.
I want to reduce the usage of memory usage, so I need to know when is the audio received by the client, so that I can decide when to discard some slow clients. But I have no idea how to achieve that with Twisted. Do anyone have idea?
And what else can I do to reduce usage of memory usage?
Thanks.
Victor Lin.
You didn't say, but I'm going to assume that you're using TCP. It would be hard to write a UDP-based system which had ever increasing memory because of clients who can't receive data as fast as you're trying to send it.
TCP has built-in flow control capabilities. If a receiver cannot read data as fast as you'd like to send it, this information will be made available to you and you can send more slowly. The way this works with the BSD socket API is that a send(2) call will block or will return 0 to indicate it cannot add any bytes to the send buffer. The way it works in Twisted is by a system called "producers and consumers". The gist of this system is that you register a producer with a consumer. The producer calls write on the consumer repeatedly. When the consumer cannot keep up, it calls pauseProducing on the producer. When the consumer is again ready for more data, it calls resumeProducing on the producer.
You can read about this system in more detail in the producer/consumer howto, part of Twisted's documentation.
Make sure you're using Python's garbage collector and then go through and delete variables you aren't using.
I have a problem as follows:
Server process 1
Constantly sends updates that occur to a datastore
Server process 2
Clients contact the server, which queries the datastore, and returns a result
The thing is, the results that process 1 and process 2 are sending back the client are totally different and unrelated.
How does one decompose this?
Do you just have one process constantly sending data, and define the protocol to have a bit which corresponds to whether the return type is 1 or 2?
Do you have two processes? How do they share the datastore then (it is just a structure not a database)?
Thanks!
It sounds like you want to stream your series of ints "somewhere" and also collect them in a datastore. In my system I am streaming sensor readings into a database and also allowing them to go directly to web clients, giving them live power readings. I've written a blog entry on why a database is not suitable for live data - though it is perfect for saving the data for later analysis.
I'd have the first server process be a twisted server that uses txamp to stream the ints to RabbitMQ. Any clients that want live data can subscribe to the stream in RabbitMQ, also using Txamp. Web browser clients can use Orbited here is a worked example.
In your design server 1 saves to the database. You could instead have server3 collect data from RabbitMQ and stream it to the database. I plan to have a server that collects chunks of data and render graphs to store to a central fileshare.
Don't create your own messaging system, RabbitMQ is well tested, scalable, and can persist your "messages" (raw data) if something goes wrong.
If you can restrict yourself to Twisted, I recommend to use Perspective Broker. It's essentially an RPC system, and doesn't care much about the notion of "client" and "server" - either the initiator of a TCP connection or the responder can start RPC calls in PB.
So server 1 would accept registration calls with a callback object, and call the callback whenever it has new data available. Server 2 provides various RPC operations as clients require them. If they operate on the very same data, I would put both servers into a single process.
Why not use a database instead of "just a structure"? Both relational and non-relational DBs offer many practical advantages (separate processes using them, take care of replication [[and/or snapshots, backups, ...]], rich functionality if you need it for the "queries", and so on, and so forth).
Worst case, the "just a structure" can be handled by a third process that's entirely dedicated to it (basically mimicking what any DB engine would offer -- though the engine would probably do it better and faster;-), allowing you to at least keep a good decomposition (with the two server processes both interacting with the "datastore process").