Similarly to Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable I cannot start my container because it does not (need to) listen on a port. It is a Discord bot that just needs outbound connections for the APIs.
Is there a way I can get around this error? I've tried listening on port 0.0.0.0:8080 using socket module with
import socket
s = socket.socket()
s.bind(("0.0.0.0", 8080))
s.listen()
Cloud Run is oriented to request-driven tasks and this explains Cloud Run's listen requirement.
Generally (!) clients make requests to your Cloud Run service endpoint triggering the creation of instances to process the requests and generate responses.
Generally (!) if there are no outstanding responses to be sent, the service scales down to zero instances.
Running a bot, you will need to configure Cloud Run artificially to:
Always run (and pay for) at least one instance (so that the bot isn't terminated)
Respond (!) to incoming requests (on one thread|process)
Run your bot (on one thread|process)
To do both #2 and #3 you'll need to consider Python multithreading|multiprocessing.
For #2, the code in your question is insufficient. You can use low-level sockets, but it will need to respond to incoming requests and so you will need to implement a server. It would be simpler to use e.g. Flask which gives you an HTTP server with very little code.
And this server code only exists to satisfy the Cloud Run requirement, it is not required for your bot.
If I were you, I'd run the bot on a Compute Engine VM. You can do this for Free.
If your bot is already packaged using a container, you can deploy the container directly to a VM.
Related
I can't find the exact guide of what I want to do, it's more of a structural and architectural issue:
Tooling:
Python 3.9
FastAPI
Uvicorn
Some scripts to monitor the folders
It'll run under docker when its done
The exact task:
I want to build a web-app that lists the photos in a directory and shows them in a grid in the browser.
The key points here:
It will use watchdog to immediately get any added or removed items.
Clients will connect with a web socket (I've followed those tutorials)
Deltas will be send to observing clients
The last bit is my issue, and to the point of the question:
What is a "accepted/best practice" for having my watchdog script send the added/removed items to the connected client web sockets.
I can't for the life of me work out how they communicate, running in uvicorn I just can't start an arbitrary background job.... I know in a dev environment I can start uvicorn manually, but I want it to follow scalable patterns even if it's just for light usage.
In short: How can a listening python script inform my fastAPI there's new data
The easy/obvious answer is to expose a management API that the wathcdog script can send... but is there any sort of message bus that fastAPI can listen to?
AWSGI is new to me, I have some experience with python async/scheduler, but have mostly used WSGI frameworks like Bottle where scheduling/threading isn't an issue.
Ok, unless anyone has any amazing ideas, the best solution is:
Connect to REDIS, pull existing values at the time the client web socket connects.
The worker process(es) can push new values via REDIS.
Since the connected client handler can use asyncio, they can subscribe to the pub/sun model.
Problem solved, yes it requires REDIS but that’s easy enough in docker.
Why REDIS?
Low boiler plate code needed, full pub/sub support and low setup pain.
My client built with pyzmq, will connect to a service that will provide it with the correct address it needs to connect to. It might do this several times, each time having to connect to a different worker.
What I have created until now, based on the zguide, is a simple broker that will accept connections from clients on a frontend port and then it will connect with one of the workers and make a question (right now its just a random choice of yes and no). If the client replies with 'yes' then my idea was to let the client know that that specific worker is ready and have it connect directly to the worker.
In the examples that I have seen clients mostly connect to a single server or broker once. What would be the best way to connect with an address given to me on runtime potentially multiple times?
I have a few servers that require executing commands on other servers. For example a Bitbucket Server post receive hook executing a git pull on another server. Another example is the CI server pulling a new docker image and restarting an instance on another server.
I would normally use ssh for this, creating a user/group specifically for the job with limited permission.
A few downsides with ssh:
Synchronous ssh call means a git push will have to wait until complete.
If a host is not contactable for whatever reason, the ssh command will fail.
Maintaining keys, users, and sudoers permissions can become unwieldy.
Few possibilities:
Find an open source out of the box solution (I have tried with no luck so far)
Set up an REST API on each server that accepts calls with some type of authentication, e.g. POST https://server/git/pull/?apikey=a1b2c3
Set up Python/Celery to execute tasks on a different queue for each host. This means a celery worker on each server that can execute commands and possibly a service that accepts REST API calls, converting them to Celery tasks.
Is there a nice solution to this problem?
Defining the problem
You want to be able to trigger a remote task without waiting for it to complete.
This can be achieved in any number of ways, including with SSH. You can execute a remote command without waiting for it to complete by closing or redirecting all I/O streams, e.g. like this:
ssh user#host "/usr/bin/foobar </dev/null >/dev/null 2>&1"
You want to be able to defer the task if the host is currently unavailable.
This requires a queuing/retry system of some kind. You will also need to decide whether the target hosts will be querying for messages ("pull") or whether messages will be sent to the target hosts from elsewhere ("push").
You want to simplify access control as much as possible.
There's no way to completely avoid this issue. One solution would be to put most of the authentication logic in a centralized task server. This splits the problem into two parts: configuring access rights in the task server, and configuring authentication between the task server and the target hosts.
Example solutions
Hosts attempt to start tasks over SSH using method above for asynchrony. If host is unavailable, task is written to local file. Cron job periodically retries sending failed tasks. Access control via SSH keys.
Hosts add tasks by writing commands to files on an SFTP server. Cron job on target hosts periodically checks for new commands and executes them if found. Access control managed via SSH keys on the SFTP server.
Hosts post tasks to REST API which adds them to queue. Celery daemon on each target host consumes from queue and executes tasks. Access managed primarily by credentials sent to the task queuing server.
Hosts post tasks to API which adds tasks to queue. Task consumer nodes pull tasks off the queue and send requests to API on target hosts. Authentication managed by cryptographic signature of sender appended to request, verified by task server on target host.
You can also look into tools that do some or all of the required functions out of the box. For example, some Google searching came up with Rundeck which seems to have some job scheduling capabilities and a REST API. You should also consider whether you can leverage any existing automated deployment or management tools already present in your system.
Conclusions
Ultimately, there's no single right answer to this question. It really depends on your particular needs. Ask yourself: How much time and effort do you want to spend creating this system? What about maintenance? How reliable does it need to be? How much does it need to scale? And so on, ad infinitum...
I have a python application , to be more precise a Network Application that can't go down this means i can't kill the PID since it actually talks with other servers and clients and so on ... many € per minute of downtime , you know the usual 24/7 system.
Anyway in my hobby projects i also work a lot with WSGI frameworks and i noticed that i have the same problem even during off-peak hours.
Anyway imagine a normal server using TCP/UDP ( put here your favourite WSGI/SIP/Classified Information Server/etc).
Now you perform a git pull in the remote server and there goes the new python files into the server (these files will of course ONLY affect the data processing and not the actual sockets so there is no need to re-raise the sockets or touch in any way the network part).
I don't usually use File monitors since i prefer to use SIGNAL to wakeup the internal app updater.
Now imagine the following code
from mysuper.app import handler
while True:
data = socket.recv()
if data:
socket.send(handler(data))
Lets imagine that handler is a APP with DB connections, cache connections , etc.
What is the best way to update the handler.
Is it safe to call reload(handler) ?
Will this break DB connections ?
Will DB Connections survive to this restart ?
Will current transactions be lost ?
Will this create anti-matter ?
What is the best-pratice patterns that you guys usually use if there are any ?
It's safe to call reload(handler).
Depends where you initialize your connections. If you make the connections inside handler(), then yes, they'll be garbage collected when the handler() object falls out of scope. But you wouldn't be connecting inside your main loop, would you? I'd highly recommend something like:
dbconnection = connect(...)
while True:
...
socket.send(handler(data, dbconnection))
if for no other reason than that you won't be making an expensive connection inside a tight loop.
That said, I'd recommend going with an entirely different architecture. Make a listener process that does basically nothing more than listen for UDP datagrams, sends them to a messaging queue like RabbitMQ, then waits for the reply message to send the results back to the client. Then write your actual servers that get their requests from the messaging queue, process them, and send a reply message back.
If you want to upgrade the UDP server, launch the new instance listening on another port. Update your firewall rules to redirect incoming traffic to the new port. Reload the rules. Kill the old process. Voila: seamless cutover.
The real win is from uncoupling your backend. Since multiple processes can listen for the same messages from your frontend "proxy" service, you can run several in parallel - on different machines, if you want to. To upgrade the backend, start a new instance then kill the old one so that there's no time when at least one instance isn't running.
To scale your proxy, have multiple instances running on different ports or different hosts, and configure your firewall to randomly redirect incoming datagrams to one of the proxies.
To scale your backend, run more instances.
A typical situation with a server/web application is that the application needs to be shut down and restarted to implement an upgrade.
What are the possible/common schemes (and available software) to avoid losing data that clients sent to the server during the short time the application was gone?
An example scheme that could work is: For a simple web server where the client connects to port 80, rather than the client connecting directly to the web server application, there could be a simple application in between that listens to port 80 and seamlessly forwards/returns data to/from the "Actual" web server application (on some other port). When the web server needs to be shut down and restarted, the relay app could detect this and buffer all incoming data until the webserver comes back to life. This way there is always an application listening to port 80 and data is never lost (within buffer-size and time reason, of course). Does such a simple intermediate buffer-on-recipient-unavailable piece of software exist already?
I'm mostly interested in solutions for a single application instance and not one where there are multiple instances (in which case a clever rolling update scheme could be used), but in the interests of having a full answer set, any response would be great!
To avoid this, have multiple application servers behind a load balancer. Before bringing one down, ensure the load balancer is not sending it new clients. Bring it down, traffic will go to the other applications servers, and when it comes back up traffic will begin getting sent to it again.
If you have only one application server, simply 'buffering' network traffic is a poor solution. When the server comes back up, it has none of the TCP state information anymore and the old incoming connections have nowhere to go anyway.