Develop Reference

How to hot reload grpc-server in python?

I'm developing some python microservices with grpc and i'm using docker for the cassandra database and the microservices. Is there a way to setup reload on change within docker-compose?
I'm guessing that first I need the code mounted as a volume but I don't see a way to reload on GRPC server like for example flask does.

We use watchdog[watchmedo] with our grpc services and Docker.
Install watchdog or add to your requirements.txt file
python -m pip install watchdog[watchmedo]
Then in your docker-compose.yml add watchmedo auto-restart --recursive --pattern="*.py" --directory="/usr/src/app/" python -- -m app to your container where --directory is the directory to where your app is contained inside the docker container, and python -- -m app is the file that starts your grpc Server. In this example the file that starts the server is called app.py:
app:
build:
context: ./app/
dockerfile: ./Dockerfile
target: app
command: watchmedo auto-restart --recursive --pattern="*.py" --directory="/usr/src/app/" python -- -m app
volumes:
- ./app/:/usr/src/app/

No module named flask with waitress-serve and virtualenv (App engine flex)

I'm working on a project which used to be run on Linux for test. It's an App Engine flex project, run with gunicorn. Gunicorn does not work on Windows if I understood well, so I've been adviced to use waitress.
I also use virtualenv in my project.
So when I'm in my virtualenv, I run waitress-serve main:app (the gunicorn cmd was gunicorn -b :8080 main:app). I get an error: It had these arguments:
1. No module named flask.
I use flask. I can see the flask folder in my virtualenv folder. And when I run python then from flask import Flask I have no error.
Is there compat issue between waitress and virtualenv ? Or I'm doing something else wrong ? (already tried to delete virtualenv folder and install all the things again)

Python modules are case sensitive
Try Flask not flask.

Can't get heroku procfile

I'm trying to build up a deep-learning-based facebook chatbot (using Python). I'm trying to deploy it on Heroku firstly, but as I'm using the command web: gunicorn echoserver:app the terminal says web: command not found. Howevere, I've installed gunicorn already.

This is because you are typing web:, which is not a command line interface (CLI) command.
If you've installed gunicorn, then the command (from the CLI) is gunicorn. Something like, for instance
gunicorn echoserver:app
I suppose it's also possible that you have a Windows machine. gunicorn does not work on Windows, so you would need to use something like waitress. With waitress, you would type web: on a Windows machine, so that it would be something like
web: waitress-serve echoserver:app

Note that the procfile is literally a file you place in your repository. The contents of the procfile should contain the command you want Heroku to run to start your server.
So in the root directory of your repo, you should have a Procfile (named exactly that with no file extension) with the following contents:
web: gunicorn echoserver:app
The first part (web:) is just used to tell Heroku which dyno to run the second part (the command) on. So Heroku will only run the command on web dynos and not on background dynos.
More info here: https://devcenter.heroku.com/articles/procfile

Supervisor cannot start Gunicorn: ENOENT

I have deployed Django using gunicorn and nginx. The django project is located in a virtual environment. Everything is working perfectly when I run -
gunicorn mydjangoproject.wsgi -c gunicorn_config.py
I am running the above command inside my Django project folder containing manage.py with the virtual environment active.
However now i want to close the server terminal and want gunicorn to run automatically. For this I am using Supervisor. I have installed supervisor using apt-get and created a gunicorn.conf file in supervisor's conf.d.
But when I run supervisorctl start gunicorn I am getting a fatal error-
gunicorn: ERROR (abnormal termination)
So checked the log file and it says-
supervisor:couldn't exec root/ervirtualenvpy2/bin/gunicorn: ENOENT
child process was not spawned
My configuration file for supervisor's gunicorn.conf looks like this-
[program:gunicorn]
command = root/ervirtualenvpy2/bin/gunicorn myproject.wsgi -c root/path/to/the/gunicorn_conf.py/file
directory = root/ervirtualenvpy2/path/to/myproject/
user=root
autorestart=true

Going by what you said and your config everything seems right except that you have specified relative path rather than absolute path:
see gunicorn docs
Instead it should be:
[program:gunicorn]
command = /root/ervirtualenvpy2/bin/gunicorn myproject.wsgi -c /root/path/to/the/gunicorn_conf.py/file
directory = /root/ervirtualenvpy2/path/to/myproject
user=root
autorestart=true

Also Check your gunicorn file path in env/bin/gunicorn file
In my case, I changed my env directory to another place, so please be sure with that
Wrong path: #!/home/ubuntu/nikhil_project/env/bin/python
Correct path: #!/home/ubuntu/env/bin/python

Start python flask webserver automatically after booting the system and keep it on till the end

I'm using flask as a webserver for my UI (it's a simple web interface which controls the recording using gstreamer on ubuntu from a webcam and a framegrabber simultaneously / kinda simple player)
Every time I need to run the command "python main.py" to run the server from command prompt manually.
I've tried the init.d solution or even writing a simple shell script and launching it every time after rebooting the system on start up but it fails to keep the server up and running till the end (just invokes the server and terminates it I guess)
is there any solution that could help me to start the webserver every time after booting the system on startup and keep it on and running?
I'd like to configure my system to boot directly into the browser so don't wanna have any need for more actions by the user.
Any Kind of suggestion/help is appreciated.

I'd like to suggest using supervisor, the documentation is here
for a very simple demo purpose, after you installed it and finish the set up, touch a new a file like this:
[program:flask_app]
command = python main.py
directory = /dir/to/your/app
autostart = true
autorestart = true
then
$ sudo supervisorctl update
Now, you should be good to go. The flask app will start every time after you boot you machine.(note: distribution package has already integrated into the service management infrastructure, if you're using others, see here)
to check whether you app is running:
$ sudo supervisorctl status
For production, you can use nginx+uwsgi+supervisor. The flask deployment documentation is here

One well documented solution is to use Gunicorn and Nginx server:
Install Components and setup a Python virtualenv with dependencies
Create the wsgi.py file :
from myproject import application
if __name__ == "__main__":
application.run()
That will be handled by Gunicorn :
gunicorn --bind 0.0.0.0:8000 wsgi
Configure Gunicorn with setting up a systemd config file: /etc/systemd/system/myproject.service :
[Unit]
Description=Gunicorn instance to serve myproject
After=network.target
[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn
--workers 3 --bind unix:myproject.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
Start the Gunicorn service at boot :
sudo systemctl start myproject
sudo systemctl enable myproject