Develop Reference

How to configure a Django application in a virtual env served by uswgi and started by a systemd service?

I'm trying to configure a basic a Django application served by uwsgi so that it can be started from systemd on a CentoS 7.X server.
So far my SystemD service is started (active and in running state) but application is not reachable on the configured port. Note that application is running inside a Python virtual environment.
Systemd service is active and running
uwsgi worker processes are
active
socket bind to TCP 8003 is in LISTEN state
SystemD unit file
# /etc/systemd/system/django_03.service
[Unit]
Description=My Django app
Requires=network.target
After=network.target
After=syslog.target
[Service]
TimeoutStartSec=0
RestartSec=10
Restart=always
User=myuser
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
StandardError=syslog
RuntimeDirectory=uwsgi
ExecStart=/bin/bash -c 'cd /opt/scripts/django/django_03; source django_03_env/bin/activate; uwsgi --ini /opt/scripts/django/django_03/django_03.ini'
[Install]
WantedBy=multi-user.target
uwsgi configuration file
# /opt/scripts/django/django_03/django_03.ini
[uwsgi]
module = wsgi:application
master = true
processes = 5
socket = 127.0.0.1:8003
chmod-socket = 664
vacuum = true
die-on-term = true
Django application
# wsgi.py
def application(environ, response):
response('200 OK', [('Content-Type', 'text/html')])
return [b"Test OK (Django 03) !!"]
Thanks for your help

In this example, systemd service is called "django_03", Python virtual env name is "django_03_env", Python application is "wsgi" with a callable named "application".
1 ] Prerequisites
Create a root directory for Python code, configuration files and virtual env:
mkdir /opt/scripts/django/django_03
Create a virtualenv
cd /opt/scripts/django/django_03
python3 -m venv django_03_env
Start virtualenv
source django_03_env/bin/activate
Install Django and uwsgi
pip install django
pip install uwsgi
2 ] Files
There are three files to create
uwsgi configuration file (INI format): django_03.ini
SystemD service unit: django_03.service
Python application: wsgi.py
2.1 SystemD unit file
# /etc/systemd/system/django_03.service
[Unit]
Description=My Django app
Requires=network.target
After=network.target
After=syslog.target
[Service]
TimeoutStartSec=0
RestartSec=10
Restart=always
User=myuser
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
StandardError=syslog
RuntimeDirectory=uwsgi
# Main call: Virtual env is activated and uwsgi is started with INI file as argument
ExecStart=/bin/bash -c 'cd /opt/scripts/django/django_03; source django_03_env/bin/activate; uwsgi --ini /opt/scripts/django/django_03/django_03.ini'
[Install]
WantedBy=multi-user.target
2.2 uwsgi configuration file
# /opt/scripts/django/django_03/django_03.ini
[uwsgi]
# "callable": Application entry point, <Python filename without extension: application method name>
module = wsgi:application
# Master process for uwsgi
master = true
# Worker processes for uwsgi
processes = 5
# uwsgi protocol modification
protocol = http
# Socket format X.X.X.X:<port number>
socket = 127.0.0.1:8003
chmod-socket = 664
vacuum = true
die-on-term = true
2.3 Django application file
# /opt/scripts/django/django_03/wsgi.py
# Application callable (main entry point)
def application(environ, response):
response('200 OK', [('Content-Type', 'text/html')])
return [b"Test OK (Django 03) !!"]
III ] Managing application
Django application can now be started by a regular systemd command:
sudo systemctl start django_03
Verify systemd service status
sudo systemctl -l status django_03
journalctl -u django_03
Verify that HTTP socket is listening
netstat -ntap | grep 8003
Verify HTTP response:
curl http://127.0.0.1:8003
Note that by default uwsgi operates with its default transfert protocol which is NOT HTTP. It is not recommended to use uwsgi HTTP socket, the best setup being the use of uswgi default protocol and a reverse proxy in front of it (NGINX, Apache HTTPd). Of course code presented here is far from perfect and might be optimized and tailored to your needs.

uWSGI runs Django project from command line but not from Emperor uwsgi.service file

I am running a virtualenv with Python3.6 on Ubuntu 16.04 for my Django project using uwsgi and NGINX.
I have uWSGI installed globally and also in the virtualenv.
I can run my project from the command line using uWSGI within the env with
/home/user/Env/myproject/bin/uwsgi --http :8080 --home /home/user/Env/myproject --chdir /home/user/myproject/src/myproject -w myproject.wsgi
and go to my domain and it loads fine.
However I am obviously running uWSGI in "Emperor mode" and when I set the service file up (along with NGINX) the domain displays internal server error.
The uWSGI logs trace to --- no python application found ---
I was having this problem when running
uwsgi --http :8080 --home /home/user/Env/myproject --chdir /home/user/myproject/src/myproject -w myproject.wsgi
because it was using the global install uwsgi instead of the virtualenv one.
I changed my StartExec to the virtualenv uwsgi path but no luck.
I can't figure out what I'm doing wrong, path error? Syntax error?
my /etc/systemd/system/uwsgi.service file
[Unit]
Description=uWSGI Emperor service
[Service]
ExecStartPre=/bin/bash -c 'mkdir -p /run/uwsgi; chown user:www-data /run/uwsgi'
ExecStart=/home/user/Env/myproject/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target

Okay bit silly but it seems I ran sudo systemctl stop uwsgi and then sudo systemctl start uwsgi and it works now.

Trying to configure gunicorn

I have been following this tutorial to deploy my django project on Digital Ocean. I am trying to configure gunicorn.
My project structure looks similar to this:
On my settings.py I use DEBUG=False
I create the gunicorn.socket and gunicorn.service.
/etc/systemd/system/gunicorn.socket
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
/etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=someuser
Group=www-data
WorkingDirectory=/home/someuser/myproject
ExecStart=/home/someuser/myproject/myprojectenv/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
Myproject.wsgi:application
[Install]
WantedBy=multi-user.target
I start and enable the Gunicorn socket:
sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket
Check the status of the process to find out whether it was able to start:
sudo systemctl status gunicorn.socket
This is what I get and then it returns to the command line.
Failed to dump process list, ignoring: No such file or directory
β—� gunicorn.socket - gunicorn socket
Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; vendor preset: enabled)
Active: active (listening) since Sat 2019-05-04 23:12:03 UTC; 13s ago
Listen: /run/gunicorn.sock (Stream)
CGroup: /system.slice/gunicorn.socket
May 04 23:12:03 myproject systemd[1]: Listening on gunicorn socket.
Next, I check for the existence of the gunicorn.sock file within the /run directory:
file /run/gunicorn.sock
/run/gunicorn.sock: socket
It seems that a file or directory doesn't exist, but it doesn't provide any more details. The gunicorn.sock seems to exist.
I am familiar with Python but not with servers and deploying so I am at a loss here.
I have successfully followed the similar tutorial for Ubuntu 16.04 a few months ago, but now I keep hitting issues.

I know, that's old question, but...
I haved some error. In my case wrong WorkingDirectory and application with modul wsgi. I think that's error in "myproject.wsgi:application"

This is actually an issue with your usage of systemd. You need to refer to the name of the service in your calls to systemctl, not the socket as you're doing. In your case the service will be called gunicorn because you've named your systemd unit file as gunicorn.service.
systemctl enable gunicorn
systemctl start gunicorn

uWSGI Emperor does not reload Vassal by touching the .ini file

I have multiple uWSGI vassals, all monitored by uwsgi emperor. I update the code for my app (Django) and I want the emperor to perform a clean reload of one of the vassals. To do that I
touch vassal-foo.ini
In the logs I see [emperor] reload the uwsgi instance vassal-foo.ini. This sounds promising, but the app is not reloaded. It continues to run the old version. Checking the process (PID) startup time, indeed, it has not been restarted.
Any hints what might cause this? Few things that might be uncommon:
Neither the emperor nor the vassal run in master mode
Emperor was installed with pip and runs under initctl
kill -9-ing the vassal triggers a correct reload (obviously)
I use symlinks
I have a secondary thread inside my python app (threading.Thread(target).start()) running with daemon=True
Things I tried and did not work:
Run the process without any additional threads (remove threading.Thread(target).start())
Touching with touch --no-dereference vassal-foo.ini
Starting emperor with --emperor-nofollow
vassal-foo.ini:
master = false
processes = 1
thunder-lock = true
enable-threads = true
socket = /tmp/%n.sock
chmod-socket = 666
vacuum = true
Emperor:
exec /tmp/uwsgi --emperor /tmp/configs/uwsgi/ --die-on-term --uid me --gid me --logto /tmp/logs/uwsgi-emperor.log
uWSGI version
$ uwsgi --version
2.0.17

The problem is that your vassal not run in master mode.
All the ways uwsgi reload a requires master process.
In emperor mode, when you touch the ini, the emperor send a sighup to the vassal and record in the log that the vassal is reload. But if the vassal doesn't has a master it just ignore the sighup. So that is why you see reload in log but nothing happened.

Start python flask webserver automatically after booting the system and keep it on till the end

I'm using flask as a webserver for my UI (it's a simple web interface which controls the recording using gstreamer on ubuntu from a webcam and a framegrabber simultaneously / kinda simple player)
Every time I need to run the command "python main.py" to run the server from command prompt manually.
I've tried the init.d solution or even writing a simple shell script and launching it every time after rebooting the system on start up but it fails to keep the server up and running till the end (just invokes the server and terminates it I guess)
is there any solution that could help me to start the webserver every time after booting the system on startup and keep it on and running?
I'd like to configure my system to boot directly into the browser so don't wanna have any need for more actions by the user.
Any Kind of suggestion/help is appreciated.

I'd like to suggest using supervisor, the documentation is here
for a very simple demo purpose, after you installed it and finish the set up, touch a new a file like this:
[program:flask_app]
command = python main.py
directory = /dir/to/your/app
autostart = true
autorestart = true
then
$ sudo supervisorctl update
Now, you should be good to go. The flask app will start every time after you boot you machine.(note: distribution package has already integrated into the service management infrastructure, if you're using others, see here)
to check whether you app is running:
$ sudo supervisorctl status
For production, you can use nginx+uwsgi+supervisor. The flask deployment documentation is here

One well documented solution is to use Gunicorn and Nginx server:
Install Components and setup a Python virtualenv with dependencies
Create the wsgi.py file :
from myproject import application
if __name__ == "__main__":
application.run()
That will be handled by Gunicorn :
gunicorn --bind 0.0.0.0:8000 wsgi
Configure Gunicorn with setting up a systemd config file: /etc/systemd/system/myproject.service :
[Unit]
Description=Gunicorn instance to serve myproject
After=network.target
[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn
--workers 3 --bind unix:myproject.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
Start the Gunicorn service at boot :
sudo systemctl start myproject
sudo systemctl enable myproject