Develop Reference

Supervisor cannot start Gunicorn: ENOENT

I have deployed Django using gunicorn and nginx. The django project is located in a virtual environment. Everything is working perfectly when I run -
gunicorn mydjangoproject.wsgi -c gunicorn_config.py
I am running the above command inside my Django project folder containing manage.py with the virtual environment active.
However now i want to close the server terminal and want gunicorn to run automatically. For this I am using Supervisor. I have installed supervisor using apt-get and created a gunicorn.conf file in supervisor's conf.d.
But when I run supervisorctl start gunicorn I am getting a fatal error-
gunicorn: ERROR (abnormal termination)
So checked the log file and it says-
supervisor:couldn't exec root/ervirtualenvpy2/bin/gunicorn: ENOENT
child process was not spawned
My configuration file for supervisor's gunicorn.conf looks like this-
[program:gunicorn]
command = root/ervirtualenvpy2/bin/gunicorn myproject.wsgi -c root/path/to/the/gunicorn_conf.py/file
directory = root/ervirtualenvpy2/path/to/myproject/
user=root
autorestart=true

Going by what you said and your config everything seems right except that you have specified relative path rather than absolute path:
see gunicorn docs
Instead it should be:
[program:gunicorn]
command = /root/ervirtualenvpy2/bin/gunicorn myproject.wsgi -c /root/path/to/the/gunicorn_conf.py/file
directory = /root/ervirtualenvpy2/path/to/myproject
user=root
autorestart=true

Also Check your gunicorn file path in env/bin/gunicorn file
In my case, I changed my env directory to another place, so please be sure with that
Wrong path: #!/home/ubuntu/nikhil_project/env/bin/python
Correct path: #!/home/ubuntu/env/bin/python

Start python flask webserver automatically after booting the system and keep it on till the end

I'm using flask as a webserver for my UI (it's a simple web interface which controls the recording using gstreamer on ubuntu from a webcam and a framegrabber simultaneously / kinda simple player)
Every time I need to run the command "python main.py" to run the server from command prompt manually.
I've tried the init.d solution or even writing a simple shell script and launching it every time after rebooting the system on start up but it fails to keep the server up and running till the end (just invokes the server and terminates it I guess)
is there any solution that could help me to start the webserver every time after booting the system on startup and keep it on and running?
I'd like to configure my system to boot directly into the browser so don't wanna have any need for more actions by the user.
Any Kind of suggestion/help is appreciated.

I'd like to suggest using supervisor, the documentation is here
for a very simple demo purpose, after you installed it and finish the set up, touch a new a file like this:
[program:flask_app]
command = python main.py
directory = /dir/to/your/app
autostart = true
autorestart = true
then
$ sudo supervisorctl update
Now, you should be good to go. The flask app will start every time after you boot you machine.(note: distribution package has already integrated into the service management infrastructure, if you're using others, see here)
to check whether you app is running:
$ sudo supervisorctl status
For production, you can use nginx+uwsgi+supervisor. The flask deployment documentation is here

One well documented solution is to use Gunicorn and Nginx server:
Install Components and setup a Python virtualenv with dependencies
Create the wsgi.py file :
from myproject import application
if __name__ == "__main__":
application.run()
That will be handled by Gunicorn :
gunicorn --bind 0.0.0.0:8000 wsgi
Configure Gunicorn with setting up a systemd config file: /etc/systemd/system/myproject.service :
[Unit]
Description=Gunicorn instance to serve myproject
After=network.target
[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn
--workers 3 --bind unix:myproject.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
Start the Gunicorn service at boot :
sudo systemctl start myproject
sudo systemctl enable myproject

Nginx Django and Gunicorn. Gunicorn sock file is missing?

I have an ansible provisioned VM based on this one https://github.com/jcalazan/ansible-django-stack but for some reason trying to start Gunicorn gives the following error:
Can't connect to /path/to/my/gunicorn.sock
and in nginx log file:
connect() to unix:/path/to/my/gunicorn.sock failed (2: No such file or directory) while connecting to upstream
And actually the socket file is missing in the specified directory. I have checked the permissions of the directory and they are fine.
Here is my gunicorn_start script:
NAME="{{ application_name }}"
DJANGODIR={{ application_path }}
SOCKFILE={{ virtualenv_path }}/run/gunicorn.sock
USER={{ gunicorn_user }}
GROUP={{ gunicorn_group }}
NUM_WORKERS={{ gunicorn_num_workers }}
# Set this to 0 for unlimited requests. During development, you might want to
# set this to 1 to automatically restart the process on each request (i.e. your
# code will be reloaded on every request).
MAX_REQUESTS={{ gunicorn_max_requests }}
echo "Starting $NAME as `whoami`"
# Activate the virtual environment.
cd $DJANGODIR
. ../../bin/activate
# Set additional environment variables.
. ../../bin/postactivate
# Create the run directory if it doesn't exist.
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Programs meant to be run under supervisor should not daemonize themselves
# (do not use --daemon).
exec gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--max-requests $MAX_REQUESTS \
--user $USER --group $GROUP \
--log-level debug \
--bind unix:$SOCKFILE \
{{ application_name }}.wsgi
Can anyone suggest what else could cause the missing socket file?
Thanks

Well, since I don't have enough rep to comment, I'll mention here that there is not a lot of specificity suggested by the missing socket, but I can tell you a bit about how I started in your shoes and got things to work.
The long and short of it is that gunicorn has encountered a problem when run by upstart and either never got up and running or shut down. Here are some steps that may help you get more info to track down your issue:
In my case, when this happened, gunicorn never got around to doing any error logging, so I had to look elsewhere. Try ps auxf | grep gunicorn to see if you have any workers going. I didn't.
Looking in the syslog for complaints from upstart, grep init: /var/log/syslog, showed me that my gunicorn service had been stopped because it was respawning too fast, though I doubt that'll be your problem since you don't have a respawn in your conf. Regardless, you might find something there.
After seeing gunicorn was failing to run or log errors, I decided to try running it from the command line. Go to the directory where your manage.py lives and run the expanded version of your upstart command against your gunicorn instance. Something like (Replace all of the vars with the appropriate litterals instead of the garbage I use.):
/path/to/your/virtualenv/bin/gunicorn --name myapp --workers 4 --max-requests 10 --user appuser --group webusers --log-level debug --error-logfile /somewhere/I/can/find/error.log --bind unix:/tmp/myapp.socket myapp.wsgi
If you're lucky, you may get a python traceback or find something in your gunicorn error log after running the command manually. Some things that can go wrong:
django errors (maybe problems loading your settings module?). Make sure your wsgi.py is referencing the appropriate settings module on the server.
whitespace issues in your upstart script. I had a tab hiding among spaces that munged things up.
user/permission issues. Finally, I was able to run gunicorn as root on the command line but not as a non-root user via the upstart config.
Hope that helps. It's been a couple of long days tracking this stuff down.

I encountered the same problem after following Michal Karzynski's great guide 'Setting up Django with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL'.
And this is how I solved it.
I had this variable in the bash script used to start gunicorn via Supervisor (myapp/bin/gunicorn_start):
SOCKFILE={{ myapp absolute path }}/run/gunicorn.sock
Which, when you run the bash script for the first time, creates a 'run' folder and a sock file using root privileges. So I sudo deleted the run folder, and then recreated it without sudo privileges and voila! Now if you rerun Gunicorn or Supervisor you won't have the annoying missing sock file error message anymore!
TL;DR
Sudo delete run folder.
Recreate it without sudo privileges.
Run Gunicorn again.
????
Profit

The error could also arise when you haven't pip installed a requirement. In my case, looking at the gunicorn error logs, I found that there was a missing module. Usually happens when you forget to pip install new requirements.

Well, I worked on this issue for more than a week and finally was able to figure it out.
Please follow links from digital ocean , but they did not pinpoint important issues one which includes
no live upstreams while connecting to upstream
*4 connect() to unix:/myproject.sock failed (13: Permission denied) while connecting to upstream
gunicorn OSError: [Errno 1] Operation not permitted
*1 connect() to unix:/tmp/myproject.sock failed (2: No such file or directory)
etc.
These issues are basically permission issue for connection between Nginx and Gunicorn.
To make things simple, I recommend to give same nginx permission to every file/project/python program you create.
To solve all the issue follow this approach:
First thing is :
Log in to the system as a root user
Create /home/nginx directory.
After doing this, follow as per the website until Create an Upstart Script.
Run chown -R nginx:nginx /home/nginx
For upstart script, do the following change in the last line :
exec gunicorn --workers 3 --bind unix:myproject.sock -u nginx -g nginx wsgi
DONT ADD -m permission as it messes up the socket. From the documentation of Gunicorn, when -m is default, python will figure out the best permission
Start the upstart script
Now just go to /etc/nginx/nginx.conf file.
Go to the server module and append:
location / {
include proxy_params;
proxy_pass http<>:<>//unix:/home/nginx/myproject.sock;
}
REMOVE <>
Do not follow the digitalocean aricle from here on
Now restart nginx server and you are good to go.

I had the same problem and found out that I had set the DJANGO_SETTINGS_MODULE to production settings in the the gunicorn script and the wsgi settings were using dev.
I pointed the DJANGO_SETTINGS_MODULE to dev and everything worked.

UWSGI, runing as service - no app loaded [duplicate]

Ubuntu 12.04, nginx 1.2.0, uwsgi 1.0.3.
I start uwsgi with the following command:
uwsgi -s 127.0.0.1:9010 -M -t 30 -A 4 -p 4 -d /var/log/uwsgi.log
On each request nginx replies with 502 and uwsgi writes to log the following line:
-- unavailable modifier requested: 0 --

Original answer
For Python 2 on Ubuntu 11.10, using upstart, install the python plugin for uWSGI with apt-get install uwsgi-plugin-python and if you're using an ini file to configure your uWSGI app, then add plugins = python to the [uwsgi] section and it should solve this problem.
Edit: Updated for Python 3 and Ubuntu 17.10
For Python 3 on Ubuntu 17.10, using systemd, install the python plugin for uWSGI with apt-get install uwsgi-plugin-python3 and if you're using an ini file to configure your uWSGI app, then add plugins = python to the [uwsgi] section and it should solve this problem.
For more information on getting started with python/uWSGI apps, including how to configure them using an ini file then please take a look at this handy guide

Solved by installing uwsgi-plugin-python3 plugin and adding --plugin python3 option to uwsgi start command

Im starting uwsgi from upstart on Ubuntu. I solved the problem by running apt-get install uwsgi-plugin-python, and then adding plugins=python to my application.ini in /etc/uwsgi/applications-available.

from http://uwsgi-docs.readthedocs.org/en/latest/ThingsToKnow.html, "To route requests to a specific plugin, the webserver needs to pass a magic number known as a modifier to the uWSGI instances. By default this number is set to 0, which is mapped to Python."
I'm using 9 for a bash script and it's working. the numbers and their meanings are on this page: http://uwsgi-docs.readthedocs.org/en/latest/Protocol.html
in my nginx configuration:
location ~ .cgi$ {
include uwsgi_params;
uwsgi_modifier1 9;
uwsgi_pass 127.0.0.1:3031;
}

Modify your ini file by added plugins line.
[uwsgi]
plugins = python3

I'm using Ubuntu 18.04 with Python 3. Below is the exact config I used to get it working.
You must have the Python 3 uWSGI plugin installed:
apt install uwsgi-plugin-python3
Your Nginx site configuration should point to your uWSGI socket. Make sure the port matches the configuration in the later steps.
location / {
uwsgi_pass 127.0.0.1:9090;
include uwsgi_params;
}
Reload the Nginx config to reflect the changes you just made:
systemctl reload nginx
You can use command-line arguments or an ini file for configuration. I created uwsgi.ini. Make sure the socket address matches your nginx config.
[uwsgi]
socket = 127.0.0.1:9090
chdir = /var/www
processes = 4
threads = 2
plugins = python3
wsgi-file = /var/www/app.py
My app.py just has a basic example:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/plain')])
return [b"Hello World!"]
Now start the uWSGI server from the command line:
uwsgi uwsgi.ini

Nginx(Django) ImportError: cannot import name celeryd

I tested my project in my local machine, and it worked fine. But after uploading to a remote server(CentOS), I cannot execute celerybeat.
Here is my command.
python manage.py celeryd --events --loglevel=INFO -c 5 --settings=[settings-directory].production
This command works in the local machine(with --settings=[settings-directory].local), but in the remote server, ImportError: cannot import name celeryd occured.
Setting about celery is in base.py. local.py and production.py import the file. In production.py, there are just DEBUG, static, database settings.
I can import djcelery and celery in shell of the remote machine.
How could I solve this?
--
I think this is a version problem.. I'm reading about celery3.1

It turned out I used different version of Django in my remote server.
In Celery 3.1, there is no command named celeryd.