I'm deploying a Django project on an ubuntu machine using gunicorn and nginx, following this tutorial to do it.
I have a gunicorn service that looks like this, similar to the one in the tutorial:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/project/mysite
EnvironmentFile=/home/ubuntu/.virtualenvs/projectenv/bin/postactivate
ExecStart=/home/ubuntu/.virtualenvs/projectenv/bin/gunicorn --access-logfile - --workers 10 --bind unix:/home/ubuntu/project/mysite.sock mysite.wsgi:application
[Install]
WantedBy=multi-user.target
Since, as I understand, the gunicorn service is run in a completely new environment instead of my project's virtual environment I need to find another way to pass it environment variables. I found this page which says I can add an EnvironmentFile to my service file, which I did above.
I directed the environment file to my virtualenv's postactivate script, which looks something like this:
#!/bin/bash
# This hook is sourced after this virtualenv is activated.
export DJANGO_DEBUG=False
...
Which doesn't work, unsurprisingly.
What is the correct format for this EnvironmentFile to be in?
You just need to pass something like:
DJANGO_DEBUG=False
PYTHON_PATH=:bla
documentation here
Related
I am trying to deploy python app in my centos7 using flask and gunicorn.
but i found that the app is not able to retrive variables from .env file.
It is working on my local vm.
I am using gunicorn --bind 0.0.0.0:5000 app:app
pls help.
So, just configure a service file app.service in /etc/systemd/system and add EnvironmentFile=/path_to_env_file under [service] and start the service.
For more info on configuring the env file in gunicorn service, you can see
here.
I've followed this guide (https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04), but I'm presently seeing the following when trying to run gunicorn via the gunicorn service file (/etc/systemd/system/gunicorn.service):
Oct 04 11:30:22 ukgcdeploy01 gunicorn[8095]: File "/opt/envs/automation-console-env/bin/gunicorn", line 5, in <module>
Oct 04 11:30:22 ukgcdeploy01 gunicorn[8095]: from gunicorn.app.wsgiapp import run
Oct 04 11:30:22 ukgcdeploy01 gunicorn[8095]: ModuleNotFoundError: No module named 'gunicorn'
The gunicorn.service file contains the following:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/opt/envs/automation-console-env
ExecStart=/opt/envs/automation-console-env/bin/gunicorn --timeout 600 --log-level debug --error-log /var/log/automation-console-env_error.log --access-logfile /var/log/automation-console-env_access.log --workers 3 --bind unix:/opt/envs/automation-console-env/automation-console-env.sock django_forms.wsgi:application
[Install]
WantedBy=multi-user.target
Running gunicorn manually works fine:
gunicorn --bind 0.0.0.0:8000 myproject.wsgi
This was previously working before I had to upgrade my Python version from 3.5.2 to 3.9, and due to some issues I ended up having to recreate the virtual environment, so I don't think it's necessarily an issue with the service file, but rather my Python/Gunicorn installation.
If anyone could offer some advice, it would be greatly appreciated :)
Does the automation-console-env environment exist? If so, is gunicorn installed there? In either case, since it is working as gunicorn ... on the command line, you should be able to use this installation.
which gunicorn
Then change your systemd unit file to point to this executable.
It seems that you either are defining the wrong path or you might be facing a permission issue in gunicorn.service.
First try to enable the service by using systemctl enable gunicorn.service. If it didn't work, then try the following:
Edit gunicorn.service as follows:
from: --bind unix:/opt/envs/automation-console-env/automation-console-env.sock django_forms.wsgi:application
to: --bind unix:/opt/envs/automation-console-env/automation-console-env.sock django_forms.wsgi
I am deploying a flask app that is running inside virtual env
I have the systemd file as follow:
[Unit]
Description=Gunicorn instance to serve my-page
After=network.target
[Service]
User=jb
Group=www-data
WorkingDirectory=/home/jb/webjosue/my-page
Environment="PATH=/home/jb/webjosue/FlaskENV/bin"
ExecStart=/home/jb/webjosue/FlaskENV/bin/gunicorn --workers 3 --bind unix:my-page.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
In my main.py I have
settings = os.environ['APP_SETTINGS']
and I get this error
raise KeyError(key)
Sep 24 00:10:13 ubuntu-512mb-nyc3-01 gunicorn[23439]: KeyError: 'settings'
I am guessing since I have my environment pointing to the virtualenv (FlaskENV) the other environment variables that are in my .profile are not being recognized.
Any ideas?
As #Joe Doherty said here, you can use Environment directive in Service section to add environment variables, more information. For instance:
[Service]
Environment="PATH=/xx/yy/zz/venv/bin"
Environment="FLASK_ENV=development"
Environment="APP_SETTINGS=config.DevelopmentConfig"
I was using virtualenv so /xx/yy/zz/venv/bin is the path of virtualenv folder.
Somehow Environment="FLASK_ENV=development" with quotes wasn't working
Environment=FLASK_ENV=development
with no quotes was working just fine
I'm using flask as a webserver for my UI (it's a simple web interface which controls the recording using gstreamer on ubuntu from a webcam and a framegrabber simultaneously / kinda simple player)
Every time I need to run the command "python main.py" to run the server from command prompt manually.
I've tried the init.d solution or even writing a simple shell script and launching it every time after rebooting the system on start up but it fails to keep the server up and running till the end (just invokes the server and terminates it I guess)
is there any solution that could help me to start the webserver every time after booting the system on startup and keep it on and running?
I'd like to configure my system to boot directly into the browser so don't wanna have any need for more actions by the user.
Any Kind of suggestion/help is appreciated.
I'd like to suggest using supervisor, the documentation is here
for a very simple demo purpose, after you installed it and finish the set up, touch a new a file like this:
[program:flask_app]
command = python main.py
directory = /dir/to/your/app
autostart = true
autorestart = true
then
$ sudo supervisorctl update
Now, you should be good to go. The flask app will start every time after you boot you machine.(note: distribution package has already integrated into the service management infrastructure, if you're using others, see here)
to check whether you app is running:
$ sudo supervisorctl status
For production, you can use nginx+uwsgi+supervisor. The flask deployment documentation is here
One well documented solution is to use Gunicorn and Nginx server:
Install Components and setup a Python virtualenv with dependencies
Create the wsgi.py file :
from myproject import application
if __name__ == "__main__":
application.run()
That will be handled by Gunicorn :
gunicorn --bind 0.0.0.0:8000 wsgi
Configure Gunicorn with setting up a systemd config file: /etc/systemd/system/myproject.service :
[Unit]
Description=Gunicorn instance to serve myproject
After=network.target
[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn
--workers 3 --bind unix:myproject.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
Start the Gunicorn service at boot :
sudo systemctl start myproject
sudo systemctl enable myproject
I have an ansible provisioned VM based on this one https://github.com/jcalazan/ansible-django-stack but for some reason trying to start Gunicorn gives the following error:
Can't connect to /path/to/my/gunicorn.sock
and in nginx log file:
connect() to unix:/path/to/my/gunicorn.sock failed (2: No such file or directory) while connecting to upstream
And actually the socket file is missing in the specified directory. I have checked the permissions of the directory and they are fine.
Here is my gunicorn_start script:
NAME="{{ application_name }}"
DJANGODIR={{ application_path }}
SOCKFILE={{ virtualenv_path }}/run/gunicorn.sock
USER={{ gunicorn_user }}
GROUP={{ gunicorn_group }}
NUM_WORKERS={{ gunicorn_num_workers }}
# Set this to 0 for unlimited requests. During development, you might want to
# set this to 1 to automatically restart the process on each request (i.e. your
# code will be reloaded on every request).
MAX_REQUESTS={{ gunicorn_max_requests }}
echo "Starting $NAME as `whoami`"
# Activate the virtual environment.
cd $DJANGODIR
. ../../bin/activate
# Set additional environment variables.
. ../../bin/postactivate
# Create the run directory if it doesn't exist.
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Programs meant to be run under supervisor should not daemonize themselves
# (do not use --daemon).
exec gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--max-requests $MAX_REQUESTS \
--user $USER --group $GROUP \
--log-level debug \
--bind unix:$SOCKFILE \
{{ application_name }}.wsgi
Can anyone suggest what else could cause the missing socket file?
Thanks
Well, since I don't have enough rep to comment, I'll mention here that there is not a lot of specificity suggested by the missing socket, but I can tell you a bit about how I started in your shoes and got things to work.
The long and short of it is that gunicorn has encountered a problem when run by upstart and either never got up and running or shut down. Here are some steps that may help you get more info to track down your issue:
In my case, when this happened, gunicorn never got around to doing any error logging, so I had to look elsewhere. Try ps auxf | grep gunicorn to see if you have any workers going. I didn't.
Looking in the syslog for complaints from upstart, grep init: /var/log/syslog, showed me that my gunicorn service had been stopped because it was respawning too fast, though I doubt that'll be your problem since you don't have a respawn in your conf. Regardless, you might find something there.
After seeing gunicorn was failing to run or log errors, I decided to try running it from the command line. Go to the directory where your manage.py lives and run the expanded version of your upstart command against your gunicorn instance. Something like (Replace all of the vars with the appropriate litterals instead of the garbage I use.):
/path/to/your/virtualenv/bin/gunicorn --name myapp --workers 4 --max-requests 10 --user appuser --group webusers --log-level debug --error-logfile /somewhere/I/can/find/error.log --bind unix:/tmp/myapp.socket myapp.wsgi
If you're lucky, you may get a python traceback or find something in your gunicorn error log after running the command manually. Some things that can go wrong:
django errors (maybe problems loading your settings module?). Make sure your wsgi.py is referencing the appropriate settings module on the server.
whitespace issues in your upstart script. I had a tab hiding among spaces that munged things up.
user/permission issues. Finally, I was able to run gunicorn as root on the command line but not as a non-root user via the upstart config.
Hope that helps. It's been a couple of long days tracking this stuff down.
I encountered the same problem after following Michal Karzynski's great guide 'Setting up Django with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL'.
And this is how I solved it.
I had this variable in the bash script used to start gunicorn via Supervisor (myapp/bin/gunicorn_start):
SOCKFILE={{ myapp absolute path }}/run/gunicorn.sock
Which, when you run the bash script for the first time, creates a 'run' folder and a sock file using root privileges. So I sudo deleted the run folder, and then recreated it without sudo privileges and voila! Now if you rerun Gunicorn or Supervisor you won't have the annoying missing sock file error message anymore!
TL;DR
Sudo delete run folder.
Recreate it without sudo privileges.
Run Gunicorn again.
????
Profit
The error could also arise when you haven't pip installed a requirement. In my case, looking at the gunicorn error logs, I found that there was a missing module. Usually happens when you forget to pip install new requirements.
Well, I worked on this issue for more than a week and finally was able to figure it out.
Please follow links from digital ocean , but they did not pinpoint important issues one which includes
no live upstreams while connecting to upstream
*4 connect() to unix:/myproject.sock failed (13: Permission denied) while connecting to upstream
gunicorn OSError: [Errno 1] Operation not permitted
*1 connect() to unix:/tmp/myproject.sock failed (2: No such file or directory)
etc.
These issues are basically permission issue for connection between Nginx and Gunicorn.
To make things simple, I recommend to give same nginx permission to every file/project/python program you create.
To solve all the issue follow this approach:
First thing is :
Log in to the system as a root user
Create /home/nginx directory.
After doing this, follow as per the website until Create an Upstart Script.
Run chown -R nginx:nginx /home/nginx
For upstart script, do the following change in the last line :
exec gunicorn --workers 3 --bind unix:myproject.sock -u nginx -g nginx wsgi
DONT ADD -m permission as it messes up the socket. From the documentation of Gunicorn, when -m is default, python will figure out the best permission
Start the upstart script
Now just go to /etc/nginx/nginx.conf file.
Go to the server module and append:
location / {
include proxy_params;
proxy_pass http<>:<>//unix:/home/nginx/myproject.sock;
}
REMOVE <>
Do not follow the digitalocean aricle from here on
Now restart nginx server and you are good to go.
I had the same problem and found out that I had set the DJANGO_SETTINGS_MODULE to production settings in the the gunicorn script and the wsgi settings were using dev.
I pointed the DJANGO_SETTINGS_MODULE to dev and everything worked.