Develop Reference

Constantly dying workers Flask-SocketIO + uWSGI

I'm trying to implement sockets usage in my flask project and followed by flask-socketio documentation I had made it run perfectly with gunicorn, but found myself unable to make it work using uwsgi. It seems like it should be working even with the simplest app and settings but it just keeps killing the workers whatever configuration I try to use. Here's an example of an environment and code I have:
app.py:
from flask import Flask
from flask_socketio import SocketIO
app = Flask(__name__)
app.config['SECRET_KEY'] = 'extremelysecret'
socketio = SocketIO(app)
if __name__ == '__main__':
app.run()
pip freeze:
click==7.1.2
Flask==1.1.2
Flask-SocketIO==4.3.0
gevent==20.5.0
greenlet==0.4.15
gunicorn==20.0.4
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
python-engineio==3.12.1
python-socketio==4.5.1
six==1.15.0
uWSGI==2.0.18
Werkzeug==1.0.1
Running with an example from documentation:
uwsgi --http :5000 --gevent 1000 --http-websockets --master --wsgi-file app.py --callable app
Output:
your processes number limit is 63299
your memory page size is 4096 bytes
detected max file descriptor number: 1024
- async cores set to 10 - fd table size: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :5008 fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:39875 (port auto-assigned) fd 3
Python version: 3.8.0 (default, Jan 9 2020, 23:03:43) [GCC 7.4.0]
Python main interpreter initialized at 0x55899af6a860
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 501072 bytes (489 KB) for 20 cores
*** Operational MODE: preforking+async ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55899af6a860 pid: 22479
(default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 22479)
spawned uWSGI worker 1 (pid: 22481, cores: 10)
spawned uWSGI worker 2 (pid: 22482, cores: 10)
*** running gevent loop engine [addr:0x558998a7f860] ***
spawned uWSGI http 1 (pid: 22483)
DAMN ! worker 1 (pid: 22481) died :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 22484)
DAMN ! worker 2 (pid: 22482) died :( trying respawn ...
Respawned uWSGI worker 2 (new pid: 22485)
*** running gevent loop engine [addr:0x558998a7f860] ***
DAMN ! worker 1 (pid: 22484) died :( trying respawn ...
So far I've tried:
- playing with uwsgi parameters;
- using different app configurations in app.py;
- using couple different versions of python;
- figuring out is everything installed properly;
And still, my guess is that I miss something obvious here. Thanks in advance for pointing me on that.

uWSGI unable to load Flask app [duplicate]

This question already has answers here:
Flask and uWSGI - unable to load app 0 (mountpoint='') (callable not found or import error)
(3 answers)
Gunicorn can't find app when name changed from "application"
(2 answers)
Closed 5 years ago.
I have been poking around on this for a couple days and I am still having issues. My uWSGI instance is apparently not loading my Flask app. I am running a CentOS 7 Vagrant and using Ansible for config mgmt. I will post the final templated files
My file setup is as follows
#/etc/tickets/tickets.ini
[uwsgi]
module = wsgi
master = true
processes = 4
socket = tickets.sock
chmod-socket = 664
uid = uwsgi
gid = nginx
vacuum = true
die-on-term = true
logto2 = /var/log/uwsgi/tickets.log
Module
#/opt/tickets/wsgi.py
import os
from create import create_app
app = create_app(os.getenv('APP_CONFIG') or 'default')
if __name__ == '__main__':
app.run(host='0.0.0.0')
Systemd service
#/etc/systemd/system/tickets.service
[Unit]
Description=uWSGI instance to serve tickets
After=network.target
[Service]
User=uwsgi
Group=nginx
WorkingDirectory=/opt/tickets
ExecStart=/usr/bin/uwsgi --ini /etc/tickets/tickets.ini
[Install]
WantedBy=multi-user.target
Here is my apps uwsgi log
[vagrant#tickets uwsgi]$ cat tickets.log
*** Operational MODE: preforking ***
/usr/lib/python2.7/site-packages/flask_sqlalchemy/__init__.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 14820)
spawned uWSGI worker 1 (pid: 14886, cores: 1)
spawned uWSGI worker 2 (pid: 14887, cores: 1)
spawned uWSGI worker 3 (pid: 14888, cores: 1)
spawned uWSGI worker 4 (pid: 14889, cores: 1)
I'm at a loss right now. I've tried moving the .ini for the uwsgi into the directory with wysgi.py but that didn't work either. ticket.sock does get created in the /opt/tickets directory. The service is running but it again it appears it is not loading the app.
EDIT
Modeled after this tutorial.

Attempting to deploy a python pyramid application with uWSGI

I'm attempting to deploy a pyramid application using uWSGI.
The application works fine when served with the included pyramid development server.
Also, I have set this up before, and I swear it worked at one time.
However, putting in the magic phrases right now is resulting in "This webpage is not available"
I'm trying to keep all of the configuration parameters as similar as possible to what I have currently so I don't have to worry about firewall issues.
uWSGI section in development.ini looks like this (from: Setup uWSGI as webserver with pyramid (no NGINX)):
[uwsgi]
socket = localhost:8080
virtualenv = /var/www/finance/finance-env
die-on-term = 1
master = 1
#logto = /var/log/wsgi/uwsgi.log
enable-threads = true
offload-threads = N
py-autoreload = 1
wsgi-file = /var/www/finance/wsgi.py
wsgy.py looks like this:
from pyramid.paster import get_app, setup_logging
ini_path = '/var/www/finance/corefinance/development.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
Here's the output right now. Everything seems to be listening just fine on port 8080.
user1#finance1:~$ sudo /var/www/finance/finance-env/bin/uwsgi --ini-paste-logg /var/www/finance/corefinance/development.ini
[uWSGI] getting INI configuration from /var/www/finance/corefinance/development.ini
*** Starting uWSGI 2.0.11.2 (64bit) on [Fri Jan 15 21:13:31 2016] ***
compiled with version: 4.7.2 on 16 November 2015 20:13:35
os: Linux-4.1.5-x86_64-linode61 #7 SMP Mon Aug 24 13:46:31 EDT 2015
nodename: finance1
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/user1
detected binary path: /var/www/finance/finance-env/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 3934
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address localhost:8080 fd 3
Python version: 3.2.3 (default, Feb 20 2013, 14:49:46) [GCC 4.7.2]
Set PythonHome to /var/www/finance/finance-env
Python main interpreter initialized at 0xfd0a10
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xfd0a10 pid: 6275 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 6275)
spawned uWSGI worker 1 (pid: 6282, cores: 1)
Python auto-reloader enabled

Unless you are behind a proxy such as nginx, you need to use the internal http routing support in uwsgi, change
socket = localhost:8080
to
http = 0.0.0.0:8080
Here are the uwsgi http support docs

UWSGI - 500s, appears overloaded

I've built an application with Flask, and I've deployed it to servers with UWSGI and NGINX web servers on AWS.
I have 2 sets of servers, Web Servers and Application servers. Requests to the Application servers are proxied through the Web Servers.
When I do any number of API calls in a row, I start getting random 500s to some requests. I know the issue isn't related to my servers getting overloaded as I can see CPU Utilization running around 10%. I'm also pretty confident that the issue isn't related to my application code or database timeouts, as no matter how hard I try, I can't replicate the behavior on the Flask development web server.
I'm using UWSGI Emperor and checking the logs, I see no application/python errors. I just see the fact that a 500 is being returned. eg:
[pid: 2938|app: 0|req: 8/8] 86.40.78.178 () {42 vars in 1038 bytes} [Thu Sep 17 23:06:33 2015] GET /api/issue/231/workflow_items/ => generated 291 bytes in 84 msecs (HTTP/1.1 500) 6 headers in 557 bytes (1 switches on core 4)
As this is my first time working with UWSGI, I'm wondering if I'm doing anything wrong in terms of my configuration. Here's what my start up log looks like:
*** Starting uWSGI 2.0.11.1 (64bit) on [Thu Sep 17 23:06:20 2015] ***
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-16) on 17 September 2015 23:03:32
os: Linux-3.14.48-33.39.amzn1.x86_64 #1 SMP Tue Jul 14 23:43:07 UTC 2015
nodename: ip-10-0-1-90
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /etc/uwsgi/vassals
detected binary path: /var/www/workspace/venv/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your processes number limit is 7826
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/www/workspace/workspace_uwsgi.sock fd 3
Python version: 2.7.9 (default, Apr 1 2015, 18:18:03) [GCC 4.8.2 20140120 (Red Hat 4.8.2-16)]
Set PythonHome to /var/www/workspace/venv
Python main interpreter initialized at 0x1b3bdc0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 341832 bytes (333 KB) for 10 cores
*** Operational MODE: preforking+threaded ***
added /var/www/workspace/ to pythonpath.
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1b3bdc0 pid: 2933 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 2933)
spawned uWSGI worker 1 (pid: 2938, cores: 5)
spawned uWSGI worker 2 (pid: 2939, cores: 5)
Here is what my UWSGI .ini looks like:
[uwsgi]
#application's base folder
base = /var/www/workspace
#python module to import
app = issuesite
module = %(app)
home = %(base)/venv
pythonpath = %(base)
#socket file's location
socket = /var/www/workspace/%n.sock
#permissions for the socket file
chmod-socket = 666
#the variable that holds a flask application inside the module imported at line #6
callable = app
processes=2
threads=5
#location of log files
logto = /var/log/uwsgi/%n.log
and the upstart script for emperor:
description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]
respawn
env UWSGI=/var/www/workspace/venv/bin/uwsgi
env LOGTO=/var/log/uwsgi/emperor.log
exec $UWSGI --master --emperor /etc/uwsgi/vassals --die-on-term --uid nginx --gid nginx --enable-threads --logto $LOGTO
I saw something about --thunder-lock in the logs so I tried it but it made no difference, I still experience the same issues.
I wonder if anyone could give me some advice? I considered the possibility that the UWSGI is running out of memory? Note that using t2.micros on AWS.

How to get uwsgi working with nginx and Django 1.7?

This is what I tried:
I have a django.conf inside /etc/nginx/sites-available/
upstream django {
server unix:///tmp/uwsgi.sock;
}
server {
listen 80;
server_name weasyprint.django.dev;
charset utf-8;
error_log /var/log/nginx/django-weasyprint.log;
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
And then I do a sudo ln -s /etc/nginx/sites-available/django.conf inside sites-enabled
And then I restarted nginx.
I created a django folder called weasyprint_site inside /var/virtual/WebApps/virtualenvs/WeasyPrintProject
I used virtualenv on the same folder, so now my structure is like this:
WeasyPrintProject
|---------bin
|---------include
|---------lib
|---------local
|---------share
|---------weasyprint_site
|------------db.sqlite3
|------------manage.py
|------------test.py
|------------uwsgi.ini
|------------weasyprint_site
and then I also put in a uwsgi.ini as you can see above.
Its contents are:
[uwsgi]
socket=/tmp/uwsgi.sock
chmod-socket=666
uid = www-data
gid = www-data
chdir=/var/virtual/WebApps/virtualenvs/WeasyPrintProject
module=weasy_print.wsgi:application
master=true
pidfile=/var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasy_print.pid
vacuum=true
When I am at /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasyprint_site,
I run
uwsgi --ini uwsgi.ini
I get the following:
[uWSGI] getting INI configuration from uwsgi.ini
*** Starting uWSGI 2.0.9 (64bit) on [Thu Feb 19 11:59:12 2015] ***
compiled with version: 4.8.2 on 16 February 2015 05:39:16
os: Linux-3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014
nodename: vagrant-ubuntu-trusty-64
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasyprint_site
writing pidfile to /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasy_print.pid
detected binary path: /var/virtual/WebApps/virtualenvs/WeasyPrintProject/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
chdir() to /var/virtual/WebApps/virtualenvs/WeasyPrintProject
your processes number limit is 15934
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.6 (default, Mar 22 2014, 23:03:41) [GCC 4.8.2]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1915160
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
ImportError: No module named weasy_print.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1913)
spawned uWSGI worker 1 (pid: 1914, cores: 1)
I have 2 issues:
I cannot get back to bash unless I do a Ctrl+C
there is an import error saying no module named weasy_print.wsgi
I followed the instructions in here
My virtualenv for this particular project is to use python 2.7.4 and I am attempting to run django 1.7. The environment is Ubuntu 14.04
How do I get the django app to work.

Firstly, you're chdiring into WeasyPrintProject, but your project is in WeasyPrintProject/weasyprint_site. You should chdir into that directory inside your uwsgi.ini file.
Secondly, your project name is weasyprint_site, not weasy_print, so you should call module weasyprint_site.wsgi:application instead.
And last problem is: you should specify path to your virtualenv inside uwsgi.ini, so uwsgi process can now where are additional packages required by your application.
To return to your console without interrupting your uwsgi server, you must put it in background, by forking it, daemonizing it or simply launching from init script. I personally recomment using emperor/vassals system built in into wsgi.
Also having your project inside virtualenv directory is not recommended.