I've been working on a garage door automation application for the Raspberry Pi that will allow you to remote open/close the garage door as well as check the status of the door (whether open or closed). I've posted the code up on GitHub here: Link to GitHub.
From time to time, it seems that the application is simply unreachable over my network as if Flask has simply stopped responding to web requests. I can still, however, SSH into my pi just fine when this happens. A reboot brings the web interface back, with no issues.
After doing some reading, as I understand it, the built-in webserver in Flask is really not robust and shouldn't be used in a production environment, so I decided that I would try to setup Gunicorn and nginx to handle the job instead. I modified the code and tried running Gunicorn. Unfortunately, Gunicorn seems to report errors with the Threading library (which I'm using to check the status of the door in the background). Here is some of the output from Gunicorn when I try to run it:
pi#raspi-4:~/garage-pi $ sudo gunicorn app:app
[2016-07-17 11:06:08 +0000] [745] [INFO] Starting gunicorn 19.6.0
[2016-07-17 11:06:08 +0000] [745] [INFO] Listening at: http://127.0.0.1:8000 (745)
[2016-07-17 11:06:08 +0000] [745] [INFO] Using worker: sync
[2016-07-17 11:06:08 +0000] [750] [INFO] Booting worker with pid: 750
[2016-07-17 11:06:08 +0000] [750] [INFO] Worker exiting (pid: 750)
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/pi/garage-pi/app.py", line 21, in checkdoorstate
if (RPi.GPIO.input(18) == True and door_state != True):
AttributeError: 'NoneType' object has no attribute 'GPIO'
With all of that said, I'm out of my depth here, and wanted to know if this setup is at all possible, or will I continue to have issues with using the Threading Python library with Gunicorn? Any thoughts on how to resolve this issue?
Many thanks in advance.
You can start your app with multiple workers or async workers with Gunicorn.
Gunicorn with gevent async worker
gunicorn server:app -k gevent --worker-connections 1000
Gunicorn 1 worker 12 threads:
gunicorn server:`app -w 1 --threads 12
Gunicorn with 4 workers (multiprocessing):
gunicorn server:app -w 4
More information on Flask concurrency in this post: How many concurrent requests does a single Flask process receive?.
Source
How to run Flask with Gunicorn in multithreaded mode
Related
I'm trying to start the airflow webserver with an existing application and the latest version (2.2.2). The executor is set as LocalExecutor, the datastore is Postgres, Python v3.9. Upon start up, it launches 4 workers which promptly die. It then spins its wheels continually trying to restart them.
Here's an example of the messages showing a worker starting and dying with signal 11 (segmentation violation). This is all within a second of starting.
Using worker: sync
[2021-11-30 17:29:31 -0500] [12529] [INFO] Booting worker with pid: 12529
[2021-11-30 17:29:31 -0500] [12530] [INFO] Booting worker with pid: 12530
[2021-11-30 17:29:31 -0500] [12531] [INFO] Booting worker with pid: 12531
[2021-11-30 17:29:31 -0500] [12532] [INFO] Booting worker with pid: 12532
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
settings.prepare_engine_args(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=12529
[WARNING] Worker with pid 12529 was terminated due to signal 11
Any suggestions on how to debug these workers?
I gave up on trying to run Airflow directly on my M1 Mac. I suspect it has something to do with the emulator for the M1 chip, but don't know for sure. I fell back to running Airflow in Docker. It takes a very long time to start, but does run OK that way.
so I'm trying to deploy my Dash app onto Azure using Azure Web App. When I use the "Github Actions" method of deployment, I get the following (I even made my repository public for this):
Package deployment using ZIP Deploy initiated.
##[error]Failed to deploy web package to App Service.
##[error]Deployment Failed with Error: Error: Failed to deploy web package to App Service.
Unauthorized (CODE: 401)
##[warning]Error: Failed to update deployment history.
Unauthorized (CODE: 401)
App Service Application URL: http://my-site.azurewebsites.net
I've also tried deploying with "Kudu App Build" instead of the Github Actions and that "deploys successfully", giving me a status of "Success(Active)" but once I go to my website it fails, then when I go to Diagnosis and solve problems in the azure portal I receive the following error:
2020-06-05T14:32:08.567155259Z
2020-06-05T14:32:08.567159059Z Documentation: http://aka.ms/webapp-linux
2020-06-05T14:32:08.567163159Z Python 3.8.0
2020-06-05T14:32:08.567166959Z Note: Any data outside '/home' is not persisted
2020-06-05T14:32:09.126144120Z Starting OpenBSD Secure Shell server: sshd.
2020-06-05T14:32:09.177989757Z Site's appCommandLine: gunicorn --bind=0.0.0.0 --timeout 600 application:app
2020-06-05T14:32:09.179066899Z Launching oryx with: -appPath /home/site/wwwroot -output /opt/startup/startup.sh -virtualEnvName antenv -defaultApp /opt/defaultsite -userStartupCommand 'gunicorn --bind=0.0.0.0 --timeout 600 application:app'
2020-06-05T14:32:09.289253728Z Oryx Version: 0.2.20200114.13, Commit: 204922f30f8e8d41f5241b8c218425ef89106d1d, ReleaseTagName: 20200114.13
2020-06-05T14:32:09.315423056Z Found build manifest file at '/home/site/wwwroot/oryx-manifest.toml'. Deserializing it...
2020-06-05T14:32:09.322220623Z Build Operation ID: |mqhAw8K+i4s=.a5925a0c_
2020-06-05T14:32:09.937709104Z Writing output script to '/opt/startup/startup.sh'
2020-06-05T14:32:10.656204831Z Found virtual environment .tar.gz archive.
2020-06-05T14:32:10.656991662Z Removing existing virtual environment directory /antenv...
2020-06-05T14:32:10.667494675Z Extracting to directory /antenv...
2020-06-05T14:32:42.888599159Z Using packages from virtual environment antenv located at /antenv.
2020-06-05T14:32:42.891102058Z Updated PYTHONPATH to ':/antenv/lib/python3.8/site-packages'
2020-06-05T14:32:44.966320399Z [2020-06-05 14:32:44 +0000] [42] [INFO] Starting gunicorn 20.0.4
2020-06-05T14:32:44.973255871Z [2020-06-05 14:32:44 +0000] [42] [INFO] Listening at: http://0.0.0.0:8000 (42)
2020-06-05T14:32:44.974184808Z [2020-06-05 14:32:44 +0000] [42] [INFO] Using worker: sync
2020-06-05T14:32:45.047728498Z [2020-06-05 14:32:45 +0000] [44] [INFO] Booting worker with pid: 44
2020-06-05T14:32:50.315649084Z Application object must be callable.
2020-06-05T14:32:50.321677821Z [2020-06-05 14:32:50 +0000] [44] [INFO] Worker exiting (pid: 44)
2020-06-05T14:32:50.630621560Z [2020-06-05 14:32:50 +0000] [42] [INFO] Shutting down: Master
2020-06-05T14:32:50.630656561Z [2020-06-05 14:32:50 +0000] [42] [INFO] Reason: App failed to load.
That was the application error, here is the container error
Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2020-06-05T14:30:22.831Z INFO - Initiating warmup request to container my-site_0_06a70784 for site my-site
2020-06-05T14:30:38.890Z INFO - Waiting for response to warmup request for container my-site_0_06a70784. Elapsed time = 16.0594968 sec
2020-06-05T14:30:54.492Z INFO - Waiting for response to warmup request for container my-site_0_06a70784. Elapsed time = 31.6616936 sec
2020-06-05T14:31:09.663Z ERROR - Container my-site_0_06a70784 for site my-site has exited, failing site start
2020-06-05T14:31:09.695Z ERROR - Container my-site_0_06a70784 didn't respond to HTTP pings on port: 8000, failing site start. See container logs for debugging.
2020-06-05T14:31:10.217Z INFO - Stopping site my-site because it failed during startup.
2020-06-05T14:31:11.646Z INFO - Starting container for site
2020-06-05T14:31:11.647Z INFO - docker run -d -p 4040:8000 --name my-site_0_9c00d1f9 -e WEBSITE_SITE_NAME=my-site -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=my-site.azurewebsites.net -e WEBSITE_INSTANCE_ID=b78e390e9ef390f579e4e316c4d4ea6c7f187e4af48b856ebd3562fda3c5ef4f appsvc/python:3.8_20200101.1 gunicorn --bind=0.0.0.0 --timeout 600 application:app
Here is what my application.py looks like (I've tried several variations of this to no avail):
import app as application
app = application.app
if __name__ == "__main__":
app.run_server(debug=False)
And at the top of my app.py file I have:
app = dash.Dash(__name__)
This is my requirements.txt:
bs4==0.0.1
dash==1.12.0
dash-table==4.7.0
Flask==1.1.2
numpy==1.18.0
pandas==1.0.4
requests==2.12.4
Lastly, this is the startup command I use in the Configuration->General Settings->Startup Command
gunicorn --bind=0.0.0.0 --timeout 600 application:app
The application you should run when deployment is dash.Dash.server(in your case it's app.server), it's the underly Flask application. So, you have to update the second line of your application.py to this:
app = application.server
See more in Dash docs.
How to configure gunicorn to make sure my python-flask website is available 24*7 ?
The issue I am facing is: As soon I kill my terminal window, the website is no more reachable.
I am using rhel7.6 to host a website using python-flask.
I have configured nginx as the web server and gunicorn as the application server.
.
I will really appreciate if some one can help me in using/configuring gunicorn to make sure my website is available 24*7.
Please have some of my code as below:
[root#syed-dashboard-4 ~]# pwd
/root
[root#syed-dashboard-4 ~]#
[root#syed-dashboard-4 ~]# cat hello.py
#!/usr/bin/python
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello magentabox!"
#if __name__ == "__main__":
# app.run(host='10.145.29.23',port=5000)
[root#syed-dashboard-4 ~]#
[root#syed-dashboard-4 ~]# gunicorn hello:app
[2019-07-17 10:34:11 +0000] [9346] [INFO] Starting gunicorn 19.9.0
[2019-07-17 10:34:11 +0000] [9346] [INFO] Listening at: http://127.0.0.1:8000 (9346)
[2019-07-17 10:34:11 +0000] [9346] [INFO] Using worker: sync
[2019-07-17 10:34:11 +0000] [9351] [INFO] Booting worker with pid: 9351
I am pretty new to web development and as I mentioned as soon I close the terminal, the website is no more reachable. I can share the nginx configuration logs as well if that helps fixing my issue.
Thanks much.
You can use the supervisor. This is the professional way to run your server for 24*7.
Please add below file in your supervisor config to run.
[program:your_project_name]
command=/home/virtualenvpath/your_env/bin/gunicorn --log-level debug run_apiengine:main_app --bind 0.0.0.0:5006 --workers 5 --worker-class gevent
stdout_logfile=/home/your_path_to_log/supervisor_stdout.log
stderr_logfile=/home/your_path_to_log/supervisor_stderr.log
user=your_user
autostart=true
autorestart=true
environment=PYTHONPATH="$PYTHONPATH:/home/path_to_your_project";OAUTHLIB_INSECURE_TRANSPORT='1';
Configure this in supervisor it will run for 24*7. And whenever your machine restarts it will auto start.
On Linux, you can start your app in a tmux session that you detach once you started your server.
# Create a new tmux session
tmux new -s server
# Start your gunicorn server
cd /path/to/app
gunicorn hello:app
# Detach the current tmux session using Ctrl - B + D
You can close your terminal and your server will still be running.
I am running django application serving with nginx and gunicorn in supervisor,I am getting gunicorn timeout
errors in gunicorn error logs but don't know what causing the errors.
2018-10-01 20:20:19 [20529] [CRITICAL] WORKER TIMEOUT (pid:20646)
2018-10-01 20:20:19 [23948] [INFO] Booting worker with pid: 23948
Is there a way we can configure gunicorn to write logs entry before the gunicorn process is timeout and killed ?
You can try by increasing the timeout value to a bigger value than the default 30sec
Correct me if I am wrong: I can use gunicorn to deploy a django project, for instance I can deploy my app - helloapp in this way:
$ cd env
$ . bin/activate
(env) $ cd ..
(env) $ pip install -r requirements.txt
(env) root#localhost:/var/www/html/helloapp# gunicorn helloapp.wsgi:application
[2017-05-18 22:22:38 +0000] [1779] [INFO] Starting gunicorn 19.7.1
[2017-05-18 22:22:38 +0000] [1779] [INFO] Listening at: http://127.0.0.1:8000 (1779)
[2017-05-18 22:22:38 +0000] [1779] [INFO] Using worker: sync
[2017-05-18 22:22:38 +0000] [1783] [INFO] Booting worker with pid: 1783
So now my django site is running at http://127.0.0.1:8000.
But it will not be available anymore as soon as I close/ exit my terminal. So how can I have it stayed connected to the port 8000 even if I have closed my terminal?
As with any long-running process, you need to run it as a service under some kind of manager. Since you're on Ubuntu, you probably want to use systemd; full instructions are in the gunicorn deployment docs. Note, you will also need to configure nginx as a reverse proxy in front of gunicorn.