I'm trying to host flask API which is using tensorflow libraries.I installed tensorflow gpu library with CUDA and cudnn libraries.I manually checked with the following command which is working fine.
/captcha/env/bin/gunicorn captcha:app -b 0.0.0.0:5124 -k gevent --worker-connections 1000
But when i add this systemd service im getting a tensorflow gpu error
systemd service
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
PIDFile=/run/gunicorn/pid
User=root
Group=root
WorkingDirectory=/captcha/env
ExecStart=/captcha/env/bin/gunicorn captcha:app -b 0.0.0.0:5124 -k gevent --worker-connections 1000
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
Error text in Log file:
Failed to load the native TensorFlow runtime.
See
https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Can anyone point me where I'm doing wrong?
Systemd seems to strip all the environment variables and TensorFlow needs to know where to find Cuda. Without a LD_LIBRARY_PATH it fails.
There are probably a handful of ways to do this, but this worked for me.
[Service]
Environment=LD_LIBRARY_PATH=/usr/local/cuda/lib64
ExecStart=/path/to/your/app
...
I'm using PM2 to manage tensorflow-flask api process.
http://pm2.keymetrics.io/
I Create a shell file with the following command as a content.
pm2 start run.sh
Related
I use this command in docker compose:
newrelic-admin run-program gunicorn main:app -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000
but after running, I encountered with following error:
(14/ThreadPoolExecutor-0_0) newrelic.core.trace_cache ERROR - Runtime instrumentation error. An active trace already exists in the cache on thread_id 281472746332640. Report this issue to New Relic support.
How can I fix it?!
Thanks
I initialized newrelic.ini config in main.py. It works and the errors were fixed:
import newrelic.agent
newrelic.agent.initialize('newrelic.ini')
I am running a virtualenv with Python3.6 on Ubuntu 16.04 for my Django project using uwsgi and NGINX.
I have uWSGI installed globally and also in the virtualenv.
I can run my project from the command line using uWSGI within the env with
/home/user/Env/myproject/bin/uwsgi --http :8080 --home /home/user/Env/myproject --chdir /home/user/myproject/src/myproject -w myproject.wsgi
and go to my domain and it loads fine.
However I am obviously running uWSGI in "Emperor mode" and when I set the service file up (along with NGINX) the domain displays internal server error.
The uWSGI logs trace to --- no python application found ---
I was having this problem when running
uwsgi --http :8080 --home /home/user/Env/myproject --chdir /home/user/myproject/src/myproject -w myproject.wsgi
because it was using the global install uwsgi instead of the virtualenv one.
I changed my StartExec to the virtualenv uwsgi path but no luck.
I can't figure out what I'm doing wrong, path error? Syntax error?
my /etc/systemd/system/uwsgi.service file
[Unit]
Description=uWSGI Emperor service
[Service]
ExecStartPre=/bin/bash -c 'mkdir -p /run/uwsgi; chown user:www-data /run/uwsgi'
ExecStart=/home/user/Env/myproject/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target
Okay bit silly but it seems I ran sudo systemctl stop uwsgi and then sudo systemctl start uwsgi and it works now.
I have been following this tutorial to deploy my django project on Digital Ocean. I am trying to configure gunicorn.
My project structure looks similar to this:
On my settings.py I use DEBUG=False
I create the gunicorn.socket and gunicorn.service.
/etc/systemd/system/gunicorn.socket
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
/etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=someuser
Group=www-data
WorkingDirectory=/home/someuser/myproject
ExecStart=/home/someuser/myproject/myprojectenv/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
Myproject.wsgi:application
[Install]
WantedBy=multi-user.target
I start and enable the Gunicorn socket:
sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket
Check the status of the process to find out whether it was able to start:
sudo systemctl status gunicorn.socket
This is what I get and then it returns to the command line.
Failed to dump process list, ignoring: No such file or directory
β—� gunicorn.socket - gunicorn socket
Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; vendor preset: enabled)
Active: active (listening) since Sat 2019-05-04 23:12:03 UTC; 13s ago
Listen: /run/gunicorn.sock (Stream)
CGroup: /system.slice/gunicorn.socket
May 04 23:12:03 myproject systemd[1]: Listening on gunicorn socket.
Next, I check for the existence of the gunicorn.sock file within the /run directory:
file /run/gunicorn.sock
/run/gunicorn.sock: socket
It seems that a file or directory doesn't exist, but it doesn't provide any more details. The gunicorn.sock seems to exist.
I am familiar with Python but not with servers and deploying so I am at a loss here.
I have successfully followed the similar tutorial for Ubuntu 16.04 a few months ago, but now I keep hitting issues.
I know, that's old question, but...
I haved some error. In my case wrong WorkingDirectory and application with modul wsgi. I think that's error in "myproject.wsgi:application"
This is actually an issue with your usage of systemd. You need to refer to the name of the service in your calls to systemctl, not the socket as you're doing. In your case the service will be called gunicorn because you've named your systemd unit file as gunicorn.service.
systemctl enable gunicorn
systemctl start gunicorn
This is driving me nuts.
A Flask app works fine if i personally run uWSGI from the CLI:
uwsgi --emperor /etc/uwsgi/emperor.ini
but when trying to start it as a service with systemd, there is a segfault and the resulting coredump says almost nothing:
sudo systemctl start emperor.uwsgi
coredump:
[New LWP 7639]
Core was generated by `/usr/bin/uwsgi --ini website.ini'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000123c in ?? ()
So, I have no idea how to get more detailed information. This is not a script I'm running with Python app.py, but is a script being served by uWSGI.
I'm clueless and would appreciate any advice.
Thanks.
Edit I - systemd init script:
[Unit]
Description=uWSGI Emperor
After=syslog.target
[Service]
ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/emperor.ini
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/bin/kill -INT $MAINPID
Restart=always
Type=notify
StandardError=syslog
NotifyAccess=all
KillSignal=SIGQUIT
[Install]
WantedBy=multi-user.target
If i run /usr/bin/uwsgi --ini /etc/uwsgi/emperor.ini manually, works fine.
I'm using flask as a webserver for my UI (it's a simple web interface which controls the recording using gstreamer on ubuntu from a webcam and a framegrabber simultaneously / kinda simple player)
Every time I need to run the command "python main.py" to run the server from command prompt manually.
I've tried the init.d solution or even writing a simple shell script and launching it every time after rebooting the system on start up but it fails to keep the server up and running till the end (just invokes the server and terminates it I guess)
is there any solution that could help me to start the webserver every time after booting the system on startup and keep it on and running?
I'd like to configure my system to boot directly into the browser so don't wanna have any need for more actions by the user.
Any Kind of suggestion/help is appreciated.
I'd like to suggest using supervisor, the documentation is here
for a very simple demo purpose, after you installed it and finish the set up, touch a new a file like this:
[program:flask_app]
command = python main.py
directory = /dir/to/your/app
autostart = true
autorestart = true
then
$ sudo supervisorctl update
Now, you should be good to go. The flask app will start every time after you boot you machine.(note: distribution package has already integrated into the service management infrastructure, if you're using others, see here)
to check whether you app is running:
$ sudo supervisorctl status
For production, you can use nginx+uwsgi+supervisor. The flask deployment documentation is here
One well documented solution is to use Gunicorn and Nginx server:
Install Components and setup a Python virtualenv with dependencies
Create the wsgi.py file :
from myproject import application
if __name__ == "__main__":
application.run()
That will be handled by Gunicorn :
gunicorn --bind 0.0.0.0:8000 wsgi
Configure Gunicorn with setting up a systemd config file: /etc/systemd/system/myproject.service :
[Unit]
Description=Gunicorn instance to serve myproject
After=network.target
[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
Environment="PATH=/home/sammy/myproject/myprojectenv/bin"
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn
--workers 3 --bind unix:myproject.sock -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
Start the Gunicorn service at boot :
sudo systemctl start myproject
sudo systemctl enable myproject