I cannot see the tasks in admin.
I followed the steps in https://github.com/jezdez/django-celery-monitor
I used
celery==4.1.1
django-celery-results==1.0.1
django-celery-beat==1.0.1
django_celery_monitor==1.1.2
ran manage.py migrate celery_monitor The migrations went well. ran celery -A lbb events -l info --camera django_celery_monitor.camera.Camera --frequency=2.0 and celery -A lbb worker -l info in separate shells. But still cannot see the tasks I ran in celery-monitor > tasks table.
Running celery command with -E to force event worked for me.
celery -A proj worker -l info -E
Related
I actually had issues printing mine on command prompt because I was using the wrong command but I found a link to a project which I forked Project
(If on Mac ) celery -A Project worker --loglevel=info
(If on Windows) celery -A Project worker -l info --pool=solo
Run
celery -A Project worker --loglevel=info
in project folder where your celery object is created.
You can use the file name to get your tasks discovered.
For Example: if your file name is tasks.py and celery object is created in that file.
Command will be:
celery -A tasks worker --loglevel=info
celery -A app worker -Q priority_high -B -l debug --purge -n priority_high_worker
celery -A app worker -Q default -B -l debug --purge -n default_worker
celery -A app beat -l info
As of now we are running the three commands in screens. What is the more production way of running these commands?
The easiest way to create daemons is with supervisord. sentry, which also uses django and celery recommends using supervisord to run the workers - you can adjust the configuration to suit your set up:
[program:celery-priority-high]
directory=/www/my_app/
command=/path/to/celery -A app worker -Q priority_high -B -l debug --purge -n priority_high_worker
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=syslog
stderr_logfile=syslog
You can, of course, also run django itself using this method.
If supervisord is too much bloat for your needs, you can also create init scripts for your init system of choice (eg. systemd).
The scenario:
Two unrelated web apps with celery background tasks running on same server.
One RabbitMQ instance
Each web app has its own virtualenv (including celery). Same celery version in both virtualenvs.
I use the following command lines to start a worker and a beat process for each application.
celery -A firstapp.tasks worker
celery -A firstapp.tasks beat
celery -A secondapp.tasks worker --hostname foobar
celery -A secondapp.tasks beat
Now everything seems to work OK, but in the worker process of secondapp I get the following error:
Received unregistered task of type 'firstapp.tasks.do_something'
Is there a way to isolate the two celery's from each other?
I'm using Celery version 3.1.16, BTW.
I believe I fixed the problem by creating a RabbitMQ vhost and configuring the second app to use that one.
Create vhost (and set permissions):
sudo rabbitmqctl add_vhost /secondapp
sudo rabbitmqctl set_permissions -p /secondapp guest ".*" ".*" ".*"
And then change the command lines for the second app:
celery -A secondapp.tasks -b amqp://localhost//secondapp worker
celery -A secondapp.tasks -b amqp://localhost//secondapp beat
In my Procfile I have the following:
worker: cd appname && celery -A appname worker -l info --app=appname.celery_setup:app
However, when my app submits a task it never happens, but I think the celery worker is at least sort of working, because
heroku logs --app appname
every so often gives me one of these:
2016-07-22T07:53:21+00:00 app[heroku-redis]: source=REDIS sample#active-connections=14 sample#load-avg-1m=0.03 sample#load-avg-5m=0.09 sample#load-avg-15m=0.085 sample#read-iops=0 sample#write-iops=0 sample#memory-total=15664884.0kB sample#memory-free=13458244.0kB sample#memory-cached=187136kB sample#memory-redis=566800bytes sample#hit-rate=0.17778 sample#evicted-keys=0
Also, when I open up bash by running
heroku run bash --app appname
and then type in
cd appname && celery -A appname worker -l info --app=appname.celery_setup:app
It immediately tells me the task has been received and then executes it. I would like to have this happen without me having to manually log in and execute the command - is that possible? Do I need a paid account on heroku to do that?
I figured it out. Turns out you also have to do
heroku ps:scale worker=1 --app appname
Or else you won't actually be running a worker.
I use celery, supervisor and flower
supervisor conf:
command=/web/venv/bin/python manage.py celeryd -E -l info -c 2 --maxtasksperchild=1 -Ofair
directory=/web/
stdout_logfile=/web/log/
redirect_stderr=true
user=web
killasgroup=true
stopasgroup=true
Problem when start from python shell, task works very fast. If from celery, works very very long. I don't know what the problem. Maybe someone can help ?
you can achieve the same by adding celery from admin, that might solution
http://www.yourhost.com/admin/djcelery/