My django app have two task out of which one is periodic task.
normal task: AddScore
periodic task: CalculateTopScore
class CalculateTopScore(celery.Task):
default_retry_delay = settings.DEFAULT_RETRY_DELAY
max_retries = settings.DEFAULT_MAX_RETRIES
name = 'games.tasks.CalculateTopScore'
def run(self):
try:
# Code to run
except Exception, err:
logger.exception("Error in running task calculate_top_score")
self.retry(exc=err)
return True
def on_failure(self, exc, task_id):
failure = "%s task for calculate_top_score failed permanently." % task_id
logger.error(failure)
def on_success(self):
task_info = 'calculate_top_score task successfully'
logger.info(task_info)
I want to execute this task periodically every 30 minute.
Here are the settings i'm using:
#celery/settings.py
import djcelery
import kombu
from celery.schedules import crontab
from config.celery import exchanges
djcelery.setup_loader()
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
CELERYBEAT_SCHEDULE = {
"calcluate_score": {
"task": "games.tasks.CalculateTopScore",
"schedule": crontab(minute='*/30'),
"args": (),
},
}
CELERY_QUEUES = (
kombu.Queue('add_score',
exchange=exchanges.add_score_exchange,
routing_key='add.scores'),
kombu.Queue('calcluate_score',
exchange=exchanges.calcluate_score_exchange,
routing_key='calculate.scores'),
)
CELERY_ROUTES = ('config.celery.routers.CeleryTaskRouter',)
# Default delay(in seconds) for retrying tasks.
DEFAULT_RETRY_DELAY = 60
# Maximum retry count
DEFAULT_MAX_RETRIES = 6
CELERY_IGNORE_RESULT = True
exchanges.py file
#exchanges.py
from kombu import Exchange
add_score_exchange = Exchange('add_score', type='direct')
calcluate_score_exchange = Exchange('calcluate_score', type='direct')
routes.py file
ROUTES = {
'players.tasks.AddScore': {
'exchange': 'add_score',
'exchange_type': 'direct',
'routing_key': 'add.score',
},
'games.tasks.CalculateTopScore': {
'exchange': 'calculate_score',
'exchange_type': 'direct',
'routing_key': 'calculate.score',
},
}
class CeleryTaskRouter(object):
""" This is a basic celery task router.
"""
def route_for_task(self, task, arg=None, kwargs=None):
return ROUTES.get(task)
On our production server i run celeryd with following arguments: celeryd worker -B
Now in logs i observe that celerybeat schedules the task every 30 minutes but worker doesn't know at all about the task scheduled and hence its not executed.
Why? any config/settings missing? How to execute a class based task periodically?
Please help
You probably have to specify the queues you want celery to get tasks from. Default is to use the celery queue.
Try:
celeryd worker -B -Q add_score,calculate_score
Related
By default Celery send all tasks to 'celery' queue, but you can change this behavior by adding extra parameter:
#task(queue='celery_periodic')
def recalc_last_hour():
log.debug('sending new task')
recalc_hour.delay(datetime(2013, 1, 1, 2)) # for example
Scheduler settings:
CELERYBEAT_SCHEDULE = {
'installer_recalc_hour': {
'task': 'stats.installer.tasks.recalc_last_hour',
'schedule': 15 # every 15 sec for test
},
}
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
Run worker:
python manage.py celery worker -c 1 -Q celery_periodic -B -E
This scheme doesn't work as expected: this workers sends periodic tasks to 'celery' queue, not 'celery_periodic'. How can I fix that?
P.S. celery==3.0.16
Periodic tasks are sent to queues by celery beat where you can do everything you do with the Celery API. Here is the list of configurations that comes with celery beat:
https://celery.readthedocs.org/en/latest/userguide/periodic-tasks.html#available-fields
In your case:
CELERYBEAT_SCHEDULE = {
'installer_recalc_hour': {
'task': 'stats.installer.tasks.recalc_last_hour',
'schedule': 15, # every 15 sec for test
'options': {'queue' : 'celery_periodic'}, # options are mapped to apply_async options
},
}
I found solution for this problem:
1) First of all I changed the way for configuring periodic tasks. I used #periodic_task decorator like this:
#periodic_task(run_every=crontab(minute='5'),
queue='celery_periodic',
options={'queue': 'celery_periodic'})
def recalc_last_hour():
dt = datetime.utcnow()
prev_hour = datetime(dt.year, dt.month, dt.day, dt.hour) \
- timedelta(hours=1)
log.debug('Generating task for hour %s', str(prev_hour))
recalc_hour.delay(prev_hour)
2) I wrote celery_periodic twice in params to #periodic_task:
queue='celery_periodic' option is used when you invoke task from code (.delay or .apply_async)
options={'queue': 'celery_periodic'} option is used when celery beat invokes it.
I'm sure, the same thing is possible if you'd configure periodic tasks with CELERYBEAT_SCHEDULE variable.
UPD. This solution correct for both DB based and file based storage for CELERYBEAT_SCHEDULER.
And if you are using djcelery Database scheduler, you can specify the queue on the Execution Options -> queue field
I use celery as MQ for a project about face recognition.
I have three task queues "task_gpu0, task_gpu1, task_download" to provide service for gunicorn flask server, which uses redis as broker and backend.
When I use jmeter to stress the server, after about 20 min, the program raise a exception:
OperationalError: Error 104 while writing to socket. Connection reset by peer.
On checking redis log, I find client_recent_max_output_buffer and connected_clients continue to increase in info clients. But when I check the key-value by redis desktop manager, the result is ok.
I don't know why the redis output buffer and connected_clients continue to increase.
redis log:
:15:M 01 Apr 2019 09:11:18.113 # Client id=58081 addr=172.16.3.22:33832 fd=54 name= age=428 idle=0 flags=P db=1 sub=74995 psub=0 multi=-1 qbuf=79 qbuf-free=32689 obl=0 oll=452 omem=9267808 events=rw cmd=subscribe scheduled to be closed ASAP for overcoming of output buffer limits.
info clients:
connected_clients:587
client_recent_max_input_buffer:4
client_recent_max_output_buffer:55524832
blocked_clients:2
task_download.py
from celery import Celery
app = Celery()
app.config_from_object("celery_app_tmp.celeryconfig")
#app.task
def download(addImageInput, faceSetId):
download_someting()
return result_dict
task0.py is same as task1.py
from celery import Celery
app = Celery()
app.config_from_object("celery_app_tmp.celeryconfig")
#app.task
def faceRec(addImageInput, faceSetId):
do someting()
return result_dict
celeryconfig.py
from kombu import Queue
from kombu import Exchange
result_serializer = 'msgpack'
task_serializer = 'msgpack'
accept_content = ['json', 'msgpack']
broker_url = "redis://:redis#172.16.3.22:7369/1"
result_backend = "redis://:redis#172.16.3.22:7369/1"
worker_concurrency = 8
result_exchange_type = 'direct'
result_expires = 5
task_queues = (
Queue('gpu_0', exchange=Exchange('gpu_0'), routing_key='gpu_0'),
Queue('gpu_1', exchange=Exchange('gpu_1'), routing_key='gpu_1'),
)
task_routes = {
'celery_app_tmp.task0.faceRec': {'queue': 'gpu_0', 'routing_key': 'gpu_0'},
'celery_app_tmp.task1.faceRec': {'queue': 'gpu_1', 'routing_key': 'gpu_1'},
'celery_app_tmp.task_download.download': {'queue': 'download', 'routing_key': 'download'},
'celery_app_tmp.task_download.delFile': {'queue': 'download', 'routing_key': 'download'}
}
main.py
import celery_app_tmp.task0
import celery_app_tmp.task1
import celery_app_tmp.task_download
result_exc = chain(celery_app_tmp.task_download.download.s(addImageInput), random.choice(list_task).faceRec.s(faceSetId))()
while True:
if result_exc.ready():
dict_exc = result_exc.get()
break
I have my celery configured as following:
# create context tasks in celery
celery = Celery(
__name__,
# redis
backend=app.config['CELERY_RESULT_BACKEND'],
broker=app.config['CELERY_BROKER_URL'],
include=['app.celery_tasks.tasks']
)
celery.conf.timezone = 'US/Pacific'
celery.conf.broker_transport_options = {'visibility_timeout': 3600*24}
celery.conf.task_routes = {
'tasks.periodic': {
'queue': 'periodic',
'routing_key': 'tasks.periodic'
},
'tasks.generate_report': {
'queue': 'report',
'routing_key': 'tasks.generate_report'
}
}
Then I'm using this helper method to get eta for all my report tasks
def get_eta_time(time_d=8):
tz = timezone('US/Pacific')
ct = datetime.now(tz=tz)
eta = ct + timedelta(hours=time_d)
return eta
The thing I'm encountering is I can see the tasks are scheduled using the celery control but they are not executed when the eta arrives. However, when I tried to restart my celery workers, these tasks got picked up immediately. Is there anything I missed in my celery config?
My tasks are triggered as following:
eta = get_eta_time()
generate_report.apply_async(args=(log_location, repetition_count+1), queue='report', eta=eta)
My periodical queue works as expected, but my report queue is not making any sense to me.
Turned out to be some hardware issue. I switch to a different machine and it is working now.
As I know, since Celery 3.1 decorator #periodic_task is depricated.
So I am trying to run an example from celery docs, and can't realise, what am I doing wrong.
I have the following code in task_planner.py:
from celery import Celery
from kombu import Queue, Exchange
class Config(object):
CELERY_QUEUES = (
Queue(
'try',
exchange=Exchange('try'),
routing_key='try',
),
)
celery = Celery('tasks',
backend='redis://',
broker='redis://localhost:6379/0')
celery.config_from_object(Config)
celery.conf.beat_schedule = {
'planner': {
'task': 'some_task',
'schedule': 5.0,
},
}
#celery.task(queue='try')
def some_task():
print('Hooray')
And when I run: celery -A task_planner worker -l info -B, I recieve only the following: [2016-11-27 19:06:56,119: INFO/Beat] Scheduler: Sending due task planner (some_task) every 5 sec.
But I am expecting the output 'Hooray'.
So, what am I missing?
Have found the solution.
I had the task:
#celery.task(queue='try')
def some_task():
print('Hooray')
I printed it's name:
print(some_task)
Got the following:
<#task: task_planner.some_task of tasks:0x7fceaaf5b9e8>
So I just changed the name of the task from some_task to task_planner.some_task here:
celery.conf.beat_schedule = {
'planner': {
'task': 'task_planner.some_task',
'schedule': 5.0,
},
}
And it worked!
[2016-11-29 10:09:57,697: WARNING/PoolWorker-3] Hooray
Note. You should run beat with worker (if task in the same module as beat) and loglevel 'info' in order to see the results:
celery -A task_planner worker -B -l info
I use celery with redis as a broker.
I build out an add task and run two workers listen to different queues for testing.
celeryd -I tasks -l info -Q tasks
celeryd -I tasks -l info -Q count
Here is the tasks.py
from celery.task import task
#task(exchange="tasks")
def add(x, y):
result = x + y
return "I am queue 2.", result
However, no matter I assigned the queue or not, both queues run the task.
Please let me know if there is something I misunderstand. Thanks a lot.
The following is the celeryconfig.py
BROKER_URL = "redis://localhost:6379/0"
# Redis Backend
CELERY_RESULT_BACKEND = "redis"
CELERY_REDIS_HOST = "localhost"
CELERY_REDIS_PORT = 6379
CELERY_REDIS_DB = 0
CELERY_SEND_EVENTS = True
CELERY_RESULT_BACKEND = "amqp"
CELERY_RESULT_ENGINE_OPTIONS = {"echo": True}
# CELERY_DEFAULT_QUEUE = "default"
CELERY_DEFAULT_EXCHANGE = "default"
CELERY_QUEUES = {
"default": {
"exchange": "default"
},
"tasks": {
"exchange": "tasks"
},
"count": {
"exchange": "tasks"
}
}