Celery periodic task doesn't start

Question

I am using celery in my django project to run periodic tasks. Following the standard tutorial from celery's website, here is my project structure project

|_ settings.py
|_ __init__.py
|_ celery_app.py (instead of celery.py)
|_ app
 |_ tasks.py

Relevant parts in settings.py looks like this -

CELERY_RESULT_BACKEND = "amqp"
CELERY_IMPORTS = ["app.tasks"]
CELERY_ALWAYS_EAGER = True 
CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERY_DB_REUSE_MAX = 1
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'America/New York'
CELERYBEAT_SCHEDULE = {
 'test_task': {
 'task': 'tasks.test_task',
 'schedule': timedelta(seconds=5),
 'args': (),
 },
}

celery_app.py looks like this -

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
app = Celery('project')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

tasks.py looks like this -

from __future__ import absolute_import
from celery.utils.log import get_task_logger
from celery import task
from celery_app import app
logger = get_task_logger(__name__)
@app.task
def test_task():
 for i in range(0, 4):
 logger.debug("celery test task")

When I run the celery worker, I can see my task being discovered -

$ python manage.py celery -A project worker --loglevel=DEBUG --app=celery_app:app
 -------------- [email protected] v3.1.19 (Cipater)
---- **** ----- 
--- * *** * -- Darwin-15.4.0-x86_64-i386-64bit
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app: project:0x10d5b9a10
- ** ---------- .> transport: amqp://sgcelery1:**@localhost:5672/sgceleryhost
- ** ---------- .> results: djcelery.backends.database:DatabaseBackend
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery exchange=celery(direct) key=celery
[tasks]
 . celery.backend_cleanup
 . celery.chain
 . celery.chord
 . celery.chord_unlock
 . celery.chunks
 . celery.group
 . celery.map
 . celery.starmap
 . app.tasks.test_task
[2016年04月24日 23:54:55,452: INFO/MainProcess] Connected to amqp://sgcelery1:**@127.0.0.1:5672/sgceleryhost
[2016年04月24日 23:54:55,471: INFO/MainProcess] mingle: searching for neighbors
[2016年04月24日 23:54:56,481: INFO/MainProcess] mingle: all alone
[2016年04月24日 23:54:56,497: WARNING/MainProcess] [email protected] ready.

And when I run beat, it shows the task being picked up from settings.py, but it never actually runs.

$ python manage.py celery -A project beat --app=celery_app:app --loglevel=DEBUG
[2016年04月24日 23:55:04,059: DEBUG/MainProcess] Current schedule:
<ModelEntry: test_task tasks.test_task(*[], **{}) {4}>

What am I missing here ?

• DancingJohn: Even when running the worker and beat together the scheduled task doesn't get picked up - python manage.py celery -A project worker -B --loglevel=DEBUG --app=celery_app:app

Answer 1

Try setting CELERY_ALWAYS_EAGER = False.

Setting CELERY_ALWAYS_EAGER = True makes the tasks run synchronously without celery being used. By switching it to False will make celery beat pick it up and run it periodically. The False switch is used in Development mode when you don't want celery to process tasks.

Checkout the docs here

• DancingJohn: CELERY_ALWAYS_EAGER = False didn't make any difference.
• Mevin Babu: Is celery beat running on your machine ? Celery beat needs to be running to do periodic tasks.
• DancingJohn: If you see at the bottom of the original question, I am running both. Worker and then beat. Both through manage.py. Or is there something else to run ?

Answer 2

try to use command:
celery -A project worker -B -E -Q beat --concurrency=1 -l INFO
this link about celery keys and options