Async task with celery

A quick guide on how to use celery to accomplish async task in application.

How it works for celery-based async tasks

  • Decorated function as celery task
  • Broker (e.g. rabbitmq, redis, and etc.) to queue up the task
  • Celery worker on the background to actually execute the task


# install a broker
$ sudo apt-get install redis-server [or rabbitmq-server]
$ sudo /etc/init.d/redis-server start
or $ sudo /etc/init.d/rabbitmq-server start

# install celery
$ sudo pip install celery
$ sudo pip install redis [if using redis as broker]

Define a task

from celery import Celery

celery_app = Celery('celery_tasks', broker='redis://localhost:6379/0')

def sum_to_one_million():
    return sum(range(1000000))

Start worker

$ cd ~/workspace/celery_project
$ celery -A celery_tasks worker --loglevel=info

Invoke the task

  • in interactive shell
    >> from celery_tasks import sum_to_one_million
    >> print(sum_to_one_million())
  • in a flask app
from flask import Flask
from celery_tasks import sum_to_one_million

app = Flask(__name__)

def calc_sum_sync():
    return sum_to_one_million()

def calc_sum_async():
    return "Calculation started in the background."


celery python