Huey is a little task queue that supports:

  • multi-process, multi-thread or greenlet task execution models
  • schedule tasks to execute at a given time, or after a given delay
  • schedule recurring tasks, like a crontab
  • automatically retry tasks that fail
  • task result storage
  • consumer publishes event stream, allowing high-fidelity monitoring

How to use:


# install redis acts as queue for task storage in huey
$ sudo apt-get install redis-server
$ sudo /etc/init.d/redis-server start

# install huey
$ pip install huey

# [optional] install greenlet if for IO-bound tasks
$ pip install greenlet

Create huey config

from huey import RedisHuey

huey = RedisHuey('test', host='localhost')

Create tasks

from config import huey
from huey import crontab
from datetime import datetime

def print_hello():

def print_world():

def print_ok():

Create main executor

from config import huey
from tasks import print_hello, print_world, print_ok

if __name__ == "__main__":

Run scheduler

# with process for cpu-intensive tasks
$ main.huey -k process -w 4

# with gevent for io-bound tasks
$ main.huey -k greenlet -w 10

# save log to file
$ main.huey -k process -w 4 --logfile=../logs/huey.log


Post tagged with: python