Queueing Paystack Webhook Work with Celery

Django handles HTTP requests synchronously by default. When Paystack sends a webhook, your view runs from top to bottom, and Django does not send the response until the view function returns. If your view updates a database, sends an email, and calls a third-party API, all that work happens before Paystack gets its 200 response.

Paystack has a timeout. If your endpoint does not respond quickly, Paystack marks the delivery as failed and tries again later. Now you have two copies of the same event coming in, and if your handler is still slow, you get three, then four. The problem compounds.

Celery solves this by moving the heavy work out of the request-response cycle entirely. Your view does the minimum (verify signature, dispatch task, return 200), and a separate Celery worker process handles the business logic on its own schedule. If the worker fails, Celery retries the task without any involvement from Paystack.

If you are building a Paystack integration in Python, you are almost certainly using Django or Flask. Celery integrates deeply with both, but the Django integration is especially mature. This guide covers the Django setup. The Flask approach is nearly identical, just swap the view for a route handler.

Install Celery and Redis (the most common broker for Django projects):

pip install celery redis

Create the Celery application in your Django project. This file lives next to your settings.py:

# myproject/celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

app = Celery('myproject')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

Make sure your project's __init__.py loads the Celery app:

# myproject/__init__.py
from .celery import app as celery_app

__all__ = ('celery_app',)

Add Celery settings to your settings.py:

# settings.py
CELERY_BROKER_URL = os.environ.get('REDIS_URL', 'redis://localhost:6379/0')
CELERY_RESULT_BACKEND = os.environ.get('REDIS_URL', 'redis://localhost:6379/0')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT = 300  # 5-minute hard limit per task

The CELERY_TASK_TIME_LIMIT setting kills any task that runs longer than 5 minutes. This prevents a single stuck task from blocking a worker forever. For webhook processing, 5 minutes is generous. Most tasks should complete in under a second.

Create a view that receives the Paystack webhook, verifies the signature, and dispatches a Celery task:

# payments/views.py
import hashlib
import hmac
import json

from django.conf import settings
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST

from .tasks import process_paystack_event


@csrf_exempt
@require_POST
def paystack_webhook(request):
    payload = request.body
    signature = request.headers.get('X-Paystack-Signature', '')
    secret = settings.PAYSTACK_SECRET_KEY

    expected = hmac.new(
        secret.encode('utf-8'),
        payload,
        hashlib.sha512,
    ).hexdigest()

    if not hmac.compare_digest(expected, signature):
        return HttpResponse(status=401)

    event_data = json.loads(payload)

    process_paystack_event.delay(
        event_type=event_data['event'],
        data=event_data['data'],
    )

    return HttpResponse(status=200)

Key details in this view:

• @csrf_exempt is required. Paystack's webhook requests do not include Django CSRF tokens. Without this decorator, Django returns 403 Forbidden for every webhook. See Webhooks and CSRF Protection in Django for a deeper look at why this is safe.
• @require_POST rejects anything that is not a POST request. Paystack always sends POST.
• hmac.compare_digest prevents timing attacks. Do not use == to compare the signature strings. The constant-time comparison in hmac.compare_digest is the correct approach.
• .delay() dispatches the task to the Celery worker immediately and returns without waiting for the result. This is what keeps the view fast.

Wire the view into your URL configuration:

# payments/urls.py
from django.urls import path
from . import views

urlpatterns = [
    path('webhook/paystack/', views.paystack_webhook, name='paystack-webhook'),
]

Create the task that does the actual processing. This runs in the Celery worker, completely separate from the Django web process:

# payments/tasks.py
import logging
from celery import shared_task
from django.db import IntegrityError

logger = logging.getLogger(__name__)


@shared_task(
    bind=True,
    autoretry_for=(Exception,),
    retry_backoff=True,
    retry_backoff_max=600,
    max_retries=5,
    acks_late=True,
)
def process_paystack_event(self, event_type, data):
    logger.info('Processing %s: %s', event_type, data.get('reference', 'N/A'))

    if event_type == 'charge.success':
        handle_charge_success(data)
    elif event_type == 'transfer.success':
        handle_transfer_success(data)
    elif event_type == 'transfer.failed':
        handle_transfer_failed(data)
    elif event_type == 'refund.processed':
        handle_refund_processed(data)
    else:
        logger.info('Ignoring unhandled event: %s', event_type)


def handle_charge_success(data):
    reference = data['reference']
    amount_naira = data['amount'] / 100

    # Idempotency check: skip if already processed
    # order = Order.objects.filter(reference=reference).first()
    # if order and order.status == 'paid':
    #     logger.info('Already processed: %s', reference)
    #     return

    # Update order status
    # Order.objects.filter(reference=reference).update(status='paid')

    # Send confirmation
    # send_payment_receipt.delay(reference)

    logger.info('charge.success processed for %s (NGN %s)', reference, amount_naira)


def handle_transfer_success(data):
    reference = data['reference']
    logger.info('transfer.success processed for %s', reference)
    # Mark payout as completed in your database


def handle_transfer_failed(data):
    reference = data['reference']
    logger.info('transfer.failed processed for %s', reference)
    # Mark payout as failed, potentially re-queue or alert


def handle_refund_processed(data):
    reference = data.get('transaction', {}).get('reference', 'unknown')
    logger.info('refund.processed for transaction %s', reference)
    # Update refund status in your database

Let us unpack the task decorator options:

• bind=True gives the task access to self, which includes retry metadata like self.request.retries (how many times this task has been retried).
• autoretry_for=(Exception,) tells Celery to automatically retry the task if any exception is raised. For production, you might narrow this to specific exceptions like database connection errors.
• retry_backoff=True enables exponential backoff between retries: 1 second, 2 seconds, 4 seconds, 8 seconds, and so on.
• retry_backoff_max=600 caps the backoff at 10 minutes, so retries do not space out indefinitely.
• max_retries=5 stops retrying after 5 attempts. After that, the task is marked as failed.
• acks_late=True tells Celery to acknowledge the message only after the task completes successfully. If the worker crashes mid-task, the message goes back to the broker and gets redelivered to another worker.

Start the Celery worker from the command line:

celery -A myproject worker --loglevel=info

In development, this is enough. The worker prints log output to your terminal, and you can see tasks being received and processed in real time.

For production, you need the worker to run as a managed service that starts on boot and restarts if it crashes. The two most common approaches:

Supervisord configuration:

[program:celery-worker]
command=/path/to/venv/bin/celery -A myproject worker --loglevel=info --concurrency=4
directory=/path/to/your/project
user=www-data
autostart=true
autorestart=true
stdout_logfile=/var/log/celery/worker.log
stderr_logfile=/var/log/celery/worker_error.log

Systemd unit file:

[Unit]
Description=Celery Worker for Paystack Webhooks
After=network.target redis.service

[Service]
Type=forking
User=www-data
Group=www-data
WorkingDirectory=/path/to/your/project
ExecStart=/path/to/venv/bin/celery -A myproject worker \
    --loglevel=info \
    --concurrency=4 \
    --detach
Restart=always

[Install]
WantedBy=multi-user.target

The --concurrency=4 flag tells Celery to run 4 worker processes (technically, "prefork" child processes). Each can handle one task at a time. For a typical Paystack integration doing 100 to 500 transactions per day, a concurrency of 2 to 4 is plenty. Increase it if you are processing thousands of events daily.

If you are deploying with Docker, run the worker as a separate container using the same image as your Django app but with the Celery start command instead of the Gunicorn command.

By default, all Celery tasks go to a single queue called "celery." As your Paystack integration handles more event types, you may want separate queues so payment events do not get stuck behind a backlog of less urgent events.

Define your queues in settings:

# settings.py
from kombu import Queue

CELERY_TASK_QUEUES = [
    Queue('payments', routing_key='payments'),
    Queue('transfers', routing_key='transfers'),
    Queue('default', routing_key='default'),
]

CELERY_TASK_DEFAULT_QUEUE = 'default'

Route tasks based on event type in your view:

# In paystack_webhook view
event_type = event_data['event']

if event_type.startswith('charge.'):
    queue_name = 'payments'
elif event_type.startswith('transfer.'):
    queue_name = 'transfers'
else:
    queue_name = 'default'

process_paystack_event.apply_async(
    kwargs={'event_type': event_type, 'data': event_data['data']},
    queue=queue_name,
)

Then start separate workers for each queue:

# Payment worker (higher concurrency, higher priority)
celery -A myproject worker --queues=payments --concurrency=4 --loglevel=info

# Transfer worker
celery -A myproject worker --queues=transfers --concurrency=2 --loglevel=info

# Default worker (everything else)
celery -A myproject worker --queues=default --concurrency=2 --loglevel=info

This way, a sudden batch of transfer events (common if you do bulk payouts) will not delay processing of incoming customer payments.

Flower is the standard monitoring tool for Celery. It gives you a web dashboard showing real-time task status, worker health, and queue depths.

pip install flower
celery -A myproject flower --port=5555

Open http://localhost:5555 in your browser. Flower shows:

• Active tasks. What is being processed right now.
• Processed and failed counts. Overall success and failure rates.
• Worker status. Whether each worker is online, how many tasks it has processed, and its current load.
• Task details. Click into any task to see its arguments, return value, exception traceback, and retry history.

In production, protect Flower behind authentication. You do not want anyone on the internet seeing your webhook payloads. Use the built-in basic auth:

celery -A myproject flower --basic_auth=admin:your_password --port=5555

For automated monitoring, Celery also emits events that you can capture with custom scripts or forward to monitoring tools like Sentry, Datadog, or Prometheus. At minimum, set up alerts for:

• Tasks stuck in the "started" state for more than 60 seconds
• The number of failed tasks exceeding a threshold (more than 5 per hour is a warning sign)
• Workers going offline

After 5 retries (with exponential backoff), a task is marked as permanently failed. In a Paystack integration, a permanently failed task means a payment event was received but not processed. A customer might have paid and never received their product, or a transfer might have completed without your books being updated.

You need visibility into these failures. Here are your options:

1. Celery's on_failure callback. Override the task's on_failure method to trigger an alert:

@shared_task(bind=True, max_retries=5, autoretry_for=(Exception,), retry_backoff=True)
def process_paystack_event(self, event_type, data):
    # ... processing logic ...
    pass

@process_paystack_event.on_failure.connect
def handle_task_failure(sender, task_id, exception, args, kwargs, **kw):
    # Send alert to Slack, PagerDuty, or email
    event_type = kwargs.get('event_type', 'unknown')
    reference = kwargs.get('data', {}).get('reference', 'N/A')
    logger.critical(
        'PAYSTACK WEBHOOK FAILED PERMANENTLY: %s for %s. Error: %s',
        event_type, reference, str(exception)
    )

2. Sentry integration. If you use Sentry for error tracking, install sentry-sdk and it will automatically capture Celery task failures with full context.

3. Manual replay. If you are storing raw webhook payloads (and you should be), you can replay failed events manually. See Replaying Failed Webhooks Safely and Storing Raw Webhook Payloads for Audit for the complete approach.

The combination of automatic retries (for transient failures) and alerting on permanent failures (for real bugs) means you catch problems at both levels. Most transient failures resolve themselves within the 5 retries. The ones that do not are genuine issues that need a developer's attention.