Monitor a background worker from anywhere with one import

Background workers are the hardest part of a Python application to observe. They run in separate processes, produce output that disappears into a log file or a container's stdout, and when they misbehave the only symptom is a queue that won't drain or a job that silently fails.

Adding BunnyLogs to a worker takes one import and gives you a live stream URL you can watch from any device.

Celery

The cleanest approach for Celery is to configure the handler in your celery.py app setup so it applies to every worker process:

# myproject/celery.py
import os
import logging
from celery import Celery
from celery.signals import worker_process_init
from bunnylogs import BunnyLogsHandler

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

app = Celery("myproject")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()

@worker_process_init.connect
def configure_worker_logging(**kwargs):
    handler = BunnyLogsHandler(os.environ["BUNNYLOGS_UUID"])
    handler.setLevel(logging.INFO)
    logging.getLogger().addHandler(handler)

Start your worker as usual and open the stream URL. You'll see task names, results, and any logger.info() calls from within your tasks.

Per-task streams

For long-running or important tasks, you might want a dedicated stream per task run so you can share a URL with a specific job:

from celery import shared_task
import logging
from bunnylogs import BunnyLogsHandler

@shared_task
def process_report(report_id: int, bunnylogs_uuid: str | None = None):
    logger = logging.getLogger(f"tasks.process_report.{report_id}")
    if bunnylogs_uuid:
        logger.addHandler(BunnyLogsHandler(bunnylogs_uuid))
    logger.setLevel(logging.INFO)

    logger.info(f"Starting report {report_id}")
    # ... task logic ...
    logger.info(f"Report {report_id} complete")

The caller creates a stream, passes the UUID to the task, and can watch that specific run live.

RQ workers

For RQ, configure logging before starting the worker. In your worker startup script:

import logging
import os
from redis import Redis
from rq import Worker, Queue
from bunnylogs import BunnyLogsHandler

logging.getLogger().addHandler(BunnyLogsHandler(os.environ["BUNNYLOGS_UUID"]))

redis_conn = Redis()
queue = Queue(connection=redis_conn)
worker = Worker([queue], connection=redis_conn)
worker.work()

Threading workers

For custom thread pools or concurrent.futures executors, configure the root logger before spawning threads. Child threads inherit the root logger's handlers:

import logging
import concurrent.futures
from bunnylogs import BunnyLogsHandler

logging.getLogger().addHandler(BunnyLogsHandler("your-uuid"))
logging.getLogger().setLevel(logging.INFO)

logger = logging.getLogger(__name__)

def worker_fn(item_id: int) -> str:
    logger.info(f"Processing {item_id}")
    result = do_work(item_id)
    logger.info(f"Finished {item_id}: {result}")
    return result

with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:
    futures = [executor.submit(worker_fn, i) for i in range(100)]
    for f in concurrent.futures.as_completed(futures):
        f.result()  # re-raises exceptions

Distinguishing workers in the stream

If you run multiple worker processes, tag each one with its hostname or process ID so the stream tells you which worker sent each line:

import socket
import os

program = f"worker/{socket.gethostname()}/{os.getpid()}"
handler = BunnyLogsHandler("your-uuid")
# The logger name becomes the program field in the stream
logging.getLogger(program).addHandler(handler)

The live view lets you filter by program, so you can isolate a single misbehaving worker.

Start streaming worker logs →