Celery worker configuration.
Source: web/backend/tasks/celery_app.py
| Setting | Value | Description |
|---|---|---|
| Broker | Redis (from Web Config) | Task queue |
| Backend | Redis | Result storage |
| Serialization | JSON | Task/result format |
| Task tracking | Enabled | Track task state |
| Acks | Late | Acknowledge after completion |
| Timeout | 24 hours | Soft time limit |
Sets up logging via Web Logging when the worker process starts.
| Task | Module | Description |
|---|---|---|
run_pipeline_task |
Pipeline Tasks | Run a pipeline for a job |
# Start worker
celery -A web.backend.tasks.celery_app:celery_app worker --loglevel=info
# Or via Makefile
cd web && make worker
Celery App
├── Celery
├── Redis
├── Web Config (REDIS_URL)
├── Web Logging (setup_logging)
└──▶ hosts Pipeline Tasks