What this is
Swappr has a dedicated worker process (@swappr/worker) that runs separately from the API. It consumes four BullMQ queues for the heavy / scheduled / long-running work that should not block API request handlers. This page covers:
- The four queues and what they do.
- How both the producer (api) and consumer (worker) sides flip between in-memory stubs and BullMQ on the same
REDIS_URLsignal. - Scheduling — what runs on cron, what reacts to enqueue.
- The graceful shutdown order.
- The stub-mode behaviour for
pnpm devand unit tests. - The deferred-at-credentials matrix.
apps/worker/src/index.ts (bootstrap), apps/api/src/index.ts (producer selection), SWAPPR_BACKEND_PLAN §13 + §14.
The four queues
Bootstrap diagram
REDIS_URL — one signal, two sides
The producer and consumer both check the same env var:- API process (
apps/api/src/index.ts) —resolvePushQueue,resolveRetentionEnqueuer,resolveGdprExportQueue,resolveErasureEnqueuerall key offprocess.env.REDIS_URL. With it: BullMQ-backed producer. Without it: a no-op enqueuer (returnsPromise.resolve()— the API still returns202 Acceptedso the surface looks identical to the caller, but no work happens). - Worker process (
apps/worker/src/index.ts) —bootWorker({ redisUrl }). With it: constructs all fourWorkerinstances + a shared ioredis connection + schedules the two repeatable jobs. Without it: returns aWorkerHandlein'no-worker'mode and exits the bootstrap path with a warn log.
The api-side selector is required to inject a
queueFactory when REDIS_URL is present (production caller must construct the BullMQ Queue). The selector deliberately throws if REDIS_URL is set but no factory is supplied — fail loud, not silently fall back to no-op when ops thinks BullMQ is wired.Scheduling — cron vs reactive
The 03:00 / 04:00 offset is intentional — retention runs first so a tenancy document that’s both “30d post-decision” AND “owned by a soft-deleted user” is removed by retention through its dedicated path before erasure sees it. See GDPR data lifecycle.
Graceful shutdown
OnSIGTERM or SIGINT the worker process closes resources in order:
- Workers (stop accepting new jobs; let in-flight jobs finish to BullMQ’s
lockDurationceiling). - Queues (drain producer-side state).
- Shared ioredis connection (
.quit()— graceful FIN, not.disconnect()).
Stub-mode behaviour
WhenREDIS_URL is absent:
This means a developer can
pnpm dev with no Redis installed and the entire HTTP surface works — the only thing missing is the asynchronous side-effects (push notifications, retention cleanup, export ZIP generation, erasure tombstoning).
Deferred at credentials
These are wired up but gated on a production credential that isn’t in Phase 7’s scope:
RevenueCat is not in this list — Phase 7 has no RevenueCat worker. The Phase 5 RevenueCat webhook stays in stub mode (
BILLING_NOT_CONFIGURED) until the real RevenueCat secret is wired.
Why a separate process?
Three reasons (BACKEND_PLAN §13):- Blast radius. A worker crash from a malformed FCM token, a runaway ZIP, or an S3 hang must not take the API down.
- Scaling profile. The API scales on request volume (spiky). The worker scales on queue depth (smoother). They need different deployment knobs.
- Deploy cadence. Worker bug fixes (e.g. a new push payload field) can ship without an API redeploy.
See also
- Push notification fan-out — how push messages reach
push-delivery. - Tenancy retention — full lifecycle of
retention-cleanup. - GDPR data lifecycle — full lifecycle of
gdpr-export+gdpr-erasure. - Trigger retention run / Trigger erasure run — manual SUPER kicks.
apps/worker/src/index.ts—bootWorkersource of truth.apps/api/src/index.ts— the four producer selectors.