The shape of the pipeline
Push notifications in Swappr are emphatically not sent from the request thread. Every push goes through BullMQ: The contract is:- Producers (api processes) do the cheap bits: persist, broadcast over the wire, enqueue one job per non-sender recipient. Producer errors are swallowed — a failed enqueue must NEVER roll back the state change that triggered it. The message is already in Mongo, the socket broadcast has already fired; push is a latency-tolerant nice-to-have.
- Consumer (worker process) does everything else: recipient lookup, mute check, throttle / group state machine, FCM send, invalid-token cleanup. The consumer can crash, retry, run behind, fan out across multiple worker instances — the recipient’s experience just sees push latency increase.
The three producers
All three enqueue via the
PushQueuePort abstraction — a port whose production implementation is a thin BullMQ wrapper around PUSH_DELIVERY_QUEUE. The port is mockable in tests so service-level tests can assert on what was enqueued without spinning up Redis.
The chat producer passes
senderId as senderName today (and the worker uses it verbatim as the push title). A UsersRepository.findById that returns display name doesn’t exist yet — this is documented in the code as a Phase 5 follow-up, NOT a Phase 5 bug. Same caveat for match — senderName is the other user’s id.The job payload
collapseKey is what makes a burst of pushes on the same conversation/match coalesce in the OS notification tray rather than stack five-deep.
The throttle + group state machine
The bulk of the worker’s complexity sits in handling chat bursts. A user receiving five messages in quick succession should NOT get five OS-level push notifications — they get one, then a brief silence, then a coalesced “3 new messages” group push. The state machine, in words:- Every
message.newpush for a given(recipientUserId, conversationId)pair records its timestamp into a Redis sorted set with score =nowMs. Entries older than 30s are pruned in the same script. - The post-pruning count is the “count within the last 30s window”.
- If the count is
1(first hit), the push goes through as a single-shot send. - If the count is
> 1, the worker bumps a per-keypendingcounter:- If
pending < 3(GROUP_AT), the worker returns — this push is suppressed. - If
pending >= 3, the worker sends a single “N new messages” group push withdata.grouped: 'true', then resetspendingto 0.
- If
push:throttle:<userId>:<conversationId>. Bulk-clearing all throttles for a user is SCAN push:throttle:<userId>:* + DEL.
The throttle is per
(userId, conversationId), not global. Two simultaneous conversations send pushes independently. A high-volume conversation cannot starve other notifications.Mute semantics
The worker honours the user’snotificationsMuted flag BEFORE consulting tokens or throttle:
Critical is reserved for events the user MUST see regardless of preferences — account banned, password changed, security incident. Phase 5 ships the kind dispatch; no Phase 5 producer enqueues a
critical job yet.
REST + WebSocket parity
chat.message.service is called from both the REST handler (POST /chat/conversations/:id/messages) and the WebSocket handler (chat:message:send). The push enqueue happens inside the service, AFTER the dedup check — so:
- A fresh send (
deduped: false) → push enqueued. - A retry (
deduped: true) → no push enqueued (the original send already enqueued one).
(senderId, clientMessageId) pair, matching the REST/WS broadcast contract. See Socket events — At-least-once delivery and dedup for the dedup mechanics.
FCM and the platform field
Per QUESTIONS.md §7.1, Swappr uses FCM as the single push sender for both Android and iOS. iOS devices register FCM tokens via Firebase’s iOS SDK, which proxies to APNs server-side. From the worker’s perspective there is noif (platform === 'ios') sendApns(...) branch — every token is an FCM token.
The platform field on pushTokens[] is kept for analytics (split rates by platform, debug per-platform delivery issues) but plays no role in the send path.
The PushSender port
FcmPushSender(production) — wrapsfirebase-admin.messaging().sendEachForMulticast(...). RequiresFCM_SERVICE_ACCOUNT_JSONenv. ReturnsinvalidTokensbased on FCM error codes.InMemoryPushSender(dev / test) — records every send into an in-memory log, never fails, returnsinvalidTokens: []. Used whenFCM_SERVICE_ACCOUNT_JSONis unset — see Deferred items.
Invalid token cleanup
FCM returns one of three error codes for tokens it can’t deliver to:messaging/registration-token-not-registered— token was uninstalled / FCM rotated it.messaging/invalid-registration-token— malformed.messaging/sender-id-mismatch— token is for a different Firebase project.
FcmPushSender collects these into result.invalidTokens[]. The worker then calls users.removeInvalidPushTokens(userId, invalidTokens) which pulls them off the user’s pushTokens array. Next push attempt → fewer dead tokens → less wasted FCM quota.
This means the explicit DELETE /me/push-tokens/:token endpoint is rarely strictly required — the cleanup happens automatically as soon as the next push fires. Clients still SHOULD call DELETE on sign-out to stop wasted push attempts and to respect the user’s intent immediately.
Failure isolation
Three layers of failure handling, in order of likelihood:
The contract is: a successful state change in Mongo NEVER fails because push failed. This is enforced at the producer layer via
try/catch + log + swallow.
Why the worker, not the API process
Three reasons:- FCM round-trips are slow and unbounded. A single API request shouldn’t carry a 200-2000ms FCM call. Off-thread is the right place.
- Bursts need queueing. A group chat with 20 participants where one user sends a message generates 19 push jobs. Doing those inline blocks the response on 19 FCM round-trips.
- The throttle is stateful and distributed. It can’t live in-process — multiple API instances would each maintain their own counts. The worker centralises it through Redis.
Deferred items
Phase 5 explicitly does NOT ship the following — clients / operators should not depend on them existing yet:- Real FCM credentials in production. The
FCM_SERVICE_ACCOUNT_JSONenv is blank in dev → the worker usesInMemoryPushSender, which logs every push attempt but never calls a real device. The first production deploy will set this env and switch toFcmPushSender. See QUESTIONS.md §7.1 for the credential-provisioning task. - Sender display name in push titles. Today the chat producer passes
senderIdassenderName, so the push title is a 24-char ObjectId. Resolving to a display name needs either a smallUsersLookupPort.getDisplayName(userId)or a denorm field onconversations.participants[]. Phase 5 follow-up. - Trial-ending push. No periodic scan job exists;
subscriptions.trialEndsAtis captured but unused for push. Phase 7. See Paywall and launch phase. - Delivered-receipt round-trip. The
Message.deliveredTo[]field is unpopulated today. Flipping it from the FCM delivery callback is a follow-up — see Socket events — out of scope. - Per-conversation mutes. Only the global
notificationsMutedflag is wired. - Image / rich-media push. Payload data is text only. FCM supports notification images but no producer attaches them.
- Critical-kind producers. The kind is in the dispatch table but no Phase 5 producer enqueues
criticaljobs. Phase 7 will use it for account-state events.
See also
- Register push token — the per-device opt-in.
- Toggle notification mute — the global mute the worker honours.
- Socket events — the in-app live channel that runs in parallel with push.
- Paywall and launch phase — covers trial-ending push deferment.
apps/worker/src/jobs/push-delivery.ts— the consumer source of truth.apps/api/src/lib/push-queue.ts— the producer-side port.