Push notification fan-out

​

The shape of the pipeline

1. Producers (api processes) do the cheap bits: persist, broadcast over the wire, enqueue one job per non-sender recipient. Producer errors are swallowed — a failed enqueue must NEVER roll back the state change that triggered it. The message is already in Mongo, the socket broadcast has already fired; push is a latency-tolerant nice-to-have.
2. Consumer (worker process) does everything else: recipient lookup, mute check, throttle / group state machine, FCM send, invalid-token cleanup. The consumer can crash, retry, run behind, fan out across multiple worker instances — the recipient’s experience just sees push latency increase.

​

The three producers

​

The job payload

export type PushJobKind = 'message.new' | 'match.new' | 'tenancy.approved' | 'critical';

export interface PushJobPayload {
  kind: PushJobKind;
  recipientUserId: string;
  /** Required when kind === 'message.new'. */
  conversationId?: string;
  /** Required when kind === 'match.new'. */
  matchId?: string;
  /** Display name used in message.new / match.new titles. */
  senderName?: string;
  /** Override for kind === 'critical'. Required for that kind. */
  title?: string;
  /** Override body. For message.new this is the preview text. */
  body?: string;
}

​

The throttle + group state machine

1. Every message.new push for a given (recipientUserId, conversationId) pair records its timestamp into a Redis sorted set with score = nowMs. Entries older than 30s are pruned in the same script.
2. The post-pruning count is the “count within the last 30s window”.
3. If the count is 1 (first hit), the push goes through as a single-shot send.
4. If the count is > 1, the worker bumps a per-key pending counter:
   • If pending < 3 (GROUP_AT), the worker returns — this push is suppressed.
   • If pending >= 3, the worker sends a single “N new messages” group push with data.grouped: 'true', then resets pending to 0.

msg 1 → window count 1 → single send (delivered)            FCM call #1
msg 2 → window count 2, pending=1 → suppressed
msg 3 → window count 3, pending=2 → suppressed
msg 4 → window count 4, pending=3 → GROUP send "3 new messages"  FCM call #2
                                  → pending reset to 0
msg 5 → window count 5, pending=1 → suppressed   (waiting for the next group threshold)

export const MESSAGE_WINDOW_MS = 30_000;  // 30s rolling window
export const GROUP_AT = 3;                // group every Nth suppressed message

​

Mute semantics

if (user.notificationsMuted && payload.kind !== 'critical') {
  log.debug({ userId, kind }, 'push: muted, skipping');
  return;
}

​

REST + WebSocket parity

• A fresh send (deduped: false) → push enqueued.
• A retry (deduped: true) → no push enqueued (the original send already enqueued one).

​

FCM and the platform field

​

The PushSender port

export interface PushSender {
  send(req: PushSendRequest): Promise<PushResult>;
}

export interface PushResult {
  successCount: number;
  failureCount: number;
  /** Tokens FCM reported as invalid (404 UNREGISTERED / 410 GONE / SenderId mismatch). */
  invalidTokens: string[];
}

• FcmPushSender (production) — wraps firebase-admin.messaging().sendEachForMulticast(...). Requires FCM_SERVICE_ACCOUNT_JSON env. Returns invalidTokens based on FCM error codes.
• InMemoryPushSender (dev / test) — records every send into an in-memory log, never fails, returns invalidTokens: []. Used when FCM_SERVICE_ACCOUNT_JSON is unset — see Deferred items.

​

Invalid token cleanup

• messaging/registration-token-not-registered — token was uninstalled / FCM rotated it.
• messaging/invalid-registration-token — malformed.
• messaging/sender-id-mismatch — token is for a different Firebase project.

​

Failure isolation

​

Why the worker, not the API process

1. FCM round-trips are slow and unbounded. A single API request shouldn’t carry a 200-2000ms FCM call. Off-thread is the right place.
2. Bursts need queueing. A group chat with 20 participants where one user sends a message generates 19 push jobs. Doing those inline blocks the response on 19 FCM round-trips.
3. The throttle is stateful and distributed. It can’t live in-process — multiple API instances would each maintain their own counts. The worker centralises it through Redis.

​

Deferred items

• Real FCM credentials in production. The FCM_SERVICE_ACCOUNT_JSON env is blank in dev → the worker uses InMemoryPushSender, which logs every push attempt but never calls a real device. The first production deploy will set this env and switch to FcmPushSender. See QUESTIONS.md §7.1 for the credential-provisioning task.
• Sender display name in push titles. Today the chat producer passes senderId as senderName, so the push title is a 24-char ObjectId. Resolving to a display name needs either a small UsersLookupPort.getDisplayName(userId) or a denorm field on conversations.participants[]. Phase 5 follow-up.
• Trial-ending push. No periodic scan job exists; subscriptions.trialEndsAt is captured but unused for push. Phase 7. See Paywall and launch phase.
• Delivered-receipt round-trip. The Message.deliveredTo[] field is unpopulated today. Flipping it from the FCM delivery callback is a follow-up — see Socket events — out of scope.
• Per-conversation mutes. Only the global notificationsMuted flag is wired.
• Image / rich-media push. Payload data is text only. FCM supports notification images but no producer attaches them.
• Critical-kind producers. The kind is in the dispatch table but no Phase 5 producer enqueues critical jobs. Phase 7 will use it for account-state events.

​

See also

• Register push token — the per-device opt-in.
• Toggle notification mute — the global mute the worker honours.
• Socket events — the in-app live channel that runs in parallel with push.
• Paywall and launch phase — covers trial-ending push deferment.
• apps/worker/src/jobs/push-delivery.ts — the consumer source of truth.
• apps/api/src/lib/push-queue.ts — the producer-side port.