#14 Pub/Sub & Fan-out
Notification Service
Send push, SMS and email notifications for many products, respecting each user's preferences, without duplicates or floods.
What is being asked
Design a shared notification platform. Any internal service can ask to notify a user; the platform chooses channels, applies preferences and limits, sends through external providers and tracks delivery.
Requirements
Functional
- Channels: mobile push, SMS, email, in-app.
- Templates, per-user preferences and quiet hours, opt-out.
- Delivery status and retries.
Non-functional
- Millions of notifications a day, with spikes for campaigns.
- Urgent messages (a login code) must not wait behind marketing.
- Do not send the same notification twice.
Design
- A caller sends an event with a user, a template and data, plus an idempotency key.
- The API checks the key, loads the user's preferences and devices, renders the template and enqueues one message per chosen channel, on a queue by priority.
- Workers per channel call the external provider, handling its rate limits, and record the outcome.
- Failures are retried with back-off; permanent failures (an invalid device token) update the user's record.
- Provider callbacks for delivered, bounced and opened update the delivery log.
Key decisions and trade-offs
- Separate queues by priority and channel so a slow SMS provider or a marketing blast cannot delay a one-time password.
- De-duplication by idempotency key for a window, because callers retry and queues deliver at least once.
- Fan-out for broadcasts. A message to ten million users is expanded by workers in batches, not by the API in one request.
- Provider failover. Keep two providers per channel and switch on error rates; this adds cost and differing behaviour.
Follow-up questions
- How do you enforce "at most three marketing notifications a day" per user?
- How would you batch several events into one digest?
- What do you do when a provider accepts a message but never confirms delivery?
- How do you honour quiet hours across time zones?