Fan-out with SNS and SQS is the pattern I reuse most often in serverless projects: something happens once, and several parts of the system react to it without knowing about each other. Sounds trivial. Even so, I keep seeing Lambda functions that call three other services in a row after saving data and trip over the second call.
In this part of the series, I build the pattern for the fictional bookshop from the first two parts. An order comes in, and the warehouse, accounting and express shipping each get their own copy. This time everything is built with the Serverless Framework, including filters, dead-letter queues and tests. At the end, I explain when I prefer EventBridge over SNS and when combining the two makes sense.
In short
With fan-out, a service publishes an event once to an SNS topic, and every consumer reads it from its own SQS queue. No consumer fails because another one is stuck, and each gets its own retries and its own dead-letter queue. The most common pitfall is a missing queue policy: the subscription exists, but SNS is not allowed to send to the queue.
What you will learn in this article
- How fan-out with SNS and SQS works and why every queue has exactly one consumer
- How to filter on message attributes and on the message body with filter policies
- A complete, tested project with serverless.yml and TypeScript
- When EventBridge is the better choice and when to combine both
- The typical mistakes that silently swallow a delivery
What does fan-out mean?
Fan-out means spreading one thing out into many. A sender sends a message exactly once, and a piece of infrastructure hands copies of it to every interested consumer. The sender does not know the consumers. All it knows is that there is a topic it reports to.
On AWS, this role usually goes to Amazon SNS (Simple Notification Service). An SNS topic is a distributor: you publish to it, and every subscription receives a copy. Subscribers can be Lambda functions, HTTP endpoints, email addresses or SQS queues. A queue holds messages until someone picks them up and processes them.
For the bookshop, it looks like this: the function that accepts orders reports “order placed” on the topic. The warehouse reserves the books, accounting writes the invoice, and express shipping books a courier, but only for express orders.
The architecture at a glance: one topic, three queues, three workers, each queue with its own dead-letter queue.
Why fan-out with SNS and SQS instead of direct calls?
The obvious solution would be a function that calls three services one after another once the order is saved. That works fine until the invoice service is slow for a minute. Then order intake waits, the customer waits, and if the call times out, you do not know whether the warehouse has already reserved the books. On top of that, you have to touch the sender’s code for every new consumer.
Fan-out turns three dependencies into one: the sender only needs the topic. New consumers attach with a subscription, and nothing changes on the sender’s side. In addition, every consumer works at its own pace.
Why every consumer needs its own queue
A common misunderstanding: “I will just attach three Lambda functions to the same SQS queue.” That is not fan-out, it is competition. With a queue, only one consumer ever receives a given message. The three functions would split the orders between them instead of each getting all of them.
That is why every consumer gets its own queue in a fan-out setup. This gives you three more things I would not want to miss. If accounting goes down, its messages pile up in its queue while the warehouse keeps working. Each queue gets its own dead-letter queue (DLQ), a parking spot for messages that still fail after several attempts. And you can control batch size and parallelism per consumer.
Why not subscribe the Lambda functions to SNS directly? You can, and for non-critical cases that is perfectly fine. With a queue in between, though, you get a buffer: load peaks are smoothed out, you can reprocess messages calmly, and partial failures in a batch can be reported cleanly.
🎬 Animation: fan-out with a filter – this animated diagram only runs on my site: watch it on sascha-ms.de
SNS or EventBridge: which one fits when?
In the second part of the series, I distributed domain events via an EventBridge bus, and in the article on SQS and EventBridge with Lambda, a rule routes events into a queue. So both services can distribute. The difference lies in the details:
| Criterion | SNS + SQS | EventBridge |
|---|---|---|
| Consumers per distribution point | up to 12.5 million subscriptions per standard topic | at most 5 targets per rule, but many rules per bus |
| Filtering | filter policy per subscription, on attributes or body | event pattern per rule, always on the body |
| Throughput (eu-central-1) | 1,500 messages per second per account (soft limit) | 2,400 PutEvents per second (soft limit) |
| Targets | SQS, Lambda, HTTP, email, SMS, Firehose | more than 20 AWS services, API destinations, other buses and accounts |
| Extras | FIFO topics with ordering | archive and replay, schema registry, input transformation |
My rule of thumb: if there is a clearly defined event with a fixed group of consumers and a lot of load, I use SNS with SQS. It is simple, fast and cheap. If domain events flow across several teams or accounts and I want to add new rules later or replay old events, I use EventBridge. And sometimes I combine both: EventBridge as the central bus, a rule forwards to an SNS topic, and SNS fans out to many queues behind it.
If you are unsure which service fits your case, my AWS service finder helps you with a few questions.
Fan-out with SNS and SQS step by step
Time to build. I use the Serverless Framework because the previous part was built with the AWS CDK. If you want to compare the two tools, you can find my take in the article Serverless Framework vs CDK vs SST.
Prerequisites
- Node.js 22 or newer and npm
- An AWS account with credentials in your terminal (for example via
aws configureor AWS SSO) - Serverless Framework v4 (
npm install -g serverless); since v4 you need a free Serverless Inc. account on first start, and small organizations below the revenue threshold pay nothing - Basic knowledge of Lambda and SQS; if you are missing it, part 1 of the series helps
Step 1: Create the project
Create a folder and install the dependencies. Since v4, the framework builds TypeScript itself with esbuild, so you do not need a plugin for that.
mkdir bookshop-fanout && cd bookshop-fanout
npm init -y
npm install @aws-sdk/client-sns
npm install -D typescript vitest @types/aws-lambda @types/node serverless
mkdir src test
Then replace the package.json and create the tsconfig.json:
// package.json
{
"name": "bookshop-fanout",
"version": "1.0.0",
"private": true,
"scripts": {
"typecheck": "tsc --noEmit",
"test": "vitest run",
"deploy": "serverless deploy --stage dev",
"remove": "serverless remove --stage dev"
},
"dependencies": {
"@aws-sdk/client-sns": "^3.1156.0"
},
"devDependencies": {
"@types/aws-lambda": "^8.10.164",
"@types/node": "^22.20.0",
"serverless": "^4.45.0",
"typescript": "^7.0.2",
"vitest": "^5.0.5"
}
}
// tsconfig.json
{
"compilerOptions": {
"target": "ES2023",
"module": "ESNext",
"moduleResolution": "Bundler",
"strict": true,
"noEmit": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src", "test"]
}
Step 2: Define the event format
Before anything gets sent, I define what the message looks like. An event always gets a type, a unique ID and a timestamp. The ID matters later because SNS and SQS only deliver “at least once”. A worker can therefore see the same message twice and has to recognize that.
// src/order-events.ts
export type ShippingMethod = "standard" | "express";
export interface OrderItem {
isbn: string;
quantity: number;
}
export interface OrderPlaced {
eventType: "OrderPlaced";
eventId: string;
orderNo: string;
shipping: ShippingMethod;
items: OrderItem[];
totalCents: number;
placedAt: string;
}
// Validates the request body and returns an error message or the data
export function validateOrder(
body: unknown,
): { ok: true; data: Omit<OrderPlaced, "eventType" | "eventId" | "placedAt"> } | { ok: false; error: string } {
if (typeof body !== "object" || body === null) {
return { ok: false, error: "Body must be a JSON object" };
}
const b = body as Record<string, unknown>;
if (typeof b.orderNo !== "string" || b.orderNo.length === 0) {
return { ok: false, error: "orderNo is missing" };
}
if (b.shipping !== "standard" && b.shipping !== "express") {
return { ok: false, error: "shipping must be standard or express" };
}
if (!Array.isArray(b.items) || b.items.length === 0) {
return { ok: false, error: "items must not be empty" };
}
for (const p of b.items as Array<Record<string, unknown>>) {
if (typeof p?.isbn !== "string" || !Number.isInteger(p?.quantity) || (p.quantity as number) < 1) {
return { ok: false, error: "every item needs an isbn and quantity >= 1" };
}
}
if (!Number.isInteger(b.totalCents) || (b.totalCents as number) < 0) {
return { ok: false, error: "totalCents must be an integer >= 0" };
}
return {
ok: true,
data: {
orderNo: b.orderNo,
shipping: b.shipping,
items: b.items as OrderItem[],
totalCents: b.totalCents as number,
},
};
}
// Reads an event from an SQS record (raw message delivery is on)
export function parseEvent(body: string): OrderPlaced {
const event = JSON.parse(body) as OrderPlaced;
if (event.eventType !== "OrderPlaced" || typeof event.eventId !== "string") {
throw new Error("Unknown event format");
}
return event;
}
Step 3: Accept and publish the order
The intake function validates the body, builds the event and publishes it with a single PublishCommand. I also set the type as the message attribute eventType. Afterwards, it responds with 202 because the actual work is still to come.
// src/place-order.ts
import { randomUUID } from "node:crypto";
import { PublishCommand, SNSClient } from "@aws-sdk/client-sns";
import type { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from "aws-lambda";
import { validateOrder, type OrderPlaced } from "./order-events.js";
const sns = new SNSClient({});
export async function handler(event: APIGatewayProxyEventV2): Promise<APIGatewayProxyResultV2> {
let body: unknown;
try {
body = JSON.parse(event.body ?? "");
} catch {
return respond(400, { error: "Body is not valid JSON" });
}
const check = validateOrder(body);
if (!check.ok) {
return respond(400, { error: check.error });
}
const message: OrderPlaced = {
eventType: "OrderPlaced",
eventId: randomUUID(),
placedAt: new Date().toISOString(),
...check.data,
};
// One publish, any number of consumers: SNS delivers to every subscription
await sns.send(
new PublishCommand({
TopicArn: process.env.TOPIC_ARN,
Message: JSON.stringify(message),
MessageAttributes: {
eventType: { DataType: "String", StringValue: message.eventType },
},
}),
);
return respond(202, { orderNo: message.orderNo, eventId: message.eventId });
}
function respond(statusCode: number, data: unknown): APIGatewayProxyResultV2 {
return {
statusCode,
headers: { "content-type": "application/json" },
body: JSON.stringify(data),
};
}
A note on persistence: in this example, the function publishes directly without storing the order in a database first. If you need to do both, you have exactly the dual-write problem from the second part. In that case, put the outbox pattern in front, and SNS becomes the target of the pipe.
Step 4: Write workers with partial failures
All three workers share a small helper. It processes every record on its own and reports only the failed messages back. SQS then redelivers just those instead of repeating the whole batch. I explain the details in the article on SQS partial batch response.
// src/batch.ts
import type { SQSBatchResponse, SQSEvent } from "aws-lambda";
import { parseEvent, type OrderPlaced } from "./order-events.js";
// Processes each record on its own and reports only the failed ones
export async function processBatch(
event: SQSEvent,
process: (order: OrderPlaced) => Promise<void>,
): Promise<SQSBatchResponse> {
const batchItemFailures: SQSBatchResponse["batchItemFailures"] = [];
for (const record of event.Records) {
try {
await process(parseEvent(record.body));
} catch (error) {
console.error(JSON.stringify({ messageId: record.messageId, error: String(error) }));
batchItemFailures.push({ itemIdentifier: record.messageId });
}
}
return { batchItemFailures };
}
// src/workers.ts
import type { SQSEvent } from "aws-lambda";
import { processBatch } from "./batch.js";
// Warehouse: reserves the books of every order
export const reserveStock = (event: SQSEvent) =>
processBatch(event, async (b) => {
const pieces = b.items.reduce((sum, p) => sum + p.quantity, 0);
console.log(JSON.stringify({ action: "reserve", orderNo: b.orderNo, eventId: b.eventId, pieces }));
});
// Accounting: creates the invoice
export const createInvoice = (event: SQSEvent) =>
processBatch(event, async (b) => {
const amount = (b.totalCents / 100).toFixed(2);
console.log(JSON.stringify({ action: "invoice", orderNo: b.orderNo, eventId: b.eventId, amount }));
});
// Express shipping: thanks to the filter, only express orders arrive here
export const scheduleExpress = (event: SQSEvent) =>
processBatch(event, async (b) => {
if (b.shipping !== "express") {
throw new Error(`Wrong shipping method ${b.shipping} – check the filter`);
}
console.log(JSON.stringify({ action: "express", orderNo: b.orderNo, eventId: b.eventId }));
});
The workers only log here to keep the example small. In a real system, you would use the eventId to catch duplicate deliveries. I show how to do that with Powertools and DynamoDB in Idempotency in AWS Lambda.
Step 5: Topic, queues and subscriptions in serverless.yml
Now for the core. The serverless.yml describes the four functions, and the resources section holds everything else CloudFormation should create: the topic, three queues with one DLQ each, a queue for undeliverable messages, the queue policy and the three subscriptions.
# serverless.yml
service: bookshop-fanout
frameworkVersion: "4"
provider:
name: aws
runtime: nodejs22.x
region: eu-central-1
architecture: arm64
memorySize: 256
logRetentionInDays: 14
environment:
TOPIC_ARN: !Ref OrdersTopic
iam:
role:
statements:
- Effect: Allow
Action: sns:Publish
Resource: !Ref OrdersTopic
functions:
placeOrder:
handler: src/place-order.handler
events:
- httpApi:
method: POST
path: /orders
reserveStock:
handler: src/workers.reserveStock
events:
- sqs:
arn: !GetAtt WarehouseQueue.Arn
batchSize: 10
functionResponseType: ReportBatchItemFailures
createInvoice:
handler: src/workers.createInvoice
events:
- sqs:
arn: !GetAtt InvoiceQueue.Arn
batchSize: 10
functionResponseType: ReportBatchItemFailures
scheduleExpress:
handler: src/workers.scheduleExpress
events:
- sqs:
arn: !GetAtt ExpressQueue.Arn
batchSize: 5
functionResponseType: ReportBatchItemFailures
resources:
Resources:
OrdersTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: ${self:service}-${sls:stage}-orders
# One queue per consumer, each with its own dead-letter queue
WarehouseDlq:
Type: AWS::SQS::Queue
Properties:
MessageRetentionPeriod: 1209600
WarehouseQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 60
RedrivePolicy:
deadLetterTargetArn: !GetAtt WarehouseDlq.Arn
maxReceiveCount: 5
InvoiceDlq:
Type: AWS::SQS::Queue
Properties:
MessageRetentionPeriod: 1209600
InvoiceQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 60
RedrivePolicy:
deadLetterTargetArn: !GetAtt InvoiceDlq.Arn
maxReceiveCount: 5
ExpressDlq:
Type: AWS::SQS::Queue
Properties:
MessageRetentionPeriod: 1209600
ExpressQueue:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 60
RedrivePolicy:
deadLetterTargetArn: !GetAtt ExpressDlq.Arn
maxReceiveCount: 5
# Catches messages that SNS could not deliver at all
DeliveryDlq:
Type: AWS::SQS::Queue
Properties:
MessageRetentionPeriod: 1209600
# Without this policy, SNS is not allowed to send to the queues
SnsMaySendPolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref WarehouseQueue
- !Ref InvoiceQueue
- !Ref ExpressQueue
- !Ref DeliveryDlq
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: sns.amazonaws.com
Action: sqs:SendMessage
Resource:
- !GetAtt WarehouseQueue.Arn
- !GetAtt InvoiceQueue.Arn
- !GetAtt ExpressQueue.Arn
- !GetAtt DeliveryDlq.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref OrdersTopic
WarehouseSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref OrdersTopic
Protocol: sqs
Endpoint: !GetAtt WarehouseQueue.Arn
RawMessageDelivery: true
FilterPolicyScope: MessageAttributes
FilterPolicy:
eventType:
- OrderPlaced
RedrivePolicy:
deadLetterTargetArn: !GetAtt DeliveryDlq.Arn
InvoiceSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref OrdersTopic
Protocol: sqs
Endpoint: !GetAtt InvoiceQueue.Arn
RawMessageDelivery: true
FilterPolicyScope: MessageAttributes
FilterPolicy:
eventType:
- OrderPlaced
RedrivePolicy:
deadLetterTargetArn: !GetAtt DeliveryDlq.Arn
# Filter on the message body: express orders only
ExpressSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref OrdersTopic
Protocol: sqs
Endpoint: !GetAtt ExpressQueue.Arn
RawMessageDelivery: true
FilterPolicyScope: MessageBody
FilterPolicy:
eventType:
- OrderPlaced
shipping:
- express
RedrivePolicy:
deadLetterTargetArn: !GetAtt DeliveryDlq.Arn
Outputs:
OrdersTopicArn:
Value: !Ref OrdersTopic
A few spots deserve a second look, because they decide between “works” and “silently does not work”.
The queue policy. A subscription alone is not enough. SNS needs permission to send to the queue. The aws:SourceArn condition makes sure that only this exact topic can send, not some other one. The same applies to the DeliveryDlq, which is why it is part of the policy as well.
Raw message delivery. Without this setting, SNS wraps your message in its own JSON envelope with fields such as Type, TopicArn and Signature. Your event then sits as a string in the Message field. With RawMessageDelivery: true, only your event lands in the queue, and the worker can parse the body directly. There is a catch: with raw delivery, a message may have at most 10 message attributes, otherwise SNS discards it for that subscription.
Two kinds of filters. Warehouse and invoice filter on the eventType attribute (FilterPolicyScope: MessageAttributes, the default). Express shipping, on the other hand, filters on the message body (MessageBody) and only receives events with "shipping": "express". The body filter is more convenient because you do not maintain anything twice. It does cost extra, though; more on that in the costs section.
Two kinds of DLQs. The DLQs on the queues catch messages the worker failed to process five times. The DeliveryDlq on the subscription kicks in earlier: it receives messages that SNS could not even write to the queue, for example because someone changed the policy. Without it, such messages would simply be gone after SNS runs out of retries.
Step 6: Write and run the tests
The tests replace the SNS client with a mock and check that exactly one message with the right attribute goes out, that invalid orders are rejected and that the workers report only broken records as failed.
// test/fanout.test.ts
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { APIGatewayProxyEventV2, SQSEvent } from "aws-lambda";
import { handler } from "../src/place-order.js";
import { reserveStock, scheduleExpress } from "../src/workers.js";
// vi.mock is hoisted above the imports, vi.hoisted makes send available there
const send = vi.hoisted(() => vi.fn());
vi.mock("@aws-sdk/client-sns", () => ({
SNSClient: class {
send = send;
},
PublishCommand: class {
constructor(public input: unknown) {}
},
}));
const order = {
orderNo: "B-1001",
shipping: "express",
items: [{ isbn: "978-3-16-148410-0", quantity: 2 }],
totalCents: 3998,
};
function httpEvent(body: string): APIGatewayProxyEventV2 {
return { body } as APIGatewayProxyEventV2;
}
function sqsEvent(...bodies: string[]): SQSEvent {
return {
Records: bodies.map((body, i) => ({ messageId: `m-${i + 1}`, body })),
} as SQSEvent;
}
describe("placeOrder", () => {
beforeEach(() => send.mockReset());
it("publishes exactly one message with an eventType attribute", async () => {
const res = await handler(httpEvent(JSON.stringify(order)));
expect(res).toMatchObject({ statusCode: 202 });
expect(send).toHaveBeenCalledTimes(1);
const input = send.mock.calls[0][0].input;
expect(input.MessageAttributes.eventType.StringValue).toBe("OrderPlaced");
expect(JSON.parse(input.Message)).toMatchObject({ orderNo: "B-1001", shipping: "express" });
});
it("rejects an order without items with 400", async () => {
const res = await handler(httpEvent(JSON.stringify({ ...order, items: [] })));
expect(res).toMatchObject({ statusCode: 400 });
expect(send).not.toHaveBeenCalled();
});
});
describe("workers", () => {
const event = (shipping: string) =>
JSON.stringify({ ...order, shipping, eventType: "OrderPlaced", eventId: "e-1", placedAt: "2026-10-26T10:00:00Z" });
it("reports only the broken record as failed", async () => {
const res = await reserveStock(sqsEvent(event("standard"), "not json"));
expect(res.batchItemFailures).toEqual([{ itemIdentifier: "m-2" }]);
});
it("express worker raises an alarm for standard shipping", async () => {
const res = await scheduleExpress(sqsEvent(event("express"), event("standard")));
expect(res.batchItemFailures).toEqual([{ itemIdentifier: "m-2" }]);
});
});
npm run typecheck
npm test
Expected result: the type check finishes without output, and Vitest reports four passing tests. The error messages that appear in the log along the way are intentional: they come from the two tests with deliberately broken messages.
Test Files 1 passed (1)
Tests 4 passed (4)
Step 7: Deploy and try it out
On the first deploy, the framework asks for your Serverless account and creates the app. After that, it builds the TypeScript files, uploads everything and creates the CloudFormation stack.
npm run deploy
The output shows the URL of the HTTP endpoint. Send two orders, one with standard and one with express shipping. Replace the example URL with your own:
URL="https://abc123.execute-api.eu-central-1.amazonaws.com/orders"
curl -s -X POST "$URL" -H "content-type: application/json" \
-d '{"orderNo":"B-1001","shipping":"standard","items":[{"isbn":"978-3-16-148410-0","quantity":1}],"totalCents":1999}'
curl -s -X POST "$URL" -H "content-type: application/json" \
-d '{"orderNo":"B-1002","shipping":"express","items":[{"isbn":"978-3-16-148410-0","quantity":2}],"totalCents":3998}'
npx serverless logs -f scheduleExpress --stage dev
Expected result: both calls respond with an eventId. Both orders show up in the logs of reserveStock and createInvoice, but only B-1002 appears in the log of scheduleExpress. The following line is an example; your IDs will differ:
{"action":"express","orderNo":"B-1002","eventId":"5f0c2d1e-7a44-4c1b-9d0e-2b8f6a1c3e90"}
To clean up, run npm run remove. This deletes the topic, queues and functions again.
Common fan-out mistakes
The subscription exists, but the queue stays empty
Almost always, the queue policy is missing, or aws:SourceArn points to a different topic. SNS does not tell you about it when you publish, because from the sender’s point of view everything worked. Check the queue’s policy and look at the topic’s NumberOfNotificationsFailed metric in CloudWatch. This is also exactly where the delivery DLQ pays off.
The worker cannot parse the body
If JSON.parse succeeds but the result contains fields like TopicArn and Message, raw message delivery is off. Either enable it on the subscription, or parse twice: first the envelope, then the Message field.
The new filter does not apply right away
According to AWS, changes to a filter policy can take up to 15 minutes to take full effect. If you test right after the deploy and wonder about misrouted messages, give it a moment. For that reason, my tests check the policy in the template rather than the behavior right after a deploy.
Messages disappear without an error
Two classics: a body filter does not match because the field is spelled differently (Shipping instead of shipping). Or the message has more than 10 attributes with raw delivery. In both cases, nothing reaches the consumer, and the sender still gets a success response. That is why the express worker in the example deliberately raises an alarm when it receives the wrong shipping method.
A worker overwhelms a sensitive system
During load peaks, Lambda starts many workers per queue in parallel. If a worker talks to a system that only tolerates a few concurrent requests, such as an older invoicing tool, it quickly runs into trouble. So limit that worker’s concurrency. In the Serverless Framework, you do that with maximumConcurrency on the SQS event; the messages then simply wait a little longer in the queue.
Costs and security
For the example, normal testing costs you a few cents or nothing at all. For production, a few points are worth knowing. SNS bills publish requests in 64 KB chunks, so a 256 KB message counts as four requests. Delivery to SQS and Lambda costs nothing extra on the SNS side, but the queues bill their own requests. Attribute filters are free, while body filters are billed by the amount of data scanned, with a minimum of 1 KB per message and subscription. At high volume, that is an argument for attribute filters.
On size: by default, an SNS message may be up to 256 KiB. The topic attribute MaximumMessageSize lets you raise that to 1 MiB. I still keep events small and send a reference to an S3 object for large payloads.
On security: the queue policy with aws:SourceArn always belongs in the setup. If you encrypt the topic or queues with your own KMS key, the sns.amazonaws.com service needs kms:GenerateDataKey* and kms:Decrypt in the key policy. You cannot use the AWS managed key aws/sqs for this, because its key policy cannot be changed. And do not put personal data into message attributes; they end up in logs quickly.
My take
I consider SNS with SQS one of the underrated patterns, precisely because it is so unspectacular. Many teams jump straight to EventBridge because it feels more modern. For domain events across team boundaries, that is the right call. For the case of “one event, a manageable number of fixed consumers, a lot of load”, though, SNS with SQS is often the simpler solution, and simple solutions are easier to debug at three in the morning.
Where I disagree: fan-out is not a replacement for a workflow. If warehouse, invoice and shipping must run in a specific order, or if something has to be rolled back on failure, you are looking at a saga, and Step Functions is the better tool for that. Fan-out only fits when the consumers are truly independent of each other.
FAQ
What is the difference between SNS and SQS?
SNS pushes a message to all subscribers immediately, while SQS stores messages until a consumer pulls them. In a fan-out setup, both work together: SNS distributes, SQS buffers for each consumer.
Does SNS guarantee message ordering?
Standard topics do not. If you need a fixed order, use a FIFO topic with FIFO queues. These have lower limits, though, for example at most 100 subscriptions per topic.
Can a message arrive twice?
Yes. SNS and standard SQS deliver at least once. Your workers should therefore be idempotent, meaning a duplicate message must not trigger a second booking or shipment.
How many filter policies can a topic have?
According to the service quotas, it is 200 filter policies per topic and 10,000 per account. Both values can be raised on request.
Your next step
Find a place in your system where a function calls several other services in a row after finishing its work. Move one of them behind a topic with its own queue and see how error handling and tests change. For the next consumer, simply add a second subscription.
Do you tend to use SNS or go straight to EventBridge for cases like this, and why? Let me know in the comments.
Next in the series: Overview “Serverless patterns in practice” · Previous part: Outbox pattern with DynamoDB Streams and EventBridge Pipes · Next part: Lambda circuit breaker: contain slow third-party APIs
Sources
- Amazon SNS: fan-out to Amazon SQS queues
- Amazon SNS: subscribing a queue to a topic (queue policy)
- Amazon SNS: raw message delivery
- Amazon SNS: message filtering
- Amazon SNS: subscription filter policies
- Amazon SNS: encrypted topics and queues
- AWS General Reference: Amazon SNS endpoints and quotas
- Amazon EventBridge: quotas
- Amazon SNS: pricing
- AWS CloudFormation: AWS::SNS::Subscription
- Serverless Framework: SQS events
- Tested with Node.js 22.23, TypeScript 7.0.2, Vitest 5.0.5, @aws-sdk/client-sns 3.1156.0 and esbuild 0.28.2:
tsc, 4 tests from the article, a test bundle of the handlers and cfn-lint for the resources inserverless.yml(Serverless Framework 4.45.0). Deploy and log output is marked as an example.
This post was created with AI assistance.
👉 Next step: Need a schedule in EventBridge? My EventBridge cron helper builds and explains the expression for you.
Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en
Top comments (0)