Designing Reliable Event Pipelines with SNS, SQS, and EventBridge

SNS, SQS, and EventBridge overlap enough that teams often reach for whichever one they used last time. They're built for different jobs, and combining them well — rather than picking just one — is usually the right answer.

Picking the right service

SNS is a pub/sub topic: publish once, deliver to every subscriber (SQS queues, Lambda functions, HTTP endpoints, email). Use it when multiple independent consumers need the same event.

SQS is a durable queue: messages sit there until a consumer polls and deletes them. Use it as the landing point for a specific consumer, or as a subscriber behind an SNS topic (the fan-out pattern) so each consumer group gets its own buffered, retryable copy.

EventBridge adds content-based routing: define rules that match on event fields and route matching events to different targets, without every producer needing to know every consumer. It's the right choice once you have more than a couple of event types or need cross-service, cross-account routing.

The fan-out pattern: SNS → multiple SQS queues

Publish once to an SNS topic; subscribe an SQS queue per consumer group so each one processes at its own pace with its own retry and dead-letter policy.

{
  "TopicArn": "arn:aws:sns:us-east-1:123456789012:order-events",
  "Subscriptions": [
    { "Protocol": "sqs", "Endpoint": "arn:aws:sqs:...:inventory-queue" },
    { "Protocol": "sqs", "Endpoint": "arn:aws:sqs:...:notifications-queue" },
    { "Protocol": "sqs", "Endpoint": "arn:aws:sqs:...:analytics-queue" }
  ]
}

Dead-letter queues aren't optional

Configure a redrive policy on every consumer queue so a message that repeatedly fails processing moves to a DLQ instead of retrying forever and starving the queue.

{
  "RedrivePolicy": {
    "deadLetterTargetArn": "arn:aws:sqs:us-east-1:123456789012:order-events-dlq",
    "maxReceiveCount": 5
  }
}

Idempotency: assume duplicates will happen

SQS and SNS both guarantee at-least-once delivery. Somewhere between a retried publish, a consumer crash after processing but before deleting the message, or a redelivery after a visibility timeout, you will see the same message twice. Track a dedupe key so processing is safe to repeat.

public async Task HandleAsync(OrderPlacedEvent evt, CancellationToken ct)
{
    var alreadyProcessed = await _dedupeStore.ExistsAsync(evt.EventId, ct);
    if (alreadyProcessed) return;

    await _inventory.ReserveAsync(evt.Lines, ct);
    await _dedupeStore.MarkProcessedAsync(evt.EventId, ttl: TimeSpan.FromDays(7), ct);
}

Ordering: only where you need it

Standard SQS queues don't guarantee order and allow (rare) duplicate delivery beyond the at-least-once norm. FIFO queues guarantee order within a message group and exactly-once processing within a 5-minute deduplication window, at a throughput cost. Use FIFO only for the subset of your traffic that genuinely needs strict ordering — for example, state transitions for a single order — and partition by a message group ID (like the order ID) so unrelated orders still process in parallel.

{
  "QueueName": "order-state-transitions.fifo",
  "FifoQueue": true,
  "ContentBasedDeduplication": false,
  "MessageGroupId": "order-8f14e45f"
}

Routing with EventBridge rules

Once you have more than a couple of event types, EventBridge rules let you route by content without a service having to know about every downstream consumer.

{
  "EventPattern": {
    "source": ["orders.service"],
    "detail-type": ["OrderPlaced"],
    "detail": { "total": [{ "numeric": [">", 1000] }] }
  },
  "Targets": [{ "Arn": "arn:aws:lambda:...:function:fraud-review" }]
}