Event-driven plumbing
Moving events between services, buffering work, fanning out and streaming records into the lake.
My stack for this
- 01Amazon EventBridge
Routing: send each event to whoever cares, by its content. Schedules too.
- 02Amazon SQS
Work: buffer, retry, dead-letter, and let Lambda take messages in batches.
- 03Amazon SNS
Fan-out: one event to several queues or functions.
- 04Amazon Data Firehose
Streams into S3: records batched, compressed and written as files for the data lake, with no consumer to write.
- 05Amazon Kinesis
Only when several consumers must replay the same ordered stream at high volume.
Don't
- AWS Step Functions
A JSON state machine where a queue and a few lines of JavaScript do the job.
Instead: Amazon SQS or Lambda durable functions
The picks, in brief
What each one is, my verdict, and what it costs. Every name opens the full entry: when to use it and when not, the gotchas with their fixes, provisioning and alternatives.
The event bus and scheduler I use all the time.
Pricing: Classic bus: AWS management events are ingested free; custom events $1 per million (billed in 64 KB chunks). Scheduler: 14 million invocations a month free, then $1.20 per million in Sydney. Pipes: $0.40 per million requests. The new Custom Event Bus (September 2026) bills by data instead: in Sydney $0.19 per GB ingested (first 5 TB) and $0.058 per GB delivered to each subscriber.
The queue in front of Lambda. Natural back-pressure, nearly free when batched.
Pricing: Sydney: $0.40 per million requests (Standard), $0.50 (FIFO). A request carries up to 10 messages, so batching cuts the bill tenfold: 50M messages ≈ $6 batched, $60 unbatched.
Fan-out from one publisher to many subscribers. Not my ingestion path at high volume.
Pricing: Sydney: first 1M publishes a month free, then $0.50 per million (each 64 KB of payload counts as one). Deliveries to SQS and Lambda cost nothing; HTTPS deliveries $0.60 per million, email $2 per 100,000. Payload-based filtering $0.11/GB scanned; attribute-based filtering is free.
The cheapest way to land a stream of records in S3. No servers, no consumers to write, and about $0.04/GB in Sydney.
Pricing: Sydney, Direct PUT: $0.037/GB ingested (first 500 TB), billed per record in 5 KB steps. JSON to Parquet conversion $0.023/GB. Dynamic partitioning $0.025/GB plus per-object fees (avoidable). Then normal S3 storage.
For genuinely huge streams. It gets expensive fast.
Pricing: Sydney, on-demand: $0.049 per stream-hour (about $36 a month per stream, even idle), $0.098 per GB written, $0.049 per GB read. Provisioned: $0.0184 per shard-hour plus $0.0195 per million 25 KB put units. Extended retention and enhanced fan-out cost extra.
Where they sit
The layers of the stack this job touches, with every tool named on this page.
APIs, queues and events
How requests and events move.
- Default Amazon EventBridge: Events and schedules
- Default Amazon Data Firehose: Managed delivery of records into S3 (and other sinks)
- Use when Amazon Kinesis: High-throughput streaming
- Watching · Proposed Lambda durable functions: Durable multi-step jobs
- Use when Amazon SNS: Pub/sub fan-out
- Default Amazon SQS: Queues and back-pressure
- Never AWS Step Functions
Want this stack in your AWS account?
Cloudless Labs is a Sydney consultancy. We migrate on-premises systems to AWS, build serverless apps on this stack, and coach teams to ship with Claude Code and Codex.