A data lake
Analytics over everything the apps produce, cheaply, without a warehouse to run.
My stack for this
- 01Amazon S3
The lake. Parquet (Iceberg next), partitioned by date and sorted by your second key.
- 02Amazon Data Firehose
The way in. Streams records from apps and devices into S3, batched and compressed.
- 03S3 + Glue + Athena
Big scans, ad-hoc questions and reports. Zero code, $5 per TB scanned.
- 04DuckDB
App queries, writing Parquet, local dev and tests, on the same files. AWS bought the company behind it in August 2026.
- 05Aurora PostgreSQL
If the app already runs on Aurora: join live tables with the lake in one SQL query (DuckDB inside).
The picks, in brief
What each one is, my verdict, and what it costs. Every name opens the full entry: when to use it and when not, the gotchas with their fixes, provisioning and alternatives.
No complaints. Storage, data lake, archive.
Pricing: Sydney: $0.025/GB-month (Standard). Internet egress $0.114/GB (first 10 TB): that's where it hurts. Requests: $5.50 per million PUT/LIST, $0.44 per million GET. Standard-IA $0.0138/GB, Glacier Instant Retrieval $0.005/GB. Transfer from S3 to CloudFront and to AWS compute in the same region is free.
The cheapest way to land a stream of records in S3. No servers, no consumers to write, and about $0.04/GB in Sydney.
Pricing: Sydney, Direct PUT: $0.037/GB ingested (first 500 TB), billed per record in 5 KB steps. JSON to Parquet conversion $0.023/GB. Dynamic partitioning $0.025/GB plus per-object fees (avoidable). Then normal S3 storage.
The data lake that works. Not amazing. Pair it with DuckDB.
Pricing: Sydney: Athena $5 per TB scanned, 10 MB minimum per query. S3 Standard $0.025/GB-month. Glue Data Catalog free for the first million objects.
SQL over Parquet that runs anywhere, from my laptop to a Lambda. Next to Athena, not instead of it.
Pricing: Free and MIT-licensed. In Lambda you pay compute and S3 GETs: a query that reads 1 GB costs about $0.002, against $0.005 on Athena, with no 10 MB minimum per query.
Peace of mind and scale, at a price. My choice when the client must stay all-AWS.
Pricing: Serverless v2 kept at 0.5 ACU always-on ≈ $73 a month. RDS Proxy adds at least about $26 a month (provisioned) or about $146 (Serverless v2, minimum 8 ACU).
Where they sit
The layers of the stack this job touches, with every tool named on this page.
APIs, queues and events
How requests and events move.
Data
Databases, migrations and the data lake.
Storage and delivery
Files, websites and CDNs.
Want this stack in your AWS account?
Cloudless Labs is a Sydney consultancy. We migrate on-premises systems to AWS, build serverless apps on this stack, and coach teams to ship with Claude Code and Codex.