# DuckDB

> SQL over Parquet that runs anywhere, from my laptop to a Lambda. Next to Athena, not instead of it.

- Verdict: Use when it fits
- Vendor: DuckDB Foundation (DuckLabs, now part of AWS)
- Layer: [Data](https://cloudlesslabs.com/stack/layers/data.md). Job: Embedded analytics engine
- Last reviewed: 19 October 2026, by Nicolas Dao (Cloudless Labs, Sydney)
- Web page: https://cloudlesslabs.com/stack/duckdb/. The whole stack in one file: https://cloudlesslabs.com/stack/brief.md

AWS bought DuckLabs, the company behind DuckDB, in August 2026. DuckDB stays MIT-licensed and owned by the non-profit DuckDB Foundation. So far the only AWS integration is Aurora PostgreSQL querying the lake; AWS says Athena, Glue and the rest will follow "over time".

The value is developer experience. The same SQL runs on my laptop, in a test, and in a Lambda, against the same Parquet in S3. It isn't a cheaper Athena: for big scans the difference is at most about 2x, and Athena does the fan-out for you.

## Use it when

- App endpoints that run many small, selective queries over the lake (per-customer dashboards, reports): sub-second when warm, no Athena polling.
- Writing and reshaping Parquet: COPY gives full control of sort order, file size and partitions, with no 100-partition limit.
- Local development and tests: the same SQL against the same S3 data from a laptop, or against sample files in CI with no AWS at all.

## Don't use it when

- Scans of tens of GB to TBs per query: one Lambda reads roughly 50 GB per 15 minutes. Athena fans out for you.
- Ad-hoc analysis by people who just want a SQL console: Athena.

## Pricing

Free and MIT-licensed. In Lambda you pay compute and S3 GETs: a query that reads 1 GB costs about $0.002, against $0.005 on Athena, with no 10 MB minimum per query.

## Gotchas, with fixes

- **Gotcha:** Lambda has no HOME directory, so INSTALL, LOAD and CREATE SECRET fail with "Can't find the home directory".
  **Fix:** Run SET home_directory='/tmp' first.
- **Gotcha:** The S3, AWS and Iceberg extensions are downloaded at runtime unless bundled: about 100 MB on every cold start.
  **Fix:** Bundle them in the layer and SET extension_directory to it.
- **Gotcha:** DuckDB SQL isn't Athena's (Trino) SQL. Date, array and approximate functions differ.
  **Fix:** Test shared queries on both engines, or keep each query on one engine.
- **Gotcha:** The layer zip is over 50 MB.
  **Fix:** Upload the layer through S3, or use a container image.

## Instead, consider

- [S3 + Glue + Athena](https://cloudlesslabs.com/stack/aws-athena-data-lake/index.md): Big scans, ad-hoc questions and many concurrent users.

## Worked examples

- [CloudWatch and Firehose, side by side](https://cloudlesslabs.com/stack/guides/chatty-logs/index.md): How do I keep chatty logs without a CloudWatch-sized bill?
- [Athena or DuckDB?](https://cloudlesslabs.com/stack/guides/data-lake/index.md): How should I query my data lake, and what changed when AWS bought DuckDB's company?

## Where it fits

- [A data lake](https://cloudlesslabs.com/stack/use-cases/data-lake.md): pick 4. App queries, writing Parquet, local dev and tests, on the same files. AWS bought the company behind it in August 2026.
- [Logs without a shocking bill](https://cloudlesslabs.com/stack/use-cases/logging.md): pick 3. Search the S3 logs from a laptop or a Lambda when someone asks what happened.

## In The Stack Weekly

- [Week 43: Aurora PostgreSQL reads Iceberg and Parquet](https://cloudlesslabs.com/weekly/2026-w43/index.md), worth a pilot

## Sources

- [AWS and DuckLabs (AWS Big Data blog)](https://aws.amazon.com/blogs/big-data/aws-and-ducklabs-building-the-future-of-analytics-together/)
- [DuckLabs to join AWS](https://ducklabs.com/news/2026/08/26/ducklabs-to-join-aws)
- [DuckDB Iceberg catalogs (S3 Tables, Glue)](https://duckdb.org/docs/current/core_extensions/iceberg/catalogs.html)
- [Parquet tips (sorting for pruning)](https://duckdb.org/docs/current/data/parquet/tips)

## Related

- [Data](https://cloudlesslabs.com/stack/layers/data.md): every tool in this layer, with its verdict
- [The whole stack in one file](https://cloudlesslabs.com/stack/brief.md): every use case and tool, for questions that span many
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)