CloudWatch and Firehose, side by side

The two jobs are different. During an incident you want live logs, alarms and metric filters: that's CloudWatch. For the chatty DEBUG and INFO lines you want them kept, cheaply, and searchable when someone asks "what happened to order 1234 last Tuesday?": that's S3.

Lambda ─┬─ console.warn / console.error ──▶ CloudWatch   (live, alarms, ~2% of volume)
        └─ log.debug / log.info ─ batched ─▶ Firehose ─▶ S3 (gzip or Parquet) ─▶ Athena / DuckDB

What 1 TB of chatty logs costs a month (Sydney)

Add about $13 for the WARN/ERROR lines that stay in CloudWatch. The ~$75 assumes about 100M invocations with one ~10 KB record each, including about $27 of extra Lambda time for the flush. With bigger records it drops to $45–70.

Why the built-in destination isn't the cheap path: CloudWatch still handles the delivery and charges $0.335/GB in Sydney for it. Your own logger skips CloudWatch entirely and pays Firehose's $0.037/GB.

Routing: per function, or per line

• Per function, no code: each function's logging config points at a log group. Critical functions stay on Standard, a verbose one can go to an Infrequent Access group. This doesn't split lines inside one function.
• Per line, a small logger: WARN and ERROR to console, everything else to a buffer flushed to Firehose. This is the split that saves the money.

The logger (sketch, about 40 lines)

import { FirehoseClient, PutRecordBatchCommand } from '@aws-sdk/client-firehose'

const firehose = new FirehoseClient({})
const STREAM = process.env.LOG_STREAM
const MAX_RECORD = 900 * 1024 // Firehose bills each record in 5 KB steps: pack lines together
let lines = []

const line = (level, msg, data) => JSON.stringify({ ts: new Date().toISOString(), level, msg, ...data })

export const log = {
	debug: (msg, data) => lines.push(line('DEBUG', msg, data)),
	info: (msg, data) => lines.push(line('INFO', msg, data)),
	warn: (msg, data) => console.warn(line('WARN', msg, data)),   // CloudWatch
	error: (msg, data) => console.error(line('ERROR', msg, data)), // CloudWatch
}

export async function flush() {
	if (!lines.length) return
	const records = []
	let cur = ''
	for (const l of lines) {
		if (cur && Buffer.byteLength(cur) + Buffer.byteLength(l) > MAX_RECORD) { records.push(cur); cur = '' }
		cur += l + '\n'
	}
	if (cur) records.push(cur)
	lines = []
	for (let i = 0; i < records.length; i += 4) { // max 4 MiB per call
		const res = await firehose.send(new PutRecordBatchCommand({
			DeliveryStreamName: STREAM,
			Records: records.slice(i, i + 4).map(r => ({ Data: Buffer.from(r) })),
		}))
		if (res.FailedPutCount) console.warn(line('WARN', 'log records dropped', { failed: res.FailedPutCount }))
	}
}

// In each handler:
export const handler = async event => {
	try {
		log.info('received', { id: event.id })
		// ...
	} finally {
		await flush() // Lambda freezes after return: flush first
	}
}

For production, add a size-based flush, a flush shortly before the timeout (context.getRemainingTimeInMillis()), and a retry of failed records. Don't use an async pino transport here: Lambda freezes it mid-send.

The infrastructure (Pulumi)

const logs = new aws.kinesis.FirehoseDeliveryStream('api-logs', {
	destination: 'extended_s3',
	extendedS3Configuration: {
		roleArn: firehoseRole.arn,        // may write to the bucket
		bucketArn: lake.arn,
		prefix: 'logs/app=api/',          // Firehose appends YYYY/MM/DD/HH/ for free
		errorOutputPrefix: 'logs-errors/',
		bufferingSize: 64,                // MB
		bufferingInterval: 300,           // seconds
		compressionFormat: 'GZIP',
	},
})
// The function's role needs firehose:PutRecordBatch on logs.arn,
// and the bucket a lifecycle rule that expires logs/ after your retention.

Searching them

From a laptop or a Lambda, with DuckDB:

SELECT ts, level, msg
FROM read_json('s3://lake/logs/app=api/2026/10/19/*/*.gz')
WHERE id = 'order-1234'
ORDER BY ts;

Or define a Glue table over logs/ with partition projection on the date and query it from Athena.

The trade-offs, honestly

• No Live Tail, no alarms on the S3 side. That's why WARN and ERROR stay in CloudWatch.
• A small loss window: lines still in memory are lost on a crash or timeout. The error itself still reaches CloudWatch.
• A few to tens of milliseconds per invocation for the flush.
• Throughput: 1 MiB/s per stream by default in Sydney. Ask for more before a busy launch.
• Cut volume first anyway: JSON log format, no payloads, retention on every log group. Cheap logs are still logs nobody reads.