An AI chat or LLM feature
My app needs to call an LLM and stream the answer to users.
My stack for this
- 01Cloudflare Workers
The proxy. Hides your key, enforces per-user budgets, streams tokens. Waiting is free.
- 02OpenRouter
Every model behind one key. Pick by cost per intelligence.
- 03pgvector
If the answers need your data (RAG).
- 04Cloudflare AI Gateway
Caching, spend caps and fallback, on my shortlist.
Don't
- AWS Lambda
Not as the proxy. Lambda bills for every second the function waits on the model, and a streamed answer takes 10–60 seconds. A Worker bills CPU time only, so waiting is free: $33–167 vs about $0 extra for 1M chats. Keep Lambda for the work behind the proxy.
Instead: Cloudflare Workers
- Anthropic API (for client systems)
Not for high-volume features in client products: cost per intelligence decides there, and OpenAI models through OpenRouter win it. (Claude Code stays my main development tool.)
Instead: OpenRouter
The picks, in brief
What each one is, my verdict, and what it costs. Every name opens the full entry: when to use it and when not, the gotchas with their fixes, provisioning and alternatives.
The edge primitive. It sits between your users and your real backend, and it's very, very good at that.
Pricing: $5 a month per account, including 10M requests and 30M CPU-ms. Then $0.30 per million requests and $0.02 per million CPU-ms. Billed on CPU time only: waiting is free. No egress charges.
One SDK, one key, every model. Usually cheaper than running the same open models elsewhere.
Pricing: Model prices pass through; OpenRouter adds 5.5% on credit purchases. Example: DeepSeek V4 Pro at $0.66 / $1.98 per million tokens vs $1.32 / $3.96 on Cloudflare Workers AI. The Business plan's fee is 8%. Bring your own provider key (BYOK): the first $25,000 of list-price usage a month carries no fee, then 5%.
Embeddings next to the data they describe, for when a user is waiting for the answer.
Pricing: Mostly the storage on a Postgres you already pay for: about $5 a month for 1M vectors on an existing Neon database.
Free caching, rate limits, spend caps and model fallback in front of OpenRouter. Works even from Lambda.
Pricing: Core features free (analytics, caching, rate limiting). Logs: 20M events a month included on Workers Paid, then $0.60 per million, for accounts whose first gateway dates from 24 September 2026 or later; older accounts keep 10M stored logs per gateway (pricing changes 1 December 2026). Unified Billing credits carry a 5% fee; your own provider keys carry none.
Where they sit
The layers of the stack this job touches, with every tool named on this page.
Want this stack in your AWS account?
Cloudless Labs is a Sydney consultancy. We migrate on-premises systems to AWS, build serverless apps on this stack, and coach teams to ship with Claude Code and Codex.