# An AI chat or LLM feature

> My app needs to call an LLM and stream the answer to users. Nicolas Dao's picks for this job (Cloudless Labs, Sydney), in order of preference, what he would avoid, and what to use instead.

- Use case 2 of 21 in The Stack, last reviewed 21 October 2026
- Web page: https://cloudlesslabs.com/stack/use-cases/ai-chat/

Picks, in order of preference:

1. **[Cloudflare Workers](https://cloudlesslabs.com/stack/cloudflare-workers/index.md)** (My default): The proxy. Hides your key, enforces per-user budgets, streams tokens. Waiting is free.
2. **[OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md)** (My default): Every model behind one key. Pick by cost per intelligence.
3. **[pgvector](https://cloudlesslabs.com/stack/pgvector/index.md)** (My default): If the answers need your data (RAG).
4. **[Cloudflare AI Gateway](https://cloudlesslabs.com/stack/cloudflare-ai-gateway/index.md)** (Watching (Proposed: research-backed, not yet confirmed by Nic)): Caching, spend caps and fallback, on my shortlist.

Don't:

- ~~AWS Lambda~~: Not as the proxy. Lambda bills for every second the function waits on the model, and a streamed answer takes 10–60 seconds. A Worker bills CPU time only, so waiting is free: $33–167 vs about $0 extra for 1M chats. Keep Lambda for the work behind the proxy. Instead: [Cloudflare Workers](https://cloudlesslabs.com/stack/cloudflare-workers/index.md).
- ~~Anthropic API (for client systems)~~: Not for high-volume features in client products: cost per intelligence decides there, and OpenAI models through OpenRouter win it. (Claude Code stays my main development tool.) Instead: [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md).

Worked example: [The LLM streaming proxy](https://cloudlesslabs.com/stack/guides/llm-proxy/index.md)

## The picks in brief

- [Cloudflare Workers](https://cloudlesslabs.com/stack/cloudflare-workers/index.md): My default. The edge primitive. It sits between your users and your real backend, and it's very, very good at that. Pricing: $5 a month per account, including 10M requests and 30M CPU-ms. Then $0.30 per million requests and $0.02 per million CPU-ms. Billed on CPU time only: waiting is free. No egress charges.
- [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md): My default. One SDK, one key, every model. Usually cheaper than running the same open models elsewhere. Pricing: Model prices pass through; OpenRouter adds 5.5% on credit purchases. Example: DeepSeek V4 Pro at $0.66 / $1.98 per million tokens vs $1.32 / $3.96 on Cloudflare Workers AI. The Business plan's fee is 8%. Bring your own provider key (BYOK): the first $25,000 of list-price usage a month carries no fee, then 5%.
- [pgvector](https://cloudlesslabs.com/stack/pgvector/index.md): My default. Embeddings next to the data they describe, for when a user is waiting for the answer. Pricing: Mostly the storage on a Postgres you already pay for: about $5 a month for 1M vectors on an existing Neon database.
- [Cloudflare AI Gateway](https://cloudlesslabs.com/stack/cloudflare-ai-gateway/index.md): Watching (Proposed: research-backed, not yet confirmed by Nic). Free caching, rate limits, spend caps and model fallback in front of OpenRouter. Works even from Lambda. Pricing: Core features free (analytics, caching, rate limiting). Logs: 20M events a month included on Workers Paid, then $0.60 per million, for accounts whose first gateway dates from 24 September 2026 or later; older accounts keep 10M stored logs per gateway (pricing changes 1 December 2026). Unified Billing credits carry a 5% fee; your own provider keys carry none.

## Related

- [The LLM streaming proxy](https://cloudlesslabs.com/stack/guides/llm-proxy/index.md): the worked example for this job, with costs
- [Edge](https://cloudlesslabs.com/stack/layers/edge/index.md): every tool in this layer
- [Compute](https://cloudlesslabs.com/stack/layers/compute/index.md): every tool in this layer
- [AI](https://cloudlesslabs.com/stack/layers/ai/index.md): every tool in this layer
- [Every use case and tool in one file](https://cloudlesslabs.com/stack/brief.md)
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)