# The LLM streaming proxy

> How do I add an AI chat feature without leaking my API key or paying for the model's thinking time? Never call an LLM from the browser. A Worker holds the key, checks the user's budget and streams tokens back. Waiting costs nothing on Workers.

- A decision guide from The Stack by Nicolas Dao (Cloudless Labs, Sydney)
- Web page: https://cloudlesslabs.com/stack/guides/llm-proxy/. The whole stack in one file: https://cloudlesslabs.com/stack/brief.md
- Tools: [Cloudflare Workers](https://cloudlesslabs.com/stack/cloudflare-workers/index.md) (My default), [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md) (My default), [Cloudflare AI Gateway](https://cloudlesslabs.com/stack/cloudflare-ai-gateway/index.md) (Watching (Proposed: research-backed, not yet confirmed by Nic))

## The problem

If the browser calls OpenRouter directly, your API key has to be in the app. Anyone can open DevTools, copy it and run up your bill, and you can't limit what each user does.

## The fix

```
Browser ──(user's login token)──▶ Worker ──(your OpenRouter key)──▶ OpenRouter
        ◀──── tokens streamed back word by word ◀────────────────
```

The Worker:
1. checks the user is logged in (a Cognito JWT works);
2. checks that user's quota or budget ("50 messages a day", "$2 a month");
3. adds your key on the server side, so it never reaches the browser;
4. streams the answer back as it's generated;
5. records token usage for billing or analytics.

## Why a Worker

The proxy does about 2 ms of work, then waits 10–60 seconds while the model writes.

| 1M chats a month × 20 s | Monthly |
|---|---|
| Classic Lambda | $33.50 (128 MB) to $167 (512 MB) |
| Lambda Web Functions | ~$3.40 (not in Sydney yet) |
| Cloudflare Workers | ~$0 extra, inside the $5 plan |

On Lambda you pay memory × wall-clock time for the waiting, and response streaming through a function URL doesn't work inside a VPC. Workers bill CPU time only.

Add Cloudflare AI Gateway in front of OpenRouter for caching, per-client spend caps and fallback when a provider is down.

## Related

- [An AI chat or LLM feature](https://cloudlesslabs.com/stack/use-cases/ai-chat.md): the use case this guide works through
- [Cloudflare Workers](https://cloudlesslabs.com/stack/cloudflare-workers/index.md): the full entry (verdict, pricing, gotchas)
- [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md): the full entry (verdict, pricing, gotchas)
- [Cloudflare AI Gateway](https://cloudlesslabs.com/stack/cloudflare-ai-gateway/index.md): the full entry (verdict, pricing, gotchas)
- [Every decision guide](https://cloudlesslabs.com/stack/brief.md): listed at the end of the brief
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)