# Baseten decision models

> My pick when a yes, a no or a label has to come back with a probability you can act on: the best score and calibration of the decision models, with open weights to leave with. I haven't run them in production yet.

- Verdict: Use when it fits
- Vendor: Baseten
- Layer: [AI](https://cloudlesslabs.com/stack/layers/ai/index.md). Job: Decision models for client systems
- Last reviewed: 2 November 2026, by Nicolas Dao (Cloudless Labs, Sydney)
- Web page: https://cloudlesslabs.com/stack/baseten-decision-models/. The whole stack in one file: https://cloudlesslabs.com/stack/brief.md

A decision model takes your data and typed questions, and returns a probability in a fraction of a second, with no text to parse. [Which decision model?](https://cloudlesslabs.com/stack/guides/which-decision-model/index.md) compares all of them. This is the family I'd try first on a client system, for three reasons that have little to do with the benchmark.

The first is the exit. The category is weeks old, and I don't want a client depending on a closed model from a company that may look different in a year. Baseten's weights are MIT, so the worst case is a GPU of your own. The second is the paperwork: zero data retention by default, SOC 2 Type II and HIPAA, which shortens procurement. The third is that the three models share one API, so you can start high and move down by changing a name.

Then there's the score. On JevBench, mammoth-1 leads the decision models I compared at 85.2, with the best calibration at 94.3. That benchmark is one person's project, so treat it as a lead, not a verdict, and test on the client's own labelled cases.

This is how I'd wire it for a client: mammoth-1 behind a thin adapter, called from a Lambda with the key in Secrets Manager, and a threshold per question. Above it, the code acts. Below it, an LLM or a person looks. If the volume makes the bill matter, test elephant-1 on the same cases and move down only if it holds.

I'd stop short of calling it my default. I haven't run it in production, nobody has tested these models against injection, and they've only been out for weeks. That's what would have to change.

## Use it when

- A wrong answer costs money, and the data may leave Australia: mammoth-1 has the best text score and calibration in the comparison.
- The same job at high volume: elephant-1 has the same API and terms, at about a third of the price and a lower score.
- A client needs zero data retention, SOC 2 Type II or HIPAA without a long negotiation: Baseten's zero retention is on by default for Model APIs.
- You want a way out: the weights are MIT, so the same model can run on your own GPU in Sydney if the client's rules change.

## Don't use it when

- Data that must stay in Australia on the hosted API: no Sydney region is documented. Self-host the weights in ap-southeast-2, or use Claude on Amazon Bedrock's AU profile with a schema.
- Text written by people trying to fool you, until it's been tested: none of these models was in DecisionRobust, the one independent injection test. d1-ultra did best there.
- joey-1 as the only gate on anything: it scores 35.91 on the Decision Index's private tests, against 63.3 on its public one.
- Long documents: the hosted limit is 16,384 tokens, and the state counts in every question. Perplexity's Decider takes 256K.
- You need the reason as well as the answer: decision models don't write text. Use an LLM.

## Pricing

Per million input tokens, output free: mammoth-1 $0.10, elephant-1 $0.035, joey-1 $0.01. At 500 tokens a decision, a million decisions cost $50, $17.50 and $5. Dedicated GPUs are billed by the minute.

## Gotchas, with fixes

- **Gotcha:** Hosted rate limits are low: 120 requests a minute on Pro, about 2 a second. A request can also get a 429 when there's no capacity.
  **Fix:** Ask for Enterprise limits or a dedicated deployment before launch, put several questions in one request, and retry with backoff.
- **Gotcha:** Slugs end in -latest, so the weights can change under you, and no deprecation policy is published.
  **Fix:** Keep a labelled test set and re-run it on any change. If you need frozen behaviour, self-host a pinned revision of the weights.
- **Gotcha:** A request over the 16,384 token limit is refused whole.
  **Fix:** Count tokens first, keep the state short, and use max_len to fail fast. Refusal beats Clef's silent truncation, but your code has to handle it.
- **Gotcha:** Gateways don't carry everything: OpenRouter doesn't list these models, and Vercel lists them as text-only, though Baseten documents images.
  **Fix:** Call Baseten directly for now, behind a thin adapter so you can switch.
- **Gotcha:** Image limits and the token cost of an image aren't published, and none of these models is on an image benchmark.
  **Fix:** Measure usage.input_tokens on your own images, and keep a person behind any image decision.

## Provisioning

Pulumi via a Terraform provider · Community Terraform provider

basetenlabs/baseten v0.1.0, community tier, for dedicated deployments. The shared Model APIs need only an API key: keep it in AWS Secrets Manager.

## Instead, consider

- [Cloudflare Workers AI](https://cloudlesslabs.com/stack/cloudflare-workers-ai/index.md): Your code already runs in a Worker: Clef Flash is one binding away, at $0.038 per million, with a lower score.
- [Jev](https://cloudlesslabs.com/stack/typesafe-jev/index.md): To see what started this category, in an experiment. Early access and closed, so there's no way out.
- [Anthropic API (for client systems)](https://cloudlesslabs.com/stack/anthropic-api/index.md): You need more accuracy or a written reason, and can afford seconds: an LLM still wins on accuracy.
- [Amazon Bedrock](https://cloudlesslabs.com/stack/aws-bedrock/index.md): The data must stay in Australia and you won't run a GPU: Claude on the AU profile with a schema, with no probabilities.

## Decision guides

- [Which decision model?](https://cloudlesslabs.com/stack/guides/which-decision-model/index.md): Jev, OpenAI Decisions, Clef, Baseten, Perplexity or Liquid? Which decision model fits my job, my volume and my risk?

## The concept behind it

- [Ask an LLM, or call a decision API?](https://cloudlesslabs.com/concepts/llm-or-decision-api/index.md): When my app needs a yes, a no, a label or a score, should it prompt a general model or call a model built to decide?

## In The Stack Weekly

- [Week 46: Sarvam Prism, and Perplexity's decisions that search the web](https://cloudlesslabs.com/weekly/2026-w46/index.md), watching

## Sources

- [Baseten decision models docs](https://docs.baseten.co/inference/model-apis/decision-models.md)
- [Baseten pricing and limits](https://docs.baseten.co/inference/model-apis/pricing-and-limits.md)
- [mammoth-1 in Baseten's library](https://www.baseten.co/library/mammoth-1/)
- [JevBench, open-weights board](https://benchmarkheaven.com/jev-models)
- [Decision Index](https://huggingface.co/datasets/multimodalart/decision-index-leaderboard)

## Related

- [AI](https://cloudlesslabs.com/stack/layers/ai/index.md): every tool in this layer, with its verdict
- [The whole stack in one file](https://cloudlesslabs.com/stack/brief.md): every use case and tool, for questions that span many
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)