Which decision model?

The short answer

This space is seven weeks old. Jev opened it on 15 September, and nothing here has a long record. So treat every pick as a pilot, on your own labelled data.

Jev started all this and isn't a pick for client work yet, because it's closed, in early access and has an unanswered leak report.

Prices are per million input tokens, and every decision model here charges nothing for output.

The one thing the vendors don't tell you: on the fullest independent benchmark, decision models win on speed and price, not accuracy. GPT-6 Luna used as an ordinary chat model scores 98.4, 13 points above the best decision model, at about eight times the latency. OpenAI's own Decisions product, built on that same model, scores 73.5. You're trading accuracy for speed, price and a probability you can set a threshold on. Often that's a great trade. It's still a trade.

The whole field in two pictures

The map is the quickest way to see who's worth a look. Perplexity's Decider and Liquid's d1-ultra sit near the top at a fraction of the price of mammoth-1. Clef costs twelve times what Decider costs for a lower score, which is hard to defend unless you need it inside a Worker, and then Clef Flash is the cheaper route.

The shapes show the trade each one makes. Readiness is my own judgement: GA from a vendor with a record scores highest, Jev scores lowest for being in early access with no published deprecation policy and an unanswered leak report, and everything else sits in between because it's weeks old.

Prices

The last two columns are my arithmetic. For the LLMs, I assumed 60 output tokens per answer. Clef Flash's free allowance covers about 5,800 of these decisions a day.

At 10 million decisions a month, that's about $210 a month on Jev, and $50 on joey-1. Your own GPU costs $764 a month whatever you do with it, so hosted wins on price until a very high volume. The real reasons to self-host are residency and an exit, not the bill.

What they can do

What happens over the limit matters more than the limit. Clef silently truncates your text to fit, which can change the answer with no error. Jev and Baseten refuse the request, which is what you want. Count your tokens before you send anything.

Most of them share one request shape, called System One, so switching is cheap. OpenAI doesn't: it uses input and predicate, so plan for an adapter.

How good they are

No score here is comparable across columns. Each benchmark tests different things in a different way.

• JevBench (Benchmark Heaven) covers the most of them on one scale. It's a one-person project that runs the models itself and says it's independent of TypeSafe, and anyone can pay to be scheduled sooner.
• The Decision Index runs private tests, which is where joey-1 falls down. Its public score of 63.3 becomes 35.91 on private tests, and 15.25 on domains it hasn't seen. Check any headline number against what it does on data it wasn't trained near.
• Artificial Analysis tests 20 of them independently, on knowledge, safety and tool choice rather than business decisions. d1-ultra leads at 65.25, then elephant-1 at 64.69, and Jev 1.13 scores 57.5.

Can you trust the probability? Roughly. On the Decision Index, when these models say about 0.97 to 0.99, they're right about 93% to 97% of the time. That's overconfident, and Jev is the most overconfident of them. On harder tasks the gap grows. So tune your threshold on your own labelled data, and never read 0.95 as 95%.

How fast they are

Never compare speed across harnesses. Clef Flash alone reads 38.8 ms in Cloudflare's own test, 53 ms on the Decision Index's GPU and 298 ms through its hosted API on Artificial Analysis.

Artificial Analysis doesn't state request sizes or its network path, so read this as a rough order. mammoth-1 and joey-1 aren't on either list. Baseten claims 20 ms for joey-1 and 49 ms for elephant-1, on its own setup. From Sydney, add your trip to the US, because most of these run there.

Can you rely on them?

• No named company has published what it runs on any of these at volume, and I found no public uptime numbers. Vendor claims such as "13% of paid gateway teams" measure interest, not production.
• Limits can bite early. Baseten's Pro tier is 120 requests a minute, about 2 a second, and more needs an Enterprise deal. Perplexity publishes 10 a second. Clef's limit isn't published, and OpenAI's and Jev's aren't either. Ask before you launch.
• The floor can move. Jev's jev-latest moved to 1.14 on 24 October, and Baseten's slugs end in -latest. Pin a version where you can, and re-run your test set when it changes. No vendor here has a published deprecation policy for these models.
• Jev's leak report: a third-party report on 24 October says Jev's outputs were exposed for every customer. I found no root cause or formal statement from TypeSafe. Until there is one, keep personal data away from it.
• Cloudflare's incident wasn't Clef, but it was the same shared Workers AI platform: 155 GLM-5.3 responses carried other customers' data.
• There's no decision model on Amazon Bedrock, and none from Anthropic or Google. Bedrock's custom import doesn't run in Sydney either. OpenRouter lists only a Jev router, not these models, so my usual gateway doesn't help you here.

Can they be fooled?

Yes, and this is the part to read before you put one in front of the public. DecisionRobust, an academic test, plants an attacker's chosen answer and counts how often the model gives it:

Lower is better, but don't rank on this alone: a model that reads well also reads the attacker's text well, and attacks were tuned on some of these. Clef, joey-1, elephant-1, mammoth-1 and Microsoft weren't tested, which includes my own pick. One tester's subtle "the support lead already decided" flipped 147 of 200 tickets on Jev, so its confidence is no alarm either. Keep untrusted text in the state, never in the instructions, and never let one of these be the only gate on anything that moves money.

Images

Perplexity's models aren't on this board, but they lead the other one, the Decision Index Vision board: Decider v1.2 scores 71.72 and Flash 66.02, against 59.54 for Clef-omni. joey-1, elephant-1 and mammoth-1 aren't on either. Calibration falls on images for all of them: Clef drops from 91.6 to 68.8, and Jev 1.14 from 90.1 to 70.5. So thresholds set on text don't carry over. The full comparison, with limits and costs, is in The Stack Weekly.

My advice: use OpenAI Decisions if you need GA today, Perplexity if its data terms suit you, and a review queue either way. To experiment, start with Jev: 1.13 scores highest on this board and 1.14 is the only one that returns a bounding box, though TypeSafe says neither is production-ready. Pin the version, because jev-latest has already moved to the lower-scoring 1.14.

Use cases

Everything here is also weak at counting, dates and arithmetic, and tends to favour the first option in a list. Do those in code, and shuffle your options.

How I'd set this up for a client

1. Label 200 of their real cases, with the right answer and what a wrong one costs.
2. Run three models against it: mammoth-1, elephant-1 and Decider v1.2, or Clef Flash if they're on Workers. Measure accuracy, and how often a stated 0.9 is right.
3. Set a threshold per question. Above it, the code acts. Below it, an LLM or a person looks.
4. Try to fool it with the client's own kind of bad input, before launch.
5. Keep an exit: a thin adapter, a pinned version, and open weights, so you're never stuck with a vendor that's weeks old.

Sources

Checked on 2 November 2026. Every figure, with its status and link, is in the research record in this site's repository.

• Benchmarks: JevBench, its API board and images board; the Decision Index and its vision board; Artificial Analysis.
• Robustness: DecisionRobust, with its results file.
• Vendors: Jev, OpenAI Decisions, Clef Flash, Workers AI pricing, Baseten and its limits, Perplexity, Liquid AI, Microsoft, Strands Decider.
• Tests by others: a phishing benchmark and a reranking test.