92.2% vs 90.0%: Google's decision API edges out Jev

Google's new Decisions API beats TypeSafe's Jev on 5 of the 7 benchmarks they share — and Google's own chart shows Jev is the cheaper of the two. The JevBench gap is 92.2% to 90.0%, and the calibration gap is bigger than the accuracy gap.

What launched

On November 11, Google announced the Gemini Decisions API, powered by a new model called Gemini Decision 1, available through Google AI Studio. The pitch: bounded decisions instead of generated text. You send one multimodal input — text, images, audio, video, or PDF, up to 256K tokens — plus up to 1,000 questions, and the model scores every candidate answer in a single forward pass instead of decoding tokens one by one. Median model compute is 50–100 ms, end-to-end latency 150–250 ms. Pricing is $0.10 per million input tokens with zero output tokens billed, and the shared input is billed once no matter how many questions you attach.

The three question types are noul (yes/no as a probability), choice (pick from 2 to 255 options), and score (an ordered rubric starting at 0). If those names feel familiar, they should be: they are Jev's three primitives, word for word. Jev launched September 15 — 57 days before Google's announcement — with noul, choice, and score, a 255-option cap on choice, and an ordered rubric starting at 0. Google's API is the same shape with a bigger model behind it.

Google's own chart, read row by row

Google published a benchmark table comparing Gemini Decision 1 against TypeSafe Jev 1.14, OpenAI's GPT-6 Luna (via the OpenAI Decisions API), and Gemini 3.5 Flash-Lite (via the Interactions API). I read the numbers off the chart and re-computed the comparisons rather than taking the blog's summary at face value:

benchmark              Decision 1   Jev 1.14   GPT-6 Luna   Flash-Lite
JevBench               92.2%        90.0%      90.9%        86.6%
Amazon ESCI (Macro-F1) 58.7%        56.7%      54.7%        47.2%
New Yorker Caption     79.7%        72.0%      68.4%        73.7%
Mind2Web Multimodal    86.2%        86.8%      81.8%        78.0%
PubMedQA               79.8%        78.2%      74.4%        77.6%
Global-MMLU-Lite       86.0%        85.0%      79.6%        78.8%
When2Call              79.9%        82.0%      70.4%        60.9%
MMAU                   71.8%        —          —            67.7%
Video-MME              69.1%        —          —            63.3%
Top-1 ECE (lower better) 0.0250     0.0494     0.0721       0.2104
Total price (7 shared)  $0.88       $0.68      $0.93        $4.14

Three findings fall out that the blog post doesn't headline.

First, Jev still wins two of the seven shared accuracy rows. Mind2Web Multimodal goes to Jev, 86.8% to 86.2%, and When2Call goes to Jev by a wider margin, 82.0% to 79.9%. Decision 1 leads the other five. Counting the two rows where only Google and Flash-Lite reported scores (MMAU, Video-MME), Decision 1 leads 7 of 9 — but the honest denominator against Jev is 5 of 7, and two of those five wins are by less than 2 points (PubMedQA +1.6, Global-MMLU-Lite +1.0).

Second, calibration is where Google actually pulls away. Top-1 expected calibration error across the 7 shared evals: Decision 1 at 0.0250, Jev at 0.0494, Luna at 0.0721, Flash-Lite at 0.2104. Decision 1 is roughly 2x better calibrated than Jev, 2.9x better than Luna, and 8.4x better than Flash-Lite. For a decision API, that is arguably the more important row — a routing or gating decision is only as good as its confidence number, and Jev's whole pitch is that a 0.7 means 70%. Google's chart says its 0.7 means 0.7 more often than Jev's does.

Third, the price row undercuts the pricing story. Google's blog says Decision 1 is "nearly 5x cheaper across the shared benchmark suite than Gemini 3.5 Flash-Lite." That checks out: $4.14 / $0.88 = 4.7x. But the same row shows Jev at $0.68 against Google's $0.88 — Jev is 23% cheaper on the identical seven evals, or put the other way, Google costs 29% more than Jev. The per-token prices explain only part of it: Google charges $0.10 per million input tokens against Jev's $0.042, a 2.4x gap, but the totals differ by only 1.3x. The arithmetic implies the token counts weren't equal — $0.88 at Google's rate is 8.8 million tokens, $0.68 at Jev's rate is 16.2 million. Either Jev's eval prompts were nearly twice as long, or the two vendors didn't run byte-identical inputs. Google doesn't say which, and that is the question I'd want answered before treating the price row as a head-to-head.

The category is now four players deep

Jev was the first dedicated decision model, announced September 15 by TypeSafe with a $40M seed round. OpenAI followed: its Decisions API, powered by GPT-6 Luna, entered public beta on October 6 and went generally available on October 22, at the same $0.10 per million input tokens Google now charges, with up to 400 questions per request and a claim of up to 10x faster than calling Luna through the Responses API. Google is third, with the 1,000-question cap and the multimodal input. And Amazon's Decider — a small model you run yourself, which I measured last week at $0.0005 per decision in practice — is the local, self-hosted answer to the same question.

Latency, for the record: Google quotes 150–250 ms end to end; TypeSafe quotes 70–500 ms for Jev; OpenAI's Japan event put its Decisions API at 130–280 ms against 1.4 seconds for the equivalent Luna call. Google's band is the narrowest, but Jev's floor is lower. Nobody here is slow; the differences are in accuracy, calibration, and price, not in whether the answer arrives in time.

What I did not do: I have no Google AI Studio API key, so I made no live calls to the Decisions API. Every number above is Google's published chart, read and re-computed by me — the ratios, the token-count implication, and the two Jev wins are my arithmetic, not Google's copy. The chart's own footnote is worth repeating: refusals, safety blocks, and requests a model could not accept are scored as incorrect, and every score uses the full benchmark as the denominator.

The interesting part isn't who won a benchmark table. It's that three vendors now ship the same API shape — noul, choice, score — with the same pricing structure and the same pitch, 57 days after a startup invented the vocabulary. Decision models stopped being a startup gimmick the day Google copied the interface. The next question is whether anyone's calibration numbers hold up outside the vendor's own chart.

Sources: Google DeepMind, "Gemini Decisions API: Fast, calibrated multimodal decisions" (blog.google, November 11, 2026) and its benchmark chart, read and re-computed by me; TypeSafe Jev documentation and pricing ($0.042 per million input tokens, 70–500 ms, announced September 15, 2026); OpenAI Decisions API general availability, October 22, 2026 (beta October 6, GPT-6 Luna, $0.10 per million input tokens, up to 400 questions per request); Amazon Strands Decider pricing from my own measurement, November 11, 2026.