Nokia's applied research team has open-sourced AnyJev, a Python library that turns any open-weight LLM into a calibrated decision model. It installs from PyPI, ships under Apache-2.0, and targets one of the most common production jobs in agentic systems: picking a single answer from a fixed set of options, with a confidence score that actually means something.
AnyJev borrows its interface from Jev, the System One decision model launched by TypeSafe AI in September 2026. You give it a typed question and receive a Decision with a probability you can threshold. That probability is read directly from the model's next-token distribution in a single prefill. Nothing is generated, parsed, or fine-tuned, which makes the layer cheap enough to sit in front of a general-purpose LLM without adding a training pipeline to the stack.
The authors are Jiamu Zhang, Tianze Yang, and Liang Wu from Nokia's Sunnyvale research office, with Yucheng Shi of Tencent Hunyuan. The project is deliberately positioned as independent of Jev and TypeSafe AI: it reimplements the interface, publishes its own benchmarks, and credits the original work without claiming affiliation.
Inside the Levels: L0, L1, and L2
Raw logits fail at this job in two ways, and AnyJev exists to fix both. First, the answer changes when the options are reordered: on Qwen3-8B over BANKING77, reversing the option order flips the raw logit readout's choice 23.0% of the time. Second, the confidence values are not calibrated: the same model carries an expected calibration error of 0.240, so a reported 0.9 confidence is not trustworthy enough to act on.
Level L0 fixes the first flaw with zero labels. It reads the decision over every cyclic rotation of the option list, averages out the position bias across the K rotations, and divides out the label prior estimated without any labels. The result is dramatic for such a small intervention: order-flip rate drops from 0.230 to 0.073, and accuracy climbs from 0.747 to 0.803 on the same 20-way task, at a cost of K prefills per decision batched over a shared prefix. The team measures about 0.25 seconds per decision on one H100 at batch 32 with K set to 20.
Level L1 adds temperature scaling on top of L0, fitted on 100 to 500 labels per question. The fitted values are saved as a small JSON artifact and reused at serving time. L1's job is not to change the ranking of answers but to reshape confidence: calibration error falls from 0.240 to 0.095, and the share of traffic that can be safely automated at a 5% error budget jumps from 7.7% to 52.0%, a 6.8x difference on the measured task.
That last row is the point of the whole library. With raw logits, a 0.9 score is not calibrated enough to act on, so nearly everything routes to a human. Once the probability means what it says, an operator can set a threshold and safely automate the traffic above it. Calibration, in other words, is what converts a language model from a text generator into a component a system can trust by default.
Level L2, new in this release, replaces the temperature with a closed-form head per question, fit on the hidden state at roughly two thirds of the model's depth. It is solved from 100 to 300 labels in seconds on a CPU, with no gradients and the model's weights untouched. The head is served from a single prompt stopped early, so one decision costs about 0.68x of a plain forward pass on Qwen3-8B. Every Decision carries its level, letting downstream code refuse to act on a level it was never given permission to trust.
The benchmark table on Qwen3-family models shows what the head buys across sizes: Qwen3-1.7B moves from 0.494 to 0.730 at L2, Qwen3-8B from 0.647 to 0.771, and Qwen3-32B from 0.700 to 0.798, with pooled calibration error between 0.03 and 0.05. The 1.7B at 64% of its depth reaches the number Jev publishes, and the 4B ties the fine-tuned Laya model on the same set. A head is roughly 100 KB of JSON, and 23 heads per model ship for five Qwen3 sizes, each solved in 2 to 8 seconds.
L2 also maintains itself. Because only the head's feature mean and scale move after the initial solve, they are re-estimated from unlabelled traffic, so the head follows its question across rewordings without new labels. When Qwen3-8B's head meets a reworded question, accuracy dips to 0.65 to 0.70, and 30 unlabelled requests of the new wording bring it back to 0.74 to 0.75. The team also shows distillation without gradients: heads from a 32B model used to label 1,200 generated cases lift the 1.7B from 0.730 to 0.760.
For serving, the transformers backend handles every level today, with vLLM and SGLang on the roadmap. The release is scoped and honest: results are regenerated from committed JSON, and the authors note the 6.8x automation figure is a point estimate on a 300-item slice with a wide interval. What matters for practitioners is the architecture: a no-training layer that turns an open LLM into a decision engine whose probabilities can be thresholded, audited, and safely acted on.
Comments