Cygnet
Typed decisions from Google's frozen Gemma-4-12B-it, with no fine-tuning. A small shim over unmodified vLLM letters the options, reads the model's own probability for each letter at one answer position and applies one calibration temperature; one output token per decision.
Cygnet adds no weights: Blockbrain publishes the shim (MIT), its tests, one temperature (T = 3.4, fitted on 241 items of its own, none from JevBench) and the run records, and vLLM 0.30.0 fetches Gemma-4-12B-it (Apache-2.0) from Google's repository, so there are no files to pull here. With JevBench's own CLI on the 231 public items it scores 87.9% (hard tier 85/111), with the same answers on an RTX A6000 and an L40S. JevBench's maintainers rank it fifth of 90 in v1.4.2.1, with a score of 61.76 and 33.8% on the sealed items. Choice takes up to 26 options, and inputs up to 16,384 tokens. The readout follows NInfer's. Google's Gemma Prohibited Use Policy applies, which the authors note rules out automated decisions affecting material or individual rights.
What it decides
- choice — picks one option from a set
- score — places the input on an ordered scale
- noul — answers a yes/no question with one calibrated probability
At a glance
| Parameters | 12B |
| Base model | google/gemma-4-12b-it |
| Maker | Blockbrain Labs |
| Released | 2026-09-24 |
| License | mit |
| Reported accuracy | 87.9% |
| Reported latency | 50 ms p50 / 52 ms p95 on an L40S; 66 ms / 72 ms on an RTX A6000 (standard tier, serial) |
Read more
This page was opened by System One for Blockbrain Labs, who can claim the organisation and take it over at any time.