Metask Lab's calibrated decision model in 16 languages. Qwen3.5-4B with a merged LoRA trained with the Nimble candidate-logit objective; one forward pass and a softmax over at most 26 answer-letter logits, with one temperature per question type.
Upstage's structured decision model, a System One endpoint on Solar Mini 4. Returns a choice, a score or a yes/no answer with a probability read from the model, in one forward pass, on the same /v1/systemone schema as Jev.
Decides
choice, score, noul, classify, route
choice, score, noul, classify, route
Architecture
metask-jev
solar
Fine-tuned from
qwen/qwen3.5-4b
—
License
apache-2.0
proprietary
Availability
Open weights
Hosted API
Hosted by
—
Upstage
Input price
—
$0.10/MTok
Decision accuracy
80.1%
—
Calibration error
—
—
Valid action rate
—
—
Median latency
62.8 ms
—
p95 latency
—
—
Figures are from each model’s manifest; accuracy and latency are what the publishers report, on their own suites and hardware. Add a third model.
Questions
What is the difference between metask-jev and solar-decide?
metask-jev is from Metask Lab and solar-decide from Upstage. metask-jev has open weights you can download and run; solar-decide is only available as a hosted API. Both answer choice, score, noul, classify and route questions. solar-decide reads up to 512K tokens of state, against 4K tokens for metask-jev. metask-jev is licensed apache-2.0; solar-decide, proprietary.
Which is more accurate, metask-jev or solar-decide?
Only metask-jev publishes an accuracy figure (80.1% on JevBench v1.2 public set (231 items) at 4,096 tokens, maker's run); solar-decide does not, so there is no comparison to make without your own test.
Which is cheaper, metask-jev or solar-decide?
metask-jev: Free (open weights). solar-decide: $0.1 / $0 per 1M. Open weights cost nothing per call beyond your own hardware.
Can I run metask-jev or solar-decide locally?
metask-jev yes — systemone pull metask-lab/metask-jev downloads its weights. The other is only served as a hosted API.
Evaluation suite
JevBench v1.2 public set (231 items) at 4,096 tokens, maker's run
