LESSON 2 OF 9 · 6 MIN
System One models and LLMs
When a decision model is the right tool, when a language model is, and how the two work together.
Which way should the snake go next?
The snake is heading east, and the food looks to be above it and slightly to the left. Turning left would bring it closer, but it is worth checking first whether its own tail could block the way back. On balance, the most reasonable choice is probably to turn left — although carrying straight on for one more step would also be defensible.
{"action": "right",
"confidence": 0.89}Two tools for two jobs
A large language model (LLM) answers "what should I say about this?". A System One model answers "what should I do about this?". Both are useful. Problems start when one is used for the other's job.
| A language model | A System One model | |
|---|---|---|
| Output | Free text | A typed answer, with a probability per option |
| Possible answers | Anything | Only the options you send |
| Time to answer | Grows with every word it writes | Nothing is written: often tens of milliseconds |
| Runs on | Usually a GPU | Often a CPU |
| Judged on | How good the text is | Accuracy and calibration |
When a System One model fits
Use one when the possible answers are known in advance and the decision comes
up often: routing a ticket, scoring a lead, flagging a message, choosing the
next move in a game. If the code after the model would be a switch
statement anyway, a decision model fits.
- The answer is always valid. The model scores the options you sent and picks one of them. It cannot invent an option, so there is no parser to fail.
- The confidence is usable. Set a threshold on the probability, act above it, and send the rest to a person.
- It is small. Many System One models run on an ordinary CPU, inside a request, at a cost that stays low over millions of calls.
When a language model fits
Use an LLM when the output really is language: a reply to a customer, a summary, an explanation. Use one too when the possible answers cannot be listed in advance.
Using both
A common pattern:
- A System One model reads each request and decides where it goes.
- Confident decisions are acted on at once.
- Unsure ones go to a person, or to a language model that has room to reason.
- When someone needs a written reply, a language model writes it.
Most requests take the fast path, and the language model is used only where writing is needed.
Try it
Find a decision in your own code
Look for a place where your code asks an LLM to "answer with one of" a few words and then parses the reply, or where a person picks from a short list many times a day. Write down the state, the question and the options.
Decide what happens when the model is unsure
Pick a starting threshold, such as 0.9, and decide where the other cases go: a person, a safe default, or a language model. Lesson 4 gives you real probabilities to test it against.
Read more
Every lesson is open to read. Sign in to tick them off and claim the certificate at the end.