Fine-tune on your own data

Why fine-tune

A base model such as Laya answers general questions well. Your decisions are narrower: your queues, your products, the words your customers use. Fine-tuning trains the base model further on examples of your own decisions, so it learns where your lines are.

You are not teaching it language. You are moving a decision boundary, and a few thousand examples are often enough.

Your data

Each example is a row: the state, the question with its options, and the right answer. For the support inbox, that is a ticket, the list of queues, and the queue a person chose. Decisions your team has already made are the best source.

Before training, set part of the rows aside and never train on them. This held-out set tells you whether the fine-tune helped.

System One Studio

System One Studio is an open-source app that fine-tunes Laya on your own computer. It runs on Apple silicon Macs with MLX, and on Windows and Linux with PyTorch, on the GPU or the CPU. Start it with the CLI:

systemone run studio

The first run downloads the studio and sets up its Python environment. Later runs update it and start in seconds. It opens in your browser, and offers to sign you in first so that it can publish your results. Your datasets and runs stay on your machine, in ~/.layastudio/workspace (the studio used to be called Laya Studio).

Check that it got better

Accuracy alone can mislead. A model that is right 94% of the time but badly calibrated can be worse in production than one at 91% that knows when it is unsure, because every threshold you set from its probabilities is wrong.

System One Studio scores the base model and your fine-tune on the same held-out rows, and reports the accuracy and calibration error of both. It refits the temperature after every run. Keep the fine-tune only if it beats the base model on your held-out rows.

Lineage

A fine-tune records the model it came from as its base_model. When you publish it (the next lesson), it is listed on its base model's page, so anyone can see what was built from that model and how well it did.