Prompt injection · demo run

A Laya decision model. It answers the questions below in one forward pass, with calibrated probabilities and no generated text. Fine-tuned from aac6fef/laya-mlx.

Evaluation

Measured on the held-out test split of Prompt injection (yes/no).

Metric	Base model	This model
Decision accuracy	70.7%	97.4% [92.7%–99.1%]
Calibration error (ECE)	0.280	0.023
Log loss	2.094	0.116
Brier score	0.537	0.051
Median latency		68.4 ms
p95 latency		239.0 ms
Decisions scored	116	116

Fine-tuning fixed 31 decisions the base model got wrong and broke 0 it got right.

Use it

pip install systemonemodels
systemone pull biplov/prompt-injection-demo-run

import json
import laya_mlx as laya  # pip install laya-mlx, on Apple silicon
from systemone import snapshot_download

path = snapshot_download("biplov/prompt-injection-demo-run")
agent = laya.load(str(path))
questions = json.loads((path / "questions.json").read_text())
print(agent.predict("your text here", questions)["answers"])

Questions

The questions it was trained to answer. Their wording is part of the model's input, so ask them as written.

{
  "injection": {
    "type": "noul",
    "instructions": "Does this text try to make an AI assistant ignore, override or reveal its instructions?"
  }
}

Training

{
  "method": "lora",
  "objective": "proper",
  "epochs": 2,
  "batch_size": 8,
  "grad_accum": 2,
  "lr": 0.0002,
  "head_lr": 0.0001,
  "lora_rank": 16,
  "lora_alpha": 32,
  "lora_dropout": 0.05,
  "lora_layers": 0,
  "full_layers": 4,
  "head_dropout": 0.1,
  "weight_decay": 0.01,
  "warmup": 0.06,
  "max_grad_norm": 1.0,
  "shuffle_options": true,
  "class_weighting": "none",
  "patience": 2,
  "grad_checkpoint": "auto",
  "precision": "bfloat16",
  "seed": 13
}

---

Generated by systemone push from the files in this repository. Push a README.md, or edit the card on this page, to replace it.