Banking77 · balanced (multilingual)

A Laya decision model. It answers the questions below in one forward pass, with calibrated probabilities and no generated text. Fine-tuned from aac6fef/laya-multilingual-mlx.

Evaluation

Measured on the held-out test split of Banking77 (77 intents).

Metric	Base model	This model
Decision accuracy	34.2%	64.2% [60.7%–67.5%]
Calibration error (ECE)	0.462	0.024
Log loss	5.083	1.309
Brier score	1.064	0.465
Median latency		62.2 ms
p95 latency		69.9 ms
Decisions scored	770	770

Fine-tuning fixed 246 decisions the base model got wrong and broke 15 it got right.

Use it

pip install systemonemodels
systemone pull biplov/banking77-balanced-multilingual

import json
import laya_mlx as laya  # pip install laya-mlx, on Apple silicon
from systemone import snapshot_download

path = snapshot_download("biplov/banking77-balanced-multilingual")
agent = laya.load(str(path))
questions = json.loads((path / "questions.json").read_text())
print(agent.predict("your text here", questions)["answers"])

Questions

The questions it was trained to answer. Their wording is part of the model's input, so ask them as written.

{
  "intent": {
    "type": "choice",
    "instructions": "Which banking support intent does the customer have?",
    "criteria": [
      "Refund_not_showing_up",
      "activate_my_card",
      "age_limit",
      "apple_pay_or_google_pay",
      "atm_support",
      "automatic_top_up",
      "balance_not_updated_after_bank_transfer",
      "balance_not_updated_after_cheque_or_cash_deposit",
      "beneficiary_not_allowed",
      "cancel_transfer",
      "card_about_to_expire",
      "card_acceptance",
      "card_arrival",
      "card_delivery_estimate",
      "card_linking",
      "card_not_working",
      "card_payment_fee_charged",
      "card_payment_not_recognised",
      "card_payment_wrong_exchange_rate",
      "card_swallowed",
      "cash_withdrawal_charge",
      "cash_withdrawal_not_recognised",
      "change_pin",
      "compromised_card",
      "contactless_not_working",
      "country_support",
      "declined_card_payment",
      "declined_cash_withdrawal",
      "declined_transfer",
      "direct_debit_payment_not_recognised",
      "disposable_card_limits",
      "edit_personal_details",
      "exchange_charge",
      "exchange_rate",
      "exchange_via_app",
      "extra_charge_on_statement",
      "failed_transfer",
      "fiat_currency_support",
      "get_disposable_virtual_card",
      "get_physical_card",
      "getting_spare_card",
      "getting_virtual_card",
      "lost_or_stolen_card",
      "lost_or_stolen_phone",
      "order_physical_card",
      "passcode_forgotten",
      "pending_card_payment",
      "pending_cash_withdrawal",
      "pending_top_up",
      "pending_transfer",
      "pin_blocked",
      "receiving_money",
      "request_refund",
      "reverted_card_payment?",
      "supported_cards_and_currencies",
      "terminate_account",
      "top_up_by_bank_transfer_charge",
      "top_up_by_card_charge",
      "top_up_by_cash_or_cheque",
      "top_up_failed",
      "top_up_limits",
      "top_up_reverted",
      "topping_up_by_card",
      "transaction_charged_twice",
      "transfer_fee_charged",
      "transfer_into_account",
      "transfer_not_received_by_recipient",
      "transfer_timing",
      "unable_to_verify_identity",
      "verify_my_identity",
      "verify_source_of_funds",
      "verify_top_up",
      "virtual_card_not_working",
      "visa_or_mastercard",
      "why_verify_identity",
      "wrong_amount_of_cash_received",
      "wrong_exchange_rate_for_cash_withdrawal"
    ]
  }
}

Training

{
  "method": "lora",
  "objective": "proper",
  "epochs": 4,
  "batch_size": 8,
  "grad_accum": 2,
  "lr": 0.0002,
  "head_lr": 0.0001,
  "lora_rank": 16,
  "lora_alpha": 32,
  "lora_dropout": 0.05,
  "lora_layers": 0,
  "full_layers": 4,
  "head_dropout": 0.1,
  "weight_decay": 0.01,
  "warmup": 0.06,
  "max_grad_norm": 1.0,
  "shuffle_options": true,
  "class_weighting": "none",
  "patience": 2,
  "grad_checkpoint": "auto",
  "precision": "bfloat16",
  "seed": 13
}

---

Generated by systemone push from the files in this repository. Push a README.md, or edit the card on this page, to replace it.