Method
RLCD is the objective laya is trained with, and it was open sourced under Apache 2.0 along with the weights, the dataset and the training script. It is written out here because the rest of this site claims these models are honest about their own uncertainty, and that claim is worth nothing without the thing that produces it.
Ordinary training rewards one thing: was the top answer right. A model can score perfectly on that and still report 0.99 on answers it gets wrong one time in three, because nothing in the loss ever asked about the number next to the answer. The number is a byproduct, so it behaves like one.
RLCD rewards the whole reported distribution instead, using scoring rules that are strictly proper. A scoring rule is strictly proper when its expected score is highest at exactly one report: the true probabilities. Under a rule like that, shading a number up to look confident is not a way to score better, it is the only way to score worse. Honesty stops being a thing you hope for and becomes the maximum.
Three scoring rules, summed. The log score is the strict one and does the work. The spherical score is bounded and keeps the gradient alive where the log score has already saturated. The ranked probability score is added only for ordered questions, where answering 4 when the truth is 5 should cost less than answering 1.
S_log(q, y) = sum_k y_k · ln q_k
S_sph(q, y) = (y · q) / ||q||
RPS(q, y) = sum_k (Q_k - Y_k)^2 / (K - 1)
R(q, y) = S_log
+ w_sph · S_sph
- w_rps · RPSThere is no second model and no reward model. The policy is the logit vector itself. Four perturbed copies are drawn around it, each one is scored by the rule above, and the copies that scored better than the group average pull the real logits towards themselves.
eps_g ~ N(0, sigma^2)
eps_g <- eps_g - mean(eps_g)
z_g = z + eps_g
q_g = softmax(z_g)
A_g = (R_g - mean R) / (std R + 1e-6)
ln pi(z_g | z)
= - ||z_g - z||^2 / (2 sigma^2)
L = - mean_g [ A_g · ln pi(z_g | z) ]
+ soft_CE(z, y)Two details in there are not decoration. The noise is zero meaned across the live options because a softmax is invariant to adding a constant to every logit, so any common part of a perturbation would be explored for free and learned from for nothing. And the advantage is standardised within the group, which is what removes the need for a learned value baseline: the other three samples are the baseline.
The soft cross entropy term is kept alongside the policy gradient rather than replaced by it. The reinforcement term shapes the distribution and the cross entropy term keeps the answer anchored to the label, and dropping it makes the first epochs wander.
sigma(epoch) = 0.4 -> 0.1
Exploration starts wide and narrows. Early on the four samples are far apart and the group learns the coarse shape of a good distribution; later they sit close in and the same loss resolves fine differences in confidence that a wide noise would have drowned.
421.3M parameters in total: a 395M ModernBERT-large encoder and a 26.5M decision head, trained together with decoupled learning rates, the head ten times faster than the encoder.
1. Pack
The question instruction, every candidate option and the record itself go into one sequence, split by [SEP] into three zones. Each option carries a [MASK] anchor in front of it.
2. Read
One bidirectional pass over the whole sequence, so every token sees every option and the record at the same time. 28 layers, width 1024, rotary positions, GeGLU feed forward.
3. Gather
The head reads the hidden states at the [MASK] positions only. That is what lets every option be scored in the same pass: the anchors are where each option's evidence collects.
4. Score
A two layer transformer over the gathered vectors, then a scorer down to one logit per option, then a softmax across the live options only. No tokens are generated at any point.
5. Reward
Four noisy copies of the logits are drawn, scored by the proper scoring rule below, and turned into advantages. That is the RLCD step.
6. Calibrate
After training ends, one temperature per question type and option count is fitted on a held back split by minimising negative log likelihood. Never during training.
encoder lr 1.2e-5 head lr 1.2e-4 (10x) weight decay 0.01 grad clip 1.0 AdamW beta 0.9 / 0.999 AdamW eps 1e-6
RLCD shapes the distribution while the model trains. It does not remove the need to calibrate afterwards, and the laya benchmarks are blunt about this: the published checkpoints ship over-confident, macro ECE 0.5709 across 51 languages, and the multilingual checkpoint ships with no fitted temperatures at all. On the head to head where a temperature is fitted on held out data, ECE lands at 0.081 against a hosted competitor's 0.246.
That gap between the objective and the shipped artefact is the whole reason the product exists. A training objective can make a model honest in general. Only fitting on your own decided cases, and measuring on rows nothing was fitted on, makes it honest on your traffic. That is what fit and calibrate is, and what Embedding Studio does with one CSV.
Every equation above is one of these two functions. Nothing here is a description of code that was never released.