OLIVE: Learning from Teacher Continuations at Student States

简体中文

L1 exact student-prefix/teacher-continuation loss masking, without a closed teacher call.

Paper information

Field	Details
Paper	Learning from Teacher Continuations at Student States
Institution	University of Illinois Urbana-Champaign (first author).
First public release	2026-09-28 (arXiv v1)
Author code	The source lists the author project as the release link.
Local adapter or method key	olive
Local reproduction code	src/auto_research/post_training/latest_20260930_closure.py

Paper method and evidence

Have the current student generate a prefix, then ask the teacher to continue from that student state. Train on teacher continuation tokens only.

flowchart LR
  S[Student samples prefix] --> T[Teacher continues from student state]
  T --> M[Mask loss to teacher continuation tokens]

Original paper figure

Figure 2 shows learning from teacher continuations on student states. From the original paper; copyright remains with the authors.

Core equation

Cross-entropy is masked off the student prefix. Tests check gradient isolation, type boundaries and deterministic execution.

Reported results

The paper's benchmark, speed, and learning-curve outcomes are not local capability evidence.

Local reproduction

Three-seed mechanism results are diagnostic_only=true. Fixed-budget fixtures verify the defining operator and auditable intermediate states; they are not a benchmark, production result, or capability-ranking entry.