LESSER: Post-Training Data Selection with Output-Layer Gradients¶
L1 feature and selection diagnostic. Output-layer gradients, two-sided random projection, and unchanged round-robin selection are implemented. No claim of reproducing LLM fine-tuning, RL, teacher selection, or speedups.
Paper information¶
| Field | Value |
|---|---|
| Paper | arXiv 2610.03702 v1 |
| Affiliation | University of Pennsylvania; first author Lyuxin David Zhang |
| First public release | 2026-10-02 (arXiv v1; announced October 5) |
| Authors' code | No public code found in official HTML as of 2026-10-05. |
| Adapter | lesser mechanism API, not a registered full reproduction |
| Local code | lesser.py; runner |
Original paper¶
Background and contribution¶
Targeted data selection often computes per-example full gradients for candidates and target queries. LESSER replaces features, not the selector: one forward yields hidden states and probabilities, from which output gradients are calculated analytically. SFT, RL question selection, and teacher selection rules can remain unchanged.
flowchart LR
A[Candidate pool and selection queries] --> B[Frozen forward: hidden states and probabilities]
B --> C[Weighted readout residual outer products]
C --> D[Shared two-sided random projection]
D --> E[Original query similarity and round robin]
E --> F[Training subset; test excluded from selection]Key figure from the paper¶
Original workflow from the paper; copyright belongs to the authors.
Core equations¶
For fixed token weights \(\alpha_t\), Proposition 1 gives
Appendix B.2 shares fixed Rademacher matrices \(\Pi_1,\Pi_2\) across candidate/query features. Project per-token residuals and states before constructing a full \(|V|\times d\) gradient:
Treat the output layer as an independent readout; tied embeddings can also affect hidden states, so the full gradient is not this expression. Zero weights contribute nothing; RL may use signed advantages. Use validation or target proxy queries, not held-out test for selection.
Paper results¶
Feature extraction FLOPs were 9.7×/3.0× lower than full gradients for SFT/RL. Across four LLMs, five tasks, and three budgets, LESSER and LESS averaged an absolute difference of 1.3 points. Workflows also differ outside features. No production A/B reported; these are not local timings.
Local reproduction¶
Three seeds compare analytic gradients to independent-readout autodiff and per-token projection to full-gradient projection, plus duplicate-free round robin. See mechanism results. Tests cover signed and zero weights, shapes, and budgets.
Reproduction limits¶
Hidden states and logits are fixed random tensors for deterministic formula/selector checks, not real LLM runs on Tulu V2, GSM8K, or A100 throughput. diagnostic_only=true, no formal dashboard or Evolve gains. L2 would need pinned public checkpoints/data, query/test isolation, matched-budget LESS/random controls, and multi-seed training; advertised CUDA would require an A100/A30 receipt.