When do extra reasoning, cross-language routing, and continual feedback change decisions?¶
简体中文 · English
Updated 2026-10-27 · 4 topic entries / 4 representative comparisons
Adaptive decisions operate on three timelines. Selective Reasoning allocates reasoning and readout within one question; a cross-language cascade routes requests across backends; Jev-DLC uses observed feedback to change later requests. Capability evaluations establish a baseline without automatically validating any control strategy. Freeze or make choices before revealing the current test target.
Mechanisms · Trade-offs · Cross-paper questions · Updates
How to read this card
The table compares representative methods. It is not a shared-benchmark leaderboard. Teaching examples are not measured results.
Mechanism comparison¶
Axes: Within-request control versus between-request state updates · Entropy, calibration, error ranking, and task accuracy · Validation-selected thresholds and test-time information isolation · Paper capabilities, recomputed statistics, and local real-model measurements
| Paper and source | Mechanism | Benefits and trade-offs |
|---|---|---|
| Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界与成本结论 |
A shared reader scores the empty prefix and intermediate reasoning states. A benefit gate, read trigger, and accept/stop thresholds serve distinct roles. | Low entropy indicates concentration only. Record budget-forced stops separately. The full online experiment was not reproduced; the controller and recomputed author statistics are not regenerated answers. |
| Jev-DLC: Decision-Level Continual Learning from Typed Feedback · Original source Sections in the Chinese source: 背景与主要改动, 双通道、PA-I与可靠性, 本地复现与边界 |
Maintains conservative and responsive channels above a frozen scorer. PA-I updates the correction direction, and errors in pre-feedback predictions adjust reliability. | Does not store full interactions, but still needs external representations and a fixed vocabulary. Receive feedback only after submitting the current choice; updates cannot change its score. The local state component uses random external logits. |
| Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 已验证结果与边界 |
Measures cross-language accuracy, probability calibration, and error ranking separately. Selects the low-confidence routing fraction by validation macro-F1, then freezes it for test. | Temperature calibration need not put errors first, and translation is not always beneficial. Cluster language variants of the same sentence rather than treating them as independent samples. |
| Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks · Original source Sections in the Chinese source: 背景与主要改动, 数据与评测契约, 本地验证与边界 |
Normalizes knowledge, reasoning, and multilingual tasks to finite options and scores a single choice with isolated labels. Separates vendor-reported values from measurements. | An evaluation protocol, not an adaptive algorithm. Local NanoJev ARC results do not represent closed Jev, and support for 13 task interfaces does not mean all 13 ran in full. |
Trade-off analysis¶
Compute feedback and user feedback are different loops¶
Selective Reasoning observes intermediate distributions while the answer is unknown and decides whether to keep generating or read out. Jev-DLC receives accept/revoke after a request and changes the next request's state. The former cannot require the current gold answer to stop; the latter can use feedback without retroactively changing the score. They can be combined, but log per-question compute and generalization after cross-question adaptation separately.
Sources: Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop · Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Well-calibrated probabilities are not the same as finding hard questions¶
Cross-language transfer uses ECE and Brier scores for calibration and measures error ranking with metrics such as risk–coverage. Selective Reasoning's normalized entropy measures concentration in the current distribution. A temperature can improve calibration without improving error ranking. Strong knowledge and weak math scores do not directly yield a per-question trigger. Set routing and budgets on validation, then measure tasks and languages independently.
Sources: Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages · Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop · Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks
Cross-paper questions¶
Try answering before expanding the explanation. Each question relates at least two papers.
Why do Selective Reasoning's intermediate readout and Jev-DLC's revoke update need different evaluation clocks?¶
Show answer and example
Answer: The intermediate read occurs before the current target is revealed and tests whether extra compute changes the current choice. A revoke has already revealed what should replace the old choice, so it can update only later requests. Rescoring the current response after feedback leaks the label into prediction and does not demonstrate continual adaptation.
Illustrative example: For question 10, the model first selects A and then receives “change to B.” DLC can use that for question 11; question 10 is scored on A. Reading an intermediate distribution on question 10 happens before that feedback.
Sources: Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop · Jev-DLC: Decision-Level Continual Learning from Typed Feedback
Why test low-confidence routing and entropy stopping separately after calibration improves?¶
Show answer and example
Answer: Calibration asks whether probabilities match aggregate correctness frequencies. Routing needs errors concentrated at low confidence, while entropy stopping sees only whether one distribution is concentrated. Cross-language transfer tests these separately. Selective Reasoning allocates budget with validation-selected controls; lower ECE alone does not show better early stopping or fallback.
Illustrative example: Two models may each be correct 80% of the time in their 0.8-confidence bins. Across the full confidence ranking, one places more errors at the bottom while the other spreads errors across bins. That single bin cannot determine how much a cascade saves.
Sources: Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages · Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
If a language or math task appears weak, can we try many routing fractions on test and report the best?¶
Show answer and example
Answer: No. Capability measurements suggest checking coverage and errors by task but do not turn test into a strategy-selection set. Choose the smallest routing fraction that meets the validation macro-F1 target, then freeze it. Likewise, fix reasoning triggers and entropy thresholds in advance and preserve budget-forced cases.
Illustrative example: Validation chooses to route the lowest-confidence 20% to the stronger model. Even if 30% looks better on test, it is not the predefined strategy unless evaluated on a separate confirmation set.
Sources: Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages · Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks · Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
Update log¶
Updates record new approaches, changed trade-offs, and follow-up evidence.
2026-10-27 · Initial version: compared Selective Reasoning, Jev-DLC, cross-language transfer, and the JEV capability evaluation. Covered within-request control, between-request state, entropy, calibration, error ranking, accuracy, and cross-paper questions.
Affected: Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks · Jev-DLC: Decision-Level Continual Learning from Typed Feedback · Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages · Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop
All 4 topic entries
The full catalog follows. The main comparison covers the representative papers listed above.
- Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks
- Jev-DLC: Decision-Level Continual Learning from Typed Feedback
- Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages
- Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop