Identify what self-improvement changes: answers, experience, interfaces, and system versions¶

Updated 2026-11-04 · 47 topic entries / 10 representative comparisons

Self-Refine revises the current artifact, Reflexion changes what experience the next attempt can read, HarnessLens checks local workflow changes, Agent S4 evolves interface repositories, RSIFlow chooses the intervention layer, and AgentX brings candidates into a production loop with guardrails. More rounds are not a shared algorithm: persistent state and acceptance rules define what changes. CROP edits system prompts offline and penalizes output cost. EvalAlloc allocates SEARCH evaluations instead of repairing candidate code, estimates fitness against fixed references, and runs paired confirmation.

Mechanisms · Trade-offs · Cross-paper questions · Updates

How to read this card

The table compares representative methods. It is not a shared-benchmark leaderboard. Teaching examples are not measured results.

Mechanism comparison¶

Axes: Whether the mutable object is an answer, textual experience, interface, model, or artifact · Whether a change serves only the current task or becomes a persistent version · Whether feedback proposes edits, selects parents, accepts candidates, or authorizes release · Whether failures are rolled back, retained in a population, or stored as knowledge

Paper and source	Mechanism	Benefits and trade-offs
Self-Refine · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界	One model explicitly generates a draft, task-specific feedback, and a revision, stopping when requirements are met.	Primarily a current-output loop, whose self-evaluation can repeat errors. The deterministic local critic verifies state transitions only.
Reflexion · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界	Converts execution feedback into natural-language reflections for the next trial's bounded episodic memory.	Context changes rather than gradient weights, and lessons can be wrong or stale. Interpret local template reflection separately from isolated ToolRoute.
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界	Chooses paired validation tasks by affected behavior and requires both positive gain and attribution to the candidate change.	Local acceptance saves evaluation without covering all capabilities. Local ToolRoute is not all paper benchmarks.
Evolving Agent Interfaces · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 成本与复现边界	Freezes the task agent and harness, editing interface code and usage knowledge. Mutated or crossed descendants enter the population even if temporarily low scoring.	Validation selects parents without exposing results to builders. The local path is a trusted-callback controller without desktop-interface evolution; full experiments are expensive.
RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 代码映射、成本与边界	Selects Model, Harness, or Artifacts interventions from evidence and compares the same parent and task batch. Accepts only valid positive gains.	Failures retry from the frozen parent. This is not arbitrary code self-modification. Local scope, pairing, and skill records run without model training.
AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界	Connects evidence-based proposals, repository implementation, guarded online A/B, and retention of positive and negative results in recommendation iteration.	Guardrails can veto primary metric gains. Author production evidence differs from the deterministic local mini-loop without traffic or large-model SGPO.
When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement · Original source Sections in the Chinese source: 背景与主要改动, 核心公式	Uses a six-cell joint posterior, 2,000 draws with a 0.2 quantile, an active frontier, and validation admission.	Behavior labels, executions, and validation scores are external. Statistical ranking is not a full self-evolution run.
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 高成本完整实验	Fixes objective weights, combines task critiques with over-threshold length critiques, and uses an external meta-optimizer to rewrite system prompts. Selects evaluated prompts by accuracy minus CoT-normalized token cost.	Unlike Self-Refine, it does not edit the current answer. Local validation selection explicitly tightens leakage prevention. Deterministic callback diagnostics do not show lossless LLM shortening or full evolution.
Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution? · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 高成本完整实验	Uses SEARCH observation informativeness to choose a common core and items without replacement. Infers only new candidate parameters under fixed external item references, then fills the cache on a common confirmation prefix and applies paired correction.	Predictions cannot overwrite observed cells. Fewer evaluation cells do not imply proportional token savings. Warmup MIRT, cross-fit refitting, and full successive halving are not implemented.
Strategist: Evolving Reasoning with Agents · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 数据协议与实现边界, 高成本完整实验	Freezes executor models, composes executable reasoning blueprints into a cross-task acyclic library, retrieves relevant and adjacent material, and selects by measured development accuracy and tier-weighted token costs.	Nonwinning structures can remain useful for later reuse, but final candidates must win on validation and hidden. Local AST/DAG, external retrieval-score kernels, and paired selection are not meta-agent search.

Trade-off analysis¶

Separate remembering how to change from actually changing the system¶

Self-Refine changes the current output; Reflexion persists text for later attempts. Agent S4 versions interface code and knowledge so available observations and actions can change. RSIFlow separates model, workflow, and artifact. Future outcomes may improve in each case, but do not call all of them model learning. Record what changed, how it executes, and what the next task inherits rather than one policy-update count.

Sources: Self-Refine · Reflexion · Evolving Agent Interfaces · RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent

Population admission, acceptance, and release are distinct¶

Agent S4 retains temporarily low-scoring descendants for later parent selection, a nongreedy search. RSIFlow changes current state only for valid improvements over the same parent. HarnessLens limits acceptance to attributable behavioral evidence. Production AgentX also requires guardrails. Archiving a candidate, passing local validation, and affecting live traffic are not one 'integrated' status. Compare rules and scope rather than ranking incompatible paper scores.

Sources: Evolving Agent Interfaces · RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent · Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification · AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems

October 29: New method and source changes¶

RuHarness does not rank its criterion frontier by one accuracy value. A 2×3 behavior/outcome table yields a Jeffreys–Dirichlet posterior for a conservative utility quantile, while set novelty and redundancy stay fixed from observed traces. This separates finite-sample uncertainty from candidate admission: only active criteria enter the frontier and final harness edits require strictly positive independent validation gains. Local work has no LLM judge or harness generation.

Sources: Self-Refine · When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

Track edit targets, judging criteria, and evaluation budgets separately¶

Self-Refine edits the current output. CROP instead uses training feedback to change a general system prompt for later tasks and explicitly charges output tokens; it does not claim model-weight training. RuHarness maintains a rubric frontier that distinguishes current failures. EvalAlloc decides which SEARCH items deserve real evaluation: with fixed item references it completes sparse estimates, fills only unobserved cells with predictions, then confirms on a common prefix with cached results. Rubric utility, predicted fitness, and real held-out success are different evidence. Half the cells does not mean half the LLM cost.

Sources: CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization · Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution? · Self-Refine · When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

Library admission does not make a strategy the current champion¶

Strategist turns executable structure, measurement cards, and training-trace experience into cross-task assets, unlike Reflexion's readable lessons. Evaluated nonwinners may remain available for future composition. Selecting the current deliverable separately requires validation and hidden gates, with test data excluded. EvalAlloc governs which evaluation cells receive budget rather than proposing strategy structure. All depend on real execution feedback, but experience accumulation, structural reuse, and budget allocation are distinct mechanisms. Synthetic scores passing any selection rule provide no LLM capability evidence.

Sources: Strategist: Evolving Reasoning with Agents · Reflexion · Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?

Cross-paper questions¶

Try answering before expanding the explanation. Each question relates at least two papers.

How do three revisions of one answer and recording a lesson for another attempt distinguish Self-Refine from Reflexion?¶

Show answer and example

Answer: Self-Refine generates feedback and edits the current draft, mainly changing that artifact. Reflexion turns execution evaluation into experience readable by the next trial's actor. Both can use a frozen model and imply no gradient training. Check whether feedback enters later context, applies to the new state, and actually improves a verified result; iteration count does not establish learning.

Illustrative example: Correcting a wrong number is a current revision. Saving 'check source tables before quoting numbers' for future tasks is cross-trial state.

Sources: Self-Refine · Reflexion

Why aren't retaining low-scoring Agent S4 descendants and rejecting non-improving RSIFlow candidates contradictory?¶

Show answer and example

Answer: They maintain different search states. The Agent S4 population retains exploration sources, and low-scoring descendants may help later search; private validation chooses parents. RSIFlow tests each candidate from a frozen parent and failure leaves the next starting state unchanged. One preserves diversity and the other governs the current version. Archive admission is not release, and neither needs the same monotonicity criterion.

Illustrative example: An interface variant with a new action can remain in S4 despite a low score. After a non-improving workflow patch, RSIFlow starts again from the original workflow.

Sources: Evolving Agent Interfaces · RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent

Why doesn't a paired local behavior gain in HarnessLens replace AgentX production-release judgment?¶

Show answer and example

Answer: Local validation asks whether the selected behavior gains positively and attributably. Deployment also involves business distributions, guardrails, and actual resources. AgentX can veto a primary gain and records positive and negative experiments. The stages can connect, but small local protocol success supplies neither traffic causality nor authorization to release.

Illustrative example: Offline recovery may improve while request latency rises in production. Functional evidence can hold and guardrails can still reject release.

Sources: Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification · AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems

Why doesn't RuHarness resample novelty and redundancy with each Dirichlet draw?¶

Show answer and example

Answer: They describe overlap of fixed trajectory sets, not six-cell outcome probabilities. Randomizing them changes the uncertainty model. A conservative posterior quantile also does not show that actual agents improved.

Illustrative example: Two criteria flag the same failure set. Sampling a success/failure probability should not change that overlap.

Sources: Self-Refine · When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

Why can't one correction-success metric summarize CROP and Self-Refine even though both revise text from feedback?¶

Show answer and example

Answer: Self-Refine mainly revises the current answer. CROP changes system prompts offline and selects evaluated candidates using accuracy and CoT-normalized output-token cost. A better current answer does not demonstrate prompt transfer; shorter prompts do not ensure shorter or correct answers. Real LLM callbacks and independent validation are additional evidence.

Illustrative example: Correcting one arithmetic answer and finding a concise prompt for unseen arithmetic are different tests. The last newly written prompt cannot be called best before evaluation.

Sources: CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization · Self-Refine

Why are neither high EvalAlloc predicted fitness nor high RuHarness criterion utility actual agent success rates?¶

Show answer and example

Answer: EvalAlloc estimates unobserved SEARCH outcomes against fixed references, retaining observed cells and requiring confirmation. RuHarness estimates conservative diagnostic value of criteria, and candidate harnesses still need independent validation. These govern budgets and criteria, not outcomes of unexecuted tasks.

Illustrative example: If only 20 of 300 items are evaluated, completed mean 0.8 does not mean 240 actual completions. A criterion that covers failures does not solve tasks.

Sources: Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution? · When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

How does Strategist retaining failed strategies differ from Reflexion storing lessons and EvalAlloc confirming champions?¶

Show answer and example

Answer: Library admission preserves executable structures and measurement cards without adopting them for the current answer. Final selection still requires both splits to win. Reflexion mainly stores readable experience, whereas EvalAlloc controls observations and confirmation rather than generating structure. Structural archives, lessons, and adopted strategies need separate records.

Illustrative example: Teaching example: an expensive voting strategy may remain a component for a future decomposition program, without replacing the current cheaper reference merely because it entered the library.

Sources: Strategist: Evolving Reasoning with Agents · Reflexion · Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?

Update log¶

Updates record new approaches, changed trade-offs, and follow-up evidence.

2026-11-04 · Reviewed Strategist against Reflexion and EvalAlloc to separate cross-task executable libraries, readable lessons, and evaluation budgets. Added explicit prompt boundaries for AST deduplication, validation/hidden gates, and the two-percentage-point/2%-cost bands. CPU synthetic selection kernels are not real search; the original eight-H200 matrix is not reproduced because of high cost, review DONE.

Affected: Strategist: Evolving Reasoning with Agents · Reflexion · Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution?

2026-10-30 · Reviewed CROP and EvalAlloc against Self-Refine and RuHarness, separating system-prompt changes, rubric utility, and SEARCH observation budgets. Added validation and CoT cost, observation-first completion, common-prefix confirmation, and limits of unimplemented MIRT and controller components.

Affected: CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization · Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution? · Self-Refine · When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

2026-10-29 · RuHarness does not rank its criterion frontier by one accuracy value. A 2×3 behavior/outcome table yields a Jeffreys–Dirichlet posterior for a conservative utility quantile, while set novelty and redundancy stay fixed from observed traces. This separates finite-sample uncertainty from candidate admission: only active criteria enter the frontier and final harness edits require strictly positive independent validation gains. Local work has no LLM judge or harness generation.

Affected: When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

2026-10-27 · Initial version: compared current revision, cross-trial reflection, local validation, nongreedy interface populations, layered intervention, and guarded production loops.

Covers 43 topic entries; see the full list below.

All 47 topic entries

The full catalog follows. The main comparison covers the representative papers listed above.

Paper-by-paper questions · All deep dives