Agent research by topic

简体中文 · English

Browse two levels: research area, then comparable method family. Each paper has its own entry. Open its write-up for experimental results and reproduction limits.

Other

Agent data quality

• What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents (ace-data): The paper represents agent data as \((E,q,\tau,v)\): environment, task, interaction trajectory and optional verifier. Accuracy defines trustworthy support, Complexity calibrates difficulty relative to the learner, and Diversity controls environment, task and behavior coverage.

Agentic RL

• When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation (race): RL masks removable reasoning and supervises early closure. Reasoning tokens decrease by 32.3%, 80.2%, 46.4%, and 62.0% in the paper's four agent environments; no online A/B test is reported.
• AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning (adastep): AdaStep uses no additional critic; it estimates the share of return variance explained by actions. When within-action noise dominates, downweight the local correction and keep global credit.
• Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents (depgpo): A terminal verifier rewards completed trajectories, while GRPO gives the same advantage to every generated token. DepGPO traces reads back to non-overwritten writes and builds edges for trusted stdout reuse, then walks backward from the files, ports, and processes actually inspected by the verifier.
• GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents (graphhca): GraphHCA merges rollouts into a state-transition graph and solves a discounted fixed point with success/failure terminal boundaries. Differences in log potential give step credit, normalized among actions from the same state.
• SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning (sapo): One autoregressive backbone outputs policy and value at different causal boundaries, combining PPO, on-policy SARSA, and trajectory GAE.
• RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training (rtpo): Organize multi-turn rollouts as sparse backward trees. Update turns in reverse temporal order so decisions remain on-policy with downstream continuations.
• SPADE: Self-Play in Adaptive Synthetic Executable Environments (spade): The same LLM plays environment designer and reasoning agent. Regret between runs with and without privileged hints trains it to produce executable environments at the capability boundary.
• PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs (planpo): Within successful trajectories, compare both the number of turns and the length of individual responses to build coarse-to-fine planning advantages.
• TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents (trca): Without successful anchors, derive base and breakthrough rewards for each transition from evidence, execution and invalidity rubrics.
• Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks (dmast): Unlike plain GRPO, attacked groups pool raw terminal rewards from both roles to compute mean and variance. The successful-sample length penalty appears only in the advantage numerator. Bounded component only; full experiments are not reproduced.

Self-improvement and experience reuse

• Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses (harmony-harness): Harmony Harness keeps functionality and safety as separate objectives: each preference vector has a harness, and a candidate replaces only worse neighbors. Tools, skills, and logic are stored in general or environment-specific scopes; a composer selects an execution DAG for the current task. Not reproduced due to cost (review complete); research material only, no executable entry.
• SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting (seer-forecasting): Update the two memories and knowledge separately from forecast trajectories only after outcomes become observable. Compress them at capacity instead of appending indefinitely. Not reproduced due to cost (review complete); research material only, no executable entry.
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents (agentdebugx): Repair generates suggestions only. A human or policy gate must authorize an actual rerun; imported logs do not trigger arbitrary external tool replay. Not reproduced due to cost (review complete); research material only, no executable entry.
• MARS: Modular Agent with Reflective Search for Automated AI Research (mars-research): Budget-aware MCTS chooses whether to create, improve, or debug a solution. Comparing old and new execution results and code diffs yields solution and debugging lessons reusable by other branches. Not reproduced due to cost (review complete); research material only, no executable entry.

Unified evaluation and reliability

• AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers (agentheal): It distinguishes an exact copy with the same work and version, another version of the same work, and a merely superficial look-alike. The controlled sandbox contains 675 instances: 225 goals each instantiated as barrier-free, recoverable, and unrecoverable. Bounded component only; full experiments are not reproduced.

Research automation

• Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems (agentx-model-replay): A Model Agent iteratively edits, trains, and measures a fixed business prediction task, returning the best intermediate implementation, negative results, and unresolved questions. The roles exchange proposals, not full work logs.
• AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems (agentx): Evaluation uses guarded online A/B with vetoes. SGPO updates the harness from successful and failed trajectories.

Automated research

• AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents (airs-bench): The protocol distinguishes the agent, search scaffold, and execution harness and compares sequential MLGym with tree-search AIRA-dojo. See Sections 3–5. Not reproduced due to cost (review complete); research material only, no executable entry.

Multi-agent collaboration

• World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds (world-of-swecraft): SW-Env uses Harbor sessions and Forgejo to simulate software maintenance, with independent controls for communication permissions, repository ownership, and session budgets. SW-Eval installs frozen versions in hidden downstream packages and checks correctness and instruction counts. Not reproduced due to cost (review complete); research material only, no executable entry.
• ARIA: Runtime Inspection and Amendment for Multi-Agent Systems (aria): Section 2 and Appendix B describe structure SFT, repair DPO, diagnosis transfer, repair GRPO and gate GRPO. The local component extracts a testable patch contract; it does not substitute heuristics for those learning stages. Bounded component only; full experiments are not reproduced.
• Formalizing Mathematics at Scale (autoformbot): A supervisor checks objectives after merges and feeds findings back into planning. See Sections 3–4. Not reproduced due to cost (review complete); research material only, no executable entry. Method reference 1
• PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing (paperorchestra): A multidimensional assessment accepts or reverts revisions. The system does not invent or execute new experiments for authors. Not reproduced due to cost (review complete); research material only, no executable entry.

Coding Agent

• AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents (autocompact): A judge assesses when to compact, what the working summary should say, and what the next action should be. The corrected output is executed directly in the environment, then learned with SFT and outcome RL.

Long-horizon agents

• Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States (belief-state-pos): Explicitly maintain current world facts and unresolved task requirements as a belief. Repeated actions that do not reduce unresolved requirements indicate Belief Trapping and trigger targeted recovery.

Agent safety

• One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails (option-channel-attack): With action and trusted policy fixed, replacing only the allow label can convert a correct denial into approval. Evaluate dangerous fail-open separately from fail-closed usability failures.
• OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport (ontrack): OnTrack forms a dependency graph from events and aligns the current execution prefix to successful reference trajectories with unbalanced, structure-aware optimal transport. Without references it explicitly falls back to loop, retry and information-stagnation detection.
• SpecGuard: Proving a Task Is Broken Before the Agent Cheats (specguard): The result may be inconclusive when faithful formalization or alignment is not possible. See Sections 3–4. Bounded component only; full experiments are not reproduced.
• PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents (pace-capability): Before every tool side effect, check it against authority compiled from the authenticated request and also check input provenance. Content accepted at ingestion is not necessarily trustworthy at execution.
• CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution (causal-armor): Low user attribution alone is not attack detection; LOO is an operational ablation, not a proof of safety. Sections 3–4 and Algorithm 1. Bounded component only; full experiments are not reproduced.

Agent evaluation

• UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training (user-proxy-bench): The audit focuses on premature disclosure, when the simulated user reveals private fields before the agent requests them. That can leave task success unchanged while letting agents skip the information gathering the benchmark is intended to test.
• Certified Selective Automation of LLM Agent Evaluation (certified-selective-eval): Bootstrap by task cluster rather than assuming trajectories are independent. Form an upper confidence bound on error within the automated region and take over automatically only when the certificate falls below budget.
• Verifiable Social Reasoning for LLM Assistants (fuse-evaluator): Multi-agent simulations with verifiable hidden motivations evaluate user retelling, framing bias, and social reasoning.
• TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems (tfrbench): Labels participate in selection and feedback, so construction results cannot serve as held-out capability scores. See Algorithm 1. Evaluation protocol interface; not a model capability result.
• ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders (convapparel): PLSA stands for Population-Level Statistical Alignment, not the topic model with the same abbreviation. See paper §4. Evaluation protocol interface; not a model capability result.

Skill evolution

• COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization (cobra-skills): Treat skill optimization as a budgeted contextual bandit over a dynamic candidate space. Evaluate high-reward or informative skills first and evolve them using execution feedback.

Agent evaluation and self-improvement

• HarnessMM: Recursive Self-Improvement for Harness Token Efficiency (harnessmm): Audit actual execution and a reduction of at least one slow call at the registered parent boundary. Closure alone cannot establish semantic correctness; the benchmark scorer remains necessary. Not reproduced due to cost (review complete); research material only, no executable entry.
• ImageRSI: Verification-Driven Recursive Self-Improvement for Agentic Image Generation (image-rsi): Compare with/without experience on the same request, input, generation settings and seed using a blinded independent evaluator, then test separately on a holdout. Train the judge offline between collection rounds, not within the same evaluation request or from external test verdicts. Not reproduced due to cost (review complete); research material only, no executable entry.
• RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training (rsi-opt): Inner agents cannot see final metrics, but outer scoring uses them; a suite's split named test is not proof of unseen-task transfer. Base LLM weights stay fixed. Not reproduced due to cost (review complete); research material only, no executable entry.
• CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation (coreflect): Planner, simulator, evaluator, and analyzer are core semantic models. A JSON state machine, arithmetic means, or manual rubric edits do not reproduce them, so no cheap heuristic adapter is offered. Not reproduced due to cost (review complete); research material only, no executable entry.

Paper-to-code state subplanning

• DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories (deeprepro): A one-shot global plan becomes stale as files, dependencies and interfaces change. At every stage, DeepRepro reads the current repository and runtime feedback, rewrites the detailed subplan, and advances implementation through repository-aware orchestration.

Failure attribution

• DeFA: Dependency-Guided Failure Attribution for LLM Agents (defa): Fuse protocol relations and semantic dependencies into an event graph. Trace backward from a violation to possible sources, retain downstream consequences in a failure-propagation graph, and select the decisive error.

Trajectory refinement

• Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning (dependency-refinement): Represent multi-turn trajectories as a turn-level dependency DAG. Prune leaves, merge under strict rules, then merge under looser rules to remove redundant messages without deleting final-answer dependencies.

Agent memory

• Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems (discernmem): Only harmful in-scope effects supported by a concrete trace defect justify content repair; existing avoid conditions take precedence. Evaluate the original task before updating memory, and do not retroactively replace its score with the replay outcome. Bounded component only; full experiments are not reproduced.

Tool use and reliability

• Diviner: Agentic RCA with Human-Governed Adaptation for Large-Scale Infrastructure (diviner): A timeout does not mean there is no anomaly; a similar historical incident does not establish the current root cause; and a postmortem is not evidence available during the incident. §§4–5 define an Encode/Execute/Evaluate/Adapt loop for domain knowledge, lead–worker investigation, postmortem review, and owner-approved knowledge updates. Runtime adaptation can revise only the current investigation plan, not publish shared workflows or execute mitigation. Bounded component only; full experiments are not reproduced.
• JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs (jev-agent-harness): It reports success, cost, and critical-path latency. AgentDecision-4B also trains on reached states after errors, rather than imitating only successful trajectories. §§2–4 and Appendices A/D Bounded component only; full experiments are not reproduced.

Planning and search

• Repairing LLM-Generated Specifications with Deterministic Diagnosis (drago): Reference formalizations and final labels are used only for final scoring. See Section 3 and Appendix A.2. Bounded component only; full experiments are not reproduced.

Coding agents

• Beyond Vector Similarity: Hierarchical Context-Aware Graph RAG vs Standard RAG in Enterprise Code Migration (hcrg): The vector-RAG control uses 800-character chunks, 150 overlap, top five. See Sections 3–4. Bounded component only; full experiments are not reproduced.
• Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution (pmcoder): Condition episodic retrieval on a hierarchical phase planner. Statistics from remembered trajectories detect stalls and trigger replanning, with real execution verdicts for verification.
• Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair (dynamic-brt): A generated fix agreeing with its own test is not proof of correctness. Section 3. Bounded component only; full experiments are not reproduced.

Harness self-improvement

• Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents (ecdysis): Aggregate recurring failures across tasks to distinguish one-off model mistakes from systematic harness defects. Multi-role FDCR diagnosis turns those patterns into repair specifications.

Program evolution

• EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution (evoalloc): The paper reports 59–82% fewer Full evaluations to reach baseline and 8.7–12.0% better results at the same budget. No production A/B is reported.
• FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution (frugalevo): FrugalEvo separates costly strategy proposals from cheaper execution and code refinement, then measures search efficiency over the entire best-score trajectory under the budget. Try and rank several strategies; retain the parent and add feedback after failures; reanchor after improvements; keep diverse programs in an island archive.

GUI Agent

• Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents (evoskill-gui): After execution fails, a critic with isolated information reflects on the failure and revises only the responsible component. Verified skills can be reused on other tasks.

Evaluation and verifiable agents

• FinanceHarness: Autonomous Financial Deep Research Framework (finance-harness-protocol): The agent sees only the question and cutoff. The thesis, scoring rubric, and future evidence stay with the post-submission evaluator. Bounded component only; full experiments are not reproduced.
• LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning (lifeagentbench): LifeAgent is a separate tool-reasoning baseline: decompose questions, maintain an evidence cache and intermediate statistics, revise plans based on new evidence, and synthesize answers. A benchmark scorer is not this agent. Not reproduced due to cost (review complete); research material only, no executable entry.

Workflow evolution

• It Takes Workflows to Evolve Better Workflows (flowright): Use workflow topology to turn sparse outcomes into hierarchical, structure-aware role credit. A single harness can then support single-role self-improvement, upstream/downstream collaboration or multi-agent co-evolution.

Agent memory

• Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents (mem-plus-plus): Keep full documents, dates, and authors; do not irreversibly summarize at write time. At retrieval, filter by the question's as-of date before fusing lexical and semantic scores.
• MemFit: Efficient Long-Term Agentic Memory (memfit): Append original turns without rewriting and use segment summaries only to build the index. Reads combine lexical, semantic, and reranking signals, avoiding expensive LLM writes.
• UpliftMem: Learning Set-Level Uplift for Agent Memory Retrieval (upliftmem): Learn complementarity, redundancy, and conflict at set level. When exhaustive rollouts are impractical, use expected value of sample information (EVSI) under a correlated Gaussian posterior to probe the sets most likely to change the local retrieval decision.
• Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents (grounded-memory): Give the asynchronous memory curator least-privilege, read-only tools to verify, scope, and refresh candidate memories before writing.

Programming agents

• An Empirical Study of Harness Design for Coding Agents (harness-design-study): Hold the execution loop fixed and vary planning, action space and context management independently to isolate which coding-agent harness factors affect outcomes and costs.
• RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Views (repoatlas): Under a fixed budget, select a relevant code subgraph, project it into a visual topology and exact text index, and refresh both views as the task evolves.

Harness learning

• Harness Learning Enables Generalizable Test-Time Adaptation (harness-learning): A proposer uses actual execution feedback to revise validation, interpretation, retrieval, or retries. The task model remains frozen and the loop retains better scaffolds after several rounds.

Planning, search, and reflection

• Harness²: Recursive Agent Harnessing for an Open World (harness2): A score is not treated as proof of correctness. Section 2 Bounded component only; full experiments are not reproduced.

Memory

• HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation (hymem): Separate planning, execution, and isolated reasoning into layers. Structured summaries preserve task progress across context refreshes.

Tool planning

• ToolLIFT: Lifting Tool-Specific Trajectories into Function-Level Graphs for Generalizable Tool Planning (toollift): Lift tool-specific histories into a transferable function workflow graph and separate workflow planning from concrete tool selection.
• HyperAgent: Planning and Acting over Tool-Schema Hypergraphs for Tool-Use LLM Agents (hyperagent): Represent tools as input-schema→output-schema hyperedges. Construct a task DAG, then expand producer support for missing state.

Jev

• JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces (jev-spawn): Build a finite compositional action space from the natural-language task. Use a Jev distribution to keep multiple branches in parallel, then update the posterior from feedback and retain alternatives to recover from failure.

Reflection

• LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation (loongreflect): Treat reflect and backtrack as memory-control actions on a reversible trajectory tree. Coordinate fast privileged-teacher supervision with slower outcome GRPO.

World models

• Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks (memento3): Models share history, and complete plans are sampled uniformly across distinct equivalence classes, not by counting duplicate models as votes. See Section 2.

Memory and long-term interaction

• Remembering Has a Price: Learning Memory Worth under Resource Constraints (tokenworth): Fast updates fit residuals and slow updates adapt the diagonal metric. One score drives retrieval, admission and eviction. Bounded component only; full experiments are not reproduced.
• MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents (memprice): Details deleted from the current window cannot reappear on their own; they can only be re-extracted from the external raw archive at refresh time under a fixed procedure. See Section 3 and Appendix G. Bounded component only; full experiments are not reproduced.
• MemoProAgent: Toward Memory-Driven Proactive Assistance (proper-agent): Training applies policy gradients to generated tokens only, not tool-injected evidence. See Section 5 and Appendix I. Bounded component only; full experiments are not reproduced.

Self-improvement

• OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks (openloopevolve): Version observation, planning, memory, action, verification and recovery assets in the loop policy. Govern changes by Champion–Challenger comparison, release monitoring and rollback.

Public evaluation protocols

• PLE: Physical-Engineering's Last Exam (physical-engineering-exam): Judges and references stay hidden. The local path freezes artifact bytes and checks provenance before accepting external grades. Bounded component only; full experiments are not reproduced.

Agent self-improvement

• POLCA: Stochastic Generative Optimization with LLM (polca): Farthest-first filtering recomputes remaining distances after adding each item. Section 3. Bounded component only; full experiments are not reproduced.

Training service optimization

• PROMPTS: Performance Optimization via Multi-Agent Planning for LLM Training and Serving (prompts): Coordinator, Analyzer and Proposal Agent jointly consult profiler and knowledge-base evidence, diagnose bottlenecks and propose explainable sharding changes.

Red-team skill evolution

• RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution (redevoagent): Distill cross-case experience into readable skills rather than retrieving long attack traces. Attribute only decisive tools and promote skills only if held-out validation beats the incumbent.

RSI safety

• Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover (safe-self-improvement): Revalidate all candidates in current conditions. If none passes, roll back to the founder instead of keeping an unsafe incumbent.

Planning

• SAGE: Symbolic Action-Gating and Editing for LLM Task Planners (sage-planner): A symbolic precondition gate blocks unsafe actions before execution and returns a typed reason. On failure, rewrite only the suffix for the relevant subgoal and retain the completed prefix.

Scientific agents

• ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents (sciencebuddy): The inner recursion evolves the interaction harness. The outer recursion updates the scientific agent's policy from successful trajectories.

Scientific research agents

• ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence (scientist-one): CoE links claims to verifiable sources. Four independent checks, I1–I4, audit scores, rules, citations, and method–code consistency. A source's existence does not establish support for a claim. See §§3 and 5. Evaluation protocol interface; not a model capability result.

Planning, reasoning, and tool use

• SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning (scope-replay): When candidate source changes, it selects up to three historical witnesses for local stateful replay; only a divergent instance may be scheduled for a sparse real-environment diagnostic. See Section 3. Bounded component only; full experiments are not reproduced.

Search agents

• SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs (searchatlas): Turn search trajectories into directed query–evidence–answer graphs, then audit whether evidence covers the question constraints and final answer.

Failure recovery

• Sentry: Learning to Recover from LLM Agent Failures at Test Time (sentry): An external verifier observes subsequent progress without task reward, and writes to the playbook only after confirmed recovery. Invalid action formats use a separate hard-repair path.

Agent data synthesis

• Reasoning-Driven Synthetic Data Generation and Evaluation (simula): The semantic generator and critics determine data quality and cannot be replaced by heuristics in a reproduction claim. Section 2 and Appendices B/C/E. Evaluation protocol interface; not a model capability result.

Agentic RL / efficient long context

• Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks (sinkflex-rl): SinkFlex-RL combines a Gymnasium dual-control environment, VERL-style data flow, GRPO without a value model, and sink-aware FlexAttention. Causal and sliding-window masks retain the model-specific sink scaling.

Skill retrieval

• When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents (skill-retention): Combine real-example replay with embedding anchoring/LwF-style regularization so synthetic skill tuning preserves real and OOD routing.

Tool use and environments

• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints (sopbench-protocol): It checks the final database, whether service was permitted, and whether required verification occurred in order; guessing the final answer cannot bypass the procedure. Constraints map to helpers and can be nested: Single, And (unordered children), Or (a verified true branch) and Chain (strict order). Bounded component only; full experiments are not reproduced.

Tool use

• Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation (spec-driven-tests): It records tested/untested conditions in .spec.md before generating tests. A developer may revise the specification, but the paper's automated comparison did not use human editing. Bounded component only; full experiments are not reproduced.

Harnesses and self-improvement

• Strategist: Evolving Reasoning with Agents (strategist): Each iteration performs Retrieve, Propose, Evaluate, and Accumulate. Relevance and adjacent-inspiration queries supply material for new/composed/reuse candidates. Bounded component only; full experiments are not reproduced.

Agent verification

• VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks (veriharness): Give one base model a workspace, evidence tools, and reusable verification skills. A disagreement resolver investigates conflicting claims; a consensus challenger questions shared claims and missing requirements.

Planning, search, and reflection

Harnesses and self-improvement

• When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement (ruharness): Small samples can inflate point estimates, so candidate criteria are ranked by a conservative posterior utility. Harness updates still require independent validation; a rubric is not the task objective. Bounded component only; full experiments are not reproduced.
• Where Should the Evaluation Budget Go in LLM-Driven Agent Scaffold Evolution? (evalalloc): Observed scores are then combined with predictions for unobserved cells. Final confirmation expands a common random item prefix in rounds and reuses cached observations. Bounded component only; full experiments are not reproduced.
• EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis (emma): A one-off execution slip gets a temporary plan directive. Reusable behavior belongs in an agent prompt, a missing procedure in a skill, and a decomposition or dispatch error in the controller. Bounded component only; full experiments are not reproduced.
• Evolving Agent Interfaces (agent-s4): Hold the task actor and harness fixed while the meta-agent modifies how it can observe and act, not merely composing tools. The builder sees unlabeled training traces only; validation traces, tasks, diagnostics and scores are withheld and used for parent selection. Bounded component only; full experiments are not reproduced.
• Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design (oo-harness): Improve on discovery clusters, then freeze classes and experience and route from task and observable base trajectory. Ancestor methods run in one instance, not extra agents. Bounded component only; full experiments are not reproduced.
• Athanor: An Agent-Native System for Autonomous Post-Training Research (athanor): The team shares experiment records and models and combines SFT, RL, model merging, and self-distillation. Calls to external teachers are prohibited. Not reproduced due to cost (review complete); research material only, no executable entry.
• AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning (auto-sft): Interventions can rebalance categories, delete or repair data, replace answers, synthesize targeted samples, or change ordering. These six types are a retrospective classification, not a fixed action menu. Not reproduced due to cost (review complete); research material only, no executable entry.
• RSIFlow: Adaptive Self-Improvement Workflows with an Evolving Meta-Agent (rsiflow): Meta-model parameters remain fixed. The local component follows the released skill-memory variant and does not turn it into a system for arbitrary code self-modification. Bounded component only; full experiments are not reproduced.
• Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine (terminal-rsi): Terminal-RSI alternates RSR trajectory rewriting with RPR that retains critic weights, calibrating the critic at the start of each stage. It proposes TB-GAE, which discounts on an interaction-turn clock rather than by observation text length, and uses family-calibrated FDR before switching to strictly completion-first SPR. Bounded component only; full experiments are not reproduced.
• The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock (torsi): After it is frozen, only independent validation decides whether the new guidance replaces the old. Test data never participates in revision or selection. Evaluation protocol interface; not a model capability result. Method reference 1
• RSI-Anything: Unlocking Recursive Self-Improvement with World Models (rsi-anything): Only versions that pass both component and pairwise compatibility checks advance. See the full paper.
• Recursive Self-Improvement through Multi-Agent Self-Supervision (mass): The paper reports about 1.2–1.6× performance per output token on research tasks after two rounds with Qwen3.6-27B. No online A/B test is reported.
• ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization (active-saddler): Recurring failure patterns become dynamic, nonstationary bandit arms. A curriculum controller discovers new scenarios or revisits known weaknesses, updating priorities as harness repairs succeed or fail.
• Self-Evolving Coding Rules for AI Coding Agents (rule-evolve): Maintain a pool of coding rules, generate variants with a mutator, then select and persist better rules using only an isolated validation evaluator.
• Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI (meta-skills): Distill meta-skills with when, provide and use fields from harness feedback on development tasks, then freeze the skill bank. Test tasks retrieve relevant meta-skills to build a new harness without writing test results back.
• Mixture of Self-Improving Branches for Agent Harness Optimization (branch-mixture): Remove examples every branch solves and retain examples that distinguish branches. At deployment an input-only router selects a development-set champion.
• Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution (video-rsi): Revisit training videos to collect additional evidence, test diagnoses and revise the harness. Accept only a Pareto improvement in accuracy or visual cost.
• MAPLE: Memory-Augmented Planning with Language and Evolution (maple): Represent optimization knowledge as editable program state. Propose changes, check executability, accept improvements, and carry the versioned program into later tasks rather than prompting from scratch.
• Procedural Graphs: Self-Evolving Execution Structures for LLM Agents (procedural-graphs): Turn new experiences into candidate local edits, then admit them to long-term structure only after held-out verification. This reduces contamination from bad experiences.
• Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification (harnesslens): A fixed validation set spends rollouts on unaffected behaviors and can average away regressions. Propose changes from execution traces, run paired checks only on tasks tied to affected behavior, and require evidence attributable to the change.
• JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution (jit-agent): The harness is formalized as four generatable modules: memory, planning, action protocol, and tools/skills. The model generates a harness for a task, repairs it after failure, and distills transferable patterns from an archive of earlier configurations.
• CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization (crop): This is not hard truncation, a length reward added to GRPO, or a numerical gradient of the prompt text. Here a text gradient is a critique produced by an external evaluator. Bounded component only; full experiments are not reproduced.
• HyperAgents (hyperagents): Put the task agent and meta-agent in one editable program. Accumulate variants in an open archive; favor strong parents and those with fewer descendants, allowing the improvement mechanism itself to evolve.

Scientific discovery and experimental loops

• Aptamers as a sandbox for lab-in-the-loop agentic molecular discovery (aptamer-lab-loop): The paper shows disagreement between some predictive proxies and measured outcomes. This note discusses research orchestration and evidence limits only; it does not provide laboratory procedures. Not reproduced due to cost (review complete); research material only, no executable entry.

Environment simulation and lookahead decisions

• Janus: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models (janus): After running the selected experiment, the system abstracts within-task and cross-task skills from successes and failures and updates its prediction, exploration, and coordination procedures. See the full paper.
• Code Owns the Simulation, Jev Owns the Evaluation (jev-lookahead): Executable code simulates each action's successor state; Jev only compares the progress afforded by actions. In addition to the action and next observation, the evaluator receives newly available actions after execution, addressing the difficulty of inferring long-term reachability from semantic action scores alone.

Tree search and self-improvement

• Superintelligent Retrieval Agent: The Next Frontier of Agentic Retrieval (sira): A frozen LLM proposes evidence terms missing from the original query. Filter absent or overly common terms by inverted-index document frequency; combine original and verified expansion in one weighted BM25 pass.
• LATS (lats): Use an LM as agent, value function and optimizer within MCTS. External execution supplies rewards; reflections on failed traces guide later exploration and backtracking.
• Tree of Thoughts (tree-of-thoughts): Treat intermediate thoughts as independently evaluable units. Expand multiple candidates, score the frontier with an LM value function, and search BFS or DFS with lookahead and backtracking instead of committing to one chain.
• Self-Refine (self-refine): One LLM writes a draft, produces actionable task-specific feedback, and revises until feedback says the constraints are met. No weight updates, extra training data, human feedback or external reward model.
• Reflexion (reflexion): Reflexion expands sparse scalar or binary feedback into actionable natural-language lessons stored in episodic memory. The Actor reads reflections on the next trial, and the Evaluator continues to judge success or failure.

Meta-reasoning and budget control

• Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning (meta-reasoning): Workers handle object-level tasks, while a controller splits control into Assess, Propose, Evaluate, and Dispatch. The controller carries compact state, reuses work via persistent artifact memory, and decides whether to continue, branch, or stop within a shared call budget.

Interleaved reasoning and task decomposition

• ReWOO (rewoo): Instead of reprompting a ReAct planner after every tool response, write a full plan with variable references. A Worker populates tool evidence and a Solver integrates it once at the end.
• ReAct (react): ReAct alternates natural-language reasoning with environment actions and adds each observation to the next context. Reasoning can correct errors and actions can obtain external facts.

Tool calling and environment execution

Tool selection, feedback, and program execution

• Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning (rpl): When a mutated program passes execution verification, the policy generates a continuation with its evidence, and distillation is interleaved with RL. See Section 3 and Appendix B. Bounded component only; full experiments are not reproduced.
• SSR: Structural Signal Retriever for Scalable Agent Discovery in LLM-Orchestrated Multi-Agent Systems (structural-signal-retriever): It extracts interpretable lexical signals from explicit capability fields, then uses an inverted index to narrow the candidate pool before selection. “Structural” refers to the fields name, description, skills, tags, examples, and tools, not to syntactic graphs or latent vectors. Its core lexical mechanism is a specific integration of existing IDF retrieval, not a new semantic model. Bounded component only; full experiments are not reproduced.
• TRACER: Trajectory-Aware Tool Retrieval for LLM Agents (tracer): Two five-dimensional linear heads learn a state-dependent weight between the empirical and contextual priors and a temperature. For unseen prefixes or low confidence, retrieval falls back to vector search across the full registry. Bounded component only; full experiments are not reproduced.
• RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics (rated): Execution successes, errors, or timeouts update only the actually selected arm. See the full paper.
• Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks (feedback-scaffold): When the agent is less capable, action guidance reduces the search burden. Later, observation enrichment lets the agent choose actions itself rather than relying indefinitely on demonstrations.
• AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research (adavdr): AdaVDR first filters tool calls that are unnecessary for the target model's capabilities and falls back to reflection only when intermediate evidence is unreliable. After SFT, a redundancy-aware RL reward balances video understanding, external retrieval, and call cost.
• SMITH: Self-Improving Tool-Using Agents through Multi-Aspect Verification (smith): SMITH mixes build tasks that write schemas/backends from examples with held-out tasks that use the tool pool. It rewards schema validity, code execution and final outcomes separately, with harder questions encouraging reusable abstractions.
• ToolGrad: Efficient Tool-Use Dataset Generation with Textual Gradients (toolgrad): Work backward from a target answer to propose tool traces, then use textual gradients to locate and revise failed calls, reducing manual annotation.
• ToolRL (toolrl): Jointly optimize tool selection, arguments, and execution results. Dynamic reward scaling mixes tools of different difficulty within an RL batch.
• ReTool (retool): Alternate natural-language reasoning and executable tool actions. Use execution feedback to learn tool invocation, correction and stopping.
• ToolBench (toolbench): After analyzing tool failures in open-source LLMs, the method combines programmatic usage examples, a system prompt, retrieval of in-context demonstrations, and output-format constraints.
• CRITIC (critic): Pure reflection can repeat a model's original mistake. CRITIC adds observable evidence from external tools, such as search and code execution, to each critique-and-revision iteration.
• ART (art): A frozen LLM produces a program; generation pauses at tool markers and resumes after injecting results. Humans can expand the library with a few corrections.
• Toolformer (toolformer): Sample candidate API calls from few-shot examples. Compare downstream token losses for real tool results, masked results and no call, and keep only helpful calls for continued training.
• PAL (pal): LLMs can decompose a task but make arithmetic or symbolic execution mistakes. PAL synthesizes a program with variables and control flow, then uses a deterministic runtime such as Python for exact calculation.

Public protocols and state audits

• AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations (agent-perturb): Valid JSON is not enough to accept a candidate. A semantic verifier independent of the generator must confirm all four conditions: the edit affects only the specified target, preserves the objective, remains solvable, and is executable. Evaluation protocol interface; not a model capability result.
• GACC-Bench: Benchmarking Goal-Adaptive Context Compression Ability across Large Language Models (gacc-bench): A fixed reader answers each question from the same compressed memory, then a fixed judge scores against the original source and references. This tests whether compression preserves evidence for the entire goal. Evaluation protocol interface; not a model capability result.
• Harness-Delta Attribution: Dissecting Gains From Harness Evolution (harness-decomposition): If the gain is not significant, attribution stops. Otherwise, construct a compute-matched baseline and an evolved variant with detected shortcuts removed. Both controls need task-specific human review. See Section 3. Evaluation protocol interface; not a model capability result.
• Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries (librarian-agents): CreateFile, MergeFiles, Delete, and Stop form a finite-horizon MDP. Delete can never remove raw documents. Evaluation protocol interface; not a model capability result.
• PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents (probe): Rewards combine violations, credible customer behavior, tool-workflow progress, and policy diversity. The paper compares GRPO, DPO, and GFlowNet. Bounded component only; full experiments are not reproduced.
• SciPIvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents (scipivot): Reference construction can use subsequent records. The evaluated model may see at most the project state before correction, never future traces, correct labels, or the origins of the options. Evaluation protocol interface; not a model capability result.
• Shortcut Sensitivity in Agent-Safety Evaluations (shortcut-sensitivity): The paper uses a three-level inference ladder: first, whether cues predict labels; second, whether cues correlate with guard decisions; and third, whether a paired intervention that changes only the cue changes those decisions. Each level only motivates the next test; it does not establish that the model used a shortcut. Bounded component only; full experiments are not reproduced.
• ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets (tool-bottleneck-bench): The original protocol has 36 task families and 12 enterprise intents. Routed menus use ground truth to narrow candidates, so they are not evidence of an oracle-free capability. Evaluation protocol interface; not a model capability result.
• AlphaDiana: A Measurement System for Agents in Verifiable Reasoning Tasks (alpha-diana): Reporting only successful tasks or mean tokens can hide missing grades, execution failures, and retry costs. AlphaDiana compares complete model–harness systems under common tasks, sampling, and resource policies while retaining each harness's native execution. §2 and Appendices A–B Evaluation protocol interface; not a model capability result.
• Evolving Together, Failing Apart: The Skill-Policy Co-evolution Trap in Self-Improving Agents (coda): Sections 3–4 and Appendix E retrieve once from the original bank and freeze selected IDs and retrieval-side fields. Only content is replaced; retrieving again would introduce a confound. Evaluation protocol interface; not a model capability result.
• What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports (ctxgap): The main sample has 37 tasks, 148 checkpoints, and 4,855 probes, of which 4,799 are valid. Released text probes and execution materials are only a subset. Evaluation protocol interface; not a model capability result.
• CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline (clasharness): The eight-stage pipeline builds a shared world, role rules and multi-turn scenarios. Each round fixes ground truth \(v^\star\), constructs and executes a reference plan in isolation, compiles deterministic and semantic checks from trajectory effects, then checks consistency, visuals and rendering. Evaluation protocol interface; not a model capability result.

Formal mathematics and independent verification

• AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics (autoformbench): There are two distinct settings. The paper leaderboard provides agents with the informal source text (source-present). Evaluation protocol interface; not a model capability result.

Expert routing, embodied systems, and browsing

• Disagreement-Guided Weak-to-Strong Routing (disagreement-routing): At test time it uses weak-model representations and two lightweight probes. The strong model is called only when the router escalates. Bounded component only; full experiments are not reproduced.
• LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents (leash): Initial and runtime gates decide handoff. Afterward, it periodically uses representations of unexecuted weak-model Shadow actions to monitor the stronger model and asks it to reconsider when needed. Bounded component only; full experiments are not reproduced.
• HuggingGPT (hugginggpt): ChatGPT decomposes requests into a dependency graph, selects experts by their Hugging Face descriptions, executes in topological order, and synthesizes outputs.
• MRKL (mrkl): Discrete modules provide deterministic computation; the LM handles language understanding and the interface. The contribution is routing and modular boundaries, not a new LM loss.
• SayCan (saycan): Score each pretrained skill by language relevance and value-function affordance. Execute the best product and append the action to context before replanning.
• WebGPT (webgpt): WebGPT searches, clicks, and scrolls in a text browser and collects citations for answers. It first trains with behavior cloning, then uses a human-preference reward model to reject-sample among browser and answer trajectories.

Computer use and reward evaluation

• When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows (jev-workflow): Recovery failures, premature completion, and resources spent on unsuccessful attempts all affect the resources per delivered success; a faster individual decision is not enough. See the full paper.
• HarnessOpt-Bench: Evaluating LLMs at Harness Optimization (harnessopt-bench): Topic: harness optimization evaluation. Under a fixed target-evaluation budget, optimizers modify prompts, tools, control flow, and memory. Hidden tests and a trusted runtime isolate search feedback; candidate versions remain auditable.
• OSReward / OS-Shepherd (os-shepherd): It reports separate success/failure recall, their mean balanced accuracy, and a leniency audit. OS-Shepherd-100K trains open 9B and 35B judges.
• Watch and Learn: Learning to Use Computers from Online Videos (watch-and-learn): Task retrieval turns these annotations into trajectories for in-context demonstrations and supervised computer-use agent (CUA) training. A generic LLM is not asked to guess actions, and rules do not reconstruct trajectories. Not reproduced due to cost (review complete); research material only, no executable entry.
• GAIA (gaia): Evaluate reasoning, multimodality, browsing, and tool use on 466 real problems with exact short answers and three difficulty levels.

Safe admission and verifiable execution

• SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents (silr): It admits only actions that do not worsen the product order. The same structured signal can be a process reward for GRPO.
• LogAct: Enabling Agentic Reliability via Shared Logs (logact): Safety review is separate from execution, and the protocol preserves uncertain states after a crash. Recovery of external effects still requires an explicit idempotency or transaction contract: replaying a conversation log is not enough. Bounded component only; full experiments are not reproduced.

Local tools and completion checks

• Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks (mingbird): Repeated calls or loops with no output trigger corrections, and common JSON errors get bounded recovery. This is an inference harness improvement, not a new backbone.

Agentic RL and post-training

Skill, turn, and rollout credit

• Frontier-OPD: Discovering the Edge of Agent Capability and Teaching Agents to Go Beyond (frontier-opd): The teacher takes up to \(K\) turns and the student continues for \(H\). Compare bridge plus continuation against the same-length original window at that position; require valid replay, non-terminal handoff and complete continuation. Bounded component only; full experiments are not reproduced.
• RSIPO: Recursive Self-Improving Policy Optimization for Agentic Reinforcement Learning via Skill-Policy Co-Evolution (rsipo): Static or offline-trained skills may not keep up with an improving policy, and a skill-augmented teacher can underperform its student. RSIPO produces skills from current-policy trajectories, measures their benefit with independent controls and matched initial-state replay, and jointly optimizes skill generation, RLVR, and self-distillation weighted by empirical-Bayes reliability. Not reproduced for now due to cost; validation pending, no executable entry.
• Agent-G²: Gaussian Guidance for Agentic Reinforcement Learning (agent-g2): A fixed depth ignores task difficulty; per-example probes waste rollouts. Agent-G² estimates the center and variance of a guidance band for each difficulty cluster from existing policy rollouts, then samples a prefix depth per task.
• AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization (auso): Action-level JSD between with-skill and without-skill policies yields bounded GRPO advantage weights. Bounds prevent flipping the original trajectory advantage's sign.
• AgentOPSD (agent-opsd): Aggregate privileged-replay teacher/student log-prob gaps into turn evidence. Update success belief recursively in log-odds space and use adjacent revisions to identify pivotal turns.
• OCSD (ocsd): Future-observation replay changes both information and scaffolding. Match Full and Observation-Ablated replay structure and modulate GRPO on uncertain steps only by their residual.
• CAST (cast): Use successive differences in solver state value as a solver advantage that supplies turn-level credit alongside sparse outcome reward.
• SEED (seed): Reflect on completed trajectories to extract reusable hindsight skills. Compare action log probabilities with and without skill context to supply dense on-policy distillation credit.
• TurnOPD (turn-opd): Use probe statistics to choose rollout depth adaptively, gradually moving token KL budget into turn-normalized supervision.
• SEARL (searl): Maintain tools and successful transitions in a memory graph. Each new rollout updates both the policy and graph-edge weights, closing the loop between experience, retrieval, and improvement.

General trajectories and credit assignment

• Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents (dr-credit): Compare a tool result with previously accepted support for each rubric and credit only new or partial support. Combine it with outcome advantage so repeated evidence does not earn repeated credit.
• HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning (harness-bandit): Allocate multi-harness training budget by combining learnability, transferability and an exploration bonus. The reference kernel retains gating and weight/state updates and exposes intermediate values for matched-budget comparisons.
• T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks (t1-terminal-rl): TITO trains on the actual sampled rollout token IDs; turn-boundary repair corrects drift, and R3 replays MoE routes.
• What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents (multi-harness-rl): The study freezes the same task–harness trajectories and compares only within- and cross-harness advantages. It evaluates transfer on a minimal harness not used during training.
• SPO++: Stabilizing Asynchronous Agentic Reinforcement Learning via Measure-Theoretic Token Correction (spo-plus-plus): Completion order can contaminate history, and trajectory whitening uses a different measure than token-mean actor loss. SPO++ organizes evidence by policy-generation event, freezes the baseline at dispatch, and normalizes advantages with action-token-count weights.
• Group-Reflective Self-Distillation (grsd): Final reward mixes useful actions, repeated errors, and chance. The policy reflects on verified on-policy traces for one prompt; a stop-gradient snapshot with the same parameters contrasts success/failure reflections to form training-only DO/AVOID guidance and modulates turn-level advantages.
• TAPO (tapo): TAPO reuses the same rollout and alternates policy optimization with \((s_t,a_t)\to s_{t+1}\) next-observation prediction on a shared backbone. It adds no sampling, expert data, or inference overhead.
• StepPO (steppo): The environment observation/action step is the natural agent decision unit, unlike token-level MDP credit. Estimate values and GAE at step boundaries, then aggregate token ratios within a step before clipping.
• Agent-R1 (agent-r1): Treat each agent/environment interaction as a transition. Pluggable context management, environment interfaces, and optimizers support token- or step-level credit.
• Agent Lightning (agent-lightning): Decouple execution from training. Log interactions as unified MDP transitions and assign credit to pieces of a long trace, rather than concatenating everything into one framework-specific sequence.
• GiGPO (gigpo): Sparse final rewards and whole-trajectory relative advantages cannot distinguish useful environment steps. Compute a macro advantage across full trajectories and a micro advantage across steps sharing an anchor state.

Environment models and world rehearsal

• AHEAD: Agentic Hints for Effective Agent Development (ahead): The teacher sees environmental feedback at every step but an additional LLM corrective hint only at errors. A bounded teacher–student log-probability difference injects dense step credit into GRPO advantage.
• EnvACE (envace): Separate group-relative advantages for acting and rehearsal avoid mixing reward scales. A few private rehearsals extend test-time planning.
• State2State: Environment-Derived Mid-Training for LLM Agents (state2state): Topic: environment-derived mid-training. Automatically sample initial and goal states from exploration and use a rule-based state matcher as verifier. This scales mid-training without handcrafted tasks or expert trajectories.

On-policy distillation and context exploration

• Asymmetric On-Policy Distillation Learns In-Context Exploration (asteroid): Fixed-latent BAMDP uses cross-episode history; general POMDP uses expert takeover from reached states. This is not gold in the prompt or ordinary one-step DAgger. Bounded component only; full experiments are not reproduced.

Harnesses and runtime policies

• Evo2: Evolving the Evolver through Weight-Harness Co-Evolution (evo2): EvoTIDE recovers productive edit trajectories for SFT, followed by interleaved updates to both roles. See the full paper.
• AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces (autosaddler): Small prompt, tool-config and middleware mistakes compound on long tasks, while manual harness tuning is costly. AutoSaddler treats the harness as code: diagnose minibatch failure traces, generate bounded structured patches, check causal effects on that batch, select updates on dev, and persist versions in EvoDAG.
• EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents (evoharness-rl): Expose belief, progress and experience as editable external state. SFT teaches harness actions; cost-aware GRPO learns when to read, write or merge.

Search, web, and multi-turn interaction RL

• ResidencyRL: Reinforcement Learning in Simulated Clinical Environments (residency-rl): Weighted normalized dimension scores minus truncation penalty are only reward aggregation, not the complete algorithm. Section 3 · Table 2 and Eq. 1. Not reproduced due to cost (review complete); research material only, no executable entry.
• Contextual Information Policy Optimization for Search Agents (cipo): Topic: search-agent RL. Rewarding only the final answer can reduce retrieval to confirmation bias. CIPO identifies later actions influenced by external evidence, gives dense turn-level credit, and jointly optimizes it with global outcome reward.
• HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning (hindsearch): Topic: hindsight for search trajectories. A frozen judge uses gold answers to write trajectory-level critiques for failed searches. Sparse success/failure becomes auxiliary on-policy distillation, combined with GRPO.
• PEARL (pearl): The offline stage learns tool usage and failure conditions by trial and error. The online stage separates Planner and Executor and applies GRPO with a dense reward for plan correctness, tool chains, and final outcomes rather than relying only on sparse success.
• MUA-RL (mua-rl): The agent clarifies intent in conversation and calls real MCP or database tools. User messages and tool results do not contribute to policy loss; exploration uses only the final task-completion reward.
• WebAgent-R1 (webagent-r1): WebAgent-R1 dynamically retains recent and task-relevant context, collects full multi-turn trajectories in parallel, and uses M-GRPO to update relative to group rewards for final success. The paper also emphasizes behavior-cloning warm-up and long chain-of-thought initialization.
• RAGEN (ragen): The paper observes an Echo Trap in which reward variance suddenly collapses with a gradient spike. StarPO-S uses trajectory filtering, a critic baseline, gradient stabilization, and decoupled clipping to suppress this self-reinforcing repetition.
• DeepResearcher (deepresearcher): Treat searching, browsing, collecting evidence, and citing answers as one trajectory. Train with joint answer and citation reward.
• Search-R1 (search-r1): The model emits search actions during reasoning and resumes when documents return. PPO/GRPO loss applies only to model-generated tokens; retrieved tokens are masked and a simple outcome reward scores the final answer.
• LOOP (loop): LOOP combines PPO's trust region with a leave-one-out baseline so it can update repeatedly on a batch without a critic. Token-level importance-ratio clipping limits drifting tokens without discarding whole long trajectories.

Multi-agent systems and software engineering

Role collaboration and software development

• A team of Gemini 4 Argon agents beats Minecraft (gemini-minecraft): An Observer summarizes world state, while chat and notes carry progress. The study concerns long-horizon coordination, programmatic tools, and persistent state, not a new training objective. Not reproduced due to cost (review complete); research material only, no executable entry.
• CERA-MoA: Co-Evolving Router and Agents for Mixture-of-Agents (cera-moa): Estimate each agent's familiarity with an example from intermediate-layer hidden states. Select experts adaptively via a cumulative threshold and direct training examples to the corresponding experts.
• CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents (codegrep): Train a 14B retrieval agent with GRPO to issue parallel grep/glob/read calls and narrow files over several turns. A frozen coding agent handles fixes; optimize downstream resolution, not standalone retrieval.
• MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems (manta): Initialize communication topology from task priors and monitor collaboration traces. Adapt roles, edges, ordering, visibility and verification under bounds.
• Agent0 (agent0): A proposer generates verifiable tool tasks. Multiple execution agents produce candidates and vote; increase curriculum difficulty along the current capability boundary.
• OpenHands (openhands): OpenHands is an open software agent platform. It unifies terminal, editor and browser actions in an observable event stream and isolates execution in a sandbox for bug fixes, implementation and repository maintenance.
• SWE-agent (swe-agent): A general-purpose shell exposes an overly broad action space and verbose output to an LLM. SWE-agent's specialized ACI constrains repository search, file viewing, precise editing, and testing so the model can locate the cause of an issue and validate a patch.
• AutoGen (autogen): Complex applications need models, tools, and humans to collaborate; hand-written control flow is hard to reuse. ConversableAgent and conversation programming let roles declare capabilities, reply policies, and termination conditions, then combine them through group chats or nested conversations.
• MetaGPT (metagpt): MetaGPT encodes human software-team SOPs as role-based message flows. Each role produces structured intermediates for downstream roles to consume and verify.
• CAMEL (camel): Inception prompts define user and assistant roles, goals, and constraints. Alternating messages solve tasks and produce multi-agent trajectories for study.

ML and software development trajectories

• SWE-Prime: Fewer Trajectories, Better Performance (swe-prime): Select traces for process, outcome and representativeness, then select semantic segments for contribution, learnability and risk. Keep all context but compute SFT loss only on retained segments.
• TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development (traceml): TraceML records each code version, score, action intent, edit size, and impact in a unified format, comparing 4465 human trajectories with agent trajectories. It distills how humans alternate stages and reopen previous approaches into a planning prior.
• AIRA2: Overcoming Bottlenecks in AI Research Agents (aira2): Search sees only the search score. Only after search ends does held-out validation that was not used for hill climbing select the champion.

Runtime cost and tool exposure control

• ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows under Quality-Cost Tradeoffs (progrouter): A dual-path predictor estimates the marginal progress each candidate model would yield. A meta-gate makes stepwise routing decisions under quality, time, and long-run cost budgets.
• TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving (topas): Under a shared KV-cache budget, jointly estimate the longest remaining workflow path and downstream prefix-reuse benefit. Account for prefix movement, preemption and aging rather than optimizing only local hit rate or one request's progress.
• CAM-DF (cam-df): Virtually traverse ranked prefixes before any tool execution. Payoff is sufficiency less heterogeneous cost; compare stopping with the best future prefix to label each decision and weight errors by regret.

Memory, skills, and continual learning

Active and long-term memory

• Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing (hy-memory): Then freeze the reader and judge and optimize only the writer. Chunk-level probes supplement sparse rewards; cross-chunk factual attribution locates contributions; a growth penalty discourages storing everything. Not reproduced due to cost (review complete); research material only, no executable entry.
• MemScope: Evolving Agent Memory Architectures through Localized Code Repair (memscope): It locates a cause and makes a bounded in-place change that preserves provider identity and compatibility with existing memory. If evidence is insufficient, it can explicitly select no patch instead of forcing an edit every round. §3 and Appendices A–B Bounded component only; full experiments are not reproduced.
• Test-Time Memory Construction (t2mem): Memory evolves alongside information needs. See the full paper.
• Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents (hippocam): The paper reports 97.8% ALFWorld success, 81.9% ScienceWorld progress, and 81.7% mean StreamBench accuracy. No online A/B test is reported.
• Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval (causal-memory-policy): CMP reserves balanced randomized exposure slots in a fixed candidate pool. Each memory then has estimable treatment and control samples, and utility with a standard error protects uncertain memories.
• ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents (remem): ReMem uses OCR-driven multimodal item perception for compact semantic representations, then keeps preferences in dynamic memory with a fixed budget. Multi-Memory GRPO shares policy updates across memory views.
• Mnemon: Raw Records, Fast Judgments, Slow Thoughts (mnemon): Retain dated raw records. A slower planner proposes searches, a fast Jev decides which records matter, and a compact view under a fixed budget feeds the original answering model.
• Interactive Memory Learning for Long-Term Conversations (interactive-memory): The Planner selects high-value information to write, the Trigger decides when to retrieve it, and delayed cross-session reward trains the two policies together.
• MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging (memforest): Merge nodes progressively under capacity pressure. Start retrieval from a semantic anchor and expand through neighboring nodes, retaining temporal structure and links between events.
• AtomRec: Evolving Atomic Memory for Agentic Recommendation (atomrec): Coarse profiles may overwrite old preferences, and a single collaborative edge is hard to explain. AtomRec splits item and user histories into independently evolving atomic fields, links them semantically and collaboratively, and retrieves multi-hop paths as recommendation evidence.
• When Memory Takes Gradients: Collaborative Vector Memory for Agentic Recommender Systems (covemem): CoVeMem builds a bank from frozen LightGCN states, retrieves history relevant to the current candidate set, and projects it into soft tokens. Semantic alignment and masked listwise co-training teach the LLM to read memory.
• MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off (memorycpt): Distill offline memory construction into a compact QAD model. QAR retrieves via RRF and summarizes query-relevant context with LoRA; cost-aware GRPO optimizes quality per cost.
• VerMem (vermem): VerMem uses one policy over all three states and seven atomic operations. A local verifier checks state transitions and a global verifier checks evidence consistency.
• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents (memcon): It chooses retrieval depth, plan injection, a replacement query, or memory maintenance based on task progress, repeated actions, memory size, and available successful plans. A lightweight tabular UCB controller explores and learns from end-of-task binary feedback and an efficiency reward without making extra LLM calls to choose memory operations. Bounded component only; full experiments are not reproduced.
• U-Mem (u-mem): U-Mem uses an acquisition cascade with rising cost: try self/teacher first, then research with tools, and finally consult an expert. Retrieval combines semantic similarity and Thompson sampling, and memory is validated and curated before write-back.
• Learning Personalized Agents from Human Feedback (pahf): Retrieve explicit user preferences before acting and ask for clarification when unknown or uncertain. Accept corrections after acting even when memory was confident, replacing stale preferences across new users, contexts and preference drift.
• LEGOMem (legomem): LEGOMem breaks experience into composable procedural units. Orchestrator memory stores decomposition and delegation, agent memory stores action templates, and the units are recombined at runtime for each new task.
• MemTool (memtool): MemTool compares autonomous, workflow, and hybrid management. The hybrid policy protects tools required by the current workflow and dynamically evicts other tools based on recency and historical success.
• MemGPT (memgpt): MemGPT borrows operating-system virtual memory: resident core information, active working context, and an external archive. Function calls move data, and interrupts or heartbeats control continued reasoning and interaction with the user.
• Generative Agents (generative-agents): Store observations in a memory stream and retrieve by recency, importance and relevance. Once cumulative importance crosses a threshold, generate higher-level reflections, then use memory and current state for schedules and action plans.

Memory retrieval and representations

• Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence (beyond-final-embedding): Each query token uses its worst returned similarity, clipped at zero, as a common floor for absent token–memory matches. The sparse local and full pooled branches are standardized per query before fusion.
• NEAR: Neighborhood-Aware Residuals for Memory Retrieval (near): Each layer recomputes neighborhoods on normalized residuals and preserves the original vector and multiple scoring views. Spherical K-means clusters memory inputs into at most 16 anchors.

Context, state, and branch management

• Cache Engineering for Agents (cache-engineering): AKV extends a radix structure with position versions, eviction and recovery and explores RSI-generated cache policies. A cache state depends on operation history, not merely the final token prefix. Not reproduced due to cost (review complete); research material only, no executable entry.
• Context Language Models (context-lm): Treat context as a file the model can edit directly, letting it retain, delete or reorganize information instead of following a fixed external harness policy. Multiple agents maintain separate context files, and the server prefills from the first mismatching token.
• Continuous Context Management (ccm): The next turn receives only the original task, memory, and latest observation. Full history is used only during training for privileged distillation.
• SKILL.state: Scalable Long-Horizon Agent Skills (skill-state): The runtime validates before committing, and transient reasoning is discarded. See §3 and Algorithm 1. Bounded component only; full experiments are not reproduced.

Skill graphs and cross-task accumulation

• Recursive Formation and First-Class Reuse of Compositional Capabilities (formcraft): A composite becomes an ordinary candidate for later search, which need not revisit its formation history. The composite's internal computations still execute at runtime. Bounded component only; full experiments are not reproduced.
• CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution (coskill): CoSkill lets a Reasoning Agent use task and substep skills and a trainable Meta-Skill Agent rewrite them from execution returns. They share a backbone and train jointly end to end.
• SPT: Skills as Pre-Training Data for Agentic Language Models (spt): Skill packages encode reusable workflows. Mid-train a causal LM on SkillCorpus before post-training; Reference Insert places cited files near first use.
• CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval (caskg): It calibrates the edges with three types of textual counterfactual probe: remove, substitute, and reorder. After Bayesian smoothing, it publishes only reliable relations.
• SkillForge: Automated Skill Discovery and Refinement for Tool-Using Agents (skillforge): Appending extracted skills forever can retain erroneous or stale entries. Include explicit skill invocation in RL, induce candidates from successful, failed and contrastive traces, and use environment evidence to activate, revise or deduplicate.
• Learning Globally Reusable Skills for Coding Agents (gse): GSE maintains a Skill Relation Graph. Cluster local experience to consolidate skills and use replay verification to reject overfitting and regressions.
• When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents (vag): VaG checks structure, behavioral harmlessness, and semantic consistency before committing skills. Marginal gain then selects a compatible Hot set.
• CoEvo-Mem (coevo-mem): Task outcomes update routing; trajectories update memory values and graph relationships. Alternate which side is frozen to control nonstationarity.
• SkillRise (skillrise): One policy alternates solving the current task and curating a skill document passed directly to the next. Solving uses the current outcome; curation uses discounted outcomes from later tasks.
• HiSkill (hiskill): Organize experience into high-level skills, executable AtomicOps, and typed directed edges. Retrieve only a task-relevant subgraph for action grounding.
• UniMem (unimem): Novel tasks first enter an episodic buffer. A self-routing controller consolidates recurring and reliable execution patterns into scalable parametric memory.
• Memento-Skills (memento-skills): Reflect on execution logs to extract structured skill descriptions. Retrieve them by task and write back versioned skills rather than appending raw trajectories.
• MemSkill (memskill): The controller selects memories from past episodes. The designer compiles recurring successful patterns into skills and updates their versions using new feedback.
• SAGE (sage): Abstract skills from successful trajectories, revise or retire them after failures, and use task rewards to learn retrieval and reuse.
• Voyager (voyager): Voyager uses GPT-4 to generate curricula and code as its action space. Successful programs are indexed by description in a skill library, then retrieved and composed for new tasks.

Skill acquisition, retrieval, and invocation

• Imagine Before Acting: Prospective Skill Identification for LLM Agents (skilldreamer): Plain similarity can find topical but unhelpful skills, and retrieving one does not imply it is invoked. SkillDreamer makes capability needs explicit during both retrieval and invocation.

Context compression and exact recovery

• Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents (transparent-compact): Compaction does not call a model, but the paper's context_lookup uses a model subagent to search history. These are not a single model-free full recovery capability. Bounded component only; full experiments are not reproduced.