DARS: Dependency-Aware Reward Shaping for Agentic Reinforcement Learning

简体中文

L1 dependency decay, invalidation/repair and potential differences; no 1.5B–8B training.

Paper information

Field	Details
Paper	arXiv 2610.01207
Institution	University of Illinois Urbana-Champaign (first author).
First public release	2026-10-01 (arXiv v1)
Author code	The official placeholder said the implementation was in preparation as of 2026-10-02; interface notes are not released implementation.
Local adapter or method key	dars
Local reproduction code	src/auto_research/post_training/latest_20261002.py

Paper method and evidence

Represent progress with a prerequisite graph of predicates. Verification, invalidation, and repair update it. The nearest broken dependency discounts prior credit by graph distance without penalizing independent branches.

flowchart LR
  E[Verify, invalidate, or repair] --> G[Update prerequisite graph]
  G --> D[Discount affected branches by distance]
  D --> R[Potential-difference reward]

Original paper figure

The original figure shows the paper mechanism. From the original paper; copyright remains with the authors.

Core equation

\(r_t=\Phi(s_{t+1})-\Phi(s_t)\); invalidated dependency discounts predicate weight by \(\gamma^{d}\).

Reported results

The paper reports up to 10 points over GiGPO on matched-budget ALFWorld and evaluates WebShop, Search-R1, AIME and tool-free reasoning.

Local reproduction

Compare with the corresponding operator disabled or in its default state. Three-seed metrics are diagnostic_only=true: they verify execution and invariants, not formal gains or capability ranking.