Harness² and Google/Meta historical discovery repairs¶
October 21 correction: 54 and 521 describe the original October 20 reconciliation, not additive implementation TODOs. The 521 included 31 deferred leads and missed 14 acceptances. Split the 54 into 39 pending 2026 reviews, nine earlier references, and six non-implementation dispositions. See subsequent reconciliation.
This round adds one affordable CPU controller, 12 expensive-paper references, and links one existing diagnostic. It records independent decisions for 22 paper identities whose method/resource sections were actually checked. All 22 full model/API experiments are too costly, not reproduced (review DONE). Only the bounded Harness² CPU component was implemented; new references are not implementations. No new CUDA path, A100/A30 run, paid API, or model download.
Harness²: implemented scope¶
See paper and one-command run:
- Fix model, environment, starting harness, and improver versions. Execute within-task Propose–Probe–Compose.
- Widen in parallel and deepen sequentially; verify \(4k\) executions and \(3k\) edit stages.
- Selector sees public tasks and anonymized prior final artifacts, not gold, rubrics, or source harness.
- Persist prior same-domain experience; full harness adoption, referenced execution comparisons, conflict rejection, and failures retained.
Seeds 42/43/44, two modes, and two pipelined tasks per mode produced 96 actual Python subprocesses and 72 edits in about 1.78 seconds. Deterministic editors/selectors are mechanism substitutes: L1 evidence, not LLM capability, official benchmark results, or Evolve integration. See paper metrics.
Discovery defects repaired¶
| Old behavior | New behavior |
|---|---|
| Any ledger entry counted as reviewed | Date/identity screening does not close method review, and generic new remains unreviewed |
| Pre-window release closed later discovery | Official Google/Meta evidence can recall older papers separately, not count them as today's v1; explicitly future releases stay outside backfill |
| Priority relied on named queries | Recognize official sources, exact GitHub owners, and organization fields; do not infer Meta from “meta-learning” |
| Generic known/review_done hid accepted but undelivered | Show implementation-pending separately in candidate summary |
| First ID observation won | Catalog first, then substantive method/cost/source closure, then pending delivery, then generic labels; keep frozen snapshots |
Explicit high-cost decisions remain final, even with older metadata-only labels. record_discovery_review.py rejects date screening and pending delivery as new terminal decisions. Google/Meta full-text source requirements no longer depend only on query names.
Added or linked references¶
Historical omissions, not today's releases. Full experiments were not reproduced.
| Paper | Delivered |
|---|---|
| Harmony Harness | Multi-objective bounded harness evolution and function/safety trade-offs, with costs |
| SEER | Retrieval, reflective memory, matured feedback, frozen test, and costs |
| MARS | Modular research, budgeted search, execution experience, and costs |
| ECO | Momentum with quantization error and no master weights, with costs |
| SA-DPO | Semantic-distance adaptive preference margins and costs |
| FUSE | Unlabeled verifier ensemble and independence assumptions |
| Prompt-Level Distillation | Teacher-rule abstraction, clustering, conflict resolution, and costs |
| RL for Code Optimization | Code optimization RL and multi-node costs |
| AutoformBot | Math formalization agent and resource costs |
| AIRS-Bench | AI research evaluation protocol, not an integrated evaluation |
| MetaEncoder | Dual-tower natural-language decisions and official 30B evaluation costs |
| Raw Video Mid-Training | Continuous visual next-token training and matched controls |
| MaD-RL | Correct first date, link old character-model/A100 diagnostics and negatives; no measurement rewrite, implementation duplication, or new GPU run |
Existing Meta Logbook, Render2Reason, PAT, CanonQ, Mixture of Layers, Omni-Diffusion-Distill, X-AgentRed, and Compute Optimal Tokenization also gained resource decisions. Per-paper closure closes only full experiments without claiming components or a detail page for all.
Coverage and remaining work¶
- Google: enumerated 102 saved historical leads and 27 DeepMind 2026 catalog records, checked 198 date-archive identities, and read targeted method/experiment/resource/release sections for seven papers.
- Meta: reconciled 48 saved 2026 official catalog records, reviewing nine historical accepted but undelivered entries with source-specific costs.
- 83 identified papers: seven existing component/implementation pages and 22 review decisions, leaving 54 without substantive closure. This is neither eligibility nor full reading.
- The original ledger had 521 purported accepted-undelivered items. This made them visible without implementing them or mass-labeling high cost. They were not commitments; prioritize institutions, topic, and cost.
See Google/DeepMind audit, Meta audit, reconciliation, and 22 decisions. Unverified identity/date/full text remains an explicit exception, not acceptance, full-text closure, or implementation. Not exhaustive across history or sources; watermark remains 2026-10-15.
Verification and rerun¶
PYTHONPATH=src python -m pytest tests/agent_research/test_harness2_component.py \
tests/test_historical_priority_discovery.py tests/test_harness2_priority_delivery.py \
tests/test_harness2_meta_reference_catalog.py
PYTHONPATH=src python scripts/audit_historical_priority_backfill.py --check
Shared discovery, source, review-write, and watermark tests are also required. Commands are not evidence of execution; check delivered results and PR CI.
Actual checks passed: 196 discovery/review regressions, 95 new targeted checks, 30 docs tests, minimal wheel contract, metrics/GPU evidence validation, and strict build. Removed about 478 MiB of temporary sites, isolated installs, build caches, and upstream clones. Kept raw CPU runs (including early diagnostics), metrics, figures, and shared public data/model caches.