HFEPX Hub

Long Horizon Or Multi Agent Papers

Updated from current HFEPX corpus (Feb 27, 2026). 109 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequent quality control: Calibration. Frequently cited benchmark: Retrieval. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 25, 2026.

Papers: 109 Last published: Feb 25, 2026 Global RSS Tag RSS

Long HorizonMulti Agent

Research Narrative

Grounded narrative Model: deterministic-grounded

Updated from current HFEPX corpus (Feb 27, 2026). This page covers 109 papers centered on Long Horizon Or Multi Agent Papers. Common evaluation modes include Automatic Metrics, Simulation Env, with benchmark emphasis on Retrieval, MATH. Metric concentration includes accuracy, cost, and the agentic footprint highlights Long Horizon, Multi Agent. Use the anchored takeaways below to compare protocol choices, quality-control patterns, and evidence depth before allocating new eval budget.

Why This Matters For Eval Research

Evaluation emphasis: Automatic Metrics and Simulation Env appear frequently in this slice.

Evidence: GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL , SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents , D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
Benchmark concentration: Retrieval, MATH helps control cross-paper variance.

Evidence: SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents , D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models , Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning
Metric concentration: accuracy, cost is repeatedly reported in this group.

Evidence: D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models , Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning , Self-Correcting VLA: Online Action Refinement via Sparse World Imagination

Protocol Takeaways

Compare papers that report both human_eval and llm_as_judge to quantify judge-human agreement drift.

Evidence: Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning , Self-Correcting VLA: Online Action Refinement via Sparse World Imagination , Structurally Aligned Subtask-Level Memory for Software Engineering Agents
Stratify by benchmark (Retrieval vs MATH) before comparing methods.

Evidence: Self-Correcting VLA: Online Action Refinement via Sparse World Imagination , Structurally Aligned Subtask-Level Memory for Software Engineering Agents , ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning
Papers with explicit human feedback is visible in approximately 17.4% of papers in this set.

Evidence: Structurally Aligned Subtask-Level Memory for Software Engineering Agents , ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning , LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies

Benchmark Interpretation

Retrieval appears as a recurring benchmark anchor in this page.
14 papers (12.8%) mention Retrieval.
Most common evaluation modes: Automatic Metrics, Simulation Env.

Metric Interpretation

accuracy is a common reported metric and should be paired with protocol context before ranking methods.
22 papers (20.2%) mention accuracy.
Most common evaluation modes: Automatic Metrics, Simulation Env.

Researcher Checklist

Papers with explicit human feedback: Coverage is a replication risk (17.4% vs 45% target).
Papers reporting quality controls: Coverage is a replication risk (2.8% vs 30% target).
Papers naming benchmarks/datasets: Coverage is usable but incomplete (33% vs 35% target).
Papers naming evaluation metrics: Coverage is strong (45.9% vs 35% target).
Papers with known rater population: Coverage is a replication risk (11.9% vs 35% target).
Papers with known annotation unit: Coverage is strong (36.7% vs 35% target).

Papers with explicit human feedback

Coverage is a replication risk (17.4% vs 45% target).

Papers reporting quality controls

Coverage is a replication risk (2.8% vs 30% target).

Papers naming benchmarks/datasets

Coverage is usable but incomplete (33% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (45.9% vs 35% target).

Papers with known rater population

Coverage is a replication risk (11.9% vs 35% target).

Papers with known annotation unit

Coverage is strong (36.7% vs 35% target).

Known Limitations

Narrative synthesis is grounded in metadata and abstracts only; full-paper method details may be missing.
Extraction fields are conservative and can under-report implicit protocol details.
Cross-page comparisons should control for benchmark and metric mismatch.

Research Utility Links

Judge vs Human Agreement - Compares papers that evaluate with both human raters and LLM judges.
Benchmark Slice: Retrieval - Prioritizes benchmark-specific protocol comparisons.
Metric Slice: accuracy - Finds papers where reported metrics are directly comparable.
Recent High-Signal Papers - Keeps the hub connected to the latest HFEPX corpus updates.

human_eval vs llm_as_judge

both=0, left_only=2, right_only=2

0 papers use both Human Eval and Llm As Judge.

human_eval vs automatic_metrics

both=0, left_only=2, right_only=77

0 papers use both Human Eval and Automatic Metrics.

llm_as_judge vs automatic_metrics