Skip to content
← Back to explorer

HFEPX Hub

Multi Agent Papers (Last 90 Days)

Updated from current HFEPX corpus (Mar 1, 2026). 29 papers are grouped in this hub page.

Read Full Context

Updated from current HFEPX corpus (Mar 1, 2026). 29 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Pairwise. Frequent quality control: Calibration. Frequently cited benchmark: Lawbench. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 18, 2026.

Papers: 29 Last published: Feb 18, 2026 Global RSS Tag RSS
Multi AgentLast 90d

Researcher Quick Triage

This hub is best used for protocol triage and replication planning from abstract-level evidence. Quality band: Developing .

High-Signal Coverage

100.0%

29 / 29 sampled papers are not low-signal flagged.

Replication-Ready Set

0

Benchmark + metric + eval mode explicitly present.

Judge/Human Comparability

0

Papers containing both `human_eval` and `llm_as_judge`.

  • 0 papers are replication-ready (benchmark + metric + explicit evaluation mode).
  • 0 papers support judge-vs-human agreement analysis.
  • 1 papers report explicit quality controls (calibration/adjudication/IAA).

Primary action: Use this page for scouting only; collect additional papers before attempting replication-critical comparisons.

Why This Matters (Expanded)

Why This Matters For Eval Research

  • 34.6% of papers report explicit human-feedback signals, led by expert verification.
  • automatic metrics appears in 41.4% of papers in this hub.
  • Lawbench is a recurring benchmark anchor for cross-paper comparisons in this page.
Protocol Notes (Expanded)

Protocol Takeaways

  • Most common quality-control signal is rater calibration (3.4% of papers).
  • Rater context is mostly domain experts, and annotation is commonly pairwise annotation; use this to scope replication staffing.
  • Pair this hub with a human_eval-heavy hub to validate judge-model calibration.

Benchmark Interpretation

  • Lawbench appears in 3.8% of hub papers (1/29); use this cohort for benchmark-matched comparisons.
  • LiveCodeBench appears in 3.8% of hub papers (1/29); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • accuracy is reported in 23.1% of hub papers (6/29); compare with a secondary metric before ranking methods.
  • cost is reported in 7.7% of hub papers (2/29); compare with a secondary metric before ranking methods.
Researcher Checklist (Expanded)

Researcher Checklist

  • Moderate: Papers with explicit human feedback

    Coverage is usable but incomplete (34.6% vs 45% target).

  • Gap: Papers reporting quality controls

    Coverage is a replication risk (3.8% vs 30% target).

  • Gap: Papers naming benchmarks/datasets

    Coverage is a replication risk (15.4% vs 35% target).

  • Strong: Papers naming evaluation metrics

    Coverage is strong (38.5% vs 35% target).

  • Moderate: Papers with known rater population

    Coverage is usable but incomplete (26.9% vs 35% target).

  • Moderate: Papers with known annotation unit

    Coverage is usable but incomplete (23.1% vs 35% target).

Strengths

  • Agentic evaluation appears in 100% of papers.

Known Gaps

  • Only 3.8% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Annotation unit is under-specified (23.1% coverage).
  • Benchmark coverage is thin (15.4% of papers mention benchmarks/datasets).

Suggested Next Analyses

  • Pair this hub with a human_eval-heavy hub to validate judge-model calibration.
  • Stratify by benchmark (Lawbench vs LiveCodeBench) before comparing methods.
  • Track metric sensitivity by reporting both accuracy and cost.
  • Add inter-annotator agreement checks when reproducing these protocols.
Recommended Queries (Expanded)

Recommended Queries

Start Here (Best First 6)

Ranked for protocol completeness (human signal, benchmark + metric anchors, quality controls, and judge/human overlap).

Protocol Matrix (Top 12)

Use this to quickly compare protocol ingredients instead of scanning long prose.

Paper HF Signal Eval Modes Benchmarks Metrics QC
Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling

Feb 18, 2026

Yes Not Reported LiveCodeBench Not Reported Calibration
SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery

Feb 24, 2026

Yes Automatic Metrics Not Reported Cost Not Reported
PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training

Feb 14, 2026

Yes Automatic Metrics Not Reported Helpfulness Not Reported
TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation

Feb 26, 2026

Yes Simulation Env Not Reported Not Reported Not Reported
Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation

Feb 16, 2026

Yes Not Reported Not Reported Not Reported Not Reported
World-Model-Augmented Web Agents with Action Correction

Feb 17, 2026

No
Not Reported
Llm As Judge , Simulation Env VisualWebArena , Mind2Web Not Reported Not Reported
Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents

Feb 15, 2026

No
Not Reported
Simulation Env WebArena , OSWorld Not Reported Not Reported
Toward Expert Investment Teams:A Multi-Agent LLM System with Fine-Grained Trading Tasks

Feb 26, 2026

Yes Not Reported Not Reported Not Reported Not Reported
Decentralized Ranking Aggregation: Gossip Algorithms for Borda and Copeland Consensus

Feb 26, 2026

Yes Not Reported Not Reported Not Reported Not Reported
The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems

Feb 17, 2026

Yes Not Reported Not Reported Not Reported Not Reported
OMGs: A multi-agent system supporting MDT decision-making across the ovarian tumour care continuum

Feb 14, 2026

Yes Not Reported Not Reported Not Reported Not Reported
The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI

Feb 19, 2026

No
Not Reported
Llm As Judge , Automatic Metrics Not Reported Accuracy Not Reported

Protocol Diff (Top Papers)

Fast side-by-side comparison for the highest-ranked papers in this hub.

Signal Team of Thoughts: Efficient Test-time Scaling of Ag… SparkMe: Adaptive Semi-Structured Interviewing for… PrivAct: Internalizing Contextual Privacy Preservat…
Human Feedback Expert VerificationExpert VerificationPairwise Preference
Evaluation Modes Not reportedAutomatic MetricsAutomatic Metrics
Benchmarks LiveCodeBenchNot reportedNot reported
Metrics Not reportedCostHelpfulness
Quality Controls CalibrationNot reportedNot reported
Rater Population Domain ExpertsDomain ExpertsUnknown
Annotation Unit UnknownTrajectoryUnknown
Suggested Reading Order (Extended)

This section is intentionally expanded only when needed; use “Start Here” above for a faster pass.

Suggested Reading Order

  1. Toward Expert Investment Teams:A Multi-Agent LLM System with Fine-Grained Trading Tasks

    Start here for detailed protocol reporting and quality-control evidence. Signals: pairwise preferences. Abstract: Crucially, further analysis of intermediate agent outputs suggests that alignment between analytical outputs and downstream.

  2. AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning

    Start here for detailed protocol reporting and quality-control evidence. Signals: automatic metrics. Focus: accuracy. Abstract: While Multi-Agent Systems (MAS) excel in complex reasoning, they suffer from the cascading.

  3. Decentralized Ranking Aggregation: Gossip Algorithms for Borda and Copeland Consensus

    Start here for detailed protocol reporting and quality-control evidence. Signals: pairwise preferences. Abstract: The concept of ranking aggregation plays a central role in preference analysis, and numerous algorithms.

  4. Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling

    Include a human-eval paper to calibrate against judge-based evaluation settings. Signals: expert verification. Focus: LiveCodeBench. Abstract: Existing Multi-Agent Systems (MAS) typically rely on static, homogeneous model configurations, limiting.

  5. TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation

    Include a human-eval paper to calibrate against judge-based evaluation settings. Signals: simulation environments + expert verification. Abstract: As mental health chatbots proliferate to address the global treatment gap,.

  6. World-Model-Augmented Web Agents with Action Correction

    Include an LLM-as-judge paper to test judge design and agreement assumptions. Signals: LLM-as-judge. Focus: VisualWebArena. Abstract: A world model, specialized in environmental state transitions, simulates action outcomes, which.

  7. The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI

    Include an LLM-as-judge paper to test judge design and agreement assumptions. Signals: LLM-as-judge. Focus: accuracy. Abstract: As Large Language Models (LLMs) transition from standalone chat interfaces to foundational.

  8. SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery

    Adds automatic metrics with expert verification for broader protocol coverage within this hub. Signals: automatic metrics + expert verification. Focus: cost. Abstract: The code, datasets, and evaluation protocols.

Known Limitations

Known Limitations

  • Only 3.8% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Annotation unit is under-specified (23.1% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.
Research Utility Snapshot (Detailed)

Research Utility Snapshot

Human Feedback Mix

  • Expert Verification (5)
  • Pairwise Preference (5)
  • Rubric Rating (1)

Evaluation Modes

  • Automatic Metrics (12)
  • Simulation Env (8)
  • Llm As Judge (2)

Top Benchmarks

  • Lawbench (1)
  • LiveCodeBench (1)
  • Mind2Web (1)
  • OSWorld (1)

Top Metrics

  • Accuracy (6)
  • Cost (2)
  • Bleu (1)
  • Error rate (1)

Rater Population Mix

  • Domain Experts (7)

Quality Controls

  • Calibration (1)
Coverage diagnostics (sample-based): human-feedback 31.0% · benchmarks 17.2% · metrics 34.5% · quality controls 3.4%.

Top Papers

Related Hubs

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.