Automatic Metrics + Multi Agent Papers

Why This Matters For Eval Research

• Common evaluation patterns here: Automatic Metrics.
• Benchmark signals emphasize: retrieval, livecodebench.
• Top reported metrics include: accuracy, cost.

Research Utility Snapshot

Top Papers

• Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning

  Tomoya Kawabe, Rin Takano · Feb 25, 2026 · Citations: 0

  Automatic Metrics Long Horizon

  We present a hierarchical multi-agent LLM-based planner with prompt optimization: an upper layer decomposes tasks and assigns them to lower-layer agents, which generate PDDL problems solved by a classical planner.

• Training Generalizable Collaborative Agents via Strategic Risk Aversion

  Chengrui Qu, Yizhou Zhang, Nicholas Lanzetti, Eric Mazumdar · Feb 25, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  Many emerging agentic paradigms require agents to collaborate with one another (or people) to achieve shared goals.

• The Headless Firm: How AI Reshapes Enterprise Boundaries

  Tassilo Klein, Sebastian Wieczorek · Feb 24, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  We argue that agentic AI induces a structural change in how coordination costs scale: in prior modular systems, integration cost grew with interaction topology (O(n^2) in the number of components); in protocol-mediated agentic systems, inte

• A Hierarchical Multi-Agent System for Autonomous Discovery in Geoscientific Data Archives

  Dmitrii Pantiukhin, Ivan Kuznetsov, Boris Shapkin, Antonia Anna Jost, Thomas Jung · Feb 24, 2026 · Citations: 0

  Automatic Metrics Long Horizon

  Here we present PANGAEA-GPT, a hierarchical multi-agent framework designed for autonomous data discovery and analysis.

• SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery

  David Anugraha, Vishakh Padmakumar, Diyi Yang · Feb 24, 2026 · Citations: 0

  Expert Verification Automatic Metrics Multi Agent

  Based on this formulation, we introduce SparkMe, a multi-agent LLM interviewer that performs deliberative planning via simulated conversation rollouts to select questions with high expected utility.

• Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination

  Rakshit Trivedi, Kartik Sharma, David C Parkes · Feb 24, 2026 · Citations: 0

  Demonstrations Automatic Metrics Multi Agent

  Effective human-AI coordination requires artificial agents capable of exhibiting and responding to human-like behaviors while adapting to changing contexts.

• SAMAS: A Spectrum-Guided Multi-Agent System for Achieving Style Fidelity in Literary Translation

  Jingzhuo Wu, Jiajun Zhang, Keyan Jin, Dehua Ma, Junbo Wang · Feb 23, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  This limitation stems from the inability of current single-model and static multi-agent systems to perceive and adapt to stylistic variations.

• Whisper: Courtside Edition Enhancing ASR Performance Through LLM-Driven Context Generation

  Yonathan Ron, Shiri Gilboa, Tammuz Dubnov · Feb 21, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  We introduce Whisper: Courtside Edition, a novel multi-agent large language model (LLM) pipeline that enhances Whisper transcriptions without retraining.

• Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

  Lichang Song, Ting Long, Yi Chang · Feb 21, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-ma

• Evaluating Chain-of-Thought Reasoning through Reusability and Verifiability

  Shashank Aggarwal, Ram Vikas Mishra, Amit Awekar · Feb 19, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  In multi-agent IR pipelines for tasks such as search and ranking, LLM-based agents exchange intermediate reasoning in terms of Chain-of-Thought (CoT) with each other.

• The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI

  Dusan Bosnjakovic · Feb 19, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  As Large Language Models (LLMs) transition from standalone chat interfaces to foundational reasoning layers in multi-agent systems and recursive evaluation loops (LLM-as-a-judge), the detection of durable, provider-level behavioral signatur

• Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling

  Jeffrey T. H. Wong, Zixi Zhang, Junyi Liu, Yiren Zhao · Feb 18, 2026 · Citations: 0

  Expert Verification Automatic Metrics Multi Agent

  Existing Multi-Agent Systems (MAS) typically rely on static, homogeneous model configurations, limiting their ability to exploit the distinct strengths of differently post-trained models.

• The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems

  Xiaoze Liu, Ruowang Zhang, Weichen Yu, Siheng Xiong, Liu He · Feb 17, 2026 · Citations: 0

  Pairwise Preference Automatic Metrics Multi Agent

  Multi-Agent Systems (MAS) powered by Large Language Models have unlocked advanced collaborative reasoning, yet they remain shackled by the inefficiency of discrete text communication, which imposes significant runtime overhead and informati

• VILLAIN at AVerImaTeC: Verifying Image-Text Claims via Multi-Agent Collaboration

  Jaeyoon Jung, Yejun Yoon, Kunwoo Park · Feb 4, 2026 · Citations: 0

  Automatic Metrics Multi Agent

  This paper describes VILLAIN, a multimodal fact-checking system that verifies image-text claims through prompt-based multi-agent collaboration.

• From Competition to Coordination: Market Making as a Scalable Framework for Safe and Aligned Multi-Agent LLM Systems

  Brendan Gho, Suman Muppavarapu, Afnan Shaik, Tyson Tsay, Atharva Mohan · Nov 18, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  As foundation models are increasingly deployed as interacting agents in multi-agent systems, their collective behavior raises new challenges for trustworthiness, transparency, and accountability.

• From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity

  Tianxi Wan, Jiaming Luo, Siyuan Chen, Kunyao Lan, Jianhua Chen · Oct 29, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  To address this, we develop a novel approach integrating synthetic patient electronic medical record (EMR) construction and multi-agent diagnostic dialogue generation.

• CogniAlign: Survivability-Grounded Multi-Agent Moral Reasoning for Safe and Transparent AI

  Hasin Jawad Ali, Ilhamul Azam, Ajwad Abrar, Md. Kamrul Hasan, Hasan Mahmud · Sep 14, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  The challenge of aligning artificial intelligence (AI) with human values persists due to the abstract and often conflicting nature of moral principles and the opacity of existing approaches.

• Your AI Bosses Are Still Prejudiced: The Emergence of Stereotypes in LLM-Based Multi-Agent Systems

  Jingyu Guo, Yingying Xu · Aug 27, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  While stereotypes are well-documented in human social interactions, AI systems are often presumed to be less susceptible to such biases.

• CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressures

  Punya Syon Pandey, Yongjin Yang, Jiarui Liu, Zhijing Jin · Aug 16, 2025 · Citations: 0

  Pairwise Preference Automatic Metrics Multi Agent

  Game-theoretic interactions between agents with Large Language Models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been sufficiently quantified.

• 1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning

  Wenkai Li, Liwen Sun, Zhenxiang Guan, Xuhui Zhou, Maarten Sap · Aug 11, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  We introduce a multi-agent framework that decomposes privacy reasoning into specialized subtasks (extraction, classification), reducing the information load on any single agent while enabling iterative validation and more reliable adherence

• CoAct-1: Computer-using Multi-Agent System with Coding Actions

  Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi · Aug 5, 2025 · Citations: 0

  Automatic Metrics Long Horizon

  Autonomous agents that operate computers via Graphical User Interfaces (GUIs) often struggle with efficiency and reliability on complex, long-horizon tasks.

• GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning

  Jie Peng, Jiarui Ji, Runlin Lei, Zhewei Wei, Yongchao Liu · Jul 4, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  Additionally, prior work mainly focuses on discriminative tasks on DyTAGs, resulting in a lack of standardized task formulations and evaluation protocols tailored for DyTAG generation.

• Reshaping MOFs text mining with a dynamic multi-agents framework of large language model

  Zuhong Lin, Daoyuan Ren, Kai Ran, Jing Sun, Songlin Yu · Apr 26, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  Accurately identifying the synthesis conditions of metal-organic frameworks (MOFs) is essential for guiding experimental design, yet remains challenging because relevant information in the literature is often scattered, inconsistent, and di

• Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition

  Zheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao, Chen Wang · Apr 26, 2025 · Citations: 0

  Pairwise Preference Automatic Metrics Multi Agent

  These domains typically involve fixed content and passive consumption, where user preferences can be matched by genre or theme.

• Can Multimodal LLMs Perform Time Series Anomaly Detection?

  Xiongxiao Xu, Haoran Wang, Yueqing Liang, Philip S. Yu, Yue Zhao · Feb 25, 2025 · Citations: 0

  Automatic Metrics Multi Agent

  One natural way for humans to detect time series anomalies is through visualization and textual description.

Related Hubs

• Automatic Metrics Papers Hub
• Multi Agent Papers Hub
• General Papers Hub
• Coding Papers Hub
• Accuracy Metric Papers Metric
• HFEPX Daily Archive: 2026-02-24 Daily