Retrieval + General Benchmark Papers

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 62 papers for Retrieval + General Benchmark Papers. Dominant protocol signals include automatic metrics, simulation environments, human evaluation, with frequent benchmark focus on Retrieval, BrowseComp and metric focus on accuracy, recall. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Top Papers On This Benchmark

• MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations

  Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa · Feb 26, 2026

  Automatic Metrics

  We present MTRAG-UN, a benchmark for exploring open challenges in multi-turn retrieval augmented generation, a popular use of large language models.

• Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training

  Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li · Feb 26, 2026

  Automatic Metrics Long Horizon

  Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed s

• Personalized Graph-Empowered Large Language Model for Proactive Information Access

  Chia Cheng Chang, An-Zi Yen, Hen-Hsen Huang, Hsin-Hsi Chen · Feb 25, 2026

  Automatic Metrics

  Since individuals may struggle to recall all life details and often confuse events, establishing a system to assist users in recalling forgotten experiences is essential.

• FewMMBench: A Benchmark for Multimodal Few-Shot Learning

  Mustafa Dogan, Ilker Kesen, Iacer Calixto, Aykut Erdem, Erkut Erdem · Feb 25, 2026

  Automatic Metrics

  In this paper, we introduce FewMMBench, a comprehensive benchmark designed to evaluate MLLMs under few-shot conditions, with a focus on In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting.

• Prompt Architecture Determines Reasoning Quality: A Variable Isolation Study on the Car Wash Problem

  Heejin Jo · Feb 25, 2026

  Automatic Metrics

  Large language models consistently fail the "car wash problem," a viral reasoning benchmark requiring implicit physical constraint inference.

• Retrieval Challenges in Low-Resource Public Service Information: A Case Study on Food Pantry Access

  Touseef Hasan, Laila Cure, Souvika Sarkar · Feb 25, 2026

  Simulation Env

  We conduct a pilot evaluation study using community-sourced queries to examine system behavior in realistic scenarios.

• Revisiting RAG Retrievers: An Information Theoretic Benchmark

  Wenqing Zheng, Dmitri Kalaev, Noah Fatsi, Daniel Barcklow, Owen Reinert · Feb 25, 2026

  Automatic Metrics

  Existing benchmarks primarily compare entire RAG pipelines or introduce new datasets, providing little guidance on selecting or combining retrievers themselves.

• Revisiting Text Ranking in Deep Research

  Chuan Meng, Litu Ou, Sean MacAvaney, Jeff Dalton · Feb 25, 2026

  Automatic Metrics

  To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it.

• Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG

  Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni · Feb 24, 2026

  Automatic Metrics

  Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components.

• HELP: HyperNode Expansion and Logical Path-Guided Evidence Localization for Accurate and Efficient GraphRAG

  Yuqi Huang, Ning Liao, Kai Yang, Anning Hu, Shengchao Hu · Feb 24, 2026

  Automatic Metrics

  Extensive experiments demonstrate that HELP achieves competitive performance across multiple simple and multi-hop QA benchmarks and up to a 28.8$\times$ speedup over leading Graph-based RAG baselines.

• E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications

  Jiwoo Kang, Yeon-Chang Lee · Feb 24, 2026

  Automatic Metrics

  Multimodal recommender systems (MMRSs) enhance collaborative filtering by leveraging item-side modalities, but their reliance on a fixed set of modalities and task-specific objectives limits both modality extensibility and task generalizati

• RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition

  Kun Ran, Marwah Alaofi, Danula Hettiachchi, Chenglong Ma, Khoi Nguyen Dinh Anh · Feb 24, 2026

  Automatic Metrics

  R2RAG won the Best Dynamic Evaluation award in the Open Source category, demonstrating high effectiveness with careful design and efficient use of resources.

• InterviewSim: A Scalable Framework for Interview-Grounded Personality Simulation

  Yu Li, Pranav Narayanan Venkit, Yada Pruksachatkun, Chien-Sheng Wu · Feb 23, 2026

  Simulation Env

  Existing evaluation approaches rely on demographic surveys, personality questionnaires, or short AI-led interviews as proxies, but lack direct assessment against what individuals actually said.

• How Retrieved Context Shapes Internal Representations in RAG

  Samuel Yeh, Sharon Li · Feb 23, 2026

  Automatic Metrics

  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial.

• Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

  Yibo Yan, Jiahao Huo, Guanbo Feng, Mingdong Ou, Yi Cao · Feb 23, 2026

  Automatic Metrics

  We begin by examining the benchmark landscape, and subsequently dive into the methodological evolution, categorizing approaches into three primary aspects: multimodal embedding models, multimodal reranker models, and the integration of Retr

• Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework

  Yibo Yan, Mingdong Ou, Yi Cao, Xin Zou, Jiahao Huo · Feb 23, 2026

  Automatic Metrics

  Visual Document Retrieval (VDR), which aims to retrieve relevant pages within vast corpora of visually-rich documents, is of significance in current multimodal retrieval applications.

• How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1

  Yinuo Xu, Shuo Lu, Jianjie Cheng, Meng Wang, Qianlong Xie · Feb 23, 2026

  Automatic Metrics

  Deep Research agents tackle knowledge-intensive tasks through multi-round retrieval and decision-oriented generation.

• Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering

  Maryam Amirizaniani, Alireza Salemi, Hamed Zamani · Feb 22, 2026

  Automatic Metrics Long Horizon

  Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context.

• VIGiA: Instructional Video Guidance via Dialogue Reasoning and Retrieval

  Diogo Glória-Silva, David Semedo, João Maglhães · Feb 22, 2026

  Automatic Metrics Long Horizon

  Our evaluation shows that VIGiA outperforms existing state-of-the-art models on all tasks in a conversational plan guidance setting, reaching over 90\% accuracy on plan-aware VQA.

• Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

  Lichang Song, Ting Long, Yi Chang · Feb 21, 2026

  Automatic Metrics Multi Agent

  To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-ma

• Contradiction to Consensus: Dual Perspective, Multi Source Retrieval Based Claim Verification with Source Level Disagreement using LLM

  Md Badsha Biswas, Ozlem Uzuner · Feb 21, 2026

  Automatic Metrics

  Through extensive evaluation on four benchmark datasets with five LLMs, we show that knowledge aggregation not only improves claim verification but also reveals differences in source-specific reasoning.

• Diagnosing LLM Reranker Behavior Under Fixed Evidence Pools

  Baris Arat, Emre Sefer · Feb 20, 2026

  Automatic Metrics

  Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever.

• RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

  Deniz Qian, Hung-Ting Chen, Eunsol Choi · Feb 20, 2026

  Automatic Metrics

  Our method outperforms baselines, including agentic search approaches, achieving at least 10% relative and 3% absolute gain in complete recall percentage on a multi-answer retrieval dataset (QAMPARI).

• Validating Political Position Predictions of Arguments

  Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn · Feb 20, 2026

  Human Eval

  Real-world knowledge representation often requires capturing subjective, continuous attributes -- such as political positions -- that conflict with pairwise validation, the widely accepted gold standard for human evaluation.

• Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions

  Raymond Li, Amirhossein Abaskohi, Chuyuan Li, Gabriel Murray, Giuseppe Carenini · Feb 20, 2026

  Automatic Metrics

  Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity.

• Unmasking the Factual-Conceptual Gap in Persian Language Models

  Alireza Sakhaeirad, Ali Ma'manpoosh, Arshia Hemmat · Feb 19, 2026

  Automatic Metrics

  While emerging Persian NLP benchmarks have expanded into pragmatics and politeness, they rarely distinguish between memorized cultural facts and the ability to reason about implicit social norms.

• PEACE 2.0: Grounded Explanations and Counter-Speech for Combating Hate Expressions

  Greta Damo, Stéphane Petiot, Elena Cabrio, Serena Villata · Feb 19, 2026

  Automatic Metrics

  The increasing volume of hate speech on online platforms poses significant societal challenges.

• The Role of the Availability Heuristic in Multiple-Choice Answering Behaviour

  Leonidas Zotos, Hedderik van Rijn, Malvina Nissim · Feb 19, 2026

  Automatic Metrics

  When students are unsure of the correct answer to a multiple-choice question (MCQ), guessing is common practice.

• RPDR: A Round-trip Prediction-Based Data Augmentation Framework for Long-Tail Question Answering

  Yiming Zhang, Siyue Zhang, Junbo Zhao, Chen Zhao · Feb 19, 2026

  Automatic Metrics

  We evaluate RPDR on two long-tail retrieval benchmarks, PopQA and EntityQuestion, demonstrating substantial improvements over existing retrievers like BM25 and Contriver, especially on extremely long-tail categories.

• Reinforced Fast Weights with Next-Sequence Prediction

  Hee Seung Hwang, Xindi Wu, Sanghyuk Chun, Olga Russakovsky · Feb 18, 2026

  Automatic Metrics

  Fast weight architectures offer a promising alternative to attention-based transformers for long-context modeling by maintaining constant memory overhead regardless of context length.

• jina-embeddings-v5-text: Task-Targeted Embedding Distillation

  Mohammad Kalim Akram, Saba Sturua, Nastia Havriushenko, Quentin Herreros, Michael Günther · Feb 17, 2026

  Automatic Metrics

  Benchmark scores for the resulting models, jina-embeddings-v5-text-small and jina-embeddings-v5-text-nano, exceed or match the state-of-the-art for models of similar size.

• NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

  Rong Fu, Yang Li, Zeyu Zhang, Jiekai Wu, Yaohua Liu · Feb 17, 2026

  Automatic Metrics

  Empirical results on standard KGQA benchmarks show that NeuroSymActive attains strong answer accuracy while reducing the number of expensive graph lookups and model calls compared to common retrieval-augmented baselines.

• Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory

  Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li · Feb 17, 2026

  Automatic Metrics

  Mnemis achieves state-of-the-art performance across all compared methods on long-term memory benchmarks, scoring 93.9 on LoCoMo and 91.6 on LongMemEval-S using GPT-4.1-mini.

• ScrapeGraphAI-100k: A Large-Scale Dataset for LLM-Based Web Information Extraction

  William Brach, Francesco Zuppichini, Marco Vinciguerra, Lorenzo Padoan · Feb 16, 2026

  Automatic Metrics

  ScrapeGraphAI-100k enables fine-tuning small models, benchmarking structured extraction, and studying schema induction for web IR indexing, and is publicly available on HuggingFace.

• Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness

  Pietro Bernardelle, Stefano Civelli, Kevin Roitero, Gianluca Demartini · Feb 15, 2026

  Automatic Metrics

  Large language models (LLMs) show strong reasoning abilities across diverse tasks, yet their performance on extended contexts remains inconsistent.

• OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering

  Yifan Zhu, Xinyu Mu, Tao Feng, Zhonghong Ou, Yuning Gong · Feb 3, 2026

  Automatic Metrics Tool Use

  To address these issues, we propose OmniRAG-Agent, an agentic omnimodal QA method for budgeted long audio-video reasoning.

• Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation

  Zhanghao Hu, Qinglin Zhu, Hanqi Yan, Yulan He, Lin Gui · Feb 2, 2026

  Automatic Metrics

  Agent memory systems often adopt the standard Retrieval-Augmented Generation (RAG) pipeline, yet its underlying assumptions differ in this setting.

• Symphonym: Universal Phonetic Embeddings for Cross-Script Name Matching

  Stephen Gadd · Jan 11, 2026

  Automatic Metrics

  Linking names across historical sources, languages, and writing systems remains a fundamental challenge in digital humanities and geographic information retrieval.

• Neurosymbolic Retrievers for Retrieval-augmented Generation

  Yash Saxena, Manas Gaur · Jan 8, 2026

  Automatic Metrics

  Retrieval Augmented Generation (RAG) has made significant strides in overcoming key limitations of large language models, such as hallucination, lack of contextual grounding, and issues with transparency.

• Embedding Retrofitting: Data Engineering for better RAG

  Anantha Sharma · Jan 6, 2026

  Automatic Metrics

  Embedding retrofitting adjusts pre-trained word vectors using knowledge graph constraints to improve domain-specific retrieval.

• Fast-weight Product Key Memory

  Tianyu Zhao, Llion Jones · Jan 2, 2026

  Automatic Metrics

  Notably, in Needle-in-a-Haystack evaluations, FwPKM generalizes to 128K-token contexts despite being trained on only 4K-token sequences.

• Bridging Symbolic Control and Neural Reasoning in LLM Agents: Structured Cognitive Loop with a Governance Layer

  Myung Ho Kim · Nov 21, 2025

  Automatic Metrics Long Horizon

  Large language model agents suffer from fundamental architectural problems: entangled reasoning and execution, memory volatility, and uncontrolled action sequences.

• Multimodal Peer Review Simulation with Actionable To-Do Recommendations for Community-Aware Manuscript Revisions

  Mengze Hong, Di Jiang, Weiwei Zhao, Yawen Li, Yihang Wang · Nov 14, 2025

  Simulation Env

  Experimental results highlight the effectiveness of the proposed system in generating more comprehensive and useful reviews aligned with expert standards, surpassing ablated baselines and advancing transparent, human-centered scholarly assi

• Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs

  Mohammad Tavakoli, Alireza Salemi, Carrie Ye, Mohamed Abdalla, Hamed Zamani · Oct 31, 2025

  Automatic Metrics

  Evaluating the abilities of large language models (LLMs) for tasks that require long-term memory and thus long-context reasoning, for example in conversational settings, is hampered by the existing benchmarks, which often lack narrative coh

• RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA

  Ruiyi Yang, Hao Xue, Imran Razzak, Hakim Hacid, Flora D. Salim · Oct 23, 2025

  Automatic Metrics Long Horizon

  A Head Agent provides guidance that leads retrieval, while an Iteration Agent selects and expands HSeq via structure-respecting actions (e.g., parent/child hops, table row/column neighbors, KG relations); Finally the head agent composes can

• MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning

  Xingyu Tan, Xiaoyang Wang, Qing Liu, Xiwei Xu, Xin Yuan · Oct 15, 2025

  Automatic Metrics

  Comprehensive experiments on multiple temporal QA benchmarks show that MemoTime achieves overall state-of-the-art results, outperforming the strong baseline by up to 24.0%.

• Embedding-Based Context-Aware Reranker

  Ye Yuan, Mohammad Amin Shabani, Siqi Liu · Oct 15, 2025

  Automatic Metrics

  We evaluate EBCAR against SOTA rerankers on the ConTEB benchmark, demonstrating its effectiveness for information retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.

• PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation

  Xiangjun Zai, Xingyu Tan, Xiaoyang Wang, Qing Liu, Xiwei Xu · Oct 14, 2025

  Automatic Metrics Long Horizon

  Experiments across multiple domains demonstrate that PRoH achieves state-of-the-art performance, surpassing the prior SOTA model HyperGraphRAG by an average of 19.73% in F1 and 8.41% in Generation Evaluation (G-E) score, while maintaining s

• Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents

  Yaorui Shi, Yuxin Chen, Siyuan Wang, Sihang Li, Hengxing Cai · Sep 27, 2025

  Automatic Metrics

  To tackle these challenges, we present ReMemR1, which integrates the mechanism of memory retrieval into the memory update process, enabling the agent to selectively callback historical memories for non-linear reasoning.

• ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation

  Jiho Kim, Junseong Choi, Woosog Chay, Daeun Kyung, Yeonsu Kwon · Sep 26, 2025

  Simulation Env

  In our simulation environment, a user agent with a rich persona interacts with the assistant, providing ratings on how well each suggestion aligns with its preferences and context.

• Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Models

  Yunqing Liu, Nan Zhang, Zhiming Tan · Sep 1, 2025

  Automatic Metrics Long Horizon

  We additionally contribute a CAD dataset with human preference annotations.

• Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning

  Dayoon Ko, Jihyuk Kim, Haeju Park, Sohyeon Kim, Dahyun Lee · Aug 26, 2025

  Automatic Metrics Long Horizon

  Large reasoning models (LRMs) combined with retrieval-augmented generation (RAG) have enabled deep research agents capable of multi-step reasoning with external knowledge retrieval.

• PersonalAI: A Systematic Comparison of Knowledge Graph Storage and Retrieval Approaches for Personalized LLM agents

  Mikhail Menschikov, Dmitry Evseev, Victoria Dochkina, Ruslan Kostoev, Ilia Perepechkin · Jun 20, 2025

  Automatic Metrics

  We evaluate our system on three benchmarks: TriviaQA, HotpotQA, DiaASQ and demonstrate that different memory and retrieval configurations yield optimal performance depending on the task.

• Probabilistic distances-based hallucination detection in LLMs with RAG

  Rodion Oblovatny, Alexandra Kuleshova, Konstantin Polev, Alexey Zaytsev · Jun 11, 2025

  Automatic Metrics

  Detecting hallucinations in large language models (LLMs) is critical for their safety in many applications.

• Structure-Augmented Reasoning Generation

  Jash Rajesh Parekh, Pengcheng Jiang, Jiawei Han · Jun 10, 2025

  Automatic Metrics

  Extensive experiments on open-domain QA benchmarks and specialized reasoning datasets in finance and medicine demonstrate that SARG significantly outperforms state-of-the-art flat-context RAG baselines in both factual accuracy and reasoning

• When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation

  Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, Shengyuan Chen, Zijin Hong · Jun 6, 2025

  Automatic Metrics

  To address this, we propose GraphRAG-Bench, a comprehensive benchmark designed to evaluate GraphRAG models onboth hierarchical knowledge retrieval and deep contextual reasoning.

• Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation

  Arthur S. Bianchessi, Yasmin C. Aguirre, Rodrigo C. Barros, Lucas S. Kupssinskü · May 28, 2025

  Automatic Metrics

  However, existing PE methods lack theoretical clarity and rely on limited evaluation metrics to substantiate their extrapolation claims.

• Can LLMs Simulate Human Behavioral Variability? A Case Study in the Phonemic Fluency Task

  Mengyang Qiu, Zoe Brisebois, Siena Sun · May 22, 2025

  Simulation Env

  Large language models (LLMs) are increasingly explored as substitutes for human participants in cognitive tasks, but their ability to simulate human behavioral variability remains unclear.

• Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition

  Zheng Hui, Xiaokai Wei, Yexi Jiang, Kevin Gao, Chen Wang · Apr 26, 2025

  Automatic Metrics Multi Agent

  These domains typically involve fixed content and passive consumption, where user preferences can be matched by genre or theme.

• Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning

  Yuehan Qin, Shawn Li, Yi Nian, Xinyan Velocity Yu, Yue Zhao · Apr 8, 2025

  Automatic Metrics

  Large language models (LLMs) have shown substantial capacity for generating fluent, contextually appropriate responses.

Other Benchmark Hubs