Retrieval Benchmark Papers

Research Utility Snapshot

Top Papers On This Benchmark

• DySCO: Dynamic Attention-Scaling Decoding for Long-Context LMs

  Xi Ye, Wuwei Zhang, Fangcong Yin, Howard Yen, Danqi Chen · Feb 25, 2026

  Automatic Metrics

  Across multiple instruction-tuned and reasoning models, DySCO consistently improves performance on challenging long-context reasoning benchmarks, yielding relative gains of up to 25% on MRCR and LongBenchV2 at 128K context length with modes

• Personalized Graph-Empowered Large Language Model for Proactive Information Access

  Chia Cheng Chang, An-Zi Yen, Hen-Hsen Huang, Hsin-Hsi Chen · Feb 25, 2026

  Automatic Metrics

  Since individuals may struggle to recall all life details and often confuse events, establishing a system to assist users in recalling forgotten experiences is essential.

• FewMMBench: A Benchmark for Multimodal Few-Shot Learning

  Mustafa Dogan, Ilker Kesen, Iacer Calixto, Aykut Erdem, Erkut Erdem · Feb 25, 2026

  Automatic Metrics

  In this paper, we introduce FewMMBench, a comprehensive benchmark designed to evaluate MLLMs under few-shot conditions, with a focus on In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting.

• Prompt Architecture Determines Reasoning Quality: A Variable Isolation Study on the Car Wash Problem

  Heejin Jo · Feb 25, 2026

  Automatic Metrics

  Large language models consistently fail the "car wash problem," a viral reasoning benchmark requiring implicit physical constraint inference.

• Structurally Aligned Subtask-Level Memory for Software Engineering Agents

  Kangning Shen, Jingyuan Zhang, Chenxi Sun, Wencong Zeng, Yang Yue · Feb 25, 2026

  Automatic Metrics Long Horizon

  Large Language Models (LLMs) have demonstrated significant potential as autonomous software engineering (SWE) agents.

• Retrieval Challenges in Low-Resource Public Service Information: A Case Study on Food Pantry Access

  Touseef Hasan, Laila Cure, Souvika Sarkar · Feb 25, 2026

  Simulation Env

  We conduct a pilot evaluation study using community-sourced queries to examine system behavior in realistic scenarios.

• Revisiting RAG Retrievers: An Information Theoretic Benchmark

  Wenqing Zheng, Dmitri Kalaev, Noah Fatsi, Daniel Barcklow, Owen Reinert · Feb 25, 2026

  Automatic Metrics

  Existing benchmarks primarily compare entire RAG pipelines or introduce new datasets, providing little guidance on selecting or combining retrievers themselves.

• Revisiting Text Ranking in Deep Research

  Chuan Meng, Litu Ou, Sean MacAvaney, Jeff Dalton · Feb 25, 2026

  Automatic Metrics

  To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it.

• Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG

  Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni · Feb 24, 2026

  Automatic Metrics

  Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components.

• Multi-Vector Index Compression in Any Modality

  Hanxiang Qin, Alexander Martin, Rohan Jha, Chunsheng Zuo, Reno Kriz · Feb 24, 2026

  Automatic Metrics

  We study efficient multi-vector retrieval for late interaction in any modality.

• A Benchmark for Deep Information Synthesis

  Debjit Paul, Daniel Murphy, Milan Gritta, Ronald Cardenas, Victor Prokhorov · Feb 24, 2026

  Human EvalAutomatic Metrics Tool Use

  Large language model (LLM)-based agents are increasingly used to solve complex tasks involving tool use, such as web browsing, code execution, and data analysis.

• HELP: HyperNode Expansion and Logical Path-Guided Evidence Localization for Accurate and Efficient GraphRAG

  Yuqi Huang, Ning Liao, Kai Yang, Anning Hu, Shengchao Hu · Feb 24, 2026

  Automatic Metrics

  Extensive experiments demonstrate that HELP achieves competitive performance across multiple simple and multi-hop QA benchmarks and up to a 28.8$\times$ speedup over leading Graph-based RAG baselines.

• E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications

  Jiwoo Kang, Yeon-Chang Lee · Feb 24, 2026

  Automatic Metrics

  Multimodal recommender systems (MMRSs) enhance collaborative filtering by leveraging item-side modalities, but their reliance on a fixed set of modalities and task-specific objectives limits both modality extensibility and task generalizati

• RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition

  Kun Ran, Marwah Alaofi, Danula Hettiachchi, Chenglong Ma, Khoi Nguyen Dinh Anh · Feb 24, 2026

  Automatic Metrics

  R2RAG won the Best Dynamic Evaluation award in the Open Source category, demonstrating high effectiveness with careful design and efficient use of resources.

• Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems

  Mukul Chhabra, Luigi Medrano, Arush Verma · Feb 23, 2026

  Automatic Metrics

  Enterprise Retrieval-Augmented Generation (RAG) assistants operate in multi-turn, case-based workflows such as technical support and IT operations, where evaluation must reflect operational constraints, structured identifiers (e.g., error c

• InterviewSim: A Scalable Framework for Interview-Grounded Personality Simulation

  Yu Li, Pranav Narayanan Venkit, Yada Pruksachatkun, Chien-Sheng Wu · Feb 23, 2026

  Simulation Env

  Existing evaluation approaches rely on demographic surveys, personality questionnaires, or short AI-led interviews as proxies, but lack direct assessment against what individuals actually said.

• KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration

  Mohammad Amanlou, Erfan Shafiee Moghaddam, Yasaman Amou Jafari, Mahdi Noori, Farhan Farsi · Feb 23, 2026

  Automatic Metrics

  Results show that KNIGHT enables token- and cost-efficient generation from a reusable graph representation, achieves high quality across these criteria, and yields model rankings aligned with MMLU-style benchmarks, while supporting topic-sp

• How Retrieved Context Shapes Internal Representations in RAG

  Samuel Yeh, Sharon Li · Feb 23, 2026

  Automatic Metrics

  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial.

• Structured Prompt Language: Declarative Context Management for LLMs

  Wen G. Gong · Feb 23, 2026

  Automatic Metrics

  SPL-flow extends SPL into resilient agentic pipelines with a three-tier provider fallback strategy (Ollama -> OpenRouter -> self-healing retry) fully transparent to the .spl script.

• Cross-lingual Matryoshka Representation Learning across Speech and Text

  Yaya Sy, Dioula Doucouré, Christophe Cerisara, Irina Illina · Feb 23, 2026

  Automatic Metrics

  We introduce large-scale data curation pipelines and new benchmarks, compare modeling strategies, and show that modality fusion within a frozen text Matryoshka model performs best.

• Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

  Yibo Yan, Jiahao Huo, Guanbo Feng, Mingdong Ou, Yi Cao · Feb 23, 2026

  Automatic Metrics

  We begin by examining the benchmark landscape, and subsequently dive into the methodological evolution, categorizing approaches into three primary aspects: multimodal embedding models, multimodal reranker models, and the integration of Retr

• Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework

  Yibo Yan, Mingdong Ou, Yi Cao, Xin Zou, Jiahao Huo · Feb 23, 2026

  Automatic Metrics

  Visual Document Retrieval (VDR), which aims to retrieve relevant pages within vast corpora of visually-rich documents, is of significance in current multimodal retrieval applications.

• How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1

  Yinuo Xu, Shuo Lu, Jianjie Cheng, Meng Wang, Qianlong Xie · Feb 23, 2026

  Automatic Metrics

  Deep Research agents tackle knowledge-intensive tasks through multi-round retrieval and decision-oriented generation.

• Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering

  Maryam Amirizaniani, Alireza Salemi, Hamed Zamani · Feb 22, 2026

  Automatic Metrics Long Horizon

  Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context.

• Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

  Lichang Song, Ting Long, Yi Chang · Feb 21, 2026

  Automatic Metrics Multi Agent

  To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-ma

• Contradiction to Consensus: Dual Perspective, Multi Source Retrieval Based Claim Verification with Source Level Disagreement using LLM

  Md Badsha Biswas, Ozlem Uzuner · Feb 21, 2026

  Automatic Metrics

  Through extensive evaluation on four benchmark datasets with five LLMs, we show that knowledge aggregation not only improves claim verification but also reveals differences in source-specific reasoning.

• Diagnosing LLM Reranker Behavior Under Fixed Evidence Pools

  Baris Arat, Emre Sefer · Feb 20, 2026

  Automatic Metrics

  Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever.

• RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

  Deniz Qian, Hung-Ting Chen, Eunsol Choi · Feb 20, 2026

  Automatic Metrics

  Our method outperforms baselines, including agentic search approaches, achieving at least 10% relative and 3% absolute gain in complete recall percentage on a multi-answer retrieval dataset (QAMPARI).

• Validating Political Position Predictions of Arguments

  Jordan Robinson, Angus R. Williams, Katie Atkinson, Anthony G. Cohn · Feb 20, 2026

  Human Eval

  Real-world knowledge representation often requires capturing subjective, continuous attributes -- such as political positions -- that conflict with pairwise validation, the widely accepted gold standard for human evaluation.

• Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

  Amine Kobeissi, Philippe Langlais · Feb 20, 2026

  Automatic Metrics

  Retrieval-augmented generation is increasingly used for financial question answering over long regulatory filings, yet reliability depends on retrieving the exact context needed to justify answers in high stakes settings.

• CUICurate: A GraphRAG-based Framework for Automated Clinical Concept Curation for NLP applications

  Victoria Blake, Mathew Miller, Jamie Novak, Sze-yuan Ooi, Blanca Gallego · Feb 20, 2026

  Automatic Metrics

  The framework was evaluated on five lexically heterogeneous clinical concepts against a manually curated benchmark and gold-standard concept sets.

• Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering

  Jash Rajesh Parekh, Wonbin Kweon, Joey Chan, Rezarta Islamaj, Robert Leaman · Feb 20, 2026

  Automatic Metrics

  Existing benchmarks do not evaluate such conditional reasoning, and retrieval-augmented or graph-based methods lack explicit mechanisms to ensure that retrieved knowledge is applicable to given context.

• Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions

  Raymond Li, Amirhossein Abaskohi, Chuyuan Li, Gabriel Murray, Giuseppe Carenini · Feb 20, 2026

  Automatic Metrics

  Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity.

• QueryPlot: Generating Geological Evidence Layers using Natural Language Queries for Mineral Exploration

  Meng Ye, Xiao Lin, Georgina Lukoczki, Graham W. Lederer, Yi Yao · Feb 19, 2026

  Automatic Metrics

  Mineral prospectivity mapping requires synthesizing heterogeneous geological knowledge, including textual deposit models and geospatial datasets, to identify regions likely to host specific mineral deposit types.

• Unmasking the Factual-Conceptual Gap in Persian Language Models

  Alireza Sakhaeirad, Ali Ma'manpoosh, Arshia Hemmat · Feb 19, 2026

  Automatic Metrics

  While emerging Persian NLP benchmarks have expanded into pragmatics and politeness, they rarely distinguish between memorized cultural facts and the ability to reason about implicit social norms.

• PEACE 2.0: Grounded Explanations and Counter-Speech for Combating Hate Expressions

  Greta Damo, Stéphane Petiot, Elena Cabrio, Serena Villata · Feb 19, 2026

  Automatic Metrics

  The increasing volume of hate speech on online platforms poses significant societal challenges.

• The Role of the Availability Heuristic in Multiple-Choice Answering Behaviour

  Leonidas Zotos, Hedderik van Rijn, Malvina Nissim · Feb 19, 2026

  Automatic Metrics

  When students are unsure of the correct answer to a multiple-choice question (MCQ), guessing is common practice.

• RPDR: A Round-trip Prediction-Based Data Augmentation Framework for Long-Tail Question Answering

  Yiming Zhang, Siyue Zhang, Junbo Zhao, Chen Zhao · Feb 19, 2026

  Automatic Metrics

  We evaluate RPDR on two long-tail retrieval benchmarks, PopQA and EntityQuestion, demonstrating substantial improvements over existing retrievers like BM25 and Contriver, especially on extremely long-tail categories.

• WebFAQ 2.0: A Multilingual QA Dataset with Mined Hard Negatives for Dense Retrieval

  Michael Dinzinger, Laura Caspari, Ali Salman, Irvin Topi, Jelena Mitrović · Feb 19, 2026

  Automatic Metrics

  We introduce WebFAQ 2.0, a new version of the WebFAQ dataset, containing 198 million FAQ-based natural question-answer pairs across 108 languages.

• From Labor to Collaboration: A Methodological Experiment Using AI Agents to Augment Research Perspectives in Taiwan's Humanities and Social Sciences

  Yi-Chih Huang · Feb 19, 2026

  Automatic Metrics

  Generative AI is reshaping knowledge work, yet existing research focuses predominantly on software engineering and the natural sciences, with limited methodological exploration for the humanities and social sciences.

• Reinforced Fast Weights with Next-Sequence Prediction

  Hee Seung Hwang, Xindi Wu, Sanghyuk Chun, Olga Russakovsky · Feb 18, 2026

  Automatic Metrics

  Fast weight architectures offer a promising alternative to attention-based transformers for long-context modeling by maintaining constant memory overhead regardless of context length.

• Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents

  Wenxuan Ding, Nicholas Tomlin, Greg Durrett · Feb 18, 2026

  Simulation Env

  Each problem has latent environment state that can be reasoned about via a prior which is passed to the LLM agent.

• Quecto-V1: Empirical Analysis of 8-bit Quantized Small Language Models for On-Device Legal Retrieval

  Subrit Dikshit · Feb 18, 2026

  Automatic MetricsSimulation Env

  The rapid proliferation of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP) but has simultaneously created a "resource divide." State-of-the-art legal intelligence systems typically rely on massive parameter

• Evidence-Grounded Subspecialty Reasoning: Evaluating a Curated Clinical Intelligence Layer on the 2025 Endocrinology Board-Style Examination

  Amir Hosseinian, MohammadReza Zare Shahneh, Umer Mansoor, Gilbert Szeto, Kirill Karlin · Feb 17, 2026

  Automatic Metrics

  Results: Mirror achieved 87.5% accuracy (105/120; 95% CI: 80.4-92.3%), exceeding a human reference of 62.3% and frontier LLMs including GPT-5.2 (74.6%), GPT-5 (74.0%), and Gemini-3-Pro (69.8%).

• RUVA: Personalized Transparent On-Device Graph Reasoning

  Gabriele Conte, Alessio Mattiace, Gianni Carmosino, Potito Aghilar, Giovanni Servedio · Feb 17, 2026

  Automatic Metrics

  We propose Ruva, the first "Glass Box" architecture designed for Human-in-the-Loop Memory Curation.

• jina-embeddings-v5-text: Task-Targeted Embedding Distillation

  Mohammad Kalim Akram, Saba Sturua, Nastia Havriushenko, Quentin Herreros, Michael Günther · Feb 17, 2026

  Automatic Metrics

  Benchmark scores for the resulting models, jina-embeddings-v5-text-small and jina-embeddings-v5-text-nano, exceed or match the state-of-the-art for models of similar size.

• NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering

  Rong Fu, Yang Li, Zeyu Zhang, Jiekai Wu, Yaohua Liu · Feb 17, 2026

  Automatic Metrics

  Empirical results on standard KGQA benchmarks show that NeuroSymActive attains strong answer accuracy while reducing the number of expensive graph lookups and model calls compared to common retrieval-augmented baselines.

• Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory

  Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li · Feb 17, 2026

  Automatic Metrics

  Mnemis achieves state-of-the-art performance across all compared methods on long-term memory benchmarks, scoring 93.9 on LoCoMo and 91.6 on LongMemEval-S using GPT-4.1-mini.

• AIC CTU@AVerImaTeC: dual-retriever RAG for image-text fact checking

  Herbert Ullrich, Jan Drchal · Feb 16, 2026

  Automatic Metrics

  In this paper, we present our 3rd place system in the AVerImaTeC shared task, which combines our last year's retrieval-augmented generation (RAG) pipeline with a reverse image search (RIS) module.

• ScrapeGraphAI-100k: A Large-Scale Dataset for LLM-Based Web Information Extraction

  William Brach, Francesco Zuppichini, Marco Vinciguerra, Lorenzo Padoan · Feb 16, 2026

  Automatic Metrics

  ScrapeGraphAI-100k enables fine-tuning small models, benchmarking structured extraction, and studying schema induction for web IR indexing, and is publicly available on HuggingFace.

• Seeing to Generalize: How Visual Data Corrects Binding Shortcuts

  Nicolas Buzeta, Felipe del Rio, Cristian Hinostroza, Denis Parra, Hans Lobel · Feb 16, 2026

  Automatic Metrics

  Vision Language Models (VLMs) are designed to extend Large Language Models (LLMs) with visual capabilities, yet in this work we observe a surprising phenomenon: VLMs can outperform their underlying LLMs on purely text-only tasks, particular

• Text Style Transfer with Parameter-efficient LLM Finetuning and Round-trip Translation

  Ruoxi Liu, Philipp Koehn · Feb 16, 2026

  Automatic Metrics

  This paper proposes a novel method for Text Style Transfer (TST) based on parameter-efficient fine-tuning of Large Language Models (LLMs).

• Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness

  Pietro Bernardelle, Stefano Civelli, Kevin Roitero, Gianluca Demartini · Feb 15, 2026

  Automatic Metrics

  Large language models (LLMs) show strong reasoning abilities across diverse tasks, yet their performance on extended contexts remains inconsistent.

• OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering

  Yifan Zhu, Xinyu Mu, Tao Feng, Zhonghong Ou, Yuning Gong · Feb 3, 2026

  Automatic Metrics Tool Use

  To address these issues, we propose OmniRAG-Agent, an agentic omnimodal QA method for budgeted long audio-video reasoning.

• Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation

  Zhanghao Hu, Qinglin Zhu, Hanqi Yan, Yulan He, Lin Gui · Feb 2, 2026

  Automatic Metrics

  Agent memory systems often adopt the standard Retrieval-Augmented Generation (RAG) pipeline, yet its underlying assumptions differ in this setting.

• Embodied Task Planning via Graph-Informed Action Generation with Large Language Model

  Xiang Li, Ning Yan, Masood Mortazavi · Jan 29, 2026

  Simulation Env Long Horizon

  While Large Language Models (LLMs) have demonstrated strong zero-shot reasoning capabilities, their deployment as embodied agents still faces fundamental challenges in long-horizon planning.

• AWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion Speakers

  Prachuryya Kaushik, Ashish Anand · Jan 15, 2026

  Automatic Metrics

  We introduce \textbf{AWED-FiNER}, an open-source collection of agentic tool, web application, and 53 state-of-the-art expert models that provide Fine-grained Named Entity Recognition (FgNER) solutions across 36 languages spoken by more than

• Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG

  David Samuel Setiawan, Raphaël Merx, Jey Han Lau · Jan 15, 2026

  Automatic Metrics

  Qualitative analysis confirms the LLM acts as a robust "safety net," repairing severe failures in zero-shot domains.

• Symphonym: Universal Phonetic Embeddings for Cross-Script Name Matching

  Stephen Gadd · Jan 11, 2026

  Automatic Metrics

  Linking names across historical sources, languages, and writing systems remains a fundamental challenge in digital humanities and geographic information retrieval.

• Neurosymbolic Retrievers for Retrieval-augmented Generation

  Yash Saxena, Manas Gaur · Jan 8, 2026

  Automatic Metrics

  Retrieval Augmented Generation (RAG) has made significant strides in overcoming key limitations of large language models, such as hallucination, lack of contextual grounding, and issues with transparency.

Other Benchmark Hubs