Retrieval + Coding Benchmark Papers

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 36 papers for Retrieval + Coding Benchmark Papers. Dominant protocol signals include automatic metrics, simulation environments, human evaluation, with frequent benchmark focus on Retrieval, ALFWorld and metric focus on accuracy, coherence. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Top Papers On This Benchmark

• AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning

  Yutong Wang, Siyuan Xiong, Xuebo Liu, Wenkang Zhou, Liang Ding · Feb 26, 2026

  Automatic Metrics Multi Agent

  While Multi-Agent Systems (MAS) excel in complex reasoning, they suffer from the cascading impact of erroneous information generated by individual participants.

• MoDora: Tree-Based Semi-Structured Document Analysis System

  Bangrui Xu, Qihang Yao, Zirui Tang, Xuanhe Zhou, Yeye He · Feb 26, 2026

  Automatic Metrics

  Semi-structured documents integrate diverse interleaved data elements (e.g., tables, charts, hierarchical paragraphs) arranged in various and often irregular layouts.

• Vectorizing the Trie: Efficient Constrained Decoding for LLM-based Generative Retrieval on Accelerators

  Zhengyang Su, Isay Katsman, Yueqi Wang, Ruining He, Lukasz Heldt · Feb 26, 2026

  Automatic Metrics

  In addition, evaluation on academic benchmarks demonstrates that STATIC can considerably improve cold-start performance for generative retrieval.

• Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance

  Weida Liang, Yiyou Sun, Shuyuan Nan, Chuang Li, Dawn Song · Feb 26, 2026

  Automatic Metrics

  Through a controlled analysis of paired human-written and model-generated solutions, we identify a systematic dissociation between usage and executability: human- and model-derived strategies differ in structured, domain-dependent ways, lea

• DySCO: Dynamic Attention-Scaling Decoding for Long-Context LMs

  Xi Ye, Wuwei Zhang, Fangcong Yin, Howard Yen, Danqi Chen · Feb 25, 2026

  Automatic Metrics

  Across multiple instruction-tuned and reasoning models, DySCO consistently improves performance on challenging long-context reasoning benchmarks, yielding relative gains of up to 25% on MRCR and LongBenchV2 at 128K context length with modes

• Structurally Aligned Subtask-Level Memory for Software Engineering Agents

  Kangning Shen, Jingyuan Zhang, Chenxi Sun, Wencong Zeng, Yang Yue · Feb 25, 2026

  Automatic Metrics Long Horizon

  Large Language Models (LLMs) have demonstrated significant potential as autonomous software engineering (SWE) agents.

• Multi-Vector Index Compression in Any Modality

  Hanxiang Qin, Alexander Martin, Rohan Jha, Chunsheng Zuo, Reno Kriz · Feb 24, 2026

  Automatic Metrics

  We study efficient multi-vector retrieval for late interaction in any modality.

• A Benchmark for Deep Information Synthesis

  Debjit Paul, Daniel Murphy, Milan Gritta, Ronald Cardenas, Victor Prokhorov · Feb 24, 2026

  Human EvalAutomatic Metrics Tool Use

  Large language model (LLM)-based agents are increasingly used to solve complex tasks involving tool use, such as web browsing, code execution, and data analysis.

• Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems

  Mukul Chhabra, Luigi Medrano, Arush Verma · Feb 23, 2026

  Automatic Metrics

  Enterprise Retrieval-Augmented Generation (RAG) assistants operate in multi-turn, case-based workflows such as technical support and IT operations, where evaluation must reflect operational constraints, structured identifiers (e.g., error c

• Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection

  Raihan Tanvir, Md. Golam Rabiul Alam · Feb 22, 2026

  Automatic Metrics

  Hateful content on social media increasingly appears as multimodal memes that combine images and text to convey harmful narratives.

• Facet-Level Persona Control by Trait-Activated Routing with Contrastive SAE for Role-Playing LLMs

  Wenqiu Tang, Zhen Wan, Takahiro Komamizu, Ichiro Ide · Feb 22, 2026

  Automatic Metrics

  Personality control in Role-Playing Agents (RPAs) is commonly achieved via training-free methods that inject persona descriptions and memory through prompts or retrieval-augmented generation, or via supervised fine-tuning (SFT) on persona-s

• AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG

  Qijie You, Wenkai Yu, Wentao Zhang · Feb 22, 2026

  Automatic Metrics Long Horizon

  With the rapid advancement of agent-based methods in recent years, Agentic RAG has undoubtedly become an important research direction.

• Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

  Amine Kobeissi, Philippe Langlais · Feb 20, 2026

  Automatic Metrics

  Retrieval-augmented generation is increasingly used for financial question answering over long regulatory filings, yet reliability depends on retrieving the exact context needed to justify answers in high stakes settings.

• QueryPlot: Generating Geological Evidence Layers using Natural Language Queries for Mineral Exploration

  Meng Ye, Xiao Lin, Georgina Lukoczki, Graham W. Lederer, Yi Yao · Feb 19, 2026

  Automatic Metrics

  Mineral prospectivity mapping requires synthesizing heterogeneous geological knowledge, including textual deposit models and geospatial datasets, to identify regions likely to host specific mineral deposit types.

• WebFAQ 2.0: A Multilingual QA Dataset with Mined Hard Negatives for Dense Retrieval

  Michael Dinzinger, Laura Caspari, Ali Salman, Irvin Topi, Jelena Mitrović · Feb 19, 2026

  Automatic Metrics

  We introduce WebFAQ 2.0, a new version of the WebFAQ dataset, containing 198 million FAQ-based natural question-answer pairs across 108 languages.

• From Labor to Collaboration: A Methodological Experiment Using AI Agents to Augment Research Perspectives in Taiwan's Humanities and Social Sciences

  Yi-Chih Huang · Feb 19, 2026

  Automatic Metrics

  Generative AI is reshaping knowledge work, yet existing research focuses predominantly on software engineering and the natural sciences, with limited methodological exploration for the humanities and social sciences.

• Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents

  Wenxuan Ding, Nicholas Tomlin, Greg Durrett · Feb 18, 2026

  Simulation Env

  Each problem has latent environment state that can be reasoned about via a prior which is passed to the LLM agent.

• Quecto-V1: Empirical Analysis of 8-bit Quantized Small Language Models for On-Device Legal Retrieval

  Subrit Dikshit · Feb 18, 2026

  Automatic MetricsSimulation Env

  The rapid proliferation of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP) but has simultaneously created a "resource divide." State-of-the-art legal intelligence systems typically rely on massive parameter

• AIC CTU@AVerImaTeC: dual-retriever RAG for image-text fact checking

  Herbert Ullrich, Jan Drchal · Feb 16, 2026

  Automatic Metrics

  In this paper, we present our 3rd place system in the AVerImaTeC shared task, which combines our last year's retrieval-augmented generation (RAG) pipeline with a reverse image search (RIS) module.

• Seeing to Generalize: How Visual Data Corrects Binding Shortcuts

  Nicolas Buzeta, Felipe del Rio, Cristian Hinostroza, Denis Parra, Hans Lobel · Feb 16, 2026

  Automatic Metrics

  Vision Language Models (VLMs) are designed to extend Large Language Models (LLMs) with visual capabilities, yet in this work we observe a surprising phenomenon: VLMs can outperform their underlying LLMs on purely text-only tasks, particular

• Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answering

  Tao Xu · Feb 15, 2026

  Automatic Metrics

  16.1\% (+14.5pp); on CircuitVQA, a public benchmark (9,315 questions), retrieval ImgR@3 achieves 31.2\% vs.

• Embodied Task Planning via Graph-Informed Action Generation with Large Language Model

  Xiang Li, Ning Yan, Masood Mortazavi · Jan 29, 2026

  Simulation Env Long Horizon

  While Large Language Models (LLMs) have demonstrated strong zero-shot reasoning capabilities, their deployment as embodied agents still faces fundamental challenges in long-horizon planning.

• SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation

  Hanqi Jiang, Junhao Chen, Yi Pan, Ling Chen, Weihang You · Jan 6, 2026

  Automatic Metrics

  While Large Language Models (LLMs) excel at generalized reasoning, standard retrieval-augmented approaches fail to address the disconnected nature of long-term agentic memory.

• OGD4All: A Framework for Accessible Interaction with Geospatial Open Government Data Based on Large Language Models

  Michael Siebenmann, Javier Argota Sánchez-Vaquerizo, Stefan Arisona, Krystian Samp, Luis Gisler · Nov 30, 2025

  Automatic Metrics

  The system combines semantic data retrieval, agentic reasoning for iterative code generation, and secure sandboxed execution that produces verifiable multimodal outputs.

• CLARITY: Contextual Linguistic Adaptation and Accent Retrieval for Dual-Bias Mitigation in Text-to-Speech Generation

  Crystal Min Hui Poon, Pai Chet Ng, Xiaoxiao Miao, Immanuel Jun Kai Loh, Bowen Zhang · Nov 14, 2025

  Automatic Metrics

  Instruction-guided text-to-speech (TTS) research has reached a maturity level where excellent speech generation quality is possible on demand, yet two coupled biases persist in reducing perceived quality: accent bias, where models default t

• Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces

  Shreyas Rajesh, Pavan Holur, Chenda Duan, David Chong, Vwani Roychowdhury · Nov 10, 2025

  Automatic Metrics Long Horizon

  On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}.

• FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark for Evaluating LLMs

  Yan Wang, Keyi Wang, Shanshan Yang, Jaisal Patel, Jeff Zhao · Oct 10, 2025

  Automatic Metrics

  We introduce FinAuditing, a taxonomy-aligned, structure-aware benchmark built from real XBRL filings.

• Finding Diamonds in Conversation Haystacks: A Benchmark for Conversational Data Retrieval

  Yohan Lee, Yongwoo Song, Sangyeop Kim · Oct 3, 2025

  Automatic Metrics

  We present the Conversational Data Retrieval (CDR) benchmark, the first comprehensive test set for evaluating systems that retrieve conversation data for product insights.

• LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?

  Guozhao Mo, Wenliang Zhong, Jiawei Chen, Qianhao Yuan, Xuanang Chen · Aug 3, 2025

  Automatic Metrics Tool Use

  Unfortunately, there is still a large gap between real-world MCP usage and current evaluation: they typically assume single-server settings and directly inject tools into the model's context, bypassing the challenges of large-scale retrieva

• Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation

  Jaechul Roh, Zachary Novack, Yuefeng Peng, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick · Jul 23, 2025

  Automatic Metrics

  Generative AI systems for music and video commonly use text-based filters to prevent regurgitation of copyrighted material.

• Revela: Dense Retriever Learning via Language Modeling

  Fengyu Cai, Tong Chen, Xinran Zhao, Sihao Chen, Hongming Zhang · Jun 19, 2025

  Automatic Metrics

  We evaluate Revela on domain-specific (CoIR), reasoning-intensive (BRIGHT), and general-domain (BEIR) benchmarks across various retriever backbones.

• Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement

  Chenyu Lin, Yilin Wen, Du Su, Hexiang Tan, Fei Sun · Jun 5, 2025

  Automatic Metrics

  Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors.

• Entailed Opinion Matters: Improving the Fact-Checking Performance of Language Models by Relying on their Entailment Ability

  Gaurav Kumar, Ayush Garg, Debajyoti Mazumder, Aditya Kishore, Babu kumar · May 21, 2025

  Automatic Metrics

  Automated fact-checking has been a challenging task for the research community.

• Diffusion Generative Recommendation with Continuous Tokens

  Haohao Qu, Shanru Lin, Yujuan Ding, Yiqi Wang, Wenqi Fan · Apr 16, 2025

  Automatic Metrics

  Specifically, ContRec consists of two key modules: a sigma-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference.

• LLM2CLIP: Powerful Language Model Unlocks Richer Cross-Modality Representation

  Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang · Nov 7, 2024

  Automatic Metrics

  The LLM-enhanced CLIP delivers consistent improvements across a wide range of downstream tasks, including linear-probe classification, zero-shot image-text retrieval with both short and long captions (in English and other languages), zero-s

• Multi-Head RAG: Solving Multi-Aspect Problems with LLMs

  Maciej Besta, Ales Kubicek, Robert Gerstenberger, Marcin Chrapek, Roman Niggli · Jun 7, 2024

  Automatic Metrics

  MRAG integrates seamlessly with existing RAG frameworks and benchmarks.

Other Benchmark Hubs