Benchmark Hub
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this benchmark page. Common evaluation modes: Automatic Metrics. Frequently cited benchmark: DROP. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for DROP Benchmark Papers (Last 60 Days). Dominant protocol signals include automatic metrics, with frequent benchmark focus on DROP, GSM8K and metric focus on accuracy, bleu. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
automatic metrics appears in 100% of papers in this hub.
Evidence: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
DROP is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Stratify by benchmark (DROP vs GSM8K) before comparing methods.
Evidence: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Track metric sensitivity by reporting both accuracy and bleu.
Evidence: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Papers with explicit human feedback
Coverage is a replication risk (0% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is strong (100% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (40% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
1. SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables
Start here for detailed protocol reporting, including rater and quality-control evidence.
2. Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Start here for detailed protocol reporting, including rater and quality-control evidence.
Adds automatic metrics for broader coverage within this hub.
5. Beyond Subtokens: A Rich Character Embedding for Low-resource and Morphologically Complex Languages
Adds automatic metrics for broader coverage within this hub.
6. Online Algorithms with Unreliable Guidance
Adds automatic metrics for broader coverage within this hub.
7. Subgroups of $U(d)$ Induce Natural RNN and Transformer Architectures
Adds automatic metrics for broader coverage within this hub.
8. TFL: Targeted Bit-Flip Attack on Large Language Model
Adds automatic metrics for broader coverage within this hub.
Benchmark Brief
DROP
Coverage: 10 papers (100%)
10 papers (100%) mention DROP.
Examples: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Benchmark Brief
GSM8K
Coverage: 1 papers (10%)
1 papers (10%) mention GSM8K.
Examples: TFL: Targeted Bit-Flip Attack on Large Language Model
Benchmark Brief
MMLU
Coverage: 1 papers (10%)
1 papers (10%) mention MMLU.
Examples: Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads
Metric Brief
accuracy
Coverage: 2 papers (20%)
2 papers (20%) mention accuracy.
Examples: Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models , ABCD: All Biases Come Disguised
Metric Brief
bleu
Coverage: 1 papers (10%)
1 papers (10%) mention bleu.
Metric Brief
f1
Coverage: 1 papers (10%)
1 papers (10%) mention f1.
Examples: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables , Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads , Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Sungho Park, Jueun Kim, Wook-Shin Han · Feb 26, 2026
Yet existing benchmarks are small, manually curated - and therefore error-prone - and contain shallow questions that seldom demand more than two hops or invoke aggregations, grouping, or other advanced analytical operations expressible in n
Shaswat Patel, Vishvesh Trivedi, Yue Han, Yihuai Hong, Eunsol Choi · Feb 25, 2026
Across four multilingual benchmarks (MMLU-ProX, MGSM, MLQA, and XQuaD) and two model families (Qwen-2.5 and Llama-3.1), we demonstrate that masking RTH induces bigger performance drop than masking Retrieval Heads (RH).
Christian Nickel, Laura Schrewe, Florian Mai, Lucie Flek · Feb 25, 2026
Theory of Mind (ToM) refers to an agent's ability to model the internal states of others.
Tangsang Chongbang, Pranesh Pyara Shrestha, Amrit Sarki, Anku Jaiswal · Feb 25, 2026
We first establish highly proficient ASR and NMT components: a Wav2Vec2-XLS-R-300m model achieved a state-of-the-art 2.72% CER on OpenSLR-54, and a multi-stage fine-tuned MarianMT model reached a 28.32 BLEU score on the FLORES-200 benchmark
Felix Schneider, Maria Gogolev, Sven Sickert, Joachim Denzler · Feb 24, 2026
Tokenization and sub-tokenization based models like word2vec, BERT and the GPTs are the state-of-the-art in natural language processing.
Julien Dallot, Yuval Emek, Yuval Gil, Maciej Pacut, Stefan Schmid · Feb 24, 2026
This paper introduces a new model for ML-augmented online decision making, called online algorithms with unreliable guidance (OAG).
Joshua Nunley · Feb 20, 2026
This paper presents a direct framework for sequence models with hidden states on closed subgroups of U(d).
Jingkai Guo, Chaitali Chakrabarti, Deliang Fan · Feb 19, 2026
Large language models (LLMs) are increasingly deployed in safety and security critical applications, raising concerns about their robustness to model parameter fault injection attacks.
Mateusz Nowak, Xavier Cadet, Peter Chin · Feb 19, 2026
Multiple-choice question (MCQ) benchmarks have been a standard evaluation practice for measuring LLMs' ability to reason and answer knowledge-based questions.
Seunghwan Jang, SooJean Han · Jan 6, 2026
Uniform-noise discrete diffusion and flow models (e.g., D3PM, SEDD, UDLM, DFM) generate sequences non-autoregressively by iteratively refining randomly initialized vocabulary tokens through multiple context-dependent replacements.
