Skip to content
OpenTrain AIFor AI Companies
← Back to explorer

HFEPX Benchmark Hub

AIME or HotpotQA or MMLU Benchmark Papers

Updated from current HFEPX corpus (2026-09-07). This page tracks 60 papers reporting AIME or HotpotQA or MMLU benchmark evidence, with protocol and metric context for comparison.

Papers: 60 Last published: Aug 31, 2026 Global RSS

Researcher Quick Triage

Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: High .

High-Signal Coverage

100.0%

60 / 60 sampled papers are not low-signal flagged.

Replication-Ready Set

41

Papers with explicit benchmark + metric + eval mode fields.

Quality Controls

3.3%

2 papers report calibration/adjudication/IAA controls.

  • 60 papers explicitly name benchmark datasets in the sampled set.
  • 44 papers report at least one metric term in metadata extraction.
  • Start with the ranked shortlist below before reading all papers.

Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.

Why This Matters (Expanded)

Why This Matters For Eval Research

  • Use this page to compare AIME or HotpotQA or MMLU papers by evaluation mode, metric, and evidence quality before reusing reported results.
Protocol Notes (Expanded)

Protocol Takeaways

  • AIME or HotpotQA or MMLU papers are often paired with automatic_metrics, llm_as_judge.

Benchmark Interpretation

  • MMLU: 26 papers
  • HotpotQA: 18 papers
  • AIME: 16 papers
  • GSM8K: 11 papers

Metric Interpretation

  • accuracy: 19 papers
  • cost: 14 papers
  • latency: 8 papers
  • f1: 7 papers

Start Here (Benchmark-Matched First 6)

Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.

Protocol Matrix (Top 10)

Compare protocol ingredients quickly before deep-reading full papers.

Paper Eval Modes Human Feedback Metrics Quality Controls
Decomposing Wrong-Consensus Agreement in LLM Self-Consistency

Aug 19, 2026

Automatic Metrics Pairwise Preference Accuracy Not reported
TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

Aug 27, 2026

Automatic Metrics Pairwise Preference Spearman Not reported
Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

Aug 21, 2026

Automatic Metrics Demonstrations Accuracy, Latency Not reported
Will Scaling Improve Social Simulation with LLMs?

Jul 2, 2026

Automatic Metrics, Simulation Env Not reported Accuracy Calibration
Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

Jun 24, 2026

Automatic Metrics Pairwise Preference Accuracy, Pass@64 Not reported
CAR: Query-Guided Confidence-Aware Reranking for Retrieval-Augmented Generation

May 6, 2026

Automatic Metrics Pairwise Preference F1, Ndcg Not reported
Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking

Apr 13, 2026

Llm As Judge Demonstrations Precision, Agreement Not reported
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models

Aug 21, 2026

Automatic Metrics Not reported Accuracy, Success rate Not reported
Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization

Aug 20, 2026

Automatic Metrics Not reported Accuracy Not reported
From Retrieved Context to Runtime Control: Adaptive Compression for Edge-based RAG

Aug 20, 2026

Not reported Not reported Latency Not reported
Researcher Workflow (Detailed)

Checklist

  • Gap: Human feedback

    Human feedback is present in 9 of 60 papers.

  • Gap: Quality controls

    Quality controls is present in 2 of 60 papers.

  • Strong: Benchmarks

    Benchmarks is present in 60 of 60 papers.

  • Strong: Metrics

    Metrics is present in 44 of 60 papers.

  • Gap: Known rater population

    Known rater population is present in 6 of 60 papers.

  • Gap: Known annotation unit

    Known annotation unit is present in 12 of 60 papers.

Strengths

  • Benchmarks is present in 60 of 60 papers.
  • Metrics is present in 44 of 60 papers.
  • Agentic evaluation is present in 12 of 60 papers.

Known Gaps

  • Human feedback is present in 9 of 60 papers.
  • Quality controls is present in 2 of 60 papers.
  • Known rater population is present in 6 of 60 papers.

Suggested Next Analyses

  • Review the most recent AIME or HotpotQA or MMLU papers first, then compare reported metrics and quality-control context before treating results as comparable.

Recommended Queries

Known Limitations
  • This synthetic persisted page is generated from extraction data because the cached benchmark payload was missing for either-aime-or-hotpotqa-or-mmlu.
Research Utility Snapshot (Detailed)

Evaluation Modes

  • Automatic Metrics (40)
  • Llm As Judge (3)
  • Simulation Env (2)

Human Feedback Mix

  • None (51)
  • Pairwise Preference (7)
  • Demonstrations (2)

Top Benchmarks

  • MMLU (26)
  • HotpotQA (18)
  • AIME (16)
  • GSM8K (11)

Top Metrics

  • Accuracy (19)
  • Cost (14)
  • Latency (8)
  • F1 (7)

Top Papers On This Benchmark

Related Benchmark Hubs