Skip to content
OpenTrain AIFor AI Companies
← Back to explorer

HFEPX Benchmark Hub

AIME or AlpacaEval or MMLU Benchmark Papers

Updated from current HFEPX corpus (2026-10-15). This page tracks 60 papers reporting AIME or AlpacaEval or MMLU benchmark evidence, with protocol and metric context for comparison.

Papers: 60 Last published: Oct 6, 2026 Global RSS

Researcher Quick Triage

Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: High .

High-Signal Coverage

100.0%

60 / 60 sampled papers are not low-signal flagged.

Replication-Ready Set

38

Papers with explicit benchmark + metric + eval mode fields.

Quality Controls

5.0%

3 papers report calibration/adjudication/IAA controls.

  • 60 papers explicitly name benchmark datasets in the sampled set.
  • 41 papers report at least one metric term in metadata extraction.
  • Start with the ranked shortlist below before reading all papers.

Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.

Why This Matters (Expanded)

Why This Matters For Eval Research

  • Use this page to compare AIME or AlpacaEval or MMLU papers by evaluation mode, metric, and evidence quality before reusing reported results.
Protocol Notes (Expanded)

Protocol Takeaways

  • AIME or AlpacaEval or MMLU papers are often paired with automatic_metrics, llm_as_judge.

Benchmark Interpretation

  • MMLU: 39 papers
  • AIME: 19 papers
  • GSM8K: 14 papers
  • MMLU-Pro: 9 papers

Metric Interpretation

  • accuracy: 23 papers
  • cost: 9 papers
  • latency: 6 papers
  • perplexity: 3 papers

Start Here (Benchmark-Matched First 6)

Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.

Protocol Matrix (Top 10)

Compare protocol ingredients quickly before deep-reading full papers.

Paper Eval Modes Human Feedback Metrics Quality Controls
Evaluating and Benchmarking the System One Model Jev

Sep 29, 2026

Automatic Metrics Rubric Rating Accuracy, F1 Not reported
Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable

Sep 29, 2026

Automatic Metrics Pairwise Preference Accuracy Not reported
Decomposing Wrong-Consensus Agreement in LLM Self-Consistency

Aug 19, 2026

Automatic Metrics Pairwise Preference Accuracy Not reported
Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

Aug 21, 2026

Automatic Metrics Demonstrations Accuracy, Latency Not reported
Will Scaling Improve Social Simulation with LLMs?

Jul 2, 2026

Automatic Metrics, Simulation Env Not reported Accuracy Calibration
Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

Jun 24, 2026

Automatic Metrics Pairwise Preference Accuracy, Pass@64 Not reported
Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices

Oct 6, 2026

Automatic Metrics Not reported Accuracy, Cost Not reported
Test-Time Adaptation of Reasoning Strategies with Bayesian Nonparametric Memory

Oct 5, 2026

Automatic Metrics Not reported Cost Not reported
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

Sep 29, 2026

Automatic Metrics Not reported Throughput Not reported
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning

Sep 29, 2026

Automatic Metrics Not reported Accuracy Not reported
Researcher Workflow (Detailed)

Checklist

  • Moderate: Human feedback

    Human feedback is present in 13 of 60 papers.

  • Gap: Quality controls

    Quality controls is present in 3 of 60 papers.

  • Strong: Benchmarks

    Benchmarks is present in 60 of 60 papers.

  • Strong: Metrics

    Metrics is present in 41 of 60 papers.

  • Gap: Known rater population

    Known rater population is present in 11 of 60 papers.

  • Gap: Known annotation unit

    Known annotation unit is present in 8 of 60 papers.

Strengths

  • Benchmarks is present in 60 of 60 papers.
  • Metrics is present in 41 of 60 papers.
  • Agentic evaluation is present in 10 of 60 papers.

Known Gaps

  • Quality controls is present in 3 of 60 papers.
  • Known rater population is present in 11 of 60 papers.
  • Known annotation unit is present in 8 of 60 papers.

Suggested Next Analyses

  • Review the most recent AIME or AlpacaEval or MMLU papers first, then compare reported metrics and quality-control context before treating results as comparable.

Recommended Queries

Known Limitations
  • This synthetic persisted page is generated from extraction data because the cached benchmark payload was missing for either-aime-or-alpacaeval-or-mmlu.
Research Utility Snapshot (Detailed)

Evaluation Modes

  • Automatic Metrics (37)
  • Llm As Judge (3)
  • Simulation Env (2)

Human Feedback Mix

  • None (47)
  • Pairwise Preference (9)
  • Demonstrations (2)
  • Expert Verification (1)

Top Benchmarks

  • MMLU (39)
  • AIME (19)
  • GSM8K (14)
  • MMLU-Pro (9)

Top Metrics

  • Accuracy (23)
  • Cost (9)
  • Latency (6)
  • Perplexity (3)

Top Papers On This Benchmark

Related Benchmark Hubs