HFEPX · Eval paper review
Lingkai Kong, Zijian Wu, Yuzhe Gu, Haiteng Zhao +10 more
Published
Jul 13, 2026
Citations
0
Trust level
Low
Usefulness score
5/100 (Low)
Extraction confidence
45% (Low)
Derived from extracted protocol signals and abstract evidence.
Rater population
Domain Experts
Signals refreshed
Sep 29, 2026
This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.
Use this as background context only. Do not make protocol decisions from this page alone.
All signals on this page are inferred from the abstract only and may be inaccurate. Do not use this page as a primary protocol reference.
Best use
Background context only
Use if you need
A benchmark-and-metrics comparison anchor.
What to verify
Validate the evaluation procedure and quality controls in the full paper before operational use.
Main weakness
This paper looks adjacent to evaluation work, but not like a strong protocol reference.
Treat as adjacent context, not a core eval-method reference.
If you are doing eval pipeline work, start here
Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and often rely on final-answer correctness or coarse judgments, leaving the validity of the reasoning process inadequately assessed. To bridge this gap, we introduce AdvancedMathBench, a benchmark suite designed to evaluate the reasoning capabilities of LLMs on advanced mathematical proofs. Its core generation benchmark, ProverBench, contains 245 problems spanning undergraduate (UG) and doctoral qualifying-exam (QE) levels. To reliably evaluate these proofs, we develop a dedicated automatic verification pipeline that is trained on large-scale expert annotations, produces both correctness verdicts and fine-grained analyses, and exhibits strong agreement with human experts on held-out proof trajectories. We further introduce VerifierBench, consisting of 888 model-generated proof trajectories paired with expert ground truth, to evaluate whether models can correctly judge proof validity and provide sound verification rationales. Experiments show that AdvancedMathBench remains challenging for frontier models. On proof generation, the best-performing model, GPT-5.5-xhigh, achieves only 64.5 and 48.9 on the UG and QE splits, respectively. On proof verification, the best model only attains a Balanced F1 of 65.1. Further analysis reveals a notable mismatch between proof generation and verification capabilities across models.
These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.
None explicit
No explicit feedback protocol extracted.
"Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood."
Automatic Metrics
Includes extracted eval setup.
"Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood."
Not reported
No explicit QC controls found.
"Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood."
Advancedmathbench, Proverbench, Verifierbench
Useful for quick benchmark comparison.
"To bridge this gap, we introduce AdvancedMathBench, a benchmark suite designed to evaluate the reasoning capabilities of LLMs on advanced mathematical proofs."
F1, Agreement
Useful for evaluation criteria comparison.
"To reliably evaluate these proofs, we develop a dedicated automatic verification pipeline that is trained on large-scale expert annotations, produces both correctness verdicts and fine-grained analyses, and exhibits strong agreement with human experts on held-out proof trajectories."
Domain Experts
Helpful for staffing comparability.
"To reliably evaluate these proofs, we develop a dedicated automatic verification pipeline that is trained on large-scale expert annotations, produces both correctness verdicts and fine-grained analyses, and exhibits strong agreement with human experts on held-out proof trajectories."
Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood.
Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.
Human feedback protocol is explicit
No explicit human feedback protocol detected.
Evaluation mode is explicit
Detected: Automatic Metrics
Quality control reporting appears
No calibration/adjudication/IAA control explicitly detected.
Benchmark or dataset anchors are present
Detected: Advancedmathbench, Proverbench, Verifierbench
Metric reporting is present
Detected: f1, agreement
