Math Papers (Last 90 Days)

Why This Matters For Eval Research

• 46.7% of papers report explicit human-feedback signals, led by pairwise preferences.
• automatic metrics appears in 60% of papers in this hub.
• LiveCodeBench is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways

• Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
• Rater context is mostly domain experts, and annotation is commonly trajectory-level annotation; use this to scope replication staffing.
• Compare papers that report both human_eval and llm_as_judge to quantify judge-human agreement drift.

Benchmark Interpretation

• LiveCodeBench appears in 13.3% of hub papers (2/15); use this cohort for benchmark-matched comparisons.
• AIME appears in 6.7% of hub papers (1/15); use this cohort for benchmark-matched comparisons.

Metric Interpretation

• accuracy is reported in 40% of hub papers (6/15); compare with a secondary metric before ranking methods.
• cost is reported in 20% of hub papers (3/15); compare with a secondary metric before ranking methods.

Researcher Checklist

• Strong: Papers with explicit human feedback

  Coverage is strong (46.7% vs 45% target).

• Gap: Papers reporting quality controls

  Coverage is a replication risk (0% vs 30% target).

• Moderate: Papers naming benchmarks/datasets

  Coverage is usable but incomplete (33.3% vs 35% target).

• Strong: Papers naming evaluation metrics

  Coverage is strong (60% vs 35% target).

• Gap: Papers with known rater population

  Coverage is a replication risk (13.3% vs 35% target).

• Gap: Papers with known annotation unit

  Coverage is a replication risk (20% vs 35% target).

Strengths

• Strong human-feedback signal (46.7% of papers).
• Contains both human-eval and LLM-as-judge protocols for head-to-head methodology comparison.
• Agentic evaluation appears in 60% of papers.

Known Gaps

• Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
• Rater population is under-specified (13.3% coverage).
• Annotation unit is under-specified (20% coverage).

Suggested Next Analyses

• Compare papers that report both human_eval and llm_as_judge to quantify judge-human agreement drift.
• Stratify by benchmark (LiveCodeBench vs AIME) before comparing methods.
• Track metric sensitivity by reporting both accuracy and cost.

Recommended Queries

This section is intentionally expanded only when needed; use “Start Here” above for a faster pass.

Suggested Reading Order

1. The logic of KM belief update is contained in the logic of AGM belief revision

   Start here for detailed protocol reporting and quality-control evidence. Signals: critique/edit feedback. Abstract: For each axiom of KM belief update we provide a corresponding axiom in a modal.

2. AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning

   Start here for detailed protocol reporting and quality-control evidence. Signals: automatic metrics. Focus: accuracy. Abstract: While Multi-Agent Systems (MAS) excel in complex reasoning, they suffer from the cascading.

3. Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

   Start here for detailed protocol reporting and quality-control evidence. Signals: automatic metrics. Focus: accuracy. Abstract: Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing.

4. Think$^{2}$: Grounded Metacognitive Reasoning in Large Language Models

   Include a human-eval paper to calibrate against judge-based evaluation settings. Signals: human evaluation + pairwise preferences. Focus: GSM8K. Abstract: Blinded human evaluations over 580 query pairs show an.

5. Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

   Include a human-eval paper to calibrate against judge-based evaluation settings. Signals: pairwise preferences. Focus: LiveCodeBench / latency. Abstract: To reach frontier-level intelligence, we design a scalable reinforcement learning.

6. Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards

   Include an LLM-as-judge paper to test judge design and agreement assumptions. Signals: LLM-as-judge. Focus: accuracy. Abstract: GR achieves a higher GPT-judged win-rate in RLHF, avoids overly focusing on.

7. Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences

   Adds automatic metrics with pairwise preferences for broader protocol coverage within this hub. Signals: automatic metrics + pairwise preferences. Focus: LiveCodeBench / accuracy. Abstract: Pairwise comparisons, by contrast,.

8. Unlocking Reasoning Capability on Machine Translation in Large Language Models

   Adds evaluation protocol evidence with critique/edit feedback for broader protocol coverage within this hub. Signals: critique/edit feedback. Abstract: Reasoning-oriented large language models (RLMs) achieve strong gains on tasks.

Known Limitations

• Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
• Rater population is under-specified (13.3% coverage).
• Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot