Pipeline for Verifying LLM-Generated Mathematical Solutions

Research Summary

Contribution Summary

• With the growing popularity of Large Reasoning Models and their results in solving mathematical problems, it becomes crucial to measure their capabilities.
• We introduce a pipeline for both automatic and interactive verification as a more accurate alternative to only checking the answer which is currently the most popular approach for benchmarks.
• The pipeline can also be used as a generator of correct solutions both in formal and informal languages.

Why It Matters For Eval

• We introduce a pipeline for both automatic and interactive verification as a more accurate alternative to only checking the answer which is currently the most popular approach for benchmarks.
• 3 AI agents, which can be chosen for the benchmark accordingly, are included in the structure.

Related Papers

• TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models Embedding Neighbor
• MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning Embedding Neighbor
• Bridging Gaps in Natural Language Processing for Yorùbá: A Systematic Review of a Decade of Progress and Prospects Embedding Neighbor
• The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging Embedding Neighbor
• MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation Embedding Neighbor
• Predicting Sentence Acceptability Judgments in Multimodal Contexts Embedding Neighbor
• Mixture-of-Experts as Soft Clustering: A Dual Jacobian-PCA Spectral Geometry Perspective Embedding Neighbor
• Personalized Prediction of Perceived Message Effectiveness Using Large Language Model Based Digital Twins Embedding Neighbor
• Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learning Embedding Neighbor
• GATES: Self-Distillation under Privileged Context with Consensus Gating Tag Overlap
• Proof-RM: A Scalable and Generalizable Reward Model for Math Proof Tag Overlap
• Esoteric Language Models: Bridging Autoregressive and Masked Diffusion LLMs Tag Overlap