Daily Archive
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Frequently cited benchmark: MATH. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Jun 15, 2025.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for HFEPX Weekly Archive: 2025-W24. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on MATH, MATH-500 and metric focus on accuracy, coherence. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
20% of papers report explicit human-feedback signals, led by expert verification.
Evidence: From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertise , $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR
automatic metrics appears in 100% of papers in this hub.
Evidence: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR , Probabilistic distances-based hallucination detection in LLMs with RAG
MATH is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: Spurious Rewards: Rethinking Training Signals in RLVR , AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking , $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR , Probabilistic distances-based hallucination detection in LLMs with RAG
Rater context is mostly domain experts, and annotation is commonly mixed annotation units; use this to scope replication staffing.
Evidence: ICE-ID: A Novel Historical Census Dataset for Longitudinal Identity Resolution , From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertise , $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
Stratify by benchmark (MATH vs MATH-500) before comparing methods.
Evidence: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR , Probabilistic distances-based hallucination detection in LLMs with RAG
Papers with explicit human feedback
Coverage is a replication risk (20% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is strong (50% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (60% vs 35% target).
Papers with known rater population
Coverage is a replication risk (20% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
1. $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
Start here for detailed protocol reporting, including rater and quality-control evidence.
2. Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Spurious Rewards: Rethinking Training Signals in RLVR
Start here for detailed protocol reporting, including rater and quality-control evidence.
4. Probabilistic distances-based hallucination detection in LLMs with RAG
Adds automatic metrics for broader coverage within this hub.
5. ICE-ID: A Novel Historical Census Dataset for Longitudinal Identity Resolution
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
7. Structure-Augmented Reasoning Generation
Adds automatic metrics for broader coverage within this hub.
8. AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking
Adds automatic metrics for broader coverage within this hub.
automatic_metrics vs simulation_env
both=1, left_only=9, right_only=0
1 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
MATH
Coverage: 2 papers (20%)
2 papers (20%) mention MATH.
Examples: Spurious Rewards: Rethinking Training Signals in RLVR , AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking
Benchmark Brief
MATH-500
Coverage: 2 papers (20%)
2 papers (20%) mention MATH-500.
Examples: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Spurious Rewards: Rethinking Training Signals in RLVR
Benchmark Brief
Retrieval
Coverage: 2 papers (20%)
2 papers (20%) mention Retrieval.
Examples: Probabilistic distances-based hallucination detection in LLMs with RAG , Structure-Augmented Reasoning Generation
Metric Brief
accuracy
Coverage: 3 papers (30%)
3 papers (30%) mention accuracy.
Examples: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingness , Structure-Augmented Reasoning Generation
Metric Brief
coherence
Coverage: 1 papers (10%)
1 papers (10%) mention coherence.
Examples: Structure-Augmented Reasoning Generation
Metric Brief
cost
Coverage: 1 papers (10%)
1 papers (10%) mention cost.
Examples: From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertise
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts , Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models , Spurious Rewards: Rethinking Training Signals in RLVR
Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer · Jun 15, 2025
Scaling test-time compute has driven the recent advances in the reasoning capabilities of large language models (LLMs), typically by allocating additional computation for more thorough exploration.
Maximilian Kreutner, Marlene Lutz, Markus Strohmaier · Jun 13, 2025
Large Language Models (LLMs) display remarkable capabilities to understand or even produce political discourse but have been found to consistently exhibit a progressive left-leaning bias.
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang · Jun 12, 2025
We show that reinforcement learning with verifiable rewards (RLVR) can elicit strong mathematical reasoning in certain language models even with spurious rewards that have little, no, or even negative correlation with the correct answer.
Rodion Oblovatny, Alexandra Kuleshova, Konstantin Polev, Alexey Zaytsev · Jun 11, 2025
Detecting hallucinations in large language models (LLMs) is critical for their safety in many applications.
Gonçalo Hora de Carvalho, Lazar S. Popov, Sander Kaatee, Mário S. Correia, Kristinn R. Thórisson · Jun 11, 2025
We introduce \textbf{ICE-ID}, a benchmark dataset comprising 984,028 records from 16 Icelandic census waves spanning 220 years (1703--1920), with 226,864 expert-curated person identifiers.
Jinkwan Jang, Hyungjin Park, Jinmyeong Choi, Taesup Kim · Jun 10, 2025
Extensive experiments on public benchmark datasets reflecting practical settings, along with one private real-world industrial dataset, demonstrate the superior robustness and accuracy of ChannelTokenFormer under challenging real-world cond
Jash Rajesh Parekh, Pengcheng Jiang, Jiawei Han · Jun 10, 2025
Extensive experiments on open-domain QA benchmarks and specialized reasoning datasets in finance and medicine demonstrate that SARG significantly outperforms state-of-the-art flat-context RAG baselines in both factual accuracy and reasoning
Silin Gao, Antoine Bosselut, Samy Bengio, Emmanuel Abbe · Jun 9, 2025
Our method, AbstRaL -- which promotes abstract reasoning in LLMs using RL on granular abstraction data -- significantly mitigates performance degradation on recent GSM perturbation benchmarks.
Nitin Sharma, Thomas Wolfers, Çağatay Yıldız · Jun 9, 2025
Accurate domain-specific benchmarking of LLMs is essential, specifically in domains with direct implications for humans, such as law, healthcare, and education.
Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi · Jun 9, 2025
In this work, we seek to understand whether style patterns compromise LLM safety, how superficial style alignment increases model vulnerability, and how best to mitigate these risks during alignment.
