HFEPX Benchmark Hub
Updated from current HFEPX corpus (Mar 8, 2026). 2 papers are grouped in this benchmark page.
Updated from current HFEPX corpus (Mar 8, 2026). 2 papers are grouped in this benchmark page. Frequently cited benchmark: LMSYS Chatbot Arena. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Sep 27, 2025.
Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: Developing .
High-Signal Coverage
100.0%
2 / 2 sampled papers are not low-signal flagged.
Replication-Ready Set
0
Papers with explicit benchmark + metric + eval mode fields.
Quality Controls
0.0%
0 papers report calibration/adjudication/IAA controls.
Primary action: Use this page to map benchmark mentions first; wait for stronger metric/QC coverage before strict comparisons.
Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.
Sep 27, 2025 · Citations: 0 · Score: 5.5
Eval: Not reported · Metrics: Not Reported
Jun 5, 2025 · Citations: 0 · Score: 5.0
Eval: Not reported · Metrics: Not Reported
Compare protocol ingredients quickly before deep-reading full papers.
| Paper | Eval Modes | Human Feedback | Metrics | Quality Controls |
|---|---|---|---|---|
| Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization Sep 27, 2025 | Not reported | Pairwise Preference | Not reported | Not reported |
| Search Arena: Analyzing Search-Augmented LLMs Jun 5, 2025 | Not reported | Pairwise Preference | Not reported | Not reported |
Strong: Papers with explicit human feedback
Coverage is strong (100% vs 45% target).
Gap: Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Strong: Papers naming benchmarks/datasets
Coverage is strong (100% vs 35% target).
Gap: Papers naming evaluation metrics
Coverage is a replication risk (0% vs 35% target).
Gap: Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Gap: Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
Evaluation Modes
Human Feedback Mix
Top Benchmarks
Top Metrics
Junming Yang, Ning Xu, Biao Liu, Shiqi Qiao, Xin Geng · Sep 27, 2025 · Citations: 0
To bridge this gap, we propose Meta-Weighted Adaptive Preference Optimization (MetaAPO), a novel framework that dynamically couples data generation with model training.
Mihran Miroyan, Tsung-Han Wu, Logan King, Tianle Li, Jiayi Pan · Jun 5, 2025 · Citations: 0
In this work, we introduce Search Arena, a crowd-sourced, large-scale, human-preference dataset of over 24,000 paired multi-turn user interactions with search-augmented LLMs.
Need human evaluators for your AI research? Scale annotation with expert AI Trainers.
