HFEPX Archive Slice

HFEPX Weekly Archive: 2025-W52

Updated from current HFEPX corpus (Mar 1, 2026). 6 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Mar 1, 2026). 6 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics, Human Eval. Most common rater population: Domain Experts. Frequent quality control: Gold Questions. Frequently cited benchmark: BIRD. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Dec 26, 2025.

Papers: 6 Last published: Dec 26, 2025 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: Developing .

High-Signal Coverage

100.0%

6 / 6 papers are not low-signal flagged.

Benchmark Anchors

16.7%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

50.0%

Papers with reported metric mentions in extraction output.

1 papers report explicit quality controls for this archive period.
Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice as early signal only; benchmark/metric anchoring is limited for rigorous period-over-period claims.

Why This Slice Matters (Expanded)

Why This Time Slice Matters

16.7% of papers report explicit human-feedback signals, led by expert verification.
automatic metrics appears in 50% of papers in this hub.
BIRD is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Notes (Expanded)

Protocol Takeaways For This Period

Most common quality-control signal is gold-question checks (16.7% of papers).
Rater context is mostly domain experts, and annotation is commonly mixed annotation units; use this to scope replication staffing.
Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics
Dec 26, 2025 · Citations: 0 · Score: 8.0

Eval: Automatic Metrics · Metrics: Accuracy
DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation
Dec 23, 2025 · Citations: 0 · Score: 4.5

Eval: Automatic Metrics, Simulation Env · Metrics: Accuracy, Cost
On the Existence and Behavior of Secondary Attention Sinks
Dec 22, 2025 · Citations: 0 · Score: 4.5

Eval: Automatic Metrics · Metrics: Relevance
Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation
Dec 25, 2025 · Citations: 0 · Score: 3.0

Eval: Human Eval · Metrics: Not reported
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
Dec 24, 2025 · Citations: 0 · Score: 2.0

Eval: Not reported · Metrics: Not reported
Stop saying LLM: Large Discourse Models (LDM) and Artificial Discursive Agent (ADA)?
Dec 22, 2025 · Citations: 0 · Score: 2.0

Eval: Not reported · Metrics: Not reported

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper	Eval Modes	Benchmarks	Metrics	Quality Controls
CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics Dec 26, 2025	Automatic Metrics	DROP, BIRD	Accuracy	Gold Questions
DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation Dec 23, 2025	Automatic Metrics, Simulation Env	Not reported	Accuracy, Cost	Not reported
On the Existence and Behavior of Secondary Attention Sinks Dec 22, 2025	Automatic Metrics	Not reported	Relevance	Not reported
Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation Dec 25, 2025	Human Eval	Not reported	Not reported	Not reported
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation Dec 24, 2025	Not reported	Not reported	Not reported	Not reported
Stop saying LLM: Large Discourse Models (LDM) and Artificial Discursive Agent (ADA)? Dec 22, 2025	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Gap: Papers with explicit human feedback

Coverage is a replication risk (16.7% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (16.7% vs 30% target).
Gap: Papers naming benchmarks/datasets

Coverage is a replication risk (16.7% vs 35% target).
Moderate: Papers naming evaluation metrics

Coverage is usable but incomplete (33.3% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (16.7% vs 35% target).
Gap: Papers with known annotation unit

Coverage is a replication risk (0% vs 35% target).

Strengths

This hub still surfaces a concentrated paper set for protocol triage and replication planning.

Known Gaps

Only 16.7% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (16.7% coverage).
Annotation unit is under-specified (0% coverage).

Suggested Next Analyses

Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.
Stratify by benchmark (BIRD vs Cricbench) before comparing methods.
Track metric sensitivity by reporting both accuracy and cost.
Add inter-annotator agreement checks when reproducing these protocols.

Recommended Queries

Human Eval Protocols Benchmark Slice: BIRD Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 16.7% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (16.7% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (3)
Human Eval (1)
Simulation Env (1)

Top Metrics

Accuracy (2)
Cost (1)

Top Benchmarks

BIRD (1)
Cricbench (1)
DROP (1)

Quality Controls

Gold Questions (1)

Papers In This Archive Slice

CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics
Vaibhav Devraj, Dhruv Kumar, Jagat Sesh Challa, Parth Agarwal, Navya Kommuri · Dec 26, 2025 · Citations: 0

Expert Verification

To investigate this potential capability gap, we present CricBench, a comprehensive benchmark suite for evaluating LLMs on specialized cricket data.
Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation
Abdullah Alabdullah, Lifeng Han, Chenghua Lin · Dec 25, 2025 · Citations: 0

Existing automatic evaluation metrics and general-purpose human evaluation frameworks struggle to capture dialect-specific MT errors, hindering progress in translation assessment.
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen · Dec 24, 2025 · Citations: 0

Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation
Ziyi Zhu, Olivier Tieleman, Caitlin A. Stamatis, Luka Smyth, Thomas D. Hull · Dec 23, 2025 · Citations: 0

Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge.
On the Existence and Behavior of Secondary Attention Sinks
Jeffrey T. H. Wong, Cheng Zhang, Louis Mahon, Wayne Luk, Anton Isopoussu · Dec 22, 2025 · Citations: 0

Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
Stop saying LLM: Large Discourse Models (LDM) and Artificial Discursive Agent (ADA)?
Amar Lakel · Dec 22, 2025 · Citations: 0

This paper proposes an epistemological shift in the analysis of large generative models, replacing the category ''Large Language Models'' (LLM) with that of ''Large Discourse Models'' (LDM), and then with that of Artificial Discursive Agent…

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote