HFEPX Archive Slice

HFEPX Weekly Archive: 2025-W12

Updated from current HFEPX corpus (Mar 8, 2026). 7 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Mar 8, 2026). 7 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Pairwise. Frequently cited benchmark: Re-Bench. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Mar 23, 2025.

Papers: 7 Last published: Mar 23, 2025 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: Developing .

High-Signal Coverage

100.0%

7 / 7 papers are not low-signal flagged.

Benchmark Anchors

14.3%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

57.1%

Papers with reported metric mentions in extraction output.

0 papers report explicit quality controls for this archive period.
Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice as early signal only; benchmark/metric anchoring is limited for rigorous period-over-period claims.

Why This Time Slice Matters

42.9% of papers report explicit human-feedback signals, led by expert verification.
automatic metrics appears in 57.1% of papers in this hub.
Re-Bench is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways For This Period

Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Rater context is mostly domain experts, and annotation is commonly pairwise annotation; use this to scope replication staffing.
Track metric sensitivity by reporting both accuracy and relevance.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

Measuring AI Ability to Complete Long Software Tasks
Mar 18, 2025 · Citations: 0 · Score: 6.0

Eval: Automatic Metrics · Metrics: Success rate
MedPlan: A Two-Stage RAG-Based System for Personalized Medical Plan Generation
Mar 23, 2025 · Citations: 0 · Score: 4.5

Eval: Automatic Metrics · Metrics: Accuracy
What Makes a Reward Model a Good Teacher? An Optimization Perspective
Mar 19, 2025 · Citations: 0 · Score: 4.5

Eval: Automatic Metrics · Metrics: Accuracy
EmoGRACE: Aspect-based emotion analysis for social media data
Mar 19, 2025 · Citations: 0 · Score: 3.5

Eval: Automatic Metrics · Metrics: F1
Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
Mar 23, 2025 · Citations: 0 · Score: 2.0

Eval: Not reported · Metrics: Not reported
Imitating AI agents increase diversity in homogeneous information environments but can reduce it in heterogeneous ones
Mar 20, 2025 · Citations: 0 · Score: 2.0

Eval: Simulation Env · Metrics: Not reported

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper	Eval Modes	Benchmarks	Metrics	Quality Controls
Measuring AI Ability to Complete Long Software Tasks Mar 18, 2025	Automatic Metrics	Re Bench	Success rate	Not reported
MedPlan: A Two-Stage RAG-Based System for Personalized Medical Plan Generation Mar 23, 2025	Automatic Metrics	Not reported	Accuracy	Not reported
What Makes a Reward Model a Good Teacher? An Optimization Perspective Mar 19, 2025	Automatic Metrics	Not reported	Accuracy	Not reported
EmoGRACE: Aspect-based emotion analysis for social media data Mar 19, 2025	Automatic Metrics	Not reported	F1	Not reported
Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment Mar 23, 2025	Not reported	Not reported	Not reported	Not reported
Imitating AI agents increase diversity in homogeneous information environments but can reduce it in heterogeneous ones Mar 20, 2025	Simulation Env	Not reported	Not reported	Not reported
OSCAR: Online Soft Compression And Reranking Mar 17, 2025	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Moderate: Papers with explicit human feedback

Coverage is usable but incomplete (42.9% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Gap: Papers naming benchmarks/datasets

Coverage is a replication risk (14.3% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (57.1% vs 35% target).
Moderate: Papers with known rater population

Coverage is usable but incomplete (28.6% vs 35% target).
Gap: Papers with known annotation unit

Coverage is a replication risk (14.3% vs 35% target).

Strengths

This hub still surfaces a concentrated paper set for protocol triage and replication planning.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Annotation unit is under-specified (14.3% coverage).
Benchmark coverage is thin (14.3% of papers mention benchmarks/datasets).

Suggested Next Analyses

Track metric sensitivity by reporting both accuracy and relevance.

Recommended Queries

Benchmark Slice: Re-Bench Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Annotation unit is under-specified (14.3% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (4)
Simulation Env (1)

Top Metrics

Accuracy (3)
Relevance (1)
Success rate (1)

Top Benchmarks

Re Bench (1)

Quality Controls

Papers In This Archive Slice

Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
Ruoxi Cheng, Haoxuan Ma, Weixin Wang, Ranjie Duan, Jiexi Liu · Mar 23, 2025 · Citations: 0

Pairwise PreferenceDemonstrations

Existing techniques are either reward-based (training a reward model on preference pairs and optimizing with reinforcement learning) or reward-free (directly fine-tuning on ranked outputs).
MedPlan: A Two-Stage RAG-Based System for Personalized Medical Plan Generation
Hsin-Ling Hsu, Cong-Tinh Dao, Luning Wang, Zitao Shuai, Thao Nguyen Minh Phan · Mar 23, 2025 · Citations: 0

Expert Verification

Comprehensive evaluation demonstrates that our method significantly outperforms baseline approaches in both assessment accuracy and treatment plan quality.
Imitating AI agents increase diversity in homogeneous information environments but can reduce it in heterogeneous ones
Emil Bakkensen Johansen, Oliver Baumann · Mar 20, 2025 · Citations: 0

Recent developments in large language models (LLMs) have facilitated autonomous AI agents capable of imitating human-generated content, raising fundamental questions about how AI may reshape democratic information environments such as news.
What Makes a Reward Model a Good Teacher? An Optimization Perspective
Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee · Mar 19, 2025 · Citations: 0

The success of Reinforcement Learning from Human Feedback (RLHF) critically depends on the quality of the reward model.
EmoGRACE: Aspect-based emotion analysis for social media data
Christina Zorenböhmer, Sebastian Schmidt, Bernd Resch · Mar 19, 2025 · Citations: 0

Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
Measuring AI Ability to Complete Long Software Tasks
Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia · Mar 18, 2025 · Citations: 0

Expert Verification Tool Use

Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear.
OSCAR: Online Soft Compression And Reranking
Maxime Louis, Thibault Formal, Hervé Dejean, Stéphane Clinchant · Mar 17, 2025 · Citations: 0

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote