HFEPX Archive Slice

HFEPX Daily Archive: 2025-10-29

Updated from current HFEPX corpus (Apr 9, 2026). 7 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Apr 9, 2026). 7 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequent quality control: Calibration. Frequently cited benchmark: APPS. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Oct 29, 2025.

Papers: 7 Last published: Oct 29, 2025 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: Developing .

High-Signal Coverage

100.0%

7 / 7 papers are not low-signal flagged.

Benchmark Anchors

14.3%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

57.1%

Papers with reported metric mentions in extraction output.

1 papers report explicit quality controls for this archive period.
Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice as early signal only; benchmark/metric anchoring is limited for rigorous period-over-period claims.

Get this digest every Friday →

Why This Time Slice Matters

28.6% of papers report explicit human-feedback signals, led by demonstration data.
automatic metrics appears in 57.1% of papers in this hub.
APPS is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways For This Period

Most common quality-control signal is rater calibration (14.3% of papers).
Rater context is mostly domain experts, and annotation is commonly trajectory-level annotation; use this to scope replication staffing.
Track metric sensitivity by reporting both accuracy and coherence.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution
Oct 29, 2025 · Citations: 0 · Score: 5.5

Eval: Automatic Metrics · Metrics: Success rate
RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline
Oct 29, 2025 · Citations: 0 · Score: 5.0

Eval: Automatic Metrics · Metrics: Rouge
Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
Oct 29, 2025 · Citations: 0 · Score: 5.0

Eval: Automatic Metrics · Metrics: Agreement
From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity
Oct 29, 2025 · Citations: 0 · Score: 4.0

Eval: Automatic Metrics · Metrics: Accuracy, Relevance
Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning
Oct 29, 2025 · Citations: 0 · Score: 2.5

Eval: Not reported · Metrics: Not reported
Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMs
Oct 29, 2025 · Citations: 0 · Score: 1.5

Eval: Not reported · Metrics: Not reported

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper	Eval Modes	Benchmarks	Metrics	Quality Controls
The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution Oct 29, 2025	Automatic Metrics	APPS	Success rate	Not reported
RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline Oct 29, 2025	Automatic Metrics	Not reported	Rouge	Not reported
Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters Oct 29, 2025	Automatic Metrics	Not reported	Agreement	Calibration
From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity Oct 29, 2025	Automatic Metrics	Not reported	Accuracy, Relevance	Not reported
Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning Oct 29, 2025	Not reported	Not reported	Not reported	Not reported
Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMs Oct 29, 2025	Not reported	Not reported	Not reported	Not reported
TheraMind: A Strategic and Adaptive Agent for Longitudinal Psychological Counseling Oct 29, 2025	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Moderate: Papers with explicit human feedback

Coverage is usable but incomplete (28.6% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (14.3% vs 30% target).
Gap: Papers naming benchmarks/datasets

Coverage is a replication risk (14.3% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (57.1% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (14.3% vs 35% target).
Gap: Papers with known annotation unit

Coverage is a replication risk (14.3% vs 35% target).

Strengths

Agentic evaluation appears in 42.9% of papers.

Known Gaps

Only 14.3% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (14.3% coverage).
Annotation unit is under-specified (14.3% coverage).

Suggested Next Analyses

Track metric sensitivity by reporting both accuracy and coherence.
Add inter-annotator agreement checks when reproducing these protocols.

Recommended Queries

Benchmark Slice: APPS Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 14.3% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (14.3% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (4)

Top Metrics

Accuracy (1)
Coherence (1)
Relevance (1)
Rouge (1)

Top Benchmarks

APPS (1)

Quality Controls

Calibration (1)

Papers In This Archive Slice

Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning
Yihe Deng, I-Hung Hsu, Jun Yan, Zifeng Wang, Rujun Han · Oct 29, 2025 · Citations: 0

Demonstrations Long Horizon

Beyond reasoning benchmarks, SRL generalizes effectively to agentic software engineering tasks, establishing it as a robust and versatile training framework for reasoning-oriented LLMs.
RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline
André V. Duarte, Xuying li, Bin Zeng, Arlindo L. Oliveira, Lei Li · Oct 29, 2025 · Citations: 0

Red Team

As such, we propose RECAP, an agentic pipeline designed to elicit and verify memorized training data from LLM outputs.
Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
Xingjian Zhang, Tianhong Gao, Suliang Jin, Tianhao Wang, Teng Ye · Oct 29, 2025 · Citations: 0

Large language models (LLMs) are increasingly used as raters for evaluation tasks.
TheraMind: A Strategic and Adaptive Agent for Longitudinal Psychological Counseling
He Hu, Chiyuan Ma, Qianning Wang, Lin Liu, Yucheng Zhou · Oct 29, 2025 · Citations: 0
The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution
Junlong Li, Wenshuo Zhao, Jian Zhao, Weihao Zeng, Haoze Wu · Oct 29, 2025 · Citations: 0

Long Horizon

To address this gap, we introduce the Tool Decathlon (dubbed as Toolathlon), a benchmark for language agents offering diverse Apps and tools, realistic environment setup, and reliable execution-based evaluation.
From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity
Tianxi Wan, Jiaming Luo, Siyuan Chen, Kunyao Lan, Jianhua Chen · Oct 29, 2025 · Citations: 0

Multi Agent

To address this, we develop a novel approach integrating synthetic patient electronic medical record (EMR) construction and multi-agent diagnostic dialogue generation.
Activation-Space Personality Steering: Hybrid Layer Selection for Stable Trait Control in LLMs
Pranav Bhandari, Nicolas Fay, Sanjeevan Selvaganapathy, Amitava Datta, Usman Naseem · Oct 29, 2025 · Citations: 0

We propose a novel pipeline that extracts hidden state activations from transformer layers using the Big Five Personality Traits (Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism), which is a comprehensive and…

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote