HFEPX Archive Slice

HFEPX Daily Archive: 2025-10-23

Updated from current HFEPX corpus (Apr 12, 2026). 14 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Apr 12, 2026). 14 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics, Human Eval. Common annotation unit: Multi Dim Rubric. Frequently cited benchmark: HotpotQA. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Oct 23, 2025.

Papers: 14 Last published: Oct 23, 2025 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: Medium .

High-Signal Coverage

100.0%

14 / 14 papers are not low-signal flagged.

Benchmark Anchors

7.1%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

35.7%

Papers with reported metric mentions in extraction output.

0 papers report explicit quality controls for this archive period.
Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice as early signal only; benchmark/metric anchoring is limited for rigorous period-over-period claims.

Get this digest every Friday →

Why This Time Slice Matters

14.3% of papers report explicit human-feedback signals, led by pairwise preferences.
automatic metrics appears in 35.7% of papers in this hub.
HotpotQA is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways For This Period

Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Rater context is mostly unspecified rater pools, and annotation is commonly multi-dimensional rubrics; use this to scope replication staffing.
Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA
Oct 23, 2025 · Citations: 0 · Score: 5.5

Eval: Automatic Metrics · Metrics: Accuracy, F1
Automated Coding of Communication Data Using ChatGPT: Consistency Across Subgroups
Oct 23, 2025 · Citations: 0 · Score: 5.0

Eval: Human Eval, Automatic Metrics · Metrics: Accuracy
Robust Preference Alignment via Directional Neighborhood Consensus
Oct 23, 2025 · Citations: 0 · Score: 5.0

Eval: Automatic Metrics · Metrics: Helpfulness
Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People
Oct 23, 2025 · Citations: 0 · Score: 4.0

Eval: Automatic Metrics · Metrics: Accuracy, F1
GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning
Oct 23, 2025 · Citations: 0 · Score: 4.0

Eval: Automatic Metrics · Metrics: F1
Support-Contra Asymmetry in LLM Explanations
Oct 23, 2025 · Citations: 0 · Score: 1.5

Eval: Not reported · Metrics: Not reported

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper	Eval Modes	Benchmarks	Metrics	Quality Controls
RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA Oct 23, 2025	Automatic Metrics	HotpotQA	Accuracy, F1	Not reported
Automated Coding of Communication Data Using ChatGPT: Consistency Across Subgroups Oct 23, 2025	Human Eval, Automatic Metrics	Not reported	Accuracy	Not reported
Robust Preference Alignment via Directional Neighborhood Consensus Oct 23, 2025	Automatic Metrics	Not reported	Helpfulness	Not reported
Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People Oct 23, 2025	Automatic Metrics	Not reported	Accuracy, F1	Not reported
GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning Oct 23, 2025	Automatic Metrics	Not reported	F1	Not reported
Support-Contra Asymmetry in LLM Explanations Oct 23, 2025	Not reported	Not reported	Not reported	Not reported
Co-Designing Quantum Codes with Transversal Diagonal Gates via Multi-Agent Systems Oct 23, 2025	Not reported	Not reported	Not reported	Not reported
Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset Oct 23, 2025	Not reported	Not reported	Not reported	Not reported
Evaluating Latent Knowledge of Public Tabular Datasets in Large Language Models Oct 23, 2025	Not reported	Not reported	Not reported	Not reported
CreativityPrism: A Holistic Evaluation Framework for Large Language Model Creativity Oct 23, 2025	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Gap: Papers with explicit human feedback

Coverage is a replication risk (14.3% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Gap: Papers naming benchmarks/datasets

Coverage is a replication risk (14.3% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (35.7% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (0% vs 35% target).
Gap: Papers with known annotation unit

Coverage is a replication risk (7.1% vs 35% target).

Strengths

This hub still surfaces a concentrated paper set for protocol triage and replication planning.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).
Annotation unit is under-specified (7.1% coverage).

Suggested Next Analyses

Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.
Stratify by benchmark (HotpotQA vs HR-Bench) before comparing methods.
Track metric sensitivity by reporting both accuracy and f1.

Recommended Queries

Human Eval Protocols Benchmark Slice: HotpotQA Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (5)
Human Eval (1)

Top Metrics

Accuracy (3)
F1 (2)
Agreement (1)
Cost (1)

Top Benchmarks

HotpotQA (1)
HR Bench (1)

Quality Controls

Papers In This Archive Slice

Support-Contra Asymmetry in LLM Explanations
Avinash Patil · Oct 23, 2025 · Citations: 0

Across three benchmark datasets-WIKIONTOLOGY, AG NEWS, and IMDB-we observe a consistent empirical pattern that we term support-contra asymmetry.
Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculation
Yuhan Liu, Lianhui Qin, Shengjie Wang · Oct 23, 2025 · Citations: 0
Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People
Gabriel Grand, Valerio Pepe, Jacob Andreas, Joshua B. Tenenbaum · Oct 23, 2025 · Citations: 0

Drawing on insights from human cognition, we develop methods to evaluate and enhance agentic information-seeking.
Co-Designing Quantum Codes with Transversal Diagonal Gates via Multi-Agent Systems
Xi He, Sirui Lu, Bei Zeng · Oct 23, 2025 · Citations: 0

Multi Agent

We address this gap by extending TeXRA with an independent Lean 4 verification layer, turning it into a human-guided multi-agent platform for exact scientific discovery.
Transferable Graph Learning for Transmission Congestion Management via Busbar Splitting
Ali Rajaei, Peter Palensky, Jochen L. Cremer · Oct 23, 2025 · Citations: 0
Automated Coding of Communication Data Using ChatGPT: Consistency Across Subgroups
Jiangang Hao, Wenju Cui, Patrick Kyllonen, Emily Kerzabi · Oct 23, 2025 · Citations: 0

Rubric Rating

Prior research has established that ChatGPT can be directly instructed with coding rubrics to code the communication data and achieves accuracy comparable to human raters.
GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning
Jinchang Luo, Mingquan Cheng, Fan Wan, Ni Li, Xiaoling Xia · Oct 23, 2025 · Citations: 0

Long Horizon

Extensive experiments on both in-domain and out-of-domain benchmarks demonstrate that GlobalRAG significantly outperforms strong baselines while using only 8k training data (42% of the training data used by strong baselines), achieving…
Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset
Paul Lerner, François Yvon · Oct 23, 2025 · Citations: 0

Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA
Ruiyi Yang, Hao Xue, Imran Razzak, Hakim Hacid, Flora D. Salim · Oct 23, 2025 · Citations: 0

Long Horizon

A Head Agent provides guidance that leads retrieval, while an Iteration Agent selects and expands HSeq via structure-respecting actions (e.g., parent/child hops, table row/column neighbors, KG relations); Finally the head agent composes…
Robust Preference Alignment via Directional Neighborhood Consensus
Ruochen Mao, Yuling Shi, Xiaodong Gu, Jiaheng Wei · Oct 23, 2025 · Citations: 0

Pairwise Preference

To address this challenge, we introduce Robust Preference Selection (RPS), a post-hoc, training-free method by leveraging directional neighborhood consensus.
Steering Evaluation-Aware Language Models to Act Like They Are Deployed
Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda · Oct 23, 2025 · Citations: 0
Evaluating Latent Knowledge of Public Tabular Datasets in Large Language Models
Matteo Silvestri, Fabiano Veglianti, Flavio Giorgi, Fabrizio Silvestri, Gabriele Tolomei · Oct 23, 2025 · Citations: 0

In contrast, we propose a framework for assessing contamination in tabular datasets by generating controlled queries and performing comparative evaluation.
Citation Failure: Definition, Analysis and Efficient Mitigation
Jan Buchmann, Iryna Gurevych · Oct 23, 2025 · Citations: 0
CreativityPrism: A Holistic Evaluation Framework for Large Language Model Creativity
Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei · Oct 23, 2025 · Citations: 0

Creativity is often seen as a hallmark of human intelligence.

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote