HFEPX Hub

Critique Edit Or Expert Verification Papers

Updated from current HFEPX corpus (Feb 27, 2026). 41 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Multi Dim Rubric. Frequent quality control: Calibration. Frequently cited benchmark: Retrieval. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 25, 2026.

Papers: 41 Last published: Feb 25, 2026 Global RSS Tag RSS

Critique EditExpert Verification

Research Narrative

Grounded narrative Model: deterministic-grounded

Updated from current HFEPX corpus (Feb 27, 2026). This page covers 41 papers centered on Critique Edit Or Expert Verification Papers. Common evaluation modes include Automatic Metrics, Simulation Env, with benchmark emphasis on Retrieval, AIME. Metric concentration includes accuracy, cost, and the agentic footprint highlights Multi Agent, Long Horizon. Use the anchored takeaways below to compare protocol choices, quality-control patterns, and evidence depth before allocating new eval budget.

Why This Matters For Eval Research

Evaluation emphasis: Automatic Metrics and Simulation Env appear frequently in this slice.

Evidence: MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models , SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video , Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Information
Benchmark concentration: Retrieval, AIME helps control cross-paper variance.

Evidence: SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video , Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Information , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
Metric concentration: accuracy, cost is repeatedly reported in this group.

Evidence: Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Information , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery , "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems

Protocol Takeaways

Compare papers that report both human_eval and llm_as_judge to quantify judge-human agreement drift.

Evidence: SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery , "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems , An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems
Stratify by benchmark (Retrieval vs AIME) before comparing methods.

Evidence: "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems , An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems , CAMEL: Confidence-Gated Reflection for Reward Modeling
Papers with explicit human feedback is visible in approximately 100% of papers in this set.

Evidence: An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems , CAMEL: Confidence-Gated Reflection for Reward Modeling , An artificial intelligence framework for end-to-end rare disease phenotyping from clinical notes using large language models

Benchmark Interpretation

Retrieval appears as a recurring benchmark anchor in this page.
3 papers (7.3%) mention Retrieval.
Most common evaluation modes: Automatic Metrics, Simulation Env.

Metric Interpretation

accuracy is a common reported metric and should be paired with protocol context before ranking methods.
10 papers (24.4%) mention accuracy.
Most common evaluation modes: Automatic Metrics.

Researcher Checklist

Papers with explicit human feedback: Coverage is strong (100% vs 45% target).
Papers reporting quality controls: Coverage is a replication risk (12.2% vs 30% target).
Papers naming benchmarks/datasets: Coverage is usable but incomplete (29.3% vs 35% target).
Papers naming evaluation metrics: Coverage is strong (61% vs 35% target).
Papers with known rater population: Coverage is strong (65.9% vs 35% target).
Papers with known annotation unit: Coverage is usable but incomplete (24.4% vs 35% target).

Papers with explicit human feedback

Coverage is strong (100% vs 45% target).

Papers reporting quality controls

Coverage is a replication risk (12.2% vs 30% target).

Papers naming benchmarks/datasets

Coverage is usable but incomplete (29.3% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (61% vs 35% target).

Papers with known rater population

Coverage is strong (65.9% vs 35% target).

Papers with known annotation unit

Coverage is usable but incomplete (24.4% vs 35% target).

Known Limitations

Narrative synthesis is grounded in metadata and abstracts only; full-paper method details may be missing.
Extraction fields are conservative and can under-report implicit protocol details.
Cross-page comparisons should control for benchmark and metric mismatch.

Research Utility Links

Judge vs Human Agreement - Compares papers that evaluate with both human raters and LLM judges.
Benchmark Slice: Retrieval - Prioritizes benchmark-specific protocol comparisons.
Metric Slice: accuracy - Finds papers where reported metrics are directly comparable.
Recent High-Signal Papers - Keeps the hub connected to the latest HFEPX corpus updates.

human_eval vs llm_as_judge

both=0, left_only=2, right_only=2

0 papers use both Human Eval and Llm As Judge.

human_eval vs automatic_metrics

both=0, left_only=2, right_only=34

0 papers use both Human Eval and Automatic Metrics.

llm_as_judge vs automatic_metrics