Skip to content
← Back to explorer

HFEPX Hub

General + Rubric Rating Papers

Updated from current HFEPX corpus (Feb 27, 2026). 11 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Human Eval. Most common rater population: Domain Experts. Common annotation unit: Multi Dim Rubric. Frequent quality control: Inter Annotator Agreement Reported. Frequently cited benchmark: Caparena. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 25, 2026.

Papers: 11 Last published: Feb 25, 2026 Global RSS Tag RSS
GeneralRubric Rating

Research Narrative

Grounded narrative Model: deterministic-grounded Source: persisted

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 11 papers for General + Rubric Rating Papers. Dominant protocol signals include automatic metrics, human evaluation, LLM-as-judge, with frequent benchmark focus on Caparena, Mle-Bench and metric focus on accuracy, agreement. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Why This Matters For Eval Research

Protocol Takeaways

Benchmark Interpretation

  • Caparena appears in 9.1% of hub papers (1/11); use this cohort for benchmark-matched comparisons.
  • Mle-Bench appears in 9.1% of hub papers (1/11); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • accuracy is reported in 18.2% of hub papers (2/11); compare with a secondary metric before ranking methods.
  • agreement is reported in 9.1% of hub papers (1/11); compare with a secondary metric before ranking methods.

Researcher Checklist

  • Maintain strength on Papers with explicit human feedback. Coverage is strong (100% vs 45% target).
  • Tighten coverage on Papers reporting quality controls. Coverage is usable but incomplete (18.2% vs 30% target).
  • Close gap on Papers naming benchmarks/datasets. Coverage is a replication risk (18.2% vs 35% target).
  • Maintain strength on Papers naming evaluation metrics. Coverage is strong (36.4% vs 35% target).
  • Tighten coverage on Papers with known rater population. Coverage is usable but incomplete (27.3% vs 35% target).
  • Maintain strength on Papers with known annotation unit. Coverage is strong (100% vs 35% target).

Papers with explicit human feedback

Coverage is strong (100% vs 45% target).

Papers reporting quality controls

Coverage is usable but incomplete (18.2% vs 30% target).

Papers naming benchmarks/datasets

Coverage is a replication risk (18.2% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (36.4% vs 35% target).

Papers with known rater population

Coverage is usable but incomplete (27.3% vs 35% target).

Papers with known annotation unit

Coverage is strong (100% vs 35% target).

Suggested Reading Order

  1. 1. RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  2. 2. SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  3. 3. Personalized Prediction of Perceived Message Effectiveness Using Large Language Model Based Digital Twins

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  4. 4. Discovering Implicit Large Language Model Alignment Objectives

    Include a human-eval paper to anchor calibration against automated judge settings.

  5. 5. Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation

    Include a human-eval paper to anchor calibration against automated judge settings.

  6. 6. HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue

    Include an LLM-as-judge paper to assess judge design and agreement assumptions.

  7. 7. PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions

    Include an LLM-as-judge paper to assess judge design and agreement assumptions.

  8. 8. KLong: Training LLM Agent for Extremely Long-horizon Tasks

    Adds automatic metrics with rubric ratings for broader coverage within this hub.

Known Limitations

  • Only 18.2% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Benchmark coverage is thin (18.2% of papers mention benchmarks/datasets).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Links

human_eval vs llm_as_judge

both=2, left_only=2, right_only=0

2 papers use both Human Eval and Llm As Judge.

human_eval vs automatic_metrics

both=0, left_only=4, right_only=7

0 papers use both Human Eval and Automatic Metrics.

llm_as_judge vs automatic_metrics

both=0, left_only=2, right_only=7

0 papers use both Llm As Judge and Automatic Metrics.

Benchmark Brief

Caparena

Coverage: 1 papers (9.1%)

1 papers (9.1%) mention Caparena.

Examples: PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions

Benchmark Brief

Mle-Bench

Coverage: 1 papers (9.1%)

1 papers (9.1%) mention Mle-Bench.

Examples: KLong: Training LLM Agent for Extremely Long-horizon Tasks

Benchmark Brief

Paperbench

Coverage: 1 papers (9.1%)

1 papers (9.1%) mention Paperbench.

Examples: KLong: Training LLM Agent for Extremely Long-horizon Tasks

Metric Brief

agreement

Coverage: 1 papers (9.1%)

1 papers (9.1%) mention agreement.

Examples: HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue

Metric Brief

f1

Coverage: 1 papers (9.1%)

1 papers (9.1%) mention f1.

Examples: Personalized Prediction of Perceived Message Effectiveness Using Large Language Model Based Digital Twins

Top Papers

Related Hubs