Skip to content
OpenTrain AIFor AI Companies
← Back to explorer

HFEPX Archive Slice

HFEPX Daily Archive: 2026-06-17

Updated from current HFEPX corpus (Jun 30, 2026). 81 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Jun 30, 2026). 81 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics, Llm As Judge. Most common rater population: Domain Experts. Common annotation unit: Ranking. Frequent quality control: Calibration. Frequently cited benchmark: ALFWorld. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Jun 17, 2026.

Papers: 81 Last published: Jun 17, 2026 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: High .

Analysis blocks are computed from the loaded sample (60 of 81 papers).

High-Signal Coverage

100.0%

60 / 60 papers are not low-signal flagged.

Benchmark Anchors

11.7%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

16.7%

Papers with reported metric mentions in extraction output.

  • 3 papers report explicit quality controls for this archive period.
  • Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice for trend comparison: review top papers first, then validate shifts in the protocol matrix.

Get this digest every Friday →

Subscribe

Why This Time Slice Matters

  • 2.5% of papers report explicit human-feedback signals, led by expert verification.
  • automatic metrics appears in 11.1% of papers in this hub.
  • ALFWorld is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways For This Period

  • Most common quality-control signal is rater calibration (2.5% of papers).
  • Rater context is mostly domain experts, and annotation is commonly ranking annotation; use this to scope replication staffing.
  • Pair this hub with a human_eval-heavy hub to validate judge-model calibration.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper Eval Modes Benchmarks Metrics Quality Controls
Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias

Jun 17, 2026

 Llm As Judge, Automatic Metrics MT Bench, Judgebench Exact match, Kappa Inter Annotator Agreement Reported
Displacement Is Not Direction: Evaluating Fidelity Metrics for Quantized LLM Deployment

Jun 17, 2026

 Automatic Metrics LiveCodeBench Accuracy, Precision Calibration
SAGE-OPD: Selective Agent-Guided Intervention for Multi-Turn On-Policy Distillation

Jun 17, 2026

 Simulation Env ALFWorld Success rate Not reported
Uncertainty Decomposition for Clarification Seeking in LLM Agents

Jun 17, 2026

 Automatic Metrics, Simulation Env ALFWorld, WebShop F1, Latency Not reported
From 50K to 8.2 Million in 24 Hours: Vozinha's Algorithmic Consecration and the Multilingual Making of World Cup Visibility

Jun 17, 2026

 Automatic Metrics Not reported Agreement Calibration, Inter Annotator Agreement Reported
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

Jun 17, 2026

 Automatic Metrics Not reported Success rate, Latency Not reported
MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection

Jun 17, 2026

 Automatic Metrics Not reported Recall, Recall@10 Not reported
Toten: A Knowledge-Based System For Structure-Preserving Representation Of Physical Quantities And Technical Notation In Brazilian Portuguese

Jun 17, 2026

 Automatic Metrics Not reported Recall, Spearman Not reported
A BART-based approach with hierarchical strategy for Vietnamese abstractive multi-document summarization

Jun 17, 2026

 Automatic Metrics Not reported F1 Not reported
Approximate Structured Diffusion for Sequence Labelling

Jun 17, 2026

 Automatic Metrics Not reported Accuracy Not reported
Researcher Workflow (Detailed)

Checklist

  • Gap: Papers with explicit human feedback

    Coverage is a replication risk (2.5% vs 45% target).

  • Gap: Papers reporting quality controls

    Coverage is a replication risk (3.7% vs 30% target).

  • Gap: Papers naming benchmarks/datasets

    Coverage is a replication risk (16% vs 35% target).

  • Strong: Papers naming evaluation metrics

    Coverage is strong (37% vs 35% target).

  • Gap: Papers with known rater population

    Coverage is a replication risk (2.5% vs 35% target).

  • Gap: Papers with known annotation unit

    Coverage is a replication risk (6.2% vs 35% target).

Strengths

  • This hub still surfaces a concentrated paper set for protocol triage and replication planning.

Known Gaps

  • Only 3.7% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Rater population is under-specified (2.5% coverage).
  • Annotation unit is under-specified (6.2% coverage).

Suggested Next Analyses

  • Pair this hub with a human_eval-heavy hub to validate judge-model calibration.
  • Stratify by benchmark (ALFWorld vs BBQ) before comparing methods.
  • Track metric sensitivity by reporting both accuracy and cost.

Recommended Queries

LLM-as-Judge ProtocolsBenchmark Slice: ALFWorldMetric Slice: accuracyIAA-Reported EvaluationsRecent High-Signal Papers
Known Limitations
  • Only 3.7% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Rater population is under-specified (2.5% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.
Research Utility Snapshot (Detailed)

Evaluation Modes

  • Automatic Metrics (9)
  • Llm As Judge (2)
  • Simulation Env (2)

Top Metrics

  • Accuracy (9)
  • Cost (8)
  • F1 (4)
  • Agreement (2)

Top Benchmarks

  • ALFWorld (2)
  • BBQ (1)
  • BrowseComp (1)
  • DeXposure Bench (1)

Quality Controls

  • Calibration (2)
  • Inter Annotator Agreement Reported (2)

Papers In This Archive Slice

Recent Archive Slices

2026-06-18 (57)