HFEPX Archive Slice

HFEPX Daily Archive: 2026-01-12

Updated from current HFEPX corpus (Mar 8, 2026). 6 papers are grouped in this daily page.

Read Full Context

Updated from current HFEPX corpus (Mar 8, 2026). 6 papers are grouped in this daily page. Common evaluation modes: Automatic Metrics. Most common rater population: Domain Experts. Common annotation unit: Multi Dim Rubric. Frequent quality control: Calibration. Frequently cited benchmark: SemEval. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Jan 12, 2026.

Papers: 6 Last published: Jan 12, 2026 Global RSS

Researcher Quick Triage

Use this archive page for time-slice monitoring (what changed in evaluation methods, metrics, and protocol quality this period). Quality band: Developing .

High-Signal Coverage

100.0%

6 / 6 papers are not low-signal flagged.

Benchmark Anchors

16.7%

Papers with benchmark/dataset mentions in extraction output.

Metric Anchors

16.7%

Papers with reported metric mentions in extraction output.

1 papers report explicit quality controls for this archive period.
Prioritize papers with both benchmark and metric anchors for reliable longitudinal comparisons.

Primary action: Use this slice as early signal only; benchmark/metric anchoring is limited for rigorous period-over-period claims.

Why This Time Slice Matters

50% of papers report explicit human-feedback signals, led by critique/edit feedback.
automatic metrics appears in 16.7% of papers in this hub.
SemEval is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways For This Period

Most common quality-control signal is rater calibration (16.7% of papers).
Rater context is mostly domain experts, and annotation is commonly multi-dimensional rubrics; use this to scope replication staffing.
Stratify by benchmark (SemEval vs Vulca-Bench) before comparing methods.

Start Here (Highest-Signal Papers In This Slice)

Ranked by protocol completeness and evidence density for faster period-over-period review.

FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures
Jan 12, 2026 · Citations: 0 · Score: 4.5

Eval: Automatic Metrics · Metrics: Bertscore
VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
Jan 12, 2026 · Citations: 0 · Score: 4.5

Eval: Not reported · Metrics: Not reported
Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models
Jan 12, 2026 · Citations: 0 · Score: 4.0

Eval: Not reported · Metrics: Not reported
Reward Modeling from Natural Language Human Feedback
Jan 12, 2026 · Citations: 0 · Score: 3.0

Eval: Not reported · Metrics: Not reported
NRR-Phi: Text-to-State Mapping for Ambiguity Preservation in LLM Inference
Jan 12, 2026 · Citations: 0 · Score: 0.0

Eval: Not reported · Metrics: Not reported
CascadeMind at SemEval-2026 Task 4: A Hybrid Neuro-Symbolic Cascade for Narrative Similarity
Jan 12, 2026 · Citations: 0 · Score: 0.0

Eval: Not reported · Metrics: Not reported

Protocol Matrix (Top 10)

Quickly compare method ingredients across this archive slice.

Paper	Eval Modes	Benchmarks	Metrics	Quality Controls
FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures Jan 12, 2026	Automatic Metrics	Not reported	Bertscore	Not reported
VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding Jan 12, 2026	Not reported	Vulca Bench	Not reported	Not reported
Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models Jan 12, 2026	Not reported	Not reported	Not reported	Calibration
Reward Modeling from Natural Language Human Feedback Jan 12, 2026	Not reported	Not reported	Not reported	Not reported
NRR-Phi: Text-to-State Mapping for Ambiguity Preservation in LLM Inference Jan 12, 2026	Not reported	Not reported	Not reported	Not reported
CascadeMind at SemEval-2026 Task 4: A Hybrid Neuro-Symbolic Cascade for Narrative Similarity Jan 12, 2026	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Strong: Papers with explicit human feedback

Coverage is strong (50% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (16.7% vs 30% target).
Moderate: Papers naming benchmarks/datasets

Coverage is usable but incomplete (33.3% vs 35% target).
Gap: Papers naming evaluation metrics

Coverage is a replication risk (16.7% vs 35% target).
Moderate: Papers with known rater population

Coverage is usable but incomplete (33.3% vs 35% target).
Moderate: Papers with known annotation unit

Coverage is usable but incomplete (33.3% vs 35% target).

Strengths

Strong human-feedback signal (50% of papers).

Known Gaps

Only 16.7% of papers report quality controls; prioritize calibration/adjudication evidence.
Metric coverage is thin (16.7% of papers mention reported metrics).

Suggested Next Analyses

Stratify by benchmark (SemEval vs Vulca-Bench) before comparing methods.
Add inter-annotator agreement checks when reproducing these protocols.

Recommended Queries

Benchmark Slice: SemEval Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 16.7% of papers report quality controls; prioritize calibration/adjudication evidence.
Metric coverage is thin (16.7% of papers mention reported metrics).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (1)

Top Metrics

Accuracy (1)

Top Benchmarks

SemEval (1)
Vulca Bench (1)

Quality Controls

Calibration (1)

Papers In This Archive Slice

FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures
Jifeng Song, Arun Das, Pan Wang, Hui Ji, Kun Zhao · Jan 12, 2026 · Citations: 0

To support high-quality supervision, we curate BioSci-Fig-Cap, a refined benchmark for panel-level grounding, alongside cross-disciplinary test suites in physics and chemistry.
VULCA-Bench: A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
Haorui Yu, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi · Jan 12, 2026 · Citations: 0

Critique Edit

We introduce VULCA-Bench, a multicultural art-critique benchmark for evaluating Vision-Language Models' (VLMs) cultural understanding beyond surface-level visual perception.
Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models
Haorui Yu, Xuehang Wen, Fengrui Zhang, Qiufeng Yi · Jan 12, 2026 · Citations: 0

Rubric RatingCritique Edit

Existing benchmarks assess perception without interpretation, and common evaluation proxies, such as automated metrics and LLM-judge averaging, are unreliable for culturally sensitive generative tasks.
Reward Modeling from Natural Language Human Feedback
Zongqi Wang, Rui Wang, Yuchuan Wu, Yiyao Yu, Pinyi Zhang · Jan 12, 2026 · Citations: 0

Pairwise PreferenceCritique Edit

To address this issue, we propose Reward Modeling from Natural Language Human Feedback (RM-NLHF), which leverages natural language feedback to obtain process reward signals, thereby mitigating the problem of limited solution space inherent…
NRR-Phi: Text-to-State Mapping for Ambiguity Preservation in LLM Inference
Kei Saito · Jan 12, 2026 · Citations: 0
CascadeMind at SemEval-2026 Task 4: A Hybrid Neuro-Symbolic Cascade for Narrative Similarity
Sebastien Kawada, Dylan Holyoak · Jan 12, 2026 · Citations: 0

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote