Research Utility Snapshot
Evaluation Modes
- Automatic Metrics (11)
- Human Eval (5)
- Llm As Judge (2)
Human Feedback Types
- Rubric Rating (17)
- Expert Verification (3)
- Pairwise Preference (3)
Required Expertise
- General (11)
- Coding (3)
- Law (2)
RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning Yukun Chen, Jiaming Li, Longze Chen, Ze Gong, Jingpeng Li, Zhen Qin · Feb 25, 2026 · Citations: 0
Rubric Rating Automatic Metrics General
- Extensive experiments on various visual reasoning benchmarks show that RuCL yields a remarkable +7.83% average improvement over the Qwen2.5-VL-7B model, achieving a state-of-the-art accuracy of 60.06%.
SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing Yifei Xu, Guilherme Potje, Shivam Shandilya, Tiancheng Yuan, Leonardo de Oliveira Nunes, Rakshanda Agarwal · Feb 24, 2026 · Citations: 0
Rubric RatingRed Team Automatic Metrics General
Personalized Prediction of Perceived Message Effectiveness Using Large Language Model Based Digital Twins Jasmin Han, Janardan Devkota, Joseph Waring, Amanda Luken, Felix Naughton, Roger Vilardaga · Feb 23, 2026 · Citations: 0
Rubric Rating Automatic Metrics General
KLong: Training LLM Agent for Extremely Long-horizon Tasks Yue Liu, Zhiyuan Hu, Flood Sung, Jiaheng Zhang, Bryan Hooi · Feb 19, 2026 · Citations: 0
Rubric Rating Automatic Metrics General
- This paper introduces KLong, an open-source LLM agent trained to solve extremely long-horizon tasks.
- Specifically, we first activate basic agentic abilities of a base model with a comprehensive SFT recipe.
Quantifying and Mitigating Socially Desirable Responding in LLMs: A Desirability-Matched Graded Forced-Choice Psychometric Study Kensuke Okada, Yui Furukawa, Kyosuke Bunji · Feb 19, 2026 · Citations: 0
Rubric Rating Automatic Metrics General
- Human self-report questionnaires are increasingly used in NLP to benchmark and audit large language models (LLMs), from persona consistency to safety and bias assessments.
- We propose a psychometric framework to quantify and mitigate SDR in questionnaire-based evaluation of LLMs.
Discovering Implicit Large Language Model Alignment Objectives Edward Chen, Sanmi Koyejo, Carlos Guestrin · Feb 17, 2026 · Citations: 0
Rubric Rating Human Eval General
- To address these limitations, we introduce Obj-Disco, a framework that automatically decomposes an alignment reward signal into a sparse, weighted combination of human-interpretable natural language objectives.
- Extensive evaluations across diverse tasks, model sizes, and alignment algorithms demonstrate the framework's robustness.
Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation Shiwei Hong, Lingyao Li, Ethan Z. Rong, Chenxinran Shen, Zhicong Lu · Feb 16, 2026 · Citations: 0
Pairwise PreferenceRubric Rating Human Eval General
- Prior work has explored multi-turn interaction and feedback for LLM writing, but evaluations still largely center on prompts and localized feedback, leaving persistent public reception in online communities underexamined.
- We test whether broadcast community discussion improves stand-up comedy writing in a controlled multi-agent sandbox: in the discussion condition, critic and audience threads are recorded, filtered, stored as social memory, and later retriev
Small Reward Models via Backward Inference Yike Wang, Faeze Brahman, Shangbin Feng, Teng Xiao, Hannaneh Hajishirzi, Yulia Tsvetkov · Feb 14, 2026 · Citations: 0
Rubric Rating Automatic Metrics Coding
- However, the dominant LLM-as-a-Judge paradigm relies on the strong reasoning capabilities of large models, while alternative approaches require reference responses or explicit rubrics, limiting flexibility and broader accessibility.
- Evaluations across four domains using 13 small language models show that FLIP outperforms LLM-as-a-Judge baselines by an average of 79.6%.
The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footage Preni Golazizian, Elnaz Rahmati, Jackson Trager, Zhivar Sourati, Nona Ghazizadeh, Georgios Chochlakis · Feb 10, 2026 · Citations: 0
Pairwise PreferenceRubric Rating Human Eval Law
- To this end, we (i) develop a domain-specific evaluation rubric grounded in procedural justice theory, LAPD training materials, and extensive fieldwork; (ii) introduce a rubric-driven preference data construction framework for perspective-c
Document Reconstruction Unlocks Scalable Long-Context RLVR Yao Xiao, Lei Wang, Yue Deng, Guanzheng Chen, Ziqi Jin, Jung-jae Kim · Feb 9, 2026 · Citations: 0
Rubric Rating Automatic Metrics Coding
- However, it often relies on gold-standard answers or explicit evaluation rubrics provided by powerful teacher models or human experts, which are costly and time-consuming.
- In this work, we investigate unsupervised approaches to enhance the long-context capabilities of LLMs, eliminating the need for heavy human annotations or teacher models' supervision.
APEX-Agents Bertie Vidgen, Austin Mann, Abby Fennelly, John Wright Stanly, Lucas Rothman, Marco Burstein · Jan 20, 2026 · Citations: 0
Rubric RatingExpert Verification Simulation Env Law
- We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate law
- APEX-Agents requires agents to navigate realistic work environments with files and tools.
Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models Haorui Yu, Xuehang Wen, Fengrui Zhang, Qiufeng Yi · Jan 12, 2026 · Citations: 0
Rubric RatingCritique Edit Automatic Metrics Coding
- Existing benchmarks assess perception without interpretation, and common evaluation proxies, such as automated metrics and LLM-judge averaging, are unreliable for culturally sensitive generative tasks.
- We address this measurement gap with a tri-tier evaluation framework grounded in art-theoretical constructs (Section 2).
HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue Laya Iyer, Kriti Aggarwal, Sanmi Koyejo, Gail Heyman, Desmond C. Ong, Subhabrata Mukherjee · Jan 9, 2026 · Citations: 0
Pairwise PreferenceRubric Rating Human EvalLlm As Judge General
- Despite rapid progress in language models, we still lack a clear way to understand how their abilities in these interpersonal domains compare to those of humans.
- We introduce HEART, the first-ever framework that directly compares humans and LLMs on the same multi-turn emotional-support conversations.
PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions Amith Ananthram, Elias Stengel-Eskin, Lorena A. Bradford, Julia Demarest, Adam Purvis, Keith Krut · Oct 21, 2025 · Citations: 0
Rubric Rating Human EvalLlm As Judge General
- While vision-language models (VLMs) have advanced into detailed image description, evaluation remains a challenge.
- In this work, we introduce PoSh, a metric for detailed image description that uses scene graphs as structured rubrics to guide LLMs-as-a-Judge, producing aggregate scores grounded in fine-grained errors (e.g.
Toward LLM-Supported Automated Assessment of Critical Thinking Subskills Marisa C. Peczuh, Nischal Ashok Kumar, Ryan Baker, Blair Lehman, Danielle Eisenberg, Caitlin Mills · Oct 14, 2025 · Citations: 0
Rubric Rating Automatic Metrics General
- As the world becomes increasingly saturated with AI-generated content, disinformation, and algorithmic persuasion, critical thinking - the capacity to evaluate evidence, detect unreliable claims, and exercise independent judgment - is becom
- We developed a coding rubric based on an established skills progression and completed human coding for a corpus of student essays.
Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training Junkai Zhang, Zihao Wang, Lin Gui, Swarnashree Mysore Sathyendra, Jaehwan Jeong, Victor Veitch · Sep 25, 2025 · Citations: 0
Rubric Rating Automatic Metrics General
A Scalable Framework for Evaluating Health Language Models Neil Mallinar, A. Ali Heydari, Xin Liu, Anthony Z. Faranesh, Brent Winslow, Nova Hammerquist · Mar 30, 2025 · Citations: 0
Rubric RatingExpert Verification Automatic Metrics Medicine
- As LLM-driven health applications are increasingly adopted, rigorous and efficient one-sided evaluation methodologies are crucial to ensure response quality across multiple dimensions, including accuracy, personalization and safety.
- Current evaluation practices for open-ended text responses heavily rely on human experts.
