HFEPX Metric Hub
Updated from current HFEPX corpus (2026-10-13). This page tracks 29 papers for Helpfulness.
Updated from current HFEPX corpus (2026-10-13). This page tracks 29 papers for Helpfulness. Use it to compare how helpfulness is measured across human feedback and evaluation studies.
Useful for background comparison, but still validate benchmark and protocol details in the linked papers. Quality band: Medium .
Metric Coverage
100.0%
29 sampled papers include metric names.
Benchmark Anchoring
13.8%
Papers with explicit dataset/benchmark anchors for fair comparison.
Quality Controls
0.0%
0 papers report calibration/adjudication/IAA controls.
Recommended next step: Treat this as directional signal only; metric reporting is present but benchmark anchoring is still thin.
Main limitation: Benchmark coverage is still thin, so avoid treating this page as a definitive guide to the metric.
Ranked for metric reporting completeness and comparability.
Jun 15, 2026 · Citations: 0 · Score: 8.0
Metrics: Task success, Helpfulness · Eval: Automatic Metrics
Apr 19, 2026 · Citations: 0 · Score: 8.0
Metrics: Helpfulness, Harmlessness · Eval: Automatic Metrics
Oct 5, 2026 · Citations: 0 · Score: 7.5
Metrics: Helpfulness · Eval: Automatic Metrics
Sep 29, 2026 · Citations: 0 · Score: 7.5
Metrics: Helpfulness · Eval: Automatic Metrics
Apr 8, 2026 · Citations: 0 · Score: 7.5
Metrics: Accuracy, Helpfulness · Eval: Human Eval, Automatic Metrics
Sep 17, 2025 · Citations: 0 · Score: 6.9
Metrics: Helpfulness · Eval: Automatic Metrics
Compare metric, benchmark, and evaluation context side by side.
| Paper | Metrics | Benchmarks | Eval Modes | Quality Controls |
|---|---|---|---|---|
| Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational Impact in LLM Tutors Jun 15, 2026 | Task success, Helpfulness | Mathtutorbench, Tutorbench | Automatic Metrics | Not reported |
| Cat-DPO: Category-Adaptive Safety Alignment Apr 19, 2026 | Helpfulness, Harmlessness | DROP | Automatic Metrics | Not reported |
| Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge Oct 5, 2026 | Helpfulness | Not reported | Automatic Metrics | Not reported |
| Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment Sep 29, 2026 | Helpfulness | Not reported | Automatic Metrics | Not reported |
| Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization Apr 8, 2026 | Accuracy, Helpfulness | Rewardbench | Human Eval, Automatic Metrics | Not reported |
| A Simple and Efficient Jailbreak Method Exploiting LLMs' Helpfulness Sep 17, 2025 | Helpfulness | AdvBench | Automatic Metrics | Not reported |
| MAM-AI: An On-Device Medical Retrieval-Augmented Generation System for Nurses and Midwives in Zanzibar Jun 28, 2026 | Latency, Helpfulness | Not reported | Automatic Metrics | Not reported |
| TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching May 12, 2026 | Helpfulness, Harmlessness | Not reported | Automatic Metrics | Not reported |
| How Value Induction Reshapes LLM Behaviour May 8, 2026 | Helpfulness, Harmlessness | Not reported | Automatic Metrics | Not reported |
| SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs Apr 24, 2026 | Helpfulness | Not reported | Automatic Metrics | Not reported |
Strong: Human feedback
Human feedback is present in 20 of 29 papers.
Gap: Quality controls
Quality controls is present in 0 of 29 papers.
Gap: Benchmarks
Benchmarks is present in 4 of 29 papers.
Strong: Metrics
Metrics is present in 29 of 29 papers.
Gap: Known rater population
Known rater population is present in 1 of 29 papers.
Moderate: Known annotation unit
Known annotation unit is present in 9 of 29 papers.
Top Metrics
Evaluation Modes
Top Benchmarks
Agentic Mix
Sebastian Joseph, Zichao Wang, Jennifer Healey, Alexa Siu, Junyi Jessy Li · Sep 29, 2026 · Citations: 0
On a dataset of paired submission and camera-ready papers from ICLR and NeurIPS, we compare the helpfulness of questions from GPT models with or without full paper context to that of human reviewers.
Wonjun Lee, Kyungsik Yang, Gaeun Ji, Vaidehi Patil, Haon Park · Oct 5, 2026 · Citations: 0
To address these limitations, we introduce LADE (Latent Safety Signals for Defense), which leverages latent safety signals extracted by contrasting harmful and benign queries from dark knowledge (i.e., information carried by the output…
Haotian Deng, Wenbin Xing, Gang Xu, Tao He, Jinkai Zheng · Sep 29, 2026 · Citations: 0
Existing safety methods often require large preference datasets, costly multi-rollout training, or additional safeguards at inference time.
Junyi Yao, Zihao Zheng, Baichuan Li · Jun 15, 2026 · Citations: 0
We propose a lightweight diagnostic based on the gap between solving-oriented and pedagogy-oriented benchmark performance.
Hoejoon Kwon, Byeonggeuk Lim, Kahyeon Kim, YoungBin Kim · Aug 31, 2026 · Citations: 0
Safety alignment is essential for deploying large language models, requiring systems to prevent harmful compliance while preserving helpfulness on benign requests.
Tiankai Yang, Yi Nian, Xinyuan Li, Ruiyao Xu, Henry Peng Zou · Apr 19, 2026 · Citations: 0
Aligning large language models with human preferences must balance two competing goals: responding helpfully to legitimate requests and reliably refusing harmful ones.
Yi Ren · Jun 28, 2026 · Citations: 0
We evaluate the exact deployed configuration with a layered methodology -- retriever, generator under oracle context, end-to-end, and latency -- scored by LLM judges validated against physician rubrics.
Rheeya Uppaal, Seungwoo Lyu, Selina Sung, Junjie Hu · Jul 2, 2026 · Citations: 0
We introduce OpenSafeIntent, a benchmark of controlled prompt-sets that vary intent while holding the underlying task fixed.
Shree Harsha Bokkahalli Satish, Christoph Minixhofer, Maria Teleki, James Caverlee, Ondřej Klejch · Mar 15, 2026 · Citations: 0
We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions across six English accents and two gender presentations, keeping linguistic content constant through voice…
Long Phan, Devin Kim, Alexander Pan, Alice Blair, Adam Khoja · May 21, 2026 · Citations: 0
We show that PCT preserves overall helpfulness, substantially reduces covert political bias, and generalizes to held-out benchmarks.
Sanskar Pandey, Ruhaan Chopra, Angkul Puniya, Sohom Pal · Oct 19, 2025 · Citations: 0
This latent bias, known as sycophancy, manifests as a preference for user agreement over principled reasoning.
Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van, Duy Minh Ho Nguyen, Khoa D. Doan · May 12, 2026 · Citations: 0
Direct Preference Optimization (DPO) is a widely used RL-free method for aligning language models from pairwise preferences, but it models preferences over full sequences even though generation is driven by per-token decisions.
Arnav Arora, Natalie Schluter, Katherine Metcalf, Maartje ter Hoeve · May 8, 2026 · Citations: 0
This is done to increase utility, ensure safety, and improve the experience of the people interacting with the model.
Ying Zhang, Congyu Qiao, Xin Geng, Ning Xu · May 8, 2026 · Citations: 0
Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones.
Jinyan Su, Jinpeng Zhou, Claire Cardie, Wen Sun · May 8, 2026 · Citations: 0
Personalized LLMs can significantly enhance user experiences by tailoring responses to preferences such as helpfulness, conciseness, and humor.
Sihang, Zhao, Kangrui Yu, Youliang Yuan, Pinjia He · Apr 24, 2026 · Citations: 0
To enable systematic study, we unify and formalize safe, helpful, and pedagogical behaviors with a knowledge-mastery graph and introduce SHAPE, a benchmark of 9,087 student-question pairs for evaluating tutoring behavior under adversarial…
Xiangxu Zhang, Lei Li, Yanyun Zhou, Xiao Zhou, Yingying Zhang · Oct 10, 2025 · Citations: 0
Recent dynamic evaluations offer a promising alternative, but often remain insufficient for diagnosis-oriented benchmarking, with limited coverage of clinically grounded confounders and trustworthiness beyond accuracy.
Qiyao Ma, Dechen Gao, Rui Cai, Boqi Zhao, Hanchu Zhou · Apr 8, 2026 · Citations: 0
Pluralistic alignment has emerged as a critical frontier in the development of Large Language Models (LLMs), with reward models (RMs) serving as a central mechanism for capturing diverse human values.
Simona-Vasilica Oprea, Adela Bâra · Apr 1, 2026 · Citations: 0
Using the Anthropic HHRLHF dataset, we evaluate ten diverse large language models LLMs under a standard pairwise preference setting, where baseline performance remains below 0.74 ROC AUC, highlighting the difficulty of the task.
Jin Ma, Jingwen Yan, Mohammed Aldeen, Ethan Anderson, Taran Kavuru · Mar 23, 2026 · Citations: 0
However, its reliance on human contributors limits both the timeliness and scalability.
Yaswanth Chittepu, Ativ Joshi, Rajarshi Bhattacharjee, Scott Niekum · Mar 11, 2026 · Citations: 0
Safe Reinforcement Learning from Human Feedback (RLHF) typically enforces safety through expected cost constraints, but the expectation captures only a single statistic of the cost distribution and fails to account for distributional…
Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu, Christopher A. Choquette-Choo, Steph Lin · Mar 11, 2026 · Citations: 0
IH is key to defending against jailbreaks, system prompt extractions, and agentic prompt injections.
Punyajoy Saha, Sudipta Halder, Debjyoti Mondal, Subhadarshi Panda · Mar 7, 2026 · Citations: 0
Safety alignment is critical for deploying large language models (LLMs) in real-world applications, yet most existing approaches rely on large human-annotated datasets and static red-teaming benchmarks that are costly, difficult to scale,…
Yule Wen, Yanzhe Zhang, Jianxun Lian, Xiaoyuan Yi, Xing Xie · Mar 3, 2026 · Citations: 0
LLM agents increasingly act on users' personal information, yet existing privacy defenses remain limited in both design and adaptability.
Ruochen Mao, Yuling Shi, Xiaodong Gu, Jiaheng Wei · Oct 23, 2025 · Citations: 0
To address this challenge, we introduce Robust Preference Selection (RPS), a post-hoc, training-free method by leveraging directional neighborhood consensus.
Xuan Luo, Yue Wang, Zefeng He, Geng Tu, Jing Li · Sep 17, 2025 · Citations: 0
This study reveals a critical safety blind spot in modern LLMs: learning-style queries, which closely resemble ordinary educational questions, can reliably elicit harmful responses.
Apurv Verma, NhatHai Phan, Shubhendu Trivedi · Jun 4, 2025 · Citations: 0
In practice, sampling as few as two to four candidates largely restores unwatermarked alignment performance in truthfulness, safety, and helpfulness, without hurting watermark detection.
Hanjiang Hu, Alexander Robey, Changliu Liu · Feb 28, 2025 · Citations: 0
To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues.
Yuhan Cheng, Hancheng Ye, Hai Helen Li, Jingwei Sun, Yiran Chen · Feb 14, 2026 · Citations: 0
We propose PrivAct, a contextual privacy-aware multi-agent learning framework that internalizes contextual privacy preservation directly into models' generation behavior for privacy-compliant agentic actions.
