HFEPX Hub
Updated from current HFEPX corpus (Feb 27, 2026). 20 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Pairwise. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 20 papers for CS.HC Papers (Last 90 Days). Dominant protocol signals include automatic metrics, simulation environments, human evaluation, with frequent benchmark focus on multiple benchmark families and metric focus on accuracy, cost. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
40% of papers report explicit human-feedback signals, led by expert verification.
Evidence: TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery , "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems , An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems
automatic metrics appears in 80% of papers in this hub.
Evidence: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models , Exploring Human-Machine Coexistence in Symmetrical Reality , Evaluating the Usage of African-American Vernacular English in Large Language Models
multi-agent setups appears in 15% of papers, indicating agentic evaluation demand.
Evidence: TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery , LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , Dynamic Personality Adaptation in Large Language Models via State Machines
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , Dynamic Personality Adaptation in Large Language Models via State Machines , When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models
Rater context is mostly domain experts, and annotation is commonly pairwise annotation; use this to scope replication staffing.
Evidence: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery , "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems
Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.
Evidence: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , Dynamic Personality Adaptation in Large Language Models via State Machines , When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models
Papers with explicit human feedback
Coverage is usable but incomplete (40% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is a replication risk (0% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (35% vs 35% target).
Papers with known rater population
Coverage is strong (35% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (10% vs 35% target).
1. LLM Novice Uplift on Dual-Use, In Silico Biology Tasks
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Dynamic Personality Adaptation in Large Language Models via State Machines
Start here for detailed protocol reporting, including rater and quality-control evidence.
Adds automatic metrics for broader coverage within this hub.
5. Exploring Human-Machine Coexistence in Symmetrical Reality
Adds automatic metrics for broader coverage within this hub.
6. Evaluating the Usage of African-American Vernacular English in Large Language Models
Adds automatic metrics for broader coverage within this hub.
7. SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
Adds automatic metrics with expert verification for broader coverage within this hub.
8. "Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems
Adds automatic metrics with expert verification for broader coverage within this hub.
human_eval vs automatic_metrics
both=0, left_only=1, right_only=16
0 papers use both Human Eval and Automatic Metrics.
automatic_metrics vs simulation_env
both=0, left_only=16, right_only=3
0 papers use both Automatic Metrics and Simulation Env.
simulation_env vs human_eval
both=0, left_only=3, right_only=1
0 papers use both Simulation Env and Human Eval.
Metric Brief
accuracy
Coverage: 2 papers (10%)
2 papers (10%) mention accuracy.
Examples: Modeling Distinct Human Interaction in Web Agents , What Do LLMs Associate with Your Name? A Human-Centered Black-Box Audit of Personal Data
Metric Brief
cost
Coverage: 2 papers (10%)
2 papers (10%) mention cost.
Examples: When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models , SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
Metric Brief
precision
Coverage: 2 papers (10%)
2 papers (10%) mention precision.
Examples: Dynamic Personality Adaptation in Large Language Models via State Machines , An Expert Schema for Evaluating Large Language Model Errors in Scholarly Question-Answering Systems
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , Dynamic Personality Adaptation in Large Language Models via State Machines
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , Dynamic Personality Adaptation in Large Language Models via State Machines
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: LLM Novice Uplift on Dual-Use, In Silico Biology Tasks , TherapyProbe: Generating Design Knowledge for Relational Safety in Mental Health Chatbots Through Adversarial Simulation , Dynamic Personality Adaptation in Large Language Models via State Machines
Chen Bo Calvin Zhang, Christina Q. Knight, Nicholas Kruus, Jason Hausenloy, Pedro Medeiros · Feb 26, 2026 · Citations: 0
Large language models (LLMs) perform increasingly well on biology benchmarks, but it remains unclear whether they uplift novice users -- i.e., enable humans to perform better than with internet-only resources.
Joydeep Chandra, Satyam Kumar Navneet, Yong Zhang · Feb 26, 2026 · Citations: 0
As mental health chatbots proliferate to address the global treatment gap, a critical question emerges: How do we design for relational safety the quality of interaction patterns that unfold across conversations rather than the correctness
Leon Pielage, Ole Hätscher, Mitja Back, Bernhard Marschall, Benjamin Risse · Feb 25, 2026 · Citations: 0
This work demonstrates the feasibility of modular, personality-adaptive architectures for education, customer support, and broader human-computer interaction.
Satyam Kumar Navneet, Joydeep Chandra, Yong Zhang · Feb 25, 2026 · Citations: 0
Large Language Models (LLMs) are increasingly used to ``professionalize'' workplace communication, often at the cost of linguistic identity.
Zhenliang Zhang · Feb 25, 2026 · Citations: 0
In the context of the evolution of artificial intelligence (AI), the interaction between humans and AI entities has become increasingly salient, challenging the conventional human-centric paradigms of human-machine interaction.
Deja Dunlap, R. Thomas McCoy · Feb 25, 2026 · Citations: 0
In AI, most evaluations of natural language understanding tasks are conducted in standardized dialects such as Standard American English (SAE).
David Anugraha, Vishakh Padmakumar, Diyi Yang · Feb 24, 2026 · Citations: 0
Based on this formulation, we introduce SparkMe, a multi-agent LLM interviewer that performs deliberative planning via simulated conversation rollouts to select questions with high expected utility.
Xinfeng Li, Shenyu Dai, Kelong Zheng, Yue Xiao, Gelei Deng · Feb 24, 2026 · Citations: 0
Large language model (LLM) agents are rapidly becoming trusted copilots in high-stakes domains like software development and healthcare.
Anna Martin-Boyle, William Humphreys, Martha Brown, Cara Leckey, Harmanpreet Kaur · Feb 24, 2026 · Citations: 0
Current evaluation metrics for testing LLM reliability are primarily automated approaches that prioritize efficiency and scalability, but lack contextual nuance and fail to reflect how scientific domain experts assess LLM outputs in practic
Anna Martin-Boyle, Cara A. C. Leckey, Martha C. Brown, Harmanpreet Kaur · Feb 24, 2026 · Citations: 0
Large language models (LLMs) are increasingly used in scholarly question-answering (QA) systems to help researchers synthesize vast amounts of literature.
Ian Steenstra, Paola Pedrelli, Weiyan Shi, Stacy Marsella, Timothy W. Bickmore · Feb 23, 2026 · Citations: 0
Large Language Models (LLMs) are increasingly utilized for mental health support; however, current safety benchmarks often fail to detect the complex, longitudinal risks inherent in therapeutic dialogue.
Emma Jiren Wang, Siying Hu, Zhicong Lu · Feb 23, 2026 · Citations: 0
As a primary channel for sustaining modern intimate relationships, instant messaging facilitates frequent connection across distances.
Zhiqin Qian, Ryan Diaz, Sangwon Seo, Vaibhav Unhelkar · Feb 20, 2026 · Citations: 0
When training artificial intelligence (AI) to perform tasks, humans often care not only about whether a task is completed but also how it is performed.
Seyed Hossein Alavi, Zining Wang, Shruthi Chockkalingam, Raymond T. Ng, Vered Shwartz · Feb 20, 2026 · Citations: 0
Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held
Erik Derner, Dalibor Kučera, Aditya Gulati, Ayoub Bagheri, Nuria Oliver · Feb 19, 2026 · Citations: 0
Conversational agents increasingly mediate everyday digital interactions, yet the effects of their communication style on user experience and task success remain unclear.
Faria Huq, Zora Zhiruo Wang, Zhanqiu Guo, Venu Arvind Arangarajan, Tianyue Ou · Feb 19, 2026 · Citations: 0
Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold.
Dimitri Staufer, Kirsten Morehouse · Feb 19, 2026 · Citations: 0
Large language models (LLMs), and conversational agents based on them, are exposed to personal data (PD) during pre-training and during user interactions.
Nusrat Jahan Lia, Shubhashis Roy Dipta · Feb 19, 2026 · Citations: 0
The core theme of bidirectional alignment is ensuring that AI systems accurately understand human intent and that humans can trust AI behavior.
Aruna Sankaranarayanan, Amir Zur, Atticus Geiger, Dylan Hadfield-Menell · Feb 17, 2026 · Citations: 0
Where should we intervene in a language model (LM) to control behaviors that are diffused across many tokens of a long-form response?
Shiwei Hong, Lingyao Li, Ethan Z. Rong, Chenxinran Shen, Zhicong Lu · Feb 16, 2026 · Citations: 0
Prior work has explored multi-turn interaction and feedback for LLM writing, but evaluations still largely center on prompts and localized feedback, leaving persistent public reception in online communities underexamined.
