Ilya Chekin, Vyacheslav Malyugin, Vladimir Chirkov, Mikhail Yurushkin · Oct 2, 2026 · Citations: 0
Human Eval General
- This is operational, non-blinded agreement rather than an independent human evaluation.
← Back to explorer Building Interpretable Feature Representations for Resume-Vacancy Matching by Distilling Production LLM Signals EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation TaxCE : A Framework for Automated Taxonomy Construction and Evaluation at Scale Sequential Trajectories and Simultaneous Blending: Multi-Emotion Modeling for Instruction-Following TTS Controllable Affective Generation via Latent Vector Steering AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design How China-Origin Vision-Language Models Move from Refusal to Reframing in State Alignment Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder ConVAWG: A Retrieval-Grounded Framework for Controlled Synthetic Dialogue Generation in Violence Against Women and Girls Evidence-Supported Credit Risk Report Generation Using News-Centric Financial Knowledge Graphs LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning AI translation of literary texts is "fine", but readers still prefer human translations PhoneBuddy: Training Open Models for Agentic Phone Use LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback DirectorBench: Diagnosing Long-Form Video Generation with Personalized Multi-Agent Evaluation Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs Annotation Quality in Aspect-Based Sentiment Analysis: A Case Study Comparing Experts, Students, Crowdworkers, and Large Language Model
Tag: Human Eval
Human Eval papers in the current HFEPX explorer (123 papers).
Papers in tag: 123
Running a Human Eval study?
Post a Job →Research Utility Snapshot
Evaluation Modes
- Human Eval (20)
- Llm As Judge (2)
Human Feedback Types
- Pairwise Preference (6)
- Red Team (1)
Required Expertise
- General (12)
- Multilingual (4)
- Coding (3)
Kuan-Po Huang, Haohe Liu, Puyuan Peng, Haibin Wu, Zhaoheng Ni, Hung-yi Lee · Sep 29, 2026 · Citations: 0
Pairwise Preference Human Eval General
- Human evaluation further shows a relative improvement up to 35.0% in the rate at which listeners correctly identified the dominant requested emotion and up to a 17.3% improvement in fidelity, while listeners prefer EmoRES for naturalness in…
Sandeep Sricharan Mukku, Albert Aristotle Nanda, Rohit Pyati · Aug 31, 2026 · Citations: 0
Human Eval General
- Existing approaches either produce shallow hierarchies, neglect long-tail topics, or lack rigorous evaluation frameworks.
- We also introduce three corpus-grounded evaluation metrics, Exclusivity, Exhaustivity, and Granularity (EEG), and integrate them into a metrics-in-the-loop iterative refinement mechanism that diagnoses deficiencies and applies targeted…
Yan Zhou, Yun Hong, Yang Feng · Aug 31, 2026 · Citations: 0
Pairwise Preference Human Eval General
- Human evaluation prefers HybridEmo to CosyVoice 3 and EmoVoice-0.5B, with nearly balanced preferences against Qwen3-TTS.
Xixian Yong, Siyuan Chang, Yingying Zhang, Xian Wu, Xiao Zhou · Aug 26, 2026 · Citations: 0
Human Eval General
- Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, establishing EmoVec as a practical inference-time method for affective control in deployed LLMs.
Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li · Aug 13, 2026 · Citations: 0
Pairwise Preference Human Eval Coding
- In this paper, we present AutoDesign, a framework that aligns with human design priors, where a meta-harness optimizer guides a code agent to recursively improve harness based on rollout feedback.
- Across seven controlled code-agent-model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%).
Guang Yang, Fengchen Liu, Alex Wang, Homa Hosseinmardi, Amir Ghasemian · Aug 12, 2026 · Citations: 0
Human Eval General
- We construct a balanced benchmark of 200 core entries spanning ten politically sensitive topics, plus a seven-variant visual-abstraction probe, and run nine vision-language models (VLMs), seven China-origin and two non-China, across four…
- We find that (i) Chinese-language prompting roughly triples the odds of state-aligned framing, within every model; (ii) China-origin models reframe more than non-China models (direction robust across judges and human raters; magnitude…
Huaxuan Wang, Huimin Wang, Ruiyu Zhang, Yingjie Li, Yitao Duan · Aug 12, 2026 · Citations: 0
Human Eval CodingMultilingual
- It achieves high intelligibility and speaker similarity on public benchmarks.
- On the CV3-Eval cross-lingual benchmark, Confucius4-TTS obtains an average WER of 3.73% across six directions.
Chen Lyu, Xingwei Tan, Simon Cullen, Shelley Wilson, Lois Arthurs, Arshad Jhumka · Aug 11, 2026 · Citations: 0
Human EvalLlm As Judge Law
- Extensive human evaluation, LLM-as-Judge assessment, ablations, and downstream tasks show strong dialogue quality and domain fidelity.
Rocio Jimenez-Villen, Ziwei Xu, Ying Chen, Oscar Araque, Ryutaro Ichise · Jul 1, 2026 · Citations: 0
Human Eval Coding
- Automatic and human evaluations show that automated hallucination detection and quality assessment remain unreliable, making expert judgment indispensable.
Nina Hosseini-Kivanani, Sandipana Dowerah · Jun 30, 2026 · Citations: 0
Human Eval Multilingual
- LuxEmo is derived from Radio Télévision Luxembourg (RTL) youth broadcasts, using automated detection followed by human validation.
- We propose a semi-automatic curation workflow combining voice activity detection, denoising, language identification, LuxASR-based segmentation, automatic emotion prediction, lexical cues, and targeted human review.
William Orwig, Roger E. Beaty · Jun 29, 2026 · Citations: 0
Human Eval General
- The present research asks whether multimodal large language models (LLMs) can serve as judges of visual creativity zero-shot (without any fine-tuning or examples of human ratings) and whether their "reasoning" output offers an interpretable…
- We tested six multimodal LLMs (Gemini 3 Flash, Gemma 4 31B IT, GPT-5.4 Mini, GLM-5v Turbo, Kimi K2.5, and Qwen 3.6 Plus) on 992 AI-generated images (based on human-written prompts) and 1,500 hand-drawn sketches scored for creativity by…
Yves Ferstler, Adam Podoxin, Ty Brassington, Roman Grundkiewicz, Maite Taboada, Marzena Karpinska · Jun 24, 2026 · Citations: 0
Pairwise Preference Human EvalLlm As Judge Multilingual
- While the content may be rendered adequately, we do not know enough about how readers experience it in terms of immersiveness and literary effect, aspects poorly captured by automatic machine translation metrics or human evaluation…
- We ask 15 avid readers to compare recently published human translations (HT) to machine translations (MT) generated with an agentic large language model (LLM)-based pipeline, for 15 recent novels in French, Polish, and Japanese and…
Zhengyang Tang, Xin Lai, Pengyuan Lyu, Xinyuan Wang, Tianyi Bai, Chenxin Li · Jun 22, 2026 · Citations: 0
Human Eval General
- Phones are becoming an important execution surface for general-purpose agents, but training open models for reliable phone use remains difficult because the environment that matters at deployment, real devices running real apps, is slow,…
- We present PhoneBuddy, a training recipe and open-model line for agentic phone use that combines a real-app environment with a mock-app environment, PhoneWorld, which reconstructs runnable mock apps from real GUI usage structure.
Jiwon Kim, Maya Ajit, Sherry Gong, Soorya Ram Shimgekar, Dong Whi Yoo, Eshwar Chandrasekharan · May 28, 2026 · Citations: 0
Pairwise Preference Human Eval General
- Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empathy, and safety often requires substantial compute, expert input, and labeled data.
- LLUMI consists of two complementary components: a generation model (GM), which drafts supportive responses to mental health queries, and an improvement model (IM), which revises an initial human-crafted response.
Jiamin Chen, Qianben Chen, Jiawen Zhang, Yidi Wu, Yuchen Li, Xiaokun Zhang · May 28, 2026 · Citations: 0
Pairwise Preference Human Eval Medicine
- However, evaluating such videos remains challenging, since existing benchmarks largely focus on local visual quality, short-horizon temporal consistency, or generic prompt alignment, and provide limited diagnosis of workflow failures and…
- We introduce DirectorBench, a personalized multi-agent diagnostic benchmark for long-form video generation.
Jingyi Kang, Junyu Lu, Bo Xu, Hongbo Wang, Linlin zong, Roy Ka-Wei Lee · May 21, 2026 · Citations: 0
Red Team Human Eval General
- We introduce Chinese Implicit Toxicity Attack (CITA), a controlled red-team evaluation and defense-data generation framework, not a deployable evasion tool.
- On CITA-generated evaluation samples, the seven tested detectors exhibit substantial missed-detection risks, reaching an average ASR of 69.48%; human evaluation further confirms preserved harmfulness and increased implicitness/evasiveness.
Tom Sander, Hongyan Chang, Tomáš Souček, Tuan Tran, Valeriu Lacatusu, Sylvestre-Alvise Rebuffi · May 12, 2026 · Citations: 0
Human Eval Multilingual
- TextSeal strictly dominates baselines like SynthID-text in detection strength and is robust to dilution, maintaining confident localized detection even in heavily mixed human/AI documents.
- The scheme is theoretically distortion-free, and evaluation across reasoning benchmarks confirms that it preserves downstream performance; while a multilingual human evaluation (6000 A/B comparisons, 5 languages) shows no perceptible…
Stefano Bannò, Kate Knill, Mark Gales · May 5, 2026 · Citations: 0
Human Eval General
- In this study, we propose a novel self-referential assessment evaluation framework that focuses on identifying intra-learner strengths and weaknesses rather than assessing inter-learner rankings.
- We conduct experiments on the publicly available ICNALE GRA, a uniquely dense second-language writing dataset annotated holistically and analytically by up to 80 trained raters.
Niklas Donhauser, Jakob Fehle, Nils Constantin Hellwig, Markus Weinberger, Udo Kruschwitz, Christian Wolff · May 5, 2026 · Citations: 0
Human Eval General
- Annotation quality is compared using Inter-Annotator Agreement (IAA) and its impact on downstream model performance for different ABSA subtasks.
- The evaluation focuses on Aspect Category Sentiment Analysis (ACSA) and Target Aspect Sentiment Detection (TASD).