Human Feedback and Eval Paper Explorer

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Here we present DeepRare, a multi-agent system for rare disease differential diagnosis decision support powered by large language models, integrating over 40 specialized tools and up-to-date knowledge sources.
• In human-phenotype-ontology-based tasks, it achieves an average Recall@1 of 57.18%, outperforming the next-best method by 23.79%; in multi-modal tests, it reaches 69.1% compared with Exomiser's 55.9% on 168 cases.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Comprehensive evaluation, including lexical, semantic, LLM-based, and expert-driven assessments with inter-annotator agreement, shows that the wrapper substantially improves factual accuracy, coherence, and completeness compared to…
• This work establishes both a new benchmark dataset and a generalizable generation framework, paving the way for future research in AI-assisted legal drafting.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• As LLM-driven health applications are increasingly adopted, rigorous and efficient one-sided evaluation methodologies are crucial to ensure response quality across multiple dimensions, including accuracy, personalization and safety.
• In this work, we introduce Adaptive Precise Boolean rubrics: an evaluation framework that streamlines human and automated evaluation of open-ended questions by identifying gaps in model responses using a minimal set of targeted rubrics…

Match reason: Keyword overlap 1/1 across title and protocol fields.

• To address this gap, we introduce the Business and Finance Fundamentals Benchmark (BFF-Bench), a dataset of 160 challenging questions and long-form responses authored by financial professionals.
• We demonstrate that providing the judges with expert-written references largely mitigates this issue, highlighting the limits of using LLM-as-a-Judge without any form of human verification.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• In this paper, we conduct a systematic evaluation of several LLMs, including GPT-3.5/4, FLAN-T5, and Llama 3, across zero-shot, few-shot, and explanation-based prompting settings.
• To enable principled evaluation under real-world topic diversity, we introduce a new dataset of out-of-domain news headlines covering diverse subjects.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• In our experiments, we demonstrate how standard reward modeling (e.g., Bradley-Terry) and LLM-as-Judge evaluation methods fail to account for divergence between annotators.
• To address these issues, we develop methods for identifying diverging preferences to mitigate their influence in evaluations and during LLM training.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We introduce an evaluation framework for testing affective cognition in foundation models.
• We evaluate the abilities of foundation models (GPT-4, Claude-3, Gemini-1.5-Pro) and humans (N = 567) across carefully selected conditions.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We introduce natural language unit tests, a paradigm that decomposes response quality into explicit, testable criteria, along with a unified scoring model, LMUnit, which combines multi-objective training across preferences, direct ratings,…
• Through controlled human studies, we show this paradigm significantly improves inter-annotator agreement and enables more effective LLM development workflows.

Match reason: Matched by broad semantic/index fallback.

• Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Match reason: Matched by broad semantic/index fallback.

• Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.