Human Feedback and Eval Paper Explorer

Match reason: Matches selected tags (Rubric Rating).

• To address the challenge, in this work, we propose StitchCUDA, a multi-agent framework for end-to-end GPU program generation, with three specialized agents: a Planner to orchestrate whole system design, a Coder dedicated to implementing it…
• Experiments on KernelBench show that StitchCUDA achieves nearly 100% success rate on end-to-end GPU programming tasks, with 1.72x better speedup over the multi-agent baseline and 2.73x than the RL model baselines.

Match reason: Matches selected tags (Rubric Rating).

• Effective hiring is integral to the success of an organisation, but it is very challenging to find the most suitable candidates because expert evaluation (e.g.\ interviews conducted by a technical manager) are expensive to deploy at scale.

Match reason: Matches selected tags (Rubric Rating).

• Extensive experiments on various visual reasoning benchmarks show that RuCL yields a remarkable +7.83% average improvement over the Qwen2.5-VL-7B model, achieving a state-of-the-art accuracy of 60.06%.

Match reason: Matches selected tags (Demonstrations).

Match reason: Matches selected tags (Demonstrations).

• The improvement of web agents on current benchmarks raises the question: Do today's agents perform just as well when the web changes?
• We introduce TimeWarp, a benchmark that emulates the evolving web using containerized environments that vary in UI, design, and layout.

Match reason: Matches selected tags (Rubric Rating).

• However, existing medical benchmarks remain largely static and task-isolated, failing to capture the openness, longitudinal structure, and safety-critical complexity of real-world clinical workflows.
• We introduce ClinConsensus, a Chinese medical benchmark curated, validated, and quality-controlled by clinical experts.

Match reason: Matches selected tags (Rubric Rating, Demonstrations).

• GUIDE paves the way for trusted, scalable assessment systems that align closely with human pedagogical standards.

Match reason: Matches selected tags (Rubric Rating).

• Conversational shopping assistants (CSAs) represent a compelling application of agentic AI, but moving from prototype to production reveals two underexplored challenges: how to evaluate multi-turn interactions and how to optimize tightly…
• We introduce a multi-faceted evaluation rubric that decomposes end-to-end shopping quality into structured dimensions and develop a calibrated LLM-as-judge pipeline aligned with human annotations.

Match reason: Matches selected tags (Rubric Rating).

• Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.
• We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.

Match reason: Matches selected tags (Rubric Rating).

• Empirical evaluations on teacher education and STEM datasets demonstrate that CARO significantly outperforms existing SOTA methods.

Match reason: Matches selected tags (Demonstrations).

• We present IDP (Intelligent Document Processing) Accelerator, a framework enabling agentic AI for end-to-end document intelligence with four key components: (1) DocSplit, a novel benchmark dataset and multimodal classifier using BIO tagging…

Match reason: Matches selected tags (Rubric Rating).

• Notably, our method achieves fairness gains comparable to using additional real human data, offering a cost-effective solution for ensuring equitable scoring in large-scale assessments.

Match reason: Matches selected tags (Rubric Rating).

• We decompose physician disagreement in the HealthBench medical AI evaluation dataset to understand where variance resides and what observable features can explain it.
• The agreement ceiling in medical AI evaluation is thus largely structural, but the reducible/irreducible dissociation suggests that closing information gaps in evaluation scenarios could lower disagreement where inherent clinical ambiguity…

Match reason: Matches selected tags (Demonstrations).

• We introduce AuditBench, an alignment auditing benchmark.
• To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools.

Match reason: Matches selected tags (Demonstrations).

• In this paper, we introduce FewMMBench, a comprehensive benchmark designed to evaluate MLLMs under few-shot conditions, with a focus on In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting.

Match reason: Matches selected tags (Demonstrations).

• Argumentative LLMs (ArgLLMs) are an existing approach leveraging Large Language Models (LLMs) and computational argumentation for decision-making, with the aim of making the resulting decisions faithfully explainable to and contestable by…
• Here we propose a web-based system implementing ArgLLM-empowered agents for binary tasks.

Match reason: Matches selected tags (Rubric Rating).

• We present LFQA-HP-1M, a large-scale dataset comprising 1.3M human pairwise preference annotations for LFQA.
• We propose nine rubrics for answer quality evaluation, and show that simple linear models based on these features perform comparably to state-of-the-art LLM evaluators.

Match reason: Matches selected tags (Demonstrations).

• Practically, RaWMPC leverages a world model to predict the consequences of multiple candidate actions and selects low-risk actions through explicit risk evaluation.
• Furthermore, to generate low-risk candidate actions at test time, we introduce a self-evaluation distillation method to distill riskavoidance capabilities from the well-trained world model into a generative action proposal network without…

Match reason: Matches selected tags (Demonstrations).

Match reason: Matches selected tags (Demonstrations).

• Extensive experiments on five KGQA benchmark datasets demonstrate that, to the best of our knowledge, our method achieves state-of-the-art performance, outperforming not only open-source but also even closed-source LLMs.