Human Feedback and Eval Paper Explorer

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Language model (LM) "reasoning", commonly described as Chain-of-Thought or test-time scaling, often improves benchmark performance, but the dynamics underlying this process remain poorly understood.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• The ability to reliably distinguish human-written text from that generated by large language models is of profound societal importance.
• However, we demonstrate that the token-level signal distinguishing human and machine text is non-uniform across the hidden space of the detector model, and naively averaging likelihood-based token scores across regions with fundamentally…

Match reason: Keyword overlap 1/1 across title and protocol fields.

• The resulting two-axis diagnostic (φ for capacity, G for direction) yields a falsifiable polarity prediction: bottleneck- and diffuse-dominated benchmarks should exhibit opposite polarity.
• Under length-controlled evaluation, transport features retain interpretable signal (LC-AUROC from 0.62 to 0.84) on tested models up to 8B parameters, with polarity reversing as predicted between HaluEval and MedHallu.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Across three 7-8B instruction-tuned models and two benchmarks, phi_first achieves a mean AUROC of 0.820, compared with 0.793 for semantic agreement and 0.791 for standard surface-form self-consistency.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• In agentic workflows, LLMs frequently process retrieved contexts that are legally protected from further training.
• The framework instruments preference data by pairing document triggers with feedback that rewards a distinctive stylistic response, inducing a latent trigger-conditioned preference if such data are used in training.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• This paper focuses on RuleForge's architecture and operational deployment for CVE-related threat detection, with particular emphasis on our novel LLM-as-a-judge (Large Language Model as judge) confidence validation system and systematic…
• We also present extensions enabling rule generation from unstructured data sources and demonstrate a proof-of-concept agentic workflow for multi-event-type detection.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We then use the estimates to investigate when each uncertainty type carries useful signal for predicting answer correctness in question answering with large language models, revealing a benchmark-dependent divergence: the combined estimate…

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Standard evaluation of LLM confidence relies on calibration metrics (ECE, Brier score) that conflate two distinct capacities: how much a model knows (Type-1 sensitivity) and how well it knows what it knows (Type-2 metacognitive…
• We introduce an evaluation framework based on Type-2 Signal Detection Theory that decomposes these capacities using meta-d' and the metacognitive efficiency ratio M-ratio.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We validate across 22 experiments, 5 benchmarks, 4 model families, and 3 model scales (3B-14B), with Jaccard, embedding, and NLI-based baselines at three DeBERTa scales (all ~0.51 AUROC).

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We present Cerebra, an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Post-mitigation evaluation on seven clinically distinct cohorts derived from the MIMIC-IV-ED and eICU databases demonstrates substantial bias reduction: Statistical Parity Difference decreases by 40 to 51 percent on MIMIC-IV-ED and 10 to 19…

Match reason: Keyword overlap 1/1 across title and protocol fields.

• In this pre-registered study, we treat three LLMs as observers performing factual discrimination across 168,000 trials and test whether temperature functions as a criterion shift analogous to payoff manipulations in human psychophysics.
• All models exhibited unequal-variance evidence distributions (z-ROC slopes 0.52-0.84), with instruct models showing more extreme asymmetry (0.52-0.63) than the base model (0.77-0.87) or human recognition memory (~0.80).

Match reason: Keyword overlap 1/1 across title and protocol fields.

• We introduce DECEPTGUARD, a unified framework that systematically compares three monitoring regimes: black-box monitors (actions and outputs only), CoT-aware monitors (additionally observing the agent's chain-of-thought reasoning trace),…
• We introduce DECEPTSYNTH, a scalable synthetic pipeline for generating deception-positive and deception-negative agent trajectories across a novel 12-category taxonomy spanning verbal, behavioral, and structural deception.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• As LLM-powered agents have been used for high-stakes decision-making, such as clinical diagnosis, it becomes critical to develop reliable verification of their decisions to facilitate trustworthy deployment.
• We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset, surpassing the best baseline by 12% in AUROC and 50% in Brier score reduction, which confirms the effectiveness in both…

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Evaluation uses multiple metrics, including accuracy, precision, recall, F1-score, Hamming loss, Cohens kappa, and AUC-ROC.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• The rapid advancement of large language models has increasingly blurred the boundary between human-written and AI-generated text, raising societal risks such as misinformation dissemination, authorship ambiguity, and threats to intellectual…
• Empirical evaluations demonstrate that Exons-Detect achieves state-of-the-art detection performance and exhibits strong robustness to adversarial attacks and varying input lengths.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Across three reasoning models and 17 tasks spanning mathematics, STEM, and humanities, we characterize how these signals scale.
• These effects are domain-dependent: in mathematics, the native domain of RLVR-style post-training, reasoning models achieve higher uncertainty quality and exhibit both stronger complementarity and faster scaling than in STEM or humanities.