Importance of Prompt Optimisation for Error Detection in Medical Notes Using Language Models

Research Summary

Contribution Summary

• Errors in medical text can cause delays or even result in incorrect treatment for patients.
• Recently, language models have shown promise in their ability to automatically detect errors in medical text, an ability that has the opportunity to significantly benefit healthcare systems.
• In this paper, we explore the importance of prompt optimisation for small and large language models when applied to the task of error detection.

Why It Matters For Eval

• We show that automatic prompt optimisation with Genetic-Pareto (GEPA) improves error detection over the baseline accuracy performance from 0.669 to 0.785 with GPT-5 and 0.578 to 0.690 with Qwen3-32B, approaching the performance of medical d

Related Papers

• Pushing the Frontier of Black-Box LVLM Attacks via Fine-Grained Detail Targeting Embedding Neighbor
• Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models Embedding Neighbor
• CLiMB: A Domain-Informed Novelty Detection Clustering Framework for Galactic Archaeology and Scientific Discovery Embedding Neighbor
• When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations Embedding Neighbor
• Diagnosing LLM Reranker Behavior Under Fixed Evidence Pools Embedding Neighbor
• SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models Embedding Neighbor
• Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Embedding Neighbor
• DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs Embedding Neighbor
• AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors Embedding Neighbor
• XMorph: Explainable Brain Tumor Analysis Via LLM-Assisted Hybrid Deep Intelligence Tag Overlap
• Building Safe and Deployable Clinical Natural Language Processing under Temporal Leakage Constraints Tag Overlap
• AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG Tag Overlap