Think$^{2}$: Grounded Metacognitive Reasoning in Large Language Models

Research Summary

Contribution Summary

• Large Language Models (LLMs) demonstrate strong reasoning performance, yet their ability to reliably monitor, diagnose, and correct their own errors remains limited.
• We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown's regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight
• Across diverse reasoning and diagnostic benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, and TruthfulQA) using Llama-3 and Qwen-3 (8B), explicit regulatory structuring substantially improves error diagnosis and yields a threefold incr

Why It Matters For Eval

• We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown's regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight
• Across diverse reasoning and diagnostic benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, and TruthfulQA) using Llama-3 and Qwen-3 (8B), explicit regulatory structuring substantially improves error diagnosis and yields a threefold incr

Related Papers

• Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment Embedding Neighbor
• Refusal Direction is Universal Across Safety-Aligned Languages Embedding Neighbor
• Reward Modeling from Natural Language Human Feedback Embedding Neighbor
• Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss Embedding Neighbor
• Modality-Guided Mixture of Graph Experts with Entropy-Triggered Routing for Multimodal Recommendation Embedding Neighbor
• A Multi-faceted Analysis of Cognitive Abilities: Evaluating Prompt Methods with Large Language Models on the CONSORT Checklist Embedding Neighbor
• Revisiting Northrop Frye's Four Myths Theory with Large Language Models Embedding Neighbor
• Neurosymbolic Retrievers for Retrieval-augmented Generation Embedding Neighbor
• ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference Embedding Neighbor
• Cold-Start Personalization via Training-Free Priors from Structured World Models Tag Overlap
• The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footage Tag Overlap
• Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learning Tag Overlap