Human Feedback and Eval Paper Explorer

Match reason: Keyword overlap 3/3 across title and protocol fields.

• We introduce Ratchet, a single-agent loop in which a frozen LLM writes, retrieves, curates, and retires its own natural-language skills.
• On MBPP+ hard-100 with Claude Opus 4.7, Ratchet lifts held-out pass@1 from a 0.258 \pm 0.047 baseline to a late-window rolling mean of 0.584 (peak 0.658 \pm 0.042) across 100 rounds and 3 seeds, a +0.328 \pm 0.018 rolling-mean gain where…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Agentic modeling aims to transform LLMs into autonomous agents capable of solving complex tasks through planning, reasoning, tool use, and multi-turn interaction with environments.
• We present Orchard, an open-source framework for scalable agentic modeling.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• In this paper, we present the first systematic study of token consumption patterns in agentic coding tasks.
• We find that: (1) agentic tasks are uniquely expensive, consuming 1000x more tokens than code reasoning and code chat, with input tokens rather than output tokens driving the overall cost; (2) token usage is highly variable and inherently…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Then, we introduce Research-Factory, an automated pipeline that generates high-quality training data by collecting research papers and constructing evaluation rubrics.
• Notably, our proposed KLong (106B) surpasses Kimi K2 Thinking (1T) by 11.28% on PaperBench, and the performance improvement generalizes to other coding benchmarks like SWE-bench Verified and MLE-bench.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• We propose an uncertainty-aware multi-agent scaffold that explicitly decouples underspecification detection from code execution.
• Our results demonstrate that this multi-agent system using OpenHands + Claude Sonnet 4.5 achieves a 69.40% task resolve rate, significantly outperforming a standard single-agent setup (61.20%) and closing the performance gap with agents…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• A prerequisite for coding agents to perform tasks on large repositories is code localization - the identification of relevant files, classes, and functions to work on.
• In this paper, we demonstrate that, with an effective reinforcement learning recipe, a coding agent equipped with nothing more than a standard Unix terminal can be trained to achieve strong results.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Random rules improve a coding agent's task performance as much as expert-curated ones (both +13.8pp on a discriminative subset of SWE-bench Verified), and in our data every individually beneficial rule is a negative constraint ("do not…
• We arrive at these findings through the first large-scale controlled study of agent rule files (CLAUDE.md, .cursorrules, and the broader family of agent skills, plugin manifests, and persona definitions): we scrape 679 rule files (25{,}532…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Our empirical results set a new benchmark for open-source models of comparable size.
• Furthermore, despite being trained exclusively on Python, our agents demonstrate robust zero-shot transferability on SWE-bench Multilingual, reaching 44.1% and confirming the paradigm's generalizability across diverse languages.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• We present KAT-Coder-V2, an agentic coding model developed by the KwaiKAT team at Kuaishou.
• KAT-Coder-V2 adopts a "Specialize-then-Unify" paradigm that decomposes agentic coding into five expert domains - SWE, WebCoding, Terminal, WebSearch, and General - each undergoing independent supervised fine-tuning and reinforcement…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• AI coding agents can resolve real-world software issues, yet they frequently introduce regressions -- breaking tests that previously passed.
• When deployed as an agent skill with a different model and framework, TDAD improved issue-resolution rate from 24% to 32%, confirming that surfacing contextual information outperforms prescribing procedural workflows.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• When assessing the quality of coding agents, predominant benchmarks focus on solving single issues on GitHub, such as SWE-Bench.
• Experiments show that agents trained on our synthetic tasks effectively generalize to diverse real-world tasks that are not present in training, improving a base model by 25.4% absolute gain on SWE-Bench Verified, 7.9% on SWT-Bench…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• LLM coding benchmarks face a credibility crisis: widespread solution leakage and test quality issues undermine SWE-bench Verified, while existing detection methods--paraphrase consistency, n-gram overlap, perplexity analysis--never directly…
• We introduce Cross-Context Verification (CCV), a black-box method that solves the same benchmark problem in N independent sessions and measures solution diversity, combined with the Hierarchical Cross-Context Architecture (HCCA), a…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Our approach combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• Agentic systems are evaluated on benchmarks where agents interact with environments to solve tasks.
• To enable reliable evaluation of agentic systems, we recommend three concrete practices: (1) estimate pass@1 from multiple independent runs per task, especially when measuring small improvements, (2) use statistical power analysis to…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• In this paper, we propose SWE-Pruner, a self-adaptive context pruning framework tailored for coding agents.
• Evaluations across four benchmarks and multiple models validate SWE-Pruner's effectiveness in various scenarios, achieving 23-54% token reduction on agent tasks like SWE-Bench Verified while even improving success rates, and up to 14.84x…

Match reason: Keyword overlap 3/3 across title and protocol fields.

• In this technical report, we present SWE-Master, an open-source and fully reproducible post-training framework for building effective software engineering agents.
• We evaluate SWE-Master on SWE-bench Verified, a standard benchmark for realistic software engineering tasks.

Match reason: Keyword overlap 3/3 across title and protocol fields.

• We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimizes the production pipeline for agentic model.
• Empirically, we evaluate ROME within a structured setting and introduce Terminal Bench Pro, a benchmark with improved scale and contamination control.

Match reason: Keyword overlap 2/3 across title and protocol fields.

• Large Language Models (LLMs) have demonstrated significant potential as autonomous software engineering (SWE) agents.
• Recent work has further explored augmenting these agents with memory mechanisms to support long-horizon reasoning.