Skip to content
OpenTrain AIFor AI Companies

SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

Yuqiao Tan, Jinxiang Meng, Fangyu Lei, Minzheng Wang, Shizhu He +2 morePublished Aug 3, 2026
arXiv PDF
Researcher verdict
Context only
Use as context only
Benchmark evidence
Missing
Not verified yet
Time to first repro
A few hours
Fast first run
Risk flags
1
Review before use

Abstract

Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.

Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages. This leads us to ask: how do coding agents understand and respond to code changes in a shared workspace? We introduce SWE-Touch, a framework that stress-tests this setting through validated Counter-Edits: plausible edits to task-relevant code that conflict with task completion. SWE-Touch mines task-critical regions from multiple repair trajectories, uses a separate User Patch Generator to construct the edits, and injects them with contextual user messages when agents reach the relevant code. We evaluate nine coding models on SWE-bench Verified, with additional experiments on longer-horizon tasks from SWE-Bench Pro and DeepSWE. Counter-Edit lowers average resolve rate by 7.7 percentage points on SWE-bench Verified, with degradation also persisting on both longer-horizon benchmarks. Trajectory analysis links these failures to limited awareness of the evolving workspace: agents may retain conflicting code or replace it without sufficiently re-inspecting the repository and validating the revised code with targeted tests. These findings show that strong autonomous performance does not yet ensure the state awareness and adaptive behavior needed for shared-workspace collaboration, and point to detecting workspace changes, reconciling conflicting edits with the task, and verifying the affected behavior as key capabilities for future optimization.

Results and benchmarks

Freshness tier: warm
Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages.

Implementation

No direct implementation yet

Maintained implementation evidence is not confirmed for this paper yet.

Use the implementation status and reproduction sections for the current action plan.

Implementation evidence summary
Confidence: low

This is primarily a method paper. Reproduce it within a maintained framework baseline instead of chasing paper-specific repos.

Reproduction risks
  • No maintained paper-verified implementation is currently available

Compare implementation paths

Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.

Maintenance
Active
Confidence
Medium
Reproducibility
Moderate
Stars
11
Last push
Aug 4, 2026 (21d)

Matched via arXiv identifier search · Strong overlap with paper title keywords

  • No tagged releases
  • No Docker setup
  • Dependency manifest missing
Maintenance
Active
Confidence
Low
Reproducibility
Moderate
Stars
1
Last push
Aug 20, 2026 (5d)

Matched via arXiv identifier search

  • No Docker setup
  • Dependency manifest missing
  • Low confidence match

Reproduction readiness

Time to first repro: hours
Last checked: Aug 25, 2026

No repo

No verified implementation available

  • No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.

Repositories and ecosystem

Official

No additional official repositories detected.

Community

  • Trae1ounG/SWE-Touch
    Confidence: Medium

    [arxiv:2608.02499] SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

    11 stars · 0 forks · Last push Aug 4, 2026 · MIT license

Hugging Face artifacts

No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.

Tip: start with models, then check datasets and spaces if you need evaluation data or demos.

Research context

Evaluation and human feedback data

Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.

Open in HFEPX