SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
Abstract
Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages. This leads us to ask: how do coding agents understand and respond to code changes in a shared workspace? We introduce SWE-Touch, a framework that stress-tests this setting through validated Counter-Edits: plausible edits to task-relevant code that conflict with task completion. SWE-Touch mines task-critical regions from multiple repair trajectories, uses a separate User Patch Generator to construct the edits, and injects them with contextual user messages when agents reach the relevant code. We evaluate nine coding models on SWE-bench Verified, with additional experiments on longer-horizon tasks from SWE-Bench Pro and DeepSWE. Counter-Edit lowers average resolve rate by 7.7 percentage points on SWE-bench Verified, with degradation also persisting on both longer-horizon benchmarks. Trajectory analysis links these failures to limited awareness of the evolving workspace: agents may retain conflicting code or replace it without sufficiently re-inspecting the repository and validating the revised code with targeted tests. These findings show that strong autonomous performance does not yet ensure the state awareness and adaptive behavior needed for shared-workspace collaboration, and point to detecting workspace changes, reconciling conflicting edits with the task, and verifying the affected behavior as key capabilities for future optimization.
Results and benchmarks
Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages.
Benchmark evidence is limited
Evidence graph: 2 refs, 1 links.
Utility signals: depth 55/100, grounding 58/100, status medium.
Implementation
No direct implementation yet
Maintained implementation evidence is not confirmed for this paper yet.
Use the implementation status and reproduction sections for the current action plan.
No verified maintained repo yet
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- This is primarily a method paper. Reproduce it within a maintained framework baseline instead of chasing paper-specific repos.
- Start with framework-native implementations (e.g. PyTorch optimizer module, Optax, or Transformers training loops).
- Replicate the paper ablation settings first, then compare against modern baselines.
Time to first repro: a few hours
This is primarily a method paper. Reproduce it within a maintained framework baseline instead of chasing paper-specific repos.
- No maintained paper-verified implementation is currently available
Compare implementation paths
Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.
- Maintenance
- Active
- Confidence
- Medium
- Reproducibility
- Moderate
- Stars
- 11
- Last push
- Aug 4, 2026 (21d)
Matched via arXiv identifier search · Strong overlap with paper title keywords
- No tagged releases
- No Docker setup
- Dependency manifest missing
- Maintenance
- Active
- Confidence
- Low
- Reproducibility
- Moderate
- Stars
- 1
- Last push
- Aug 20, 2026 (5d)
Matched via arXiv identifier search
- No Docker setup
- Dependency manifest missing
- Low confidence match
Reproduction readiness
No repo
No verified implementation available
- No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.
Validation caveat
Repositories and ecosystem
Official
No additional official repositories detected.
Community
- Trae1ounG/SWE-TouchConfidence: Medium
[arxiv:2608.02499] SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
11 stars · 0 forks · Last push Aug 4, 2026 · MIT license
These repositories had low-confidence matching signals and are hidden by default.
- yth1120/Auraxis-Agent
Confidence: Low · 1 stars
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.
Models
Datasets
Spaces
Tip: start with models, then check datasets and spaces if you need evaluation data or demos.
Research context
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPX