Skip to content
OpenTrain AIFor AI Companies

HFEPX · Eval paper review

Erase to Improve: Erasable Reinforcement Learning for Search-Augmented LLMs

Ziliang Wang, Kang An, Xuhui Zheng, Faqiang Qian +5 more

Published

Oct 1, 2025

Citations

0

Trust level

Moderate

Usefulness score

25/100 (Low)

Extraction confidence

55% (Moderate)

Derived from extracted protocol signals and abstract evidence.

Rater population

Not reported

Signals refreshed

Apr 20, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Tags

Automatic Metrics Law Long Horizon

Should you rely on this paper?

This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.

Use this for comparison and orientation, not as your only source.

Best use

Background context only

Use if you need

A benchmark-and-metrics comparison anchor.

What to verify

Validate the evaluation procedure and quality controls in the full paper before operational use.

Main weakness

No major weakness surfaced.

Human feedback signal
Not explicit
Not explicit in abstract metadata
Evaluation signal
Detected
Eval setup described
Usefulness for eval research
25/100
Adjacent candidate

Treat as adjacent context, not a core eval-method reference.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited. This limitation arises from three fundamental challenges: decomposition errors, where tasks are incorrectly broken down; retrieval missing, where key evidence fails to be retrieved; and reasoning errors, where flawed logic propagates through the reasoning chain. A single failure in any of these stages can derail the final answer. We propose Erasable Reinforcement Learning (ERL), a novel framework that transforms fragile reasoning into a robust process. ERL explicitly identifies faulty steps, erases them, and regenerates reasoning in place, preventing defective logic from propagating through the reasoning chain. This targeted correction mechanism turns brittle reasoning into a more resilient process. Models trained with ERL, termed ESearch, achieve substantial improvements on HotpotQA, MuSiQue, 2Wiki, and Bamboogle, with the 3B model achieving +8.48% EM and +11.56% F1, and the 7B model achieving +5.38% EM and +7.22% F1 over previous state-of-the-art(SOTA) results. These findings suggest that erasable reinforcement learning provides a powerful paradigm shift for robust multi-step reasoning in LLMs.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.

Human Feedback Types

missing

None explicit

No explicit feedback protocol extracted.

"While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited."

Evaluation Modes

strong

Automatic Metrics

Includes extracted eval setup.

"While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited."

Quality Controls

missing

Not reported

No explicit QC controls found.

"While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited."

Benchmarks / Datasets

strong

HotpotQA

Useful for quick benchmark comparison.

"Models trained with ERL, termed ESearch, achieve substantial improvements on HotpotQA, MuSiQue, 2Wiki, and Bamboogle, with the 3B model achieving +8.48% EM and +11.56% F1, and the 7B model achieving +5.38% EM and +7.22% F1 over previous state-of-the-art(SOTA) results."

Reported Metrics

strong

F1

Useful for evaluation criteria comparison.

"While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited."

Benchmarks and datasets

HotpotQA

Reported metrics

f1
Human feedback details
Uses human feedback
No
Feedback types
None
Rater population
Not reported
Expertise required
General
Evaluation details
Evaluation modes
Automatic Metrics
Agentic eval
Long Horizon
Quality controls
Not reported
Evidence quality
Moderate
Use this page as
Background context only

Research brief

Metadata summary

While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited.

Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.

Key takeaways

  • While search-augmented large language models (LLMs) exhibit impressive capabilities, their reliability in complex multi-hop reasoning remains limited.
  • This limitation arises from three fundamental challenges: decomposition errors, where tasks are incorrectly broken down; retrieval missing, where key evidence fails to be retrieved; and reasoning errors, where flawed logic propagates through the reasoning chain.
  • A single failure in any of these stages can derail the final answer.

Researcher actions

  • Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
  • Validate inferred eval signals (Automatic metrics, Long-horizon tasks) against the full paper.
  • Use related-paper links to find stronger protocol-specific references.

Caveats

  • Generated from abstract + metadata only; no PDF parsing.
  • Signals below are heuristic and may miss details reported outside the abstract.

Recommended queries

F1

Contribution summary

  • We propose Erasable Reinforcement Learning (ERL), a novel framework that transforms fragile reasoning into a robust process.
  • Models trained with ERL, termed ESearch, achieve substantial improvements on HotpotQA, MuSiQue, 2Wiki, and Bamboogle, with the 3B model achieving +8.48% EM and +11.56% F1, and the 7B model achieving +5.38% EM and +7.22% F1 over previous…

Researcher checklist

  • Human feedback protocol is explicit

    No explicit human feedback protocol detected.

  • Evaluation mode is explicit

    Detected: Automatic Metrics

  • Quality control reporting appears

    No calibration/adjudication/IAA control explicitly detected.

  • Benchmark or dataset anchors are present

    Detected: HotpotQA

  • Metric reporting is present

    Detected: f1