BFS-PO: Best-First Search for Large Reasoning Models
Fiorenzo Parascandolo, Wenhui Tan, Enver Sangineto, Ruihua Song, Rita Cucchiara
Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
Large Reasoning Models (LRMs) such as OpenAI o1 and DeepSeek-R1 have shown excellent performance in reasoning tasks using long reasoning chains. However, this has also led to a significant increase of computational costs and the generation of verbose output, a phenomenon known as overthinking. The tendency to overthinking is often exacerbated by Reinforcement Learning (RL) algorithms such as GRPO/DAPO. In this paper, ...
we propose BFS-PO, an RL algorithm which alleviates this problem using a Best-First Search exploration strategy. Specifically, BFS-PO looks for the shortest correct answer using a backtracking mechanism based on maximum entropy nodes. By generating progressively shorter responses during training, BFS-PO learns to produce concise reasoning chains. Using different benchmarks and base LRMs, we show that BFS-PO can simultaneously increase the LRM accuracy and shorten its answers.
Researcher verdict
Reference-only page for now
Use this page for paper context, links, and cautious triage only. The current benchmark signals are too weak or indirect to support a confident implementation or benchmark decision.
Why this page is still worth reading
- Some benchmark signal exists, but it is still too thin to support a confident benchmark judgment.
- Reproduction risks are surfaced explicitly, which helps decide whether the paper is worth immediate prototyping.
Benchmark trust
Some benchmark signal exists in the extracted evidence, but it is not structured strongly enough yet for a confident benchmark decision.
Use this page as
Use this page for context, citations, and paper triage rather than immediate implementation.
Results & Benchmarks
Benchmark evidence drill-down
Audit each benchmark finding before selecting an implementation path. Evidence refs map to the disclosure section below.
| Task | Dataset | Metric | Value | Source | Evidence refs |
|---|---|---|---|---|---|
| Reinforcement learning | GSM8K | Accuracy | 3.1 | paper-derived | No explicit refs |
| Reinforcement learning | MATH | Accuracy | 500 | paper-derived | No explicit refs |
Large Reasoning Models (LRMs) such as OpenAI o1 and DeepSeek-R1 have shown excellent performance in reasoning tasks using long reasoning chains.
Implementation Evidence Summary
Recommendation evidence is currently too limited for a maintained-repo choice. Use Implementation Status and Reproduction Path for a practical baseline plan.
Reproduction Risks
- Estimate is based on paper-only reproduction flow
Hardware Notes
Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Evidence disclosure
Evidence graph: 2 refs, 1 links.
Utility signals: depth 95/100, grounding 68/100, status medium.
Implementation Status
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- No direct maintained implementation was found. Use the paper PDF and citation graph to design a baseline reproduction.
- Start from this likely method family: Reinforcement learning.
- Track assumptions and missing details in an experiment log before coding.
What is known right now
This page is not strong enough for a full AI-written research brief yet, so the summary is reduced to what is evidenced, what is missing, and what to do next.
What is known
- Large Reasoning Models (LRMs) such as OpenAI o1 and DeepSeek-R1 have shown excellent performance in reasoning tasks using long reasoning chains.
- Benchmark anchor: Reinforcement learning on GSM8K using Accuracy.
What is missing
- Benchmark evidence is not yet strong enough to treat the LLM brief as fully researcher-ready.
- There is no verified maintained implementation path yet.
What to do next
- No direct maintained implementation was found. Use the paper PDF and citation graph to design a baseline reproduction.
- Start from this likely method family: Reinforcement learning.
- Track assumptions and missing details in an experiment log before coding.
Reproduction path
Follow this baseline workflow to decide if this paper is worth immediate prototyping.
- 1
Use the paper and benchmark evidence to scope a baseline reproduction plan.
- 2
Start from this likely method family: Reinforcement learning.
- 3
Track assumptions and missing details in an experiment log before coding.
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet.
Continue with targeted Hugging Face searches derived from the paper title and method context:
Models
Tip: start with models, then check datasets/spaces if you need evaluation data or demos.
Direct artifact matches are currently sparse. Use targeted Hugging Face searches to quickly locate candidate models, datasets, and demos.
Research context
Tasks
Reinforcement learning
Methods
Reinforcement learning
Domains
None detected
Evaluation & Human Feedback Data
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXExplore Similar Papers
Jump to Paper2Code search queries derived from this paper's research context.
Need human evaluators for your AI research? Scale annotation with expert AI Trainers.