Complexity-aware fine-tuning
Andrey Goncharov, Daniil Vyazhev, Petr Sychev, Edvard Khalafyan, Alexey Zaytsev
Abstract
General-purpose Large Language Models (LLMs) are frequently fine-tuned through supervised fine-tuning (SFT) to enhance performance in specific domains. Better results can be achieved by distilling the chain-of-thought of a larger model at the cost of numerous expensive calls and a much greater amount of data. We propose a novel blueprint for efficient fine-tuning that uses reasoning only for complex data identified b...
Results & Benchmarks
| Task | Dataset | Metric | Value |
|---|---|---|---|
| Natural language processing | MMLU | Accuracy | 0.39 |
| Natural language processing | GSM8K | Accuracy | 0.51 |
Hardware Requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Best Implementation
Only a historical official implementation is available.
Use with caution for new projects; verify against current tooling and maintained community alternatives.
- Only historical official repository was found: labarss/complexity-aware-fine-tuning.
- No maintained paper-verified implementation met reliability thresholds.
Reproduction Path
- 1
Only historical official repository was found (labarss/complexity-aware-fine-tuning).
- 2
Do not treat historical code as drop-in for modern stacks; verify dependencies and framework versions first.
- 3
Use citation-context and adjacent references to build a maintained reproduction baseline.
Additional Implementations
No additional verified repositories beyond the primary recommendation.
Hugging Face Artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet.
Continue with targeted Hugging Face searches: