ICARE: cross-domain text classification with incremental class-aware representation and distillation learning
Abstract
Domain fit: AI-core · Core AI workload signals detected from paper context and implementation/artifact evidence.
The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems. These NLI systems ideally train across various tasks, encompassing diverse concepts like sentiment analysis, news classification, and coreference resolution. However, this continual, multi-tasking approach can lead to catastrophic forgetting, where previously learned tasks are lost, especially in domains lacking specific labels. Rehearsing past examples with labels can further exacerbate this issue by causing overfitting, hindering the system's ability to classify new categories. To tackle these challenges, this research proposes ICARE, a novel framework with two key components: an instance selection module and a knowledge distillation method. The former is based on representation learning, which meticulously selects training examples to present to the NLI system. The latter, knowledge distillation with feature and logit-level transfer, facilitates knowledge transfer between tasks. This leverages pruning techniques and optimized objective functions to transfer both essential feature-level and decision-making (logit-level) knowledge, mitigating catastrophic forgetting while adapting to new patterns. Extensive evaluations on benchmark text classification datasets have demonstrated ICARE's effectiveness. The framework achieves a significant improvement in target accuracy and reduction in the ``forgetting metric''. These results solidify ICARE's ability to preserve and leverage learned knowledge across diverse tasks, ultimately enhancing the adaptability and performance of NLI systems. The source code is available at https://github.com/CongSon01/ICARE_NLP.git
Results and benchmarks
The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems.
Benchmark evidence is limited
Evidence graph: 2 refs, 1 links.
Utility signals: depth 65/100, grounding 58/100, status medium.
Implementation
No direct implementation yet
Maintained implementation evidence is not confirmed for this paper yet.
Use the implementation status and reproduction sections for the current action plan.
No verified maintained repo yet
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- No direct maintained implementation was found. Use the paper PDF and citation graph to design a baseline reproduction.
- Start from related paper: Continual Few-Shot Learning for Text Classification.
- Start from this likely method family: Distillation.
Time to first repro: a few days
Recommendation evidence is currently too limited for a maintained-repo choice. Use Implementation Status and Reproduction Path for a practical baseline plan.
- Estimate is based on paper-only reproduction flow
Reproduction readiness
No repo
No verified implementation available
- No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Validation caveat
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.
Models
Tip: start with models, then check datasets and spaces if you need evaluation data or demos.
Research context
4
Citations
67
References
Tasks
Computer science, Class (philosophy), Representation (politics), Computational intelligence, Domain (mathematical analysis), Pattern recognition (psychology), catastrophic forgetting, continual learning
Methods
Distillation, knowledge distillation, large language models
Domains
Artificial intelligence, Machine learning, Natural language processing
Related papers
- Continual Few-Shot Learning for Text ClassificationSearch on Paper2Code
2021 · Semantic similarity
- Mitigating Catastrophic Forgetting in Continual Learning for Natural Language Processing TasksSearch on Paper2Code
2024 · Semantic similarity
- CIL-LLM: Incremental Learning Framework Based on Large Language Models for Category ClassificationSearch on Paper2Code
2025 · Semantic similarity
- Mitigating Spurious Correlations in NLI via LLM-Synthesized Counterfactuals and Dynamic Balanced SamplingSearch on Paper2Code
2025 · Semantic similarity
- Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language TasksSearch on Paper2Code
2025 · Semantic similarity
- Lifelong Learning of Few-shot Learners across NLP Tasks.Search on Paper2Code
2021 · Semantic similarity
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.