Skip to content
implementation starting point
Benchmarks: thin evidence
Time to repro: a few hours
1 risk flag
none

Results & Benchmarks

Freshness tier: hot
Direct + Inferred Evidence
Classification
our Answerable-or-Not
Accuracy
97.98
Source: paper fulltext
Classification
LSTM
F1 Score
93.82
Source: paper fulltext
Classification
BiLSTM
F1 Score
94.40
Source: paper fulltext
Classification
CNN-LSTM
F1 Score
94.61
Source: paper fulltext

Benchmark evidence drill-down

4 findings

Audit each benchmark finding before selecting an implementation path. Evidence refs map to the disclosure section below.

Task Dataset Metric Value Source Evidence refs
Classification our Answerable-or-Not Accuracy 97.98 paper-derived No explicit refs
Classification LSTM F1 Score 93.82 paper-derived No explicit refs
Classification BiLSTM F1 Score 94.40 paper-derived No explicit refs
Classification CNN-LSTM F1 Score 94.61 paper-derived No explicit refs

LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities presents a quantization method.

Use This Implementation Because…

Confidence: high

kalyan-nakka/litelmguard_mlc-llm is the strongest maintained implementation based on ranking signals. CI workflows are present. License is declared (Apache-2.0).

Open kalyan-nakka/litelmguard_mlc-llm

Reproduction Risks

  • Top repository has low community adoption
Evidence disclosure

Evidence graph: 3 refs, 3 links.

Utility signals: depth 90/100, grounding 85/100, status high.

Implementation Comparison

Top 2 paths

Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.

kalyan-nakka/litelmguard_mlc-llm
best maintained
Maintenance: Recently updated
Confidence: High
Reproducibility: Strong

Official implementation from Papers with Code · Repository link is mentioned in the paper metadata

Stars
0
Last push
Mar 15, 2026 (37d ago)
CIDependencies

Risk flags

  • No tagged releases
  • No Docker setup
kalyan-nakka/on_device_slms_defense
historical official
Maintenance: Recently updated
Confidence: High
Reproducibility: Limited

Official implementation from Papers with Code · Repository link is mentioned in the paper metadata

Stars
0
Last push
Mar 15, 2026 (37d ago)

Risk flags

  • No CI pipeline detected
  • No tagged releases
  • No Docker setup

Best implementation now

kalyan-nakka/litelmguard_mlc-llm
Confidence: High
Reproducibility: Strong

[IJCNLP-AACL 2025] LiteLMGuard project's "MLC-LLM" copy

Stars: 0
Forks: 0
Last push: Mar 15, 2026
License: Apache-2.0
Official implementation from Papers with Code
Repository link is mentioned in the paper metadata
License ✓
CI ✓
Deps ✓
Docker –
  • Selected kalyan-nakka/litelmguard_mlc-llm as the strongest maintained implementation for new work.
  • Includes CI workflow signals.
  • Includes dependency/environment manifest signals.
  • Repository activity is within the last 24 months.

Historical official implementation

Preserved for provenance. Not recommended as the default path for new builds.

kalyan-nakka/on_device_slms_defense
Stars: 0
Last push: Mar 15, 2026

Reproduction readiness

Ready to Run
Time to first repro: hours
Last checked: Apr 18, 2026

Ready to reproduce

  • · Clone kalyan-nakka/litelmguard_mlc-llm and install dependencies from pyproject.toml.
  • · CI pipeline detected — automated tests are in place.
  • · Last updated 37 days ago.
Open kalyan-nakka/litelmguard_mlc-llm

Quick start

git clone https://github.com/kalyan-nakka/litelmguard_mlc-llm.git
pip install -e .

Hugging Face artifacts

No trustworthy direct or curated related Hugging Face artifacts were found yet.

Continue with targeted Hugging Face searches derived from the paper title and method context:

Tip: start with models, then check datasets/spaces if you need evaluation data or demos.

Direct artifact matches are currently sparse. Use targeted Hugging Face searches to quickly locate candidate models, datasets, and demos.

Search models Search datasets Search spaces

Research context

Tasks

Classification

Methods

Quantization

Domains

Natural Language Processing

Evaluation & Human Feedback Data

Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.

Open in HFEPX

Explore Similar Papers

Jump to Paper2Code search queries derived from this paper's research context.

Classification Quantization Natural Language Processing

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote