Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
Results and benchmarks
Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models presents a transformer method.
Benchmark evidence is limited
Evidence graph: 4 refs, 4 links.
Utility signals: depth 55/100, grounding 85/100, status medium.
Implementation
Best maintained implementation now
[ACL 2024] Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
64 stars · 5 forks · Last push Sep 4, 2024 · MIT license
- License
- CI
- Dependencies
- Docker
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata · Strong overlap with paper title keywords
jinhaoduan/sar is the strongest maintained implementation based on ranking signals. License is declared (MIT). Dependency/environment manifests are present.
Open jinhaoduan/sar- No CI workflows detected
- Selected jinhaoduan/sar as the strongest maintained implementation for new work.
- Includes dependency/environment manifest signals.
- Repository activity is within the last 24 months.
- Official repository is preserved separately as historical context.
Compare implementation paths
Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.
- Maintenance
- Stale
- Confidence
- High
- Reproducibility
- Moderate
- Stars
- 64
- Last push
- Sep 4, 2024 (721d)
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata
- No push in 12+ months
- No CI pipeline detected
- No tagged releases
- Maintenance
- Stale
- Confidence
- High
- Reproducibility
- Moderate
- Stars
- 64
- Last push
- Sep 4, 2024 (721d)
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata
- No push in 12+ months
- No CI pipeline detected
- No tagged releases
- Maintenance
- Active
- Confidence
- Low
- Reproducibility
- Strong
- Stars
- 501
- Last push
- Aug 19, 2026 (7d)
Community adoption signal (501 stars)
- Low confidence match
Reproduction readiness
Setup required
Dependencies pinned, manual setup needed
- jinhaoduan/sar has requirements.txt but requires manual environment setup.
- Last push was 721 days ago, so expect possible dependency version conflicts.
- No Dockerfile, so you will set up the environment manually.
- No CI pipeline, so test coverage is unknown.
Quick start
git clone https://github.com/jinhaoduan/sar.git
pip install -r requirements.txtValidation caveat
Repositories and ecosystem
No additional verified repositories beyond the primary recommendation.
These repositories had low-confidence matching signals and are hidden by default.
- iinemo/lm-polygraph
Confidence: Low · 501 stars
Hugging Face artifacts
No direct paper-linked artifacts were found. Showing strongest curated related artifacts for faster exploration.
Models
- kernels-community/paged-attention
1,768 downloads · 11 likes
Broaden model search
Datasets
No trustworthy datasets matches right now.
Search datasets on Hugging FaceSpaces
No trustworthy spaces matches right now.
Search spaces on Hugging FaceResearch context
Tasks
None detected
Methods
Transformer
Domains
Natural Language Processing
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.
Data includes links from Papers with Code ( CC-BY-SA-4.0 ).