HFEPX · Eval paper review
Riya Naik, Shivam Sharma, Swati Agarwal
Published
Nov 9, 2026
Citations
0
Trust level
Low
Usefulness score
49/100 (Medium)
Extraction confidence
45% (Low)
Derived from extracted protocol signals and abstract evidence.
Rater population
Not reported
Signals refreshed
Nov 9, 2026
This paper has useful evaluation signal, but protocol completeness is partial; pair it with related papers before deciding implementation strategy.
Use this as background context only. Do not make protocol decisions from this page alone.
Use this page for context, then validate protocol choices against stronger HFEPX references before implementation decisions.
Best use
Secondary protocol comparison source
Use if you need
A secondary eval reference to pair with stronger protocol papers.
What to verify
Validate the evaluation procedure and quality controls in the full paper before operational use.
Main weakness
The available metadata is too thin to trust this as a primary source.
Useful as a secondary reference; validate protocol details against neighboring papers.
If you are doing eval pipeline work, start here
India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art. Due to the widespread curiosity, people increasingly turn to Artificial Intelligence (AI) assistants for information about them. Current models are rarely trained or evaluated on this domain. We present BharatSynthQA, a passage grounded question answering (QA) dataset focused on Indian temples, which also covers other religious sites connected to them. Our pipeline crawls Wikipedia from topics seeded by authentic heritage sources, filters articles for topical relevance, and locations in India. Further it generates Boolean, factoid, and explanatory QA pairs with a fewshot prompted LLM. An LLM judge filters the pairs on answer correctness, retaining 98,677 of 131,494 generated pairs. Additionally, 300 human-annotated pairs are released separately as a reference set. Human evaluation of the subsample of this dataset showed 89.83% of the questions were formed correctly. Benchmarking five current LLMs shows that factoid token F1 is only 0.18-0.30 without the source passage but 0.73-0.77 with it, indicating that this domain is poorly captured in model knowledge. The dataset is available at https://huggingface.co/datasets/theshivam7/bharatsynth-qa to support research on culturally grounded QA and retrieval-augmented generation for Indian heritage.
These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.
None explicit
No explicit feedback protocol extracted.
"India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art."
Human Eval, Llm As Judge, Automatic Metrics
Includes extracted eval setup.
"India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art."
Not reported
No explicit QC controls found.
"India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art."
Not extracted
No benchmark anchors detected.
"India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art."
F1
Useful for evaluation criteria comparison.
"India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art."
No benchmark or dataset names were extracted from the available abstract.
India's temples and other places of worship are among its richest repositories of cultural heritage, bringing together religious tradition, history, architecture, and art.
Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.
Human feedback protocol is explicit
No explicit human feedback protocol detected.
Evaluation mode is explicit
Detected: Human Eval, Llm As Judge, Automatic Metrics
Quality control reporting appears
No calibration/adjudication/IAA control explicitly detected.
Benchmark or dataset anchors are present
No benchmark/dataset anchor extracted from abstract.
Metric reporting is present
Detected: f1
