Skip to content
OpenTrain AIFor AI Companies

HFEPX · Eval paper review

SurveyGraph: Generating Academic Surveys via Methods-Entity Knowledge Graphs

Yifan Yuan, Yancheng Zhu, Zuoyu Zhang, Xiaogang Peng

Published

Oct 22, 2026

Citations

0

Trust level

Low

Usefulness score

15/100 (Low)

Extraction confidence

55% (Moderate)

Derived from extracted protocol signals and abstract evidence.

Rater population

Not reported

Signals refreshed

Oct 22, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Tags

Automatic Metrics

Should you rely on this paper?

This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.

Use this as background context only. Do not make protocol decisions from this page alone.

Best use

Background context only

Use if you need

A benchmark-and-metrics comparison anchor.

What to verify

Validate the exact study setup in the full paper before operational use.

Main weakness

This paper looks adjacent to evaluation work, but not like a strong protocol reference.

Human feedback signal
Not explicit
Not explicit in abstract metadata
Evaluation signal
Detected
Eval setup described
Usefulness for eval research
15/100
Adjacent candidate

Treat as adjacent context, not a core eval-method reference.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure. We present SurveyGraph, a framework built around a Methods-Entity Knowledge Graph (MEKG) encoding method-level follow-on, competition, and shared-resource relations. The graph supports core-paper discovery, retrieval, hybrid outlining, and evidence-gated narrative synthesis. We evaluate retrieval, semantic edges, citation grounding, collection coverage, and generation-stage components. In a controlled seven-method equal-budget benchmark, MEKG-anchored fusion balances recall and structural density, with an independent co-citation check. In a separate archived graph-native diagnostic, graph-aware retrieval nearly doubles recall over flat lexical baselines across eight runs (p = 0.004). A semantic evidence gate achieves AUC = 0.75. Audits of 160 generated sentences find 80.6% anchored-or-partial citation support. Human adjudication of 108 stratified semantic edges finds 68.5% supported-or-partial relations (binary-support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%). A SurveyBench-facing collection probe reaches mean reference coverage of 0.521 across four topics with an eight-paper budget. Same-backend Qwen diagnostics on a 30-paper/74-relation graph show reduced organization signals when hybrid outlining is removed. Concurrent-system comparisons provide bounded context rather than matched-backend end-to-end evidence. The results position MEKG as an auditable structural layer for typed diagnostics, relation-specific filtering, and controlled survey generation. Code and artifacts: https://github.com/xzzvsxd/SurveyGraph.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.

Human Feedback Types

missing

None explicit

No explicit feedback protocol extracted.

"Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure."

Evaluation Modes

strong

Automatic Metrics

Includes extracted eval setup.

"Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure."

Quality Controls

strong

Adjudication, Inter Annotator Agreement Reported

Calibration/adjudication style controls detected.

"Human adjudication of 108 stratified semantic edges finds 68.5% supported-or-partial relations (binary-support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%)."

Benchmarks / Datasets

strong

Surveybench

Useful for quick benchmark comparison.

"A SurveyBench-facing collection probe reaches mean reference coverage of 0.521 across four topics with an eight-paper budget."

Reported Metrics

strong

Recall, Kappa

Useful for evaluation criteria comparison.

"In a controlled seven-method equal-budget benchmark, MEKG-anchored fusion balances recall and structural density, with an independent co-citation check."

Benchmarks and datasets

Surveybench

Reported metrics

recallkappa
Human feedback details
Uses human feedback
No
Feedback types
None
Rater population
Not reported
Expertise required
Coding
Evaluation details
Evaluation modes
Automatic Metrics
Agentic eval
None
Quality controls
Adjudication, Inter Annotator Agreement Reported
Evidence quality
Low
Use this page as
Background context only

Research brief

Metadata summary

Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure.

Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.

Key takeaways

  • Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure.
  • We present SurveyGraph, a framework built around a Methods-Entity Knowledge Graph (MEKG) encoding method-level follow-on, competition, and shared-resource relations.
  • The graph supports core-paper discovery, retrieval, hybrid outlining, and evidence-gated narrative synthesis.

Researcher actions

  • Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
  • Check the full text for explicit evaluation design choices (raters, protocol, and metrics).
  • Use related-paper links to find stronger protocol-specific references.

Caveats

  • Generated from abstract + metadata only; no PDF parsing.
  • Signals below are heuristic and may miss details reported outside the abstract.

Recommended queries

Agreement / Kappa

Contribution summary

  • We present SurveyGraph, a framework built around a Methods-Entity Knowledge Graph (MEKG) encoding method-level follow-on, competition, and shared-resource relations.
  • We evaluate retrieval, semantic edges, citation grounding, collection coverage, and generation-stage components.
  • Human adjudication of 108 stratified semantic edges finds 68.5% supported-or-partial relations (binary-support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%).

Why it matters for eval

  • In a controlled seven-method equal-budget benchmark, MEKG-anchored fusion balances recall and structural density, with an independent co-citation check.
  • Human adjudication of 108 stratified semantic edges finds 68.5% supported-or-partial relations (binary-support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%).

Researcher checklist

  • Human feedback protocol is explicit

    No explicit human feedback protocol detected.

  • Evaluation mode is explicit

    Detected: Automatic Metrics

  • Quality control reporting appears

    Detected: Adjudication, Inter Annotator Agreement Reported

  • Benchmark or dataset anchors are present

    Detected: Surveybench

  • Metric reporting is present

    Detected: recall, kappa