Skip to content
OpenTrain AIFor AI Companies

HFEPX · Eval paper review

Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark

Yuedong Tan, Lei Qi, Yu Liu, Di Wen +12 more

Published

Sep 29, 2026

Citations

0

Trust level

Low

Usefulness score

0/100 (Low)

Extraction confidence

35% (Low)

Derived from extracted protocol signals and abstract evidence.

Rater population

Not reported

Signals refreshed

Sep 29, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Tags

Automatic Metrics

Should you rely on this paper?

This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.

Use this as background context only. Do not make protocol decisions from this page alone.

Best use

Background context only

Use if you need

A secondary eval reference to pair with stronger protocol papers.

What to verify

Validate the evaluation procedure and quality controls in the full paper before operational use.

Main weakness

This paper looks adjacent to evaluation work, but not like a strong protocol reference.

Human feedback signal
Not explicit
Not explicit in abstract metadata
Evaluation signal
Detected
Eval setup described
Usefulness for eval research
0/100
Adjacent candidate

Treat as adjacent context, not a core eval-method reference.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggregate cross-video accuracy conflates failures of local perception with failures to preserve observation identity, establish correspondence, and compose evidence, obscuring whether local video understanding actually transfers. We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition. It contains 567 single-video and 1,487 multi-video questions derived from 126 human-collected egocentric recordings covering 39 outdoor routes. Repeated traversals across movement speeds and lighting conditions ground comparisons in shared physical environments; 531 questions require alignment across independent recordings. Single-video questions measure the local visual, spatial, and motion evidence available to a model, while multi-video questions test whether evidence remains bound to the correct observation and can be composed into consistent route relationships. We report 29 single-video and 31 multi-video MLLM configurations across six model families in the main leaderboard. Among the 20 configurations evaluated comparably on both splits, every model performs worse on multi-video questions, with a mean decrease of 22.5 percentage points, and the gap persists when answer format and scoring are held fixed. The gap is not explained simply by additional videos or recording boundaries. The central bottlenecks are observation--evidence binding and ordered route-state tracking. The code and benchmark are publicly available at https://github.com/lei-qi-233/EgoGears.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.

Human Feedback Types

missing

None explicit

No explicit feedback protocol extracted.

"Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination."

Evaluation Modes

partial

Automatic Metrics

Includes extracted eval setup.

"Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination."

Quality Controls

missing

Not reported

No explicit QC controls found.

"Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination."

Benchmarks / Datasets

missing

Not extracted

No benchmark anchors detected.

"Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination."

Reported Metrics

partial

Accuracy

Useful for evaluation criteria comparison.

"Yet aggregate cross-video accuracy conflates failures of local perception with failures to preserve observation identity, establish correspondence, and compose evidence, obscuring whether local video understanding actually transfers."

Benchmarks and datasets

No benchmark or dataset names were extracted from the available abstract.

Reported metrics

accuracy
Human feedback details
Uses human feedback
No
Feedback types
None
Rater population
Not reported
Expertise required
Coding
Evaluation details
Evaluation modes
Automatic Metrics
Agentic eval
None
Quality controls
Not reported
Evidence quality
Low
Use this page as
Background context only

Research brief

Metadata summary

Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination.

Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.

Key takeaways

  • Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination.
  • Yet aggregate cross-video accuracy conflates failures of local perception with failures to preserve observation identity, establish correspondence, and compose evidence, obscuring whether local video understanding actually transfers.
  • We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition.

Researcher actions

  • Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
  • Validate inferred eval signals (Automatic metrics) against the full paper.
  • Use related-paper links to find stronger protocol-specific references.

Caveats

  • Generated from abstract + metadata only; no PDF parsing.
  • Signals below are heuristic and may miss details reported outside the abstract.

Recommended queries

Accuracy

Contribution summary

  • We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition.
  • It contains 567 single-video and 1,487 multi-video questions derived from 126 human-collected egocentric recordings covering 39 outdoor routes.
  • The code and benchmark are publicly available at https://github.com/lei-qi-233/EgoGears.

Why it matters for eval

  • We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition.
  • It contains 567 single-video and 1,487 multi-video questions derived from 126 human-collected egocentric recordings covering 39 outdoor routes.

Researcher checklist

  • Human feedback protocol is explicit

    No explicit human feedback protocol detected.

  • Evaluation mode is explicit

    Detected: Automatic Metrics

  • Quality control reporting appears

    No calibration/adjudication/IAA control explicitly detected.

  • Benchmark or dataset anchors are present

    No benchmark/dataset anchor extracted from abstract.

  • Metric reporting is present

    Detected: accuracy