HFEPX · Eval paper review
Yujin Tang, Chenming Shang, Ruize Xu, Nikhil Singh
Published
Jun 25, 2026
Citations
0
Trust level
Moderate
Usefulness score
25/100 (Low)
Extraction confidence
55% (Moderate)
Derived from extracted protocol signals and abstract evidence.
Rater population
Not reported
Signals refreshed
Aug 31, 2026
This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.
Use this for comparison and orientation, not as your only source.
Best use
Background context only
Use if you need
A benchmark-and-metrics comparison anchor.
What to verify
Validate the evaluation procedure and quality controls in the full paper before operational use.
Main weakness
No major weakness surfaced.
Treat as adjacent context, not a core eval-method reference.
If you are doing eval pipeline work, start here
Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved. In other words, they assess what agents elected to record, not what they happened to see. We introduce DMV-Bench (code: https://github.com/yyyujintang/DMV-Bench), the first interactive benchmark for visual memory in multimodal agents, to study this often-neglected property. DMV-Bench is built on (1) a controlled home-furnishing e-commerce environment, supported by a catalog of 1,000 product variants, and (2) a text-leakage contract which ensures that the primary discriminative signal of each task is solely in the pixels. In DMV-Bench, agents undergo chains of autonomous shopping sessions in which every visited product image carries a unique, pre-rendered incidental cue that the agent is later asked to recall. We show that conventional solutions struggle with this task. Inspired by dual-coding theory, we propose a memory architecture that uses parallel visual and verbal codes, which we call DualMem. On DMV-Bench, DualMem outperforms a caption-only baseline and three recent multimodal agent-memory systems across multi-session chain lengths on multiple models. These gains persist even adjusting for memory-bank size and encoding-position bias. Further experiments also reveal an asymmetric division of labor between the two codes; a weighted coding scheme is often strongest. We view this as a step towards memory systems that preserve a richer record of agents' observations.
These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.
None explicit
No explicit feedback protocol extracted.
"Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved."
Automatic Metrics
Includes extracted eval setup.
"Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved."
Not reported
No explicit QC controls found.
"Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved."
Dmv Bench
Useful for quick benchmark comparison.
"We introduce DMV-Bench (code: https://github.com/yyyujintang/DMV-Bench), the first interactive benchmark for visual memory in multimodal agents, to study this often-neglected property."
Recall
Useful for evaluation criteria comparison.
"In DMV-Bench, agents undergo chains of autonomous shopping sessions in which every visited product image carries a unique, pre-rendered incidental cue that the agent is later asked to recall."
Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved.
Based on abstract + metadata only. Check the source paper before making high-confidence protocol decisions.
Human feedback protocol is explicit
No explicit human feedback protocol detected.
Evaluation mode is explicit
Detected: Automatic Metrics
Quality control reporting appears
No calibration/adjudication/IAA control explicitly detected.
Benchmark or dataset anchors are present
Detected: Dmv-Bench
Metric reporting is present
Detected: recall
