HFEPX · Eval paper review

MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents

Haozhen Zhang, Haodong Yue, Quanyu Long, Jianzhu Bao +5 more

Published

Oct 5, 2026

Citations

Trust level

Moderate

Usefulness score

55/100 (Medium)

Extraction confidence

70% (Moderate)

Derived from extracted protocol signals and abstract evidence.

Rater population

Not reported

Signals refreshed

Oct 5, 2026

Open arXiv Find implementation Shortlist (0) RSS feed

Should you rely on this paper?

This paper has useful evaluation signal, but protocol completeness is partial; pair it with related papers before deciding implementation strategy.

Use this for comparison and orientation, not as your only source.

Low-signal caution for protocol decisions

Use this page for context, then validate protocol choices against stronger HFEPX references before implementation decisions.

The abstract does not clearly name benchmarks or metrics.

Best use

Secondary protocol comparison source

Use if you need

A secondary eval reference to pair with stronger protocol papers.

What to verify

Read the full paper before copying any benchmark, metric, or protocol choices.

Main weakness

The abstract does not clearly name benchmarks or metrics.

Human feedback signal

Detected

From extracted signals

Evaluation signal

Detected

Eval setup described

Usefulness for eval research

55/100

Moderate-confidence candidate

Useful as a secondary reference; validate protocol details against neighboring papers.

If you are doing eval pipeline work, start here

Human Eval Papers LLM-as-Judge Papers Pairwise Preference Papers Tool-Use Evaluation Search

Abstract

Memory has become integral to the LLM agent ecosystem, supporting information retention and reuse across interactions. However, most existing agent memory systems construct memory in a query-agnostic manner, which can incur unnecessary preprocessing cost and discard details that later prove essential. Recent studies have begun shifting memory processing toward runtime adaptation, but typically specialize in particular operations or fixed processing schemes, leaving flexible control over performance, cost, and latency largely underexplored. To address this challenge, we present \textbf{MemPilot}, a flexible framework that orchestrates on-demand memory curation under different performance--cost--latency preferences. Specifically, we optimize a multi-step LLM policy via reinforcement learning to iteratively choose between retrieving from query-agnostic memory and delegating query-specific curation of raw multimodal history to heterogeneous LLMs and VLMs. The policy jointly controls evidence amount, curation instructions, model selection, and visual access, enabling fine-grained allocation of runtime computation. To optimize this policy under competing objectives, we adapt objective-wise advantage decoupling by separately estimating each objective's advantage before aggregation. Moreover, we introduce prefix-based marginal utility estimation for fine-grained credit assignment across multi-step rollouts. Experiments on five multimodal agent-memory benchmarks demonstrate favorable performance--cost--latency trade-offs across optimization preferences, with preference sweeps yielding broader frontiers than existing trade-off-aware baselines.

What we could verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.