HFEPX Hub

Web Browsing + General (Last 45 Days)

Updated from current HFEPX corpus (Apr 17, 2026). 12 papers are grouped in this hub page.

Read Full Context

Updated from current HFEPX corpus (Apr 17, 2026). 12 papers are grouped in this hub page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequently cited benchmark: BIRD. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Mar 22, 2026.

Papers: 12 Last published: Mar 22, 2026 Global RSS Tag RSS

Web BrowsingGeneralLast 45d

Researcher Quick Triage

This hub is best used for protocol triage and replication planning from abstract-level evidence. Quality band: Developing .

All Sampled Papers (12) Replication-Ready Only (3)

High-Signal Coverage

100.0%

12 / 12 sampled papers are not low-signal flagged.

Replication-Ready Set

Benchmark + metric + eval mode explicitly present.

Judge/Human Comparability

Papers containing both `human_eval` and `llm_as_judge`.

3 papers are replication-ready (benchmark + metric + explicit evaluation mode).
1 papers support judge-vs-human agreement analysis.
0 papers report explicit quality controls (calibration/adjudication/IAA).

Primary action: Start with the top 2 papers in “Start Here”, then validate assumptions in the protocol matrix.

Need evaluators for this research workflow?

Post a Job →

Why This Matters For Eval Research

41.7% of papers report explicit human-feedback signals, led by demonstration data.
automatic metrics appears in 58.3% of papers in this hub.
BIRD is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Takeaways

1 sampled papers report both human evaluation and LLM-as-judge, supporting direct agreement checks.
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Rater context is mostly domain experts, and annotation is commonly trajectory-level annotation; use this to scope replication staffing.

Benchmark Interpretation

BIRD appears in 8.3% of hub papers (1/12); use this cohort for benchmark-matched comparisons.
Mapg-Bench appears in 8.3% of hub papers (1/12); use this cohort for benchmark-matched comparisons.

Metric Interpretation

accuracy is reported in 58.3% of hub papers (7/12); compare with a secondary metric before ranking methods.
cost is reported in 16.7% of hub papers (2/12); compare with a secondary metric before ranking methods.

Researcher Checklist (Expanded)

Researcher Checklist

Moderate: Papers with explicit human feedback

Coverage is usable but incomplete (41.7% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Moderate: Papers naming benchmarks/datasets

Coverage is usable but incomplete (33.3% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (75% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (8.3% vs 35% target).
Moderate: Papers with known annotation unit

Coverage is usable but incomplete (25% vs 35% target).

Strengths

Contains both human-eval and LLM-as-judge protocols for head-to-head methodology comparison.
Agentic evaluation appears in 100% of papers.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (8.3% coverage).
LLM-as-judge appears without enough inter-annotator agreement reporting.

Suggested Next Analyses

Compare papers that report both human_eval and llm_as_judge to quantify judge-human agreement drift.
Stratify by benchmark (BIRD vs Mapg-Bench) before comparing methods.
Track metric sensitivity by reporting both accuracy and cost.

Recommended Queries (Expanded)

Recommended Queries

Judge vs Human Agreement Benchmark Slice: BIRD Metric Slice: accuracy Recent High-Signal Papers

Start with These 3

Use these when you need one protocol anchor, one benchmark anchor, and one recent comparison point before reading the wider hub.

Strongest protocol reference

AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabe…

Highest protocol score with explicit human/eval signal plus WebArena.

Strongest benchmark reference

SODIUM: From Open Web Data to Queryable Databases

Sodium-Bench with accuracy gives a fast comparison anchor.

Strongest recent paper

Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vi…

Useful for current practice scanning; published Mar 19, 2026.

Start Here (Best First 6)

Ranked for protocol completeness (human signal, benchmark + metric anchors, quality controls, and judge/human overlap).

AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling
Mar 22, 2026 · Citations: 0 · Score: 10.0

HF: Demonstrations · Eval: Human Eval, Llm As Judge · Benchmark: WebArena · Metric: Precision
SODIUM: From Open Web Data to Queryable Databases
Mar 19, 2026 · Citations: 0 · Score: 8.0

HF: Expert Verification · Eval: Automatic Metrics · Benchmark: Sodium Bench · Metric: Accuracy
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
Mar 19, 2026 · Citations: 0 · Score: 6.5

HF: Demonstrations · Eval: Simulation Env · Benchmark: Mapg Bench · Metric: Not Reported
BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion
Mar 10, 2026 · Citations: 0 · Score: 5.5

HF: Not reported · Eval: Automatic Metrics, Simulation Env · Benchmark: BIRD · Metric: Accuracy
Aligning Large Language Models with Searcher Preferences
Mar 11, 2026 · Citations: 0 · Score: 4.0

HF: Pairwise Preference · Eval: Not reported · Benchmark: Not Reported · Metric: Not Reported
TimeWarp: Evaluating Web Agents by Revisiting the Past
Mar 5, 2026 · Citations: 0 · Score: 4.0

HF: Demonstrations · Eval: Not reported · Benchmark: Not Reported · Metric: Not Reported

Protocol Matrix (Top 12)

Use this to quickly compare protocol ingredients instead of scanning long prose.

Paper	HF Signal	Eval Modes	Benchmarks	Metrics	QC
AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling Mar 22, 2026	Yes Demonstrations	Human Eval , Llm As Judge	WebArena , ToolBench	Precision , Pass@1	Not Reported
SODIUM: From Open Web Data to Queryable Databases Mar 19, 2026	Yes Expert Verification	Automatic Metrics	Sodium Bench	Accuracy	Not Reported
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation Mar 19, 2026	Yes Demonstrations	Simulation Env	Mapg Bench	Not Reported	Not Reported
BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion Mar 10, 2026	No Not Reported	Automatic Metrics , Simulation Env	BIRD	Accuracy	Not Reported
Aligning Large Language Models with Searcher Preferences Mar 11, 2026	Yes Pairwise Preference	Not Reported	Not Reported	Not Reported	Not Reported
TimeWarp: Evaluating Web Agents by Revisiting the Past Mar 5, 2026	Yes Demonstrations	Not Reported	Not Reported	Not Reported	Not Reported
Mind over Space: Can Multimodal Large Language Models Mentally Navigate? Mar 23, 2026	No Not Reported	Automatic Metrics , Simulation Env	Not Reported	Accuracy	Not Reported
Integrating Deep RL and Bayesian Inference for ObjectNav in Mobile Robotics Mar 26, 2026	No Not Reported	Simulation Env	Not Reported	Success rate	Not Reported
Rethinking SAR ATR: A Target-Aware Frequency-Spatial Enhancement Framework with Noise-Resilient Knowledge Guidance Mar 23, 2026	No Not Reported	Automatic Metrics	Not Reported	Accuracy , Precision	Not Reported
Beyond Explicit Edges: Robust Reasoning over Noisy and Sparse Knowledge Graphs Mar 14, 2026	No Not Reported	Automatic Metrics	Not Reported	Accuracy	Not Reported
Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections Mar 12, 2026	No Not Reported	Automatic Metrics	Not Reported	Accuracy	Not Reported
Human-Centred LLM Privacy Audits: Findings and Frictions Mar 12, 2026	No Not Reported	Automatic Metrics	Not Reported	Accuracy	Not Reported

Protocol Diff (Top Papers)

Fast side-by-side comparison for the highest-ranked papers in this hub.

Signal	AgentHER: Hindsight Experience Replay for LLM Agent…	SODIUM: From Open Web Data to Queryable Databases	Meanings and Measurements: Multi-Agent Probabilisti…
Human Feedback	Demonstrations	Expert Verification	Demonstrations
Evaluation Modes	Human Eval, Llm As Judge	Automatic Metrics	Simulation Env
Benchmarks	WebArena, ToolBench	Sodium Bench	Mapg Bench
Metrics	Precision, Pass@1	Accuracy	Not reported
Quality Controls	Not reported	Not reported	Not reported
Rater Population	Unknown	Domain Experts	Unknown
Annotation Unit	Trajectory	Unknown	Unknown

Research Utility Snapshot

Human Feedback Mix

Demonstrations (3)
Expert Verification (1)
Pairwise Preference (1)

Evaluation Modes

Automatic Metrics (7)
Simulation Env (5)
Human Eval (1)
Llm As Judge (1)

Top Benchmarks

BIRD (1)
Mapg Bench (1)
Sodium Bench (1)
ToolBench (1)

Top Metrics

Accuracy (7)
Cost (2)
Precision (2)
Pass@1 (1)

Rater Population Mix

Domain Experts (1)

Quality Controls

Coverage diagnostics (sample-based): human-feedback 41.7% · benchmarks 33.3% · metrics 75.0% · quality controls 0.0%.

Top Papers

AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling
Liang Ding · Mar 22, 2026 · Citations: 0

Demonstrations Human EvalLlm As Judge Long Horizon

LLM agents fail on the majority of real-world tasks -- GPT-4o succeeds on fewer than 15% of WebArena navigation tasks and below 55% pass@1 on ToolBench (Zhou et al., 2024; Qin et al., 2024) -- yet every failed trajectory is routinely…
SODIUM: From Open Web Data to Queryable Databases
Chuxuan Hu, Philip Li, Maxwell Yang, Daniel Kang · Mar 19, 2026 · Citations: 0

Expert Verification Automatic Metrics Multi Agent

Existing systems struggle with SODIUM tasks: we evaluate 6 advanced AI agents on SODIUM-Bench, with the strongest baseline achieving only 46.5% accuracy.
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah, Omkar Patil, Thao Nguyen · Mar 19, 2026 · Citations: 0

Demonstrations Simulation Env Multi Agent

To address this limitation, we propose MAPG (Multi-Agent Probabilistic Grounding), an agentic framework that decomposes language queries into structured subcomponents and queries a VLM to ground each component.
BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion
Xinyu Gao, Gang Chen, Javier Alonso-Mora · Mar 10, 2026 · Citations: 0

Automatic MetricsSimulation Env Web Browsing

As a result, they struggle to infer target locations in occluded regions, typically caused by furniture or moving humans.
Aligning Large Language Models with Searcher Preferences
Wei Wu, Peilun Zhou, Liyi Chen, Qimeng Wang, Chengqiang Lu · Mar 11, 2026 · Citations: 0

Pairwise Preference Web Browsing

This setting introduces challenges, including robustness to noisy retrieval, non-negotiable safety guarantees, and alignment with diverse user needs.
TimeWarp: Evaluating Web Agents by Revisiting the Past
Md Farhan Ishmam, Kenneth Marino · Mar 5, 2026 · Citations: 0

Demonstrations Web Browsing

The improvement of web agents on current benchmarks raises the question: Do today's agents perform just as well when the web changes?
Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
Qihui Zhu, Shouwei Ruan, Xiao Yang, Hao Jiang, Yao Huang · Mar 23, 2026 · Citations: 0

Automatic MetricsSimulation Env Web Browsing

Despite the widespread adoption of MLLMs in embodied agents, their capabilities remain largely confined to reactive planning from immediate observations, consistently failing in spatial reasoning across extensive spatiotemporal scales.
Integrating Deep RL and Bayesian Inference for ObjectNav in Mobile Robotics
João Castelo-Branco, José Santos-Victor, Alexandre Bernardino · Mar 26, 2026 · Citations: 0

Simulation Env Web Browsing

Autonomous object search is challenging for mobile robots operating in indoor environments due to partial observability, perceptual uncertainty, and the need to trade off exploration and navigation efficiency.
Rethinking SAR ATR: A Target-Aware Frequency-Spatial Enhancement Framework with Noise-Resilient Knowledge Guidance
Yansong Lin, Zihan Cheng, Jielei Wang, Guoming Lua, Zongyong Cui · Mar 23, 2026 · Citations: 0

Automatic Metrics Web Browsing

However, the coherent speckle noise inherent in SAR imagery often obscures salient target features, leading to degraded recognition accuracy and limited model generalization.
Beyond Explicit Edges: Robust Reasoning over Noisy and Sparse Knowledge Graphs
Hang Gao, Dimitris N. Metaxas · Mar 14, 2026 · Citations: 0

Automatic Metrics Web Browsing

INSES consistently outperforms SOTA RAG and GraphRAG baselines across multiple benchmarks.
Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections
Łukasz Borchmann, Jordy Van Landeghem, Michał Turski, Shreyansh Padarha, Ryan Othniel Kearns · Mar 12, 2026 · Citations: 0

Automatic Metrics Web Browsing

To evaluate agentic behaviour, we introduce a novel evaluation protocol measuring the accuracy-effort trade-off.
Human-Centred LLM Privacy Audits: Findings and Frictions
Dimitri Staufer, Kirsten Morehouse, David Hartmann, Bettina Berendt · Mar 12, 2026 · Citations: 0

Automatic Metrics Web Browsing

Our findings also contribute to exposing a broader generative AI evaluation crisis: when outputs are probabilistic, context-dependent, and user-mediated through elicitation, what model--individual associations even include is…

Related Hubs

Get Started

Join the #1 Platform for AI Training Talent

Where top AI builders and expert AI Trainers connect to build the future of AI.

Self-Service

Post a Job

Post your project and get a shortlist of qualified AI Trainers and Data Labelers. Hire and manage your team in the tools you already use.

Create Account & Post a Job

Managed Service

For Large Projects

Done-for-You

We recruit, onboard, and manage a dedicated team inside your tools. End-to-end operations for large or complex projects.

Learn About Managed Service

For Freelancers

Join as an AI Trainer

Find AI training and data labeling projects across platforms, all in one place. One profile, one application process, more opportunities.

Join Now