Skip to content
← Back to explorer

HFEPX Hub

Coding + Simulation Env Papers

Updated from current HFEPX corpus (Feb 27, 2026). 34 papers are grouped in this hub page. Common evaluation modes: Simulation Env, Automatic Metrics. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequent quality control: Calibration. Frequently cited benchmark: Retrieval. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 25, 2026.

Papers: 34 Last published: Feb 25, 2026 Global RSS Tag RSS
CodingSimulation Env

Research Narrative

Grounded narrative Model: deterministic-grounded Source: persisted

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 34 papers for Coding + Simulation Env Papers. Dominant protocol signals include simulation environments, automatic metrics, human evaluation, with frequent benchmark focus on Retrieval, SWE-bench and metric focus on accuracy, cost. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Why This Matters For Eval Research

Protocol Takeaways

Benchmark Interpretation

  • Retrieval appears in 8.8% of hub papers (3/34); use this cohort for benchmark-matched comparisons.
  • SWE-bench appears in 5.9% of hub papers (2/34); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • accuracy is reported in 23.5% of hub papers (8/34); compare with a secondary metric before ranking methods.
  • cost is reported in 14.7% of hub papers (5/34); compare with a secondary metric before ranking methods.

Researcher Checklist

  • Close gap on Papers with explicit human feedback. Coverage is a replication risk (5.9% vs 45% target).
  • Close gap on Papers reporting quality controls. Coverage is a replication risk (2.9% vs 30% target).
  • Tighten coverage on Papers naming benchmarks/datasets. Coverage is usable but incomplete (26.5% vs 35% target).
  • Maintain strength on Papers naming evaluation metrics. Coverage is strong (55.9% vs 35% target).
  • Close gap on Papers with known rater population. Coverage is a replication risk (8.8% vs 35% target).
  • Close gap on Papers with known annotation unit. Coverage is a replication risk (14.7% vs 35% target).

Papers with explicit human feedback

Coverage is a replication risk (5.9% vs 45% target).

Papers reporting quality controls

Coverage is a replication risk (2.9% vs 30% target).

Papers naming benchmarks/datasets

Coverage is usable but incomplete (26.5% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (55.9% vs 35% target).

Papers with known rater population

Coverage is a replication risk (8.8% vs 35% target).

Papers with known annotation unit

Coverage is a replication risk (14.7% vs 35% target).

Suggested Reading Order

  1. 1. TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  2. 2. Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  3. 3. Scalable Kernel-Based Distances for Statistical Inference and Integration

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  4. 4. MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification

    Include a human-eval paper to anchor calibration against automated judge settings.

  5. 5. Self-Correcting VLA: Online Action Refinement via Sparse World Imagination

    Adds simulation environments for broader coverage within this hub.

  6. 6. Toward an Agentic Infused Software Ecosystem

    Adds simulation environments for broader coverage within this hub.

  7. 7. SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

    Adds simulation environments for broader coverage within this hub.

  8. 8. Explicit Grammar Semantic Feature Fusion for Robust Text Classification

    Adds simulation environments for broader coverage within this hub.

Known Limitations

  • Only 2.9% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Rater population is under-specified (8.8% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Links

human_eval vs automatic_metrics

both=1, left_only=1, right_only=8

1 papers use both Human Eval and Automatic Metrics.

simulation_env vs automatic_metrics

both=9, left_only=25, right_only=0

9 papers use both Simulation Env and Automatic Metrics.

simulation_env vs human_eval

both=2, left_only=32, right_only=0

2 papers use both Simulation Env and Human Eval.

Benchmark Brief

Retrieval

Coverage: 3 papers (8.8%)

3 papers (8.8%) mention Retrieval.

Examples: Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents , Quecto-V1: Empirical Analysis of 8-bit Quantized Small Language Models for On-Device Legal Retrieval , Embodied Task Planning via Graph-Informed Action Generation with Large Language Model

Benchmark Brief

SWE-bench

Coverage: 2 papers (5.9%)

2 papers (5.9%) mention SWE-bench.

Examples: Hybrid-Gym: Training Coding Agents to Generalize Across Tasks , SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training

Benchmark Brief

SWE-bench Verified

Coverage: 2 papers (5.9%)

2 papers (5.9%) mention SWE-bench Verified.

Examples: Hybrid-Gym: Training Coding Agents to Generalize Across Tasks , SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training

Metric Brief

accuracy

Coverage: 8 papers (23.5%)

8 papers (23.5%) mention accuracy.

Examples: Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text , MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification , Counterfactual Simulation Training for Chain-of-Thought Faithfulness

Metric Brief

cost

Coverage: 5 papers (14.7%)

5 papers (14.7%) mention cost.

Examples: Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents , MAEB: Massive Audio Embedding Benchmark , Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

Metric Brief

coherence

Coverage: 2 papers (5.9%)

2 papers (5.9%) mention coherence.

Examples: From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design , Embodied Task Planning via Graph-Informed Action Generation with Large Language Model

Most Cited In This Hub

Fast path to methods with the strongest citation traction in this scope.

Papers: TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text , Scalable Kernel-Based Distances for Statistical Inference and Integration

Most Recent

Fast path to latest protocol changes and newly published evaluation setups.

Papers: TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text , Scalable Kernel-Based Distances for Statistical Inference and Integration

Best Protocol Detail

Papers with explicit rater/unit metadata and quality-control signals for reproducibility.

Papers: TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Small Wins Big: Comparing Large Language Models and Domain Fine-Tuned Models for Sarcasm Detection in Code-Mixed Hinglish Text , Scalable Kernel-Based Distances for Statistical Inference and Integration

Top Papers

Related Hubs