Skip to content
OpenTrain AIFor AI Companies
← Back to explorer

Tag: General

General papers in the current HFEPX explorer (897 papers).

Papers in tag: 897

Global RSS

Running a General study?

Post a Job →

Research Utility Snapshot

Evaluation Modes

  • Automatic Metrics (11)
  • Llm As Judge (3)
  • Simulation Env (2)

Human Feedback Types

  • Red Team (5)
  • Pairwise Preference (4)
  • Critique Edit (2)

Required Expertise

  • General (20)
Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs

Nithin Rao Koluguri, Sasha Meister, Nikolay Karpov, Piotr Zelasko, Desh Raj, Jagadeesh Balam · Jun 28, 2026 · Citations: 0

Pairwise Preference Automatic Metrics General
  • Current benchmarks therefore cannot measure whether a model follows user preferences for output style.
  • We introduce PreferenceASR, a test set evaluating ASR systems on their ability to follow natural-language preference instructions across four categories: normalization, entities, disfluencies, and case.
Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning

Craig Atkinson · Jun 28, 2026 · Citations: 0

Llm As JudgeAutomatic Metrics General
  • We identify intervention bias as a previously unquantified failure mode of zero-shot large-language-model (LLM) educational advisory agents: without task-specific training, they recommend action when a hindsight-optimal oracle policy…
  • We also show an Evaluation Gap: LLM-as-judge scoring (DeepEval G-Eval) is blind to intervention bias, rewarding fluent over-prescription rather than decision quality.
Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts

Alex Kwon · Jun 28, 2026 · Citations: 0

Demonstrations General
  • LLM agents carry conclusions across steps and sessions in compressed memory, and memory products (e.g., mem0, LangMem) rewrite conversation into stored "facts" that later steps trust.
  • We show this rewriting manufactures confidence: across our constructed agent settings, a casual, hedged remark becomes a confident, dated assertion the agent then obeys like a verified fact, granting every above-clearance request it faces.
The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling

Shubh Chapra, Dhruv Kumar, Murari Mandal, Yash Sinha · Jun 28, 2026 · Citations: 0

Automatic Metrics General
  • We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows.
  • A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning.
Selective Memory Retention for Long-Horizon LLM Agents

Pranath Reddy · Jun 28, 2026 · Citations: 0

Simulation Env General
  • When does retention matter for memory-augmented LLM agents?
  • We study this with TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents that scores entries by interpretable features (success, age, access frequency, redundancy, specificity, similarity, downstream utility)…
Fine-Tuning General-Purpose Large Language Models for Agricultural Applications:A Reproducible Framework and Evaluation Protocol Based on Qwen3-8B

Zhaoyang Li, Ruijie Zhang, Jiaqi Liu, Zhaoji Sun · Jun 27, 2026 · Citations: 0

Rubric Rating General
  • Agricultural applications, however, are domain-specific, region-dependent, time-sensitive, and safety-critical.
  • Without data governance, expert evaluation, and evidence constraints, an agricultural assistant mayproduce unreliable advice on crop diseases, pesticide use, fertilization, or policy interpretation.To avoid presenting unverified simulated…
When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search

Yiling Tao, Shihan Deng, Meiling Tao, Pengzhi Wei, Zhichao Hu, Zhihao Zhu · Jun 26, 2026 · Citations: 0

Simulation Env General
  • Search agents powered by large language models (LLMs) are increasingly used to solve complex information-seeking tasks, requiring multi-step retrieval and reasoning to fulfill user goals.
  • To address this gap, we introduce DiscoBench, a benchmark for clarification-aware deep search, designed to evaluate whether search agents can proactively identify ambiguity, ask effective clarification questions, and recover correct…
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation

Han Jeon, Shiv Medler, Joseph Voyles, Matt Wood · Jun 24, 2026 · Citations: 0

Red Team Llm As JudgeAutomatic Metrics General
  • Safety evaluation of LLM outputs has generally relied on LLM-based judges, which can be effective but are often slow and expensive to deploy at scale.
  • In this paper, we evaluate whether fine-tuned modern encoder classifiers from the ModernBERT family, including ModernBERT and Ettin, can reliably identify harmful LLM outputs in user-model conversations without substantial performance loss…
RAS: Measuring LLM Safety Through Refusal Alignment

Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, Wei-Bin Lee · Jun 24, 2026 · Citations: 0

Red Team Automatic Metrics General
  • Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.
  • We propose **SafeVec**, a white-box evaluation procedure that measures safety from internal representations rather than generated answers.
Staying In Character: Perspective-Bounded Memory For Book-Based Role-Playing Agents

Xushuo Tang, Junhe Zhang, Zihan Yang, Yifu Tang, Sichao Li, Longbin Lai · Jun 24, 2026 · Citations: 0

Pairwise Preference Automatic Metrics General
  • Recent LLM role-playing systems build character agents from novels by extracting characters, scenes, and relations.
  • To address these failures, we propose REVERIEMEM, a three-layer memory architecture for book-based character agents.
A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation

Abrar Alotaibi, Raed Mughus, Moataz Ahmed · Jun 24, 2026 · Citations: 0

Red Team Automatic Metrics General
  • Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness.
  • The approach identifies how structural constraints in summarization can shape vulnerability patterns, with format limitations yielding measurable gains in faithfulness, and shows that architectural design choices typically outweigh…
Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents

Yuxin Wang, Paul Thomas, Zhiwei Yu, Yuan Gao, Saeed Hassanpour, Soroush Vosoughi · Jun 24, 2026 · Citations: 0

Pairwise Preference Automatic Metrics General
  • Specifically, how they shape an agent's responses under varying conversational contexts and whether they lead to substantively different response behaviors.
  • We present a fine-grained taxonomy of conversational memory, classify retrieved memories into different role types, and design a user-centric evaluation framework that simulates user perspectives.
RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory

Yixun Hu, Zhicheng Zheng, Lihan Zha, Chunwei Xing, Rajdeep Singh, Omar Hossain · Jun 23, 2026 · Citations: 0

Automatic Metrics General
  • In this paper, we propose RAVEN, an agentic memory system for long-horizon robotic question answering and navigation.
  • Across several simulated and real-world video question-answering benchmarks, RAVEN consistently surpasses caption-based memory systems and matches frontier VLMs on long-horizon tasks at 10\times lower retrieval cost.
To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG

Jungseob Lee, Chanjun Park, Heuiseok Lim · Jun 23, 2026 · Citations: 0

Automatic Metrics General
  • Multi-agent document assessment for retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood.
  • We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA benchmarks, revealing a sharp dichotomy in how models benefit from assessment.
What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics

Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal · Jun 23, 2026 · Citations: 0

Red Team General
  • Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training.
  • Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training.
LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challenges

Thi Huyen Nguyen, Zahra Ahmadi · Jun 23, 2026 · Citations: 0

Critique Edit General
  • The rapid growth of scientific submissions has pushed traditional peer review toward its scalability limits, motivating the exploration of large language models (LLMs) as intelligent automated evaluation assistants.
  • We present a structured taxonomy of modeling approaches (including prompt-based, supervised, retrieval-augmented, and alignment-optimized approaches), and synthesize empirical findings across existing benchmarks.
Do Thinking Tokens Help with Safety?

Narutatsu Ri, Abhishek Panigrahi, Sanjeev Arora · Jun 23, 2026 · Citations: 0

Critique Edit Automatic Metrics General
  • Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.
  • It is also generally believed that this more "deliberative" mode should improve alignment and safety, by providing the model a safe space to consider whether its planned answer to a request violates its safety principles.