Human Feedback and Eval Paper Explorer

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Preference alignment is usually achieved by weight-updating training on preference data, which adds substantial alignment-stage compute and provides limited mechanistic visibility.
• We propose Dynamic SAE Steering for Preference Alignment (DSPA), an inference-time method that makes sparse autoencoder (SAE) steering prompt-conditional.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Recent test-time alignment methods offer a lightweight alternative, but have been explored mainly for preference alignment rather than reasoning.
• Furthermore, TARo also generalizes from small to large backbones without retraining, extending test-time alignment from preference optimization to robust, cross-domain reasoning.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• This paper proposes Group-relative Implicit Fine-Tuning (GIFT), a reinforcement learning framework for aligning large language models (LLMs) that unifies on-policy optimization with implicit preference learning.
• Results show that GIFT converges faster, generalizes better with reduced overfitting, and outperforms GRPO on mathematical reasoning benchmarks (GSM8K, MATH, AIME) as well as generation tasks' evaluations (AlpacaEval and Arena-Hard).

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Despite its small size, fine-tuning Llama-3-8B-Base on PiKa-SFT even outperforms the official Llama-3-8B-Instruct model trained on over 10M proprietary examples on widely used benchmarks such as AlpacaEval 2.0 and Arena-Hard.
• Additionally, we provide 30k high-quality preference optimization examples to further enhance alignment.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Self-play preference optimization has emerged as a prominent paradigm for aligning large language models (LLMs).
• It typically involves a language model to generate on-policy responses for prompts and a reward model (RM) to guide the selection of chosen and rejected responses, which can be further trained with direct preference optimization (DPO).

Match reason: Keyword overlap 1/1 across title and protocol fields.

• To bridge this gap, we propose Meta-Weighted Adaptive Preference Optimization (MetaAPO), a novel framework that dynamically couples data generation with model training.
• Experiments on AlpacaEval 2, Arena-Hard and MT-Bench demonstrate that MetaAPO consistently outperforms existing preference optimization approaches across various settings, while reducing 42% in online annotation costs.

Match reason: Keyword overlap 1/1 across title and protocol fields.

• Direct Preference Optimization (DPO) has emerged as a promising approach for aligning large language models with human preferences.
• To further mitigate the noise in different reward models, we propose a Bayesian Aggregation approach that unifies multiple margin sources (external and implicit) into a single preference probability.

Match reason: Matched by broad semantic/index fallback.

Match reason: Matched by broad semantic/index fallback.