Avey-B

Research Summary

Contribution Summary

• Compact pretrained bidirectional encoders remain the backbone of industrial NLP under tight compute and memory budgets.
• Their effectiveness stems from self-attention's ability to deliver high-quality bidirectional contextualization with sequence-level parallelism, as popularized by BERT-style architectures.
• Recently, Avey was introduced as an autoregressive, attention-free alternative that naturally admits an encoder-only adaptation.

Why It Matters For Eval

• Results show that this reformulated architecture compares favorably to four widely used Transformer-based encoders, consistently outperforming them on standard token-classification and information-retrieval benchmarks while scaling more eff

Related Papers

• GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning Embedding Neighbor
• Shapley Value Computation in Ontology-Mediated Query Answering Embedding Neighbor
• Knowledge Fusion of Large Language Models Via Modular SkillPacks Embedding Neighbor
• A Survey on the Optimization of Large Language Model-based Agents Embedding Neighbor
• Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task Embedding Neighbor
• Entropy-Based Data Selection for Language Models Embedding Neighbor
• RAIR: A Rule-Aware Benchmark Uniting Challenging Long-Tail and Visual Salience Subset for E-commerce Relevance Assessment Embedding Neighbor
• Uncovering Context Reliance in Unstructured Knowledge Editing Embedding Neighbor
• Chlorophyll-a Mapping and Prediction in the Mar Menor Lagoon Using C2RCC-Processed Sentinel 2 Imagery Embedding Neighbor
• Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination Tag Overlap
• Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models Tag Overlap
• EARL: Entropy-Aware RL Alignment of LLMs for Reliable RTL Code Generation Tag Overlap