Polynomial Mixing for Efficient Self-supervised Speech Encoders

Eva Feillet, Ryan Whetten, David Picard, Alexandre Allauzen · Feb 28, 2026 · Citations: 0

Abstract

State-of-the-art speech-to-text models typically employ Transformer-based encoders that model token dependencies via self-attention mechanisms. However, the quadratic complexity of self-attention in both memory and computation imposes significant constraints on scalability. In this work, we propose a novel token-mixing mechanism, the Polynomial Mixer (PoM), as a drop-in replacement for multi-head self-attention. PoM computes a polynomial representation of the input with linear complexity with respect to the input sequence length. We integrate PoM into a self-supervised speech representation learning framework based on BEST-RQ and evaluate its performance on downstream speech recognition tasks. Experimental results demonstrate that PoM achieves a competitive word error rate compared to full self-attention and other linear-complexity alternatives, offering an improved trade-off between performance and efficiency in time and memory.

HFEPX Relevance Assessment

This paper appears adjacent to HFEPX scope (human-feedback/eval), but does not show strong direct protocol evidence in metadata/abstract.

Eval-Fit Score

5/100 • Low

Treat as adjacent context, not a core eval-method reference.

Human Feedback Signal

Not explicit in abstract metadata

Evaluation Signal

Detected

HFEPX Fit

Adjacent candidate

If you are doing eval pipeline work, start here:

Human Eval Hub LLM-as-Judge Hub Pairwise Preference Hub Tool-Use Eval Hub

Human Data Lens

Uses human feedback: No
Feedback types: None
Rater population: Unknown
Unit of annotation: Unknown
Expertise required: General
Extraction source: Runtime deterministic fallback

Evaluation Lens

Evaluation modes: Automatic Metrics
Agentic eval: None
Quality controls: Not reported
Confidence: 0.45
Flags: low_signal, possible_false_positive, runtime_fallback_extraction

Protocol And Measurement Signals

Benchmarks / Datasets

DROP

Reported Metrics

error ratewer

Research Brief

Deterministic synthesis

In this work, we propose a novel token-mixing mechanism, the Polynomial Mixer (PoM), as a drop-in replacement for multi-head self-attention. HFEPX signals include Automatic Metrics with confidence 0.45. Updated from current HFEPX corpus.

Generated Mar 3, 2026, 7:29 AM · Grounded in abstract + metadata only

Key Takeaways

In this work, we propose a novel token-mixing mechanism, the Polynomial Mixer (PoM), as a drop-in replacement for multi-head self-attention.
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Researcher Actions

Treat this as method context, then pivot to protocol-specific HFEPX hubs.
Cross-check benchmark overlap: DROP.
Validate metric comparability (error rate, wer).

Caveats

Generated from title, abstract, and extracted metadata only; full-paper implementation details are not parsed.
Low-signal flag detected: protocol relevance may be indirect.

Recommended Queries

human-eval protocol design pairwise preference data quality inter-rater agreement adjudication

Research Summary

Contribution Summary

In this work, we propose a novel token-mixing mechanism, the Polynomial Mixer (PoM), as a drop-in replacement for multi-head self-attention.

Why It Matters For Eval

Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

Researcher Checklist

Gap: Human feedback protocol is explicit

No explicit human feedback protocol detected.
Pass: Evaluation mode is explicit

Detected: Automatic Metrics
Gap: Quality control reporting appears

No calibration/adjudication/IAA control explicitly detected.
Pass: Benchmark or dataset anchors are present

Detected: DROP
Pass: Metric reporting is present

Detected: error rate, wer

Category-Adjacent Papers (Broader Context)

These papers are nearby in arXiv category and useful for broader context, but not necessarily protocol-matched to this paper.

InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models Category Neighbor

Citations: 0 Relevance: 3.65
- Shared arXiv category (cs.CL, cs.LG)
- Shared terminology (efficient)
Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking Category Neighbor

Citations: 0 Relevance: 3.65
- Shared arXiv category (cs.CL, cs.LG)
- Shared terminology (rate)
Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models Category Neighbor

Citations: 0 Relevance: 3.20
- Shared arXiv category (cs.CL, cs.LG)
From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning Category Neighbor

Citations: 0 Relevance: 3.20
- Shared arXiv category (cs.CL, cs.LG)
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning Category Neighbor

Citations: 0 Relevance: 3.20
- Shared arXiv category (cs.CL, cs.LG)
SPARTA: Scalable and Principled Benchmark of Tree-Structured Multi-hop QA over Text and Tables Category Neighbor

Citations: 0 Relevance: 3.15
- Shared arXiv category (cs.CL)
- Shared benchmark mentions
- Shared terminology (drop)
A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations Category Neighbor

Citations: 0 Relevance: 2.05
- Shared arXiv category (cs.CL)
- Shared terminology (speech)
AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning Category Neighbor

Citations: 0 Relevance: 2.05
- Shared arXiv category (cs.CL)
- Shared terminology (error)

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote