Patches Are All You Need?
Asher Trockman, Kolter, J. Zico
Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
Although convolutional networks have been the dominant architecture for vision tasks for many years, recent experiments have shown that Transformer-based models, most notably the Vision Transformer (ViT), may exceed their performance in some settings. However, due to the quadratic runtime of the self-attention layers in Transformers, ViTs require the use of patch embeddings, which group together small regions of the ...
image into single input features, in order to be applied to larger image sizes. This raises a question: Is the performance of ViTs due to the inherently-more-powerful Transformer architecture, or is it at least partly due to using patches as the input representation? In this paper, we present some evidence for the latter: specifically, we propose the ConvMixer, an extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in that it operates directly on patches as input, separates the mixing of spatial and channel dimensions, and maintains equal size and resolution throughout the network. In contrast, however, the ConvMixer uses only standard convolutions to achieve the mixing steps. Despite its simplicity, we show that the ConvMixer outperforms the ViT, MLP-Mixer, and some of their variants for similar parameter counts and data set sizes, in addition to outperforming classical vision models such as the ResNet. Our code is available at https://github.com/locuslab/convmixer.
Results & Benchmarks
No concrete benchmark grounding is available yet. Treat the page as context or an implementation starting point only.
Although convolutional networks have been the dominant architecture for vision tasks for many years, recent experiments have shown that Transformer-based models, most notably the Vision Transformer (ViT), may exceed their performance in some settings.
Implementation Evidence Summary
huggingface/pytorch-image-models is the closest maintained adjacent implementation (Matches contextual method/domain keyword: transformer). It is not paper-verified; validate algorithm and evaluation setup against the paper before trusting reported metrics. Community adoption signal: 36885 GitHub stars.
Reproduction Risks
- Adjacent implementations are not paper-verified
- Recommended repository is adjacent and not paper-verified.
Hardware Notes
Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Evidence disclosure
Evidence graph: 3 refs, 3 links.
Utility signals: depth 70/100, grounding 75/100, status medium.
Implementation Status
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- No maintained paper-verified implementation was found; start with the closest related repositories below.
- Compare repo methods against the paper equations/algorithm before trusting metrics.
- Create a minimal baseline implementation from the paper and use adjacent repos as references.
Reproduction readiness
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
No verified implementation available
- · No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.
No benchmark numbers could be verified. You will not be able to validate reproduction correctness against published numbers.
Framework baselines
- Hugging Face Transformers training guide
Modern transformer training baseline.
- PyTorch nn.Transformer docs
Reference transformer building block implementation.
Closest related implementations
These are not paper-verified. Use them as reference points when no direct implementation is available.
- huggingface/pytorch-image-modelsAdjacentConfidence: MediumStars: 36,885
Matches contextual method/domain keyword: transformer
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet.
Continue with targeted Hugging Face searches derived from the paper title and method context:
Tip: start with models, then check datasets/spaces if you need evaluation data or demos.
Direct artifact matches are currently sparse. Use targeted Hugging Face searches to quickly locate candidate models, datasets, and demos.
Research context
0
Citations
0
References
Tasks
Computer science, Quadratic equation, Convolutional neural network, Pattern recognition (psychology), Computer engineering, Physical Sciences
Methods
Transformer, Architecture, Algorithm
Domains
Artificial intelligence, Computer Vision and Pattern Recognition
Evaluation & Human Feedback Data
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXExplore Similar Papers
Jump to Paper2Code search queries derived from this paper's research context.
Related papers
-
Search on Paper2Code
Framework for extracting rotation invariant features for image classification and an application using Haar wavelets (2008) Semantic similarity
-
Search on Paper2Code
A novel model of neural networks for fast data detection (2006) Semantic similarity
-
Search on Paper2Code
A Novel Hybrid Perceptron Neural Network Algorithm for Classifying Breast MRI Tumors (2014) Semantic similarity
-
Search on Paper2Code
Image Super-Resolution with Fast Approximate Convolutional Sparse Coding (2014) Semantic similarity
-
Search on Paper2Code
A Hybrid Approach for Image Edge Detection Using Neural Network and Particle Swarm Optimization (2014) Semantic similarity
-
Search on Paper2Code
A lateral contribution learning algorithm for multi MLP architecture. (1994) Semantic similarity
Need human evaluators for your AI research? Scale annotation with expert AI Trainers.