awesome-transformer-search
github.com/automl/awesome-transformer-search ↗A curated list of awesome resources combining Transformers with Neural Architecture Search
270
GitHub Stars
70
Curated Resources
4
Categories
4 hours ago
Last Refreshed
General Transformer SearchDomain Specific Transformer SearchTransformer SurveysFoundation Models
Use this list with your AI agent
Add the Context Awesome MCP server to Claude, Cursor, or any MCP client, then ask:
"Show me transformers knowledge: insights, searchable parameters, attention resources from awesome-transformer-search"
Installation instructions →What's inside
General Transformer Search
- Analyzing and Mitigating Interference in Neural Architecture Search
ICML'22
- AutoTrans: Automating Transformer Design via Reinforced Architecture Search
NLPCC'21
- BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
ICCV'21
- Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
NeurIPS'22
- Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition
arxiv [Aug'20]
- LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
AutoML Conference 2022 Workshop Track
Domain Specific Transformer Search
- Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be PrunedTransformers Knowledge: Insights, Searchable parameters, Attention
ACL'19
- AutoAttend: Automated Attention Representation SearchTransformers Knowledge: Insights, Searchable parameters, Attention
ICML'21
- AutoBERT-Zero: Evolving the BERT backbone from scratchNatural Language Processing
AAAI'22
- AutoFormer: Searching Transformers for Visual RecognitionVision
ICCV'21
- AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language ModelsNatural Language Processing
ACL'21
- BEiT: BERT Pre-Training of Image TransformersTransformers Knowledge: Insights, Searchable parameters, Attention
ICLR'22 (Oral)
Transformer Surveys
- A Survey of Vision Transformers
TPAMI'22
- Efficient Transformers: A Survey
ACM Computing Surveys'22
- Neural Architecture Search for Transformers: A Survey
IEEE xplore [Sep'22]
- Transformers in Vision: A Survey
ACM Computing Surveys'22
Foundation Models
Showing a sample of 70 resources. View the full list on GitHub →