awesome-ai-infrastructure
github.com/awesomelistsio/awesome-ai-infrastructure ↗A curated list of awesome tools, frameworks, platforms, and resources for building scalable and efficient AI infrastructure, including distributed training, model serving, MLOps, and deployment.
Use this list with your AI agent
Add the Context Awesome MCP server to Claude, Cursor, or any MCP client, then ask:
"Show me mlops and automation resources from awesome-ai-infrastructure"
Installation instructions →What's inside
MLOps and Automation
- Airflow
A platform for orchestrating complex workflows, commonly used in machine learning pipelines.
- DVC (Data Version Control)
A tool for version control and reproducibility in machine learning projects.
- Kubeflow
A platform for orchestrating machine learning workflows on Kubernetes.
- Metaflow
A human-centric framework for building and managing real-life data science projects, developed by Netflix.
- MLflow
An open-source platform for managing the end-to-end machine learning lifecycle.
- ZenML
An extensible MLOps framework for creating portable, production-ready machine learning pipelines.
Infrastructure as Code
- Ansible
An open-source automation tool for provisioning and managing infrastructure.
- AWS CloudFormation
A service for automating AWS resource deployment and management.
- Google Deployment Manager
An infrastructure management tool for Google Cloud Platform.
- Pulumi
Infrastructure as code for deploying and managing cloud infrastructure using programming languages.
- Terraform
A tool for building, changing, and versioning infrastructure safely and efficiently.
Data Management
- Apache Hudi
A data management framework that simplifies incremental data processing and streaming analytics.
- Delta Lake
An open-source storage layer that brings reliability to data lakes.
- Feast
An open-source feature store for managing and serving machine learning features.
- Great Expectations
A tool for data validation and testing in machine learning workflows.
- LakeFS
An open-source data versioning platform for managing data lakes.
Optimization Tools
- Apache TVM
A deep learning compiler stack for optimizing models on various hardware backends.
- Intel OpenVINO
A toolkit for optimizing and deploying AI inference on Intel hardware.
- NVIDIA TensorRT
A high-performance deep learning inference optimizer and runtime.
- OctoML
An AI model optimization platform for efficient deployment on edge and cloud.
- Quantization Aware Training (QAT)
Tools for optimizing model performance through quantization.
Cloud Platforms
- AWS SageMaker
A comprehensive platform for building, training, and deploying machine learning models on AWS.
- Azure Machine Learning
A cloud-based platform for training, deploying, and managing machine learning models.
- Google AI Platform
Google Cloud’s integrated environment for AI development and deployment.
- IBM Watson Studio
A suite of tools for data science, machine learning, and AI model development.
- KubeStellar Console
A CNCF Sandbox multi-cluster Kubernetes management dashboard for deploying and observing workloads across edge and cloud infrastructure.
- Lium (lium.io)
Decentralized GPU compute marketplace on Bittensor Subnet 51 for AI and machine learning workloads.
Learning Resources
- AWS SageMaker Workshops
Example projects and tutorials for using AWS SageMaker.
- Coursera: MLOps Fundamentals
A course on MLOps best practices for machine learning projects.
- Google Cloud: ML Operations
Training resources on MLOps and model deployment.
- Kubeflow Documentation
Official documentation and guides for using Kubeflow.
- PyTorch Distributed Training Guide
A tutorial on distributed training with PyTorch.
Model Serving and Deployment
- Bifrost
Self-hosted OpenAI-compatible gateway for multi-provider LLM routing, fallbacks, load balancing, and observability.
- KFServing (KServe)
A Kubernetes-based model serving solution as part of the Kubeflow project.
- NVIDIA Triton Inference Server
A scalable model serving platform supporting multiple frameworks.
- ONNX Runtime
A cross-platform, high-performance scoring engine for serving ONNX models.
- Seldon Core
An open-source platform for deploying and monitoring machine learning models on Kubernetes.
- TensorFlow Serving
A flexible, high-performance serving system for machine learning models.
Books
- Building Machine Learning Powered Applications
A guide to building robust ML applications in production.
- Designing Data-Intensive Applications
A comprehensive guide to building scalable and reliable data systems.
- Machine Learning Engineering
A book on building scalable machine learning infrastructure.
- Reliable Machine Learning
A book on creating resilient machine learning infrastructure.
Showing a sample of 53 resources. View the full list on GitHub →