awesome-production-machine-learning

Accelerate abstracts exactly and only the boilerplate code related to multi-GPU/TPU/mixed-precision and leaves the rest of your code unchanged.

Adapters is a unified library for parameter-efficient and modular transfer learning.

BitBLAS is a library to support mixed-precision BLAS operations on GPUs

Bitsandbytes library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.

Cache-DiT is built on top of Diffusers and supports nearly all DiTs, providing hybrid cache acceleration (DBCache, TaylorSeer, SCM, etc.) and comprehensive parallelism optimizations including Context Parallelism, Tensor Parallelism, and hybrid 2D/3D parallelism, with compatibility for compilation, CPU offloading, and quantization.

A unified deep learning system for big model era, which helps users to efficiently and quickly deploy large AI model training and inference.

Acme is a library of reinforcement learning (RL) building blocks that strives to expose simple, efficient, and readable agents.

AReaL is a reinforcement learning library.

ChatLearn is a flexible and efficient reinforcement learning training framework for large language models, supporting distributed training engines (FSDP2, Megatron) and inference engines (vLLM, SGLang) with modern RL algorithms such as GRPO and GSPO.

CleanRL is a Deep Reinforcement Learning library that provides high-quality single-file implementation with research-friendly features. The implementation is clean and simple, yet we can scale it to run thousands of experiments using AWS Batch.

CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks.

d3rlpy is an offline deep reinforcement learning library for practitioners and researchers.

An open-source bias audit toolkit for data scientists, machine learning researchers, and policymakers to audit machine learning models for discrimination and bias, and to make informed and equitable decisions around developing and deploying predictive risk-assessment tools.

Interpretability and explainability of data and machine learning models including a comprehensive set of algorithms that cover different dimensions of explanations along with proxy explainability metrics.

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The initial focus on the library is on black-box, instance based model explanations.

model interpretability and understanding library for PyTorch developed by Facebook. It contains general purpose implementations of integrated gradients, saliency maps, smoothgrad, vargrad and others for PyTorch models.

Agenta provides end-to-end tools for the entire LLMOps workflow: building (LLM playground, evaluation), deploying (prompt and configuration management), and (LLM observability and tracing).

AirLLM optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card without quantization, distillation and pruning.

AITemplate (AIT) is a Python framework that transforms deep neural networks into CUDA (NVIDIA GPU) / HIP (AMD GPU) C++ code for lightning-fast inference serving.

BentoML is an open source framework for high performance ML model serving.

BISHENG is an open LLM application devops platform, focusing on enterprise scenarios.

Machine Learning production server for TensorFlow, XGBoost and Cafe models written in C++ and maintained by Jolibrain.

AI2-THOR is a near photo-realistic interactable framework for AI agents.

AIDE is an open-source ML engineering agent that uses a tree search algorithm to autonomously explore, implement, and evaluate solution strategies for machine learning tasks.

Automated feature, model, and hyperparameter selection for tabular, image, and text data on top of popular machine learning libraries (Scikit-Learn, LightGBM, CatBoost, PyTorch, MXNet).

AutoML library for Keras based on

Framework to automate algorithm and hyperparameter tuning for sklearn.

Ax is an accessible, general-purpose platform for understanding, managing, deploying, and automating adaptive experiments.

BoTorch is a library for Bayesian Optimization built on PyTorch.

The AI Gateway is a blazing fast AI Gateway with integrated guardrails.

Structured, source-backed dataset tracking 96 AI-attributed workforce reductions (457K workers affected, 13 countries, 13 sectors). Every entry includes source URLs, attribution tier, and job functions.

ART (Adversarial Robustness Toolbox) provides tools that enable developers and researchers to defend and evaluate Machine Learning models and applications against the adversarial threats of Evasion, Poisoning, Extraction, and Inference.

Covers governance, compliance, and regulatory frameworks essential for responsible ML system deployment across different jurisdictions.

Focuses specifically on generative AI deployment, including LLM operations, prompt engineering, and GenAI-specific monitoring and safety tools.

Curated list of production-grade tools and best practices for building scalable RAG systems.

A super-easy way to record, search and compare AI experiments.

Auto-Magical Experiment Manager & Version Control for AI (previously Trains).

DataHub is an open-source data catalog for the modern data stack.

Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.

DVC (Data Version Control) is a git fork that allows for version management of models.