awesome-llmops

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision.

an easy-to-use and performant open-source experiment tracker.

Instruct-tune LLaMA on consumer hardware

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler.

A tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures.

A fast open framework for deep learning.

The LLMOps platform to build robust LLM apps. Easily experiment and evaluate different prompts, models, and workflows to build robust apps.

Open-source control plane for building and operating AI agents like APIs at scale, with routing, memory, observability, identity, auth, and policy controls.

Type-Safe Markdown-based Agents

Evaluation framework for automated, reproducible scoring of LLM, agent, and workflow performance.

A Reliable Open Source AI studio to build core infrastructure stack for your LLM Applications. It allows you to gain visibility, make your application reliable, and prepare it for production with features such as caching, rate limiting, exponential retry, model fallback, and more.

ML observability for LLMs, vision, language, and tabular models.

Automated optimization engine for improving agent workflows using feedback-driven iterative refinements.

Information-theoretic context optimization proxy. Cuts LLM token costs by 70–95% with zero accuracy loss using greedy submodular knapsack maximization.

FeatherCNN is a high performance inference engine for convolutional neural networks.

A library for high performance deep learning inference on NVIDIA GPUs.

Self-hostable AI Agent Workforce Platform. Multi-agent orchestration with remote AI workstations (AgentPods), PTY sandbox + git worktree isolation, built-in Kanban, and per-pod MCP server. Supports Claude Code, Codex CLI, Gemini CLI, Aider, OpenCode.

Open-source ML engineering agent that uses tree search to explore solution spaces. Automates machine learning experimentation from data analysis to model training. Paper .

Deterministic Python orchestrator for 37 CLI coding agents (Claude Code, Codex CLI, Gemini CLI, GitHub Copilot CLI, Cursor, Aider, OpenHands, OpenCode, Goose, Qwen, Ollama, ...) running in parallel git worktrees. First-class MCP server, quality gates, cost tracking with budgets.

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.

Open Code LLMs for Code Understanding and Generation.

A platform to programmatically author, schedule and monitor workflows.

An Open-Source Platform for Production Data Science

Workflow engine for Kubernetes.

Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management.

Comet is an MLOps platform that offers Model Production Management, a Model Registry, and full model lineage from training straight through to production. Use Comet for model reproducibility, model debugging, model versioning, model visibility, model auditing, model governance, and model monitoring.

Open-source confidential AI framework for secure LLM deployment with data privacy, providing hardware-enforced isolation for production ML workloads.

An easy way to turn any app into searchable data for LLMs.

An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search.

AI Native database for embedding vectors

the open source embedding database

A 10x faster, cheaper, and better vector database

The AI-native database built for LLM applications, providing incredibly fast vector and full-text search

Code and documentation to train Stanford's Alpaca models, and generate the data.

Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects.

A 7B Large Language Model fine-tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca.

BigScience Large Open-science Open-access Multilingual Language Model

ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B .

A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.

Alpaca-LoRA as Chatbot service

The Unified Model Serving Framework

serving the OpenAI CLIP model

fast inference engine for Transformer models in C++

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

fast inference engine for whisper in C++ using CTranslate2.