awesome-llmops

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision.

an easy-to-use and performant open-source experiment tracker.

Instruct-tune LLaMA on consumer hardware

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler.

A tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures.

A fast open framework for deep learning.

The LLMOps platform to build robust LLM apps. Easily experiment and evaluate different prompts, models, and workflows to build robust apps.

Zero-dependency Python CLI and library for measuring lexical convergence, exact-match rate, overlap, and divergence across parallel agent runs.

Open-source control plane for building and operating AI agents like APIs at scale, with routing, memory, observability, identity, auth, and policy controls.

Type-Safe Markdown-based Agents

Gateway-agnostic, self-hosted LLM control plane (MIT): an OpenAI-compatible governance proxy (auth, budgets, guardrails, cost, observability) that runs LiteLLM, Bifrost or Portkey as a swappable, stateless translation engine, so provider keys stay in your own control plane instead of the gateway.

Evaluation framework for automated, reproducible scoring of LLM, agent, and workflow performance.

Track and optimize LLM API costs across OpenAI, Anthropic, and LangChain.

Local security audit for AI API relays and LLM proxies; checks prompt injection, model identity drift, tool-call rewriting, error leakage, SSE anomalies, and Web3 wallet probes.

"Batteries included" logging solution for your Azure OpenAI instance.

CLI tool for running coding agents inside hardware-isolated microVMs with snapshot isolation, egress control, and MCP authorization.

Zero-infra hard spending caps for LLM APIs — blocks before overspending (atomic reservation), per-feature cost attribution, streaming usage, retry-storm detection, and adapters for Vercel AI SDK / LangChain.js / LlamaIndex.TS / Mastra.

Runtime monitoring for AI agents — heartbeat watchdog, loop detection, cost tracking, auto-restart. Python SDK or HTTP API.

Automated optimization engine for improving agent workflows using feedback-driven iterative refinements.

Browser-only calculator for estimating monthly AI agent token spend, avoidable cost, and optimization payback. Live calculator

Optimize agentic capabilities with agents. Reflective optimizer for AI agent prompts, tool code, MCP tool surfaces, and skill packages; learning from evaluation traces, val-only significance gate with a sealed test split scored once. Adapters for τ²-bench, SWE-bench, SkillsBench. Zero runtime deps, Apache-2.0.

Information-theoretic context optimization proxy. Cuts LLM token costs by 70–95% with zero accuracy loss using greedy submodular knapsack maximization.

FeatherCNN is a high performance inference engine for convolutional neural networks.

Self-hostable AI Agent Workforce Platform. Multi-agent orchestration with remote AI workstations (AgentPods), PTY sandbox + git worktree isolation, built-in Kanban, and per-pod MCP server. Supports Claude Code, Codex CLI, Gemini CLI, Aider, OpenCode.

Open-source ML engineering agent that uses tree search to explore solution spaces. Automates machine learning experimentation from data analysis to model training. Paper .

MIT harness for long-running Claude Code sessions that saves a capsule, runs /clear and resumes instead of compacting.

Local-first CLI and TUI coding agent that runs open-weight models entirely on your machine through a llama.cpp fork. No account or API key required. 56 built-in tools (browser, filesystem, git, memory, vision), MCP support, five-layer local memory, macOS/Linux/Windows.

Deterministic Python orchestrator for 37 CLI coding agents (Claude Code, Codex CLI, Gemini CLI, GitHub Copilot CLI, Cursor, Aider, OpenHands, OpenCode, Goose, Qwen, Ollama, ...) running in parallel git worktrees. First-class MCP server, quality gates, cost tracking with budgets.

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

A platform to programmatically author, schedule and monitor workflows.

An Open-Source Platform for Production Data Science

Workflow engine for Kubernetes.

One-function HITL primitive for AI agents. blocks like a Promise; a human reviews via Slack/email/dashboard; the agent resumes with the typed response. Durable across restarts via Stripe-style idempotency keys. Temporal + LangGraph adapters included.

Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management.

Comet is an MLOps platform that offers Model Production Management, a Model Registry, and full model lineage from training straight through to production. Use Comet for model reproducibility, model debugging, model versioning, model visibility, model auditing, model governance, and model monitoring.

An easy way to turn any app into searchable data for LLMs.

An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search.

AI Native database for embedding vectors

the open source embedding database

A 10x faster, cheaper, and better vector database

Code and documentation to train Stanford's Alpaca models, and generate the data.

Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects.

A 7B Large Language Model fine-tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca.

BigScience Large Open-science Open-access Multilingual Language Model

ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B .

A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.