awesome-distributed-system-projects
github.com/roma-glushko/awesome-distributed-system-projects ↗🚀 List of distributed system projects for inspiration and learning to build distributed services from real world examples
Use this list with your AI agent
Add the Context Awesome MCP server to Claude, Cursor, or any MCP client, then ask:
"Show me column databases resources from awesome-distributed-system-projects"
Installation instructions →What's inside
Queues
- Apache ActiveMQ - high performance Apache 2.0 licensed Message Broker
high performance Apache 2.0 licensed Message Broker
- Apache Pulsar - distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API
distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API
- dnpipes - distributed version of Unix named pipes comparable to AWS SQS
distributed version of Unix named pipes comparable to AWS SQS
- GatewayWorker - distributed realtime messaging framework based on workerman
distributed realtime messaging framework based on workerman
- nsq - realtime fault tolerant distributed messaging platform designed to operate at scale, handling billions of messages per day [Raft + gRPC]
realtime fault tolerant distributed messaging platform designed to operate at scale, handling billions of messages per day [Raft + gRPC]
- Sandglass - distributed, horizontally scalable, persistent, time ordered message queue
distributed, horizontally scalable, persistent, time ordered message queue
Schedulers
- Apache Airflow - a platform to programmatically author, schedule, and monitor workflows
a platform to programmatically author, schedule, and monitor workflows
- Celery - a distributed task queue
a distributed task queue
- dkron - a distributed, fault tolerant job scheduling system for cloud native environments
a distributed, fault tolerant job scheduling system for cloud native environments
Databases
- Apache Cassandra - a highly-scalable partitioned row store. Rows are organized into tables with a required primary keyColumn Databases
a highly-scalable partitioned row store. Rows are organized into tables with a required primary key
- Apache Doris - an easy-to-use, high performance and unified analytics databaseTime Series
an easy-to-use, high performance and unified analytics database
- Apache HBase - a Hadoop database, a distributed, scalable, big data store
a Hadoop database, a distributed, scalable, big data store
- AresDB - a GPU-powered real-time analytics storage and query engineAnalytical Databases
a GPU-powered real-time analytics storage and query engine
- BaikalDB is a distributed HTAP MySQL-compatible database designed for petabytes scaleAnalytical Databases
- BonsaiDB - an ACID, transactional KV or document dev-friendly database with configurable delayed on-disk data storingNoSQL, Document Databases
an ACID, transactional KV or document dev-friendly database with configurable delayed on-disk data storing
Search Engines
- Apache Lucene - a high-performance, full featured text search engine library
a high-performance, full featured text search engine library
- ElasticSearch - distributed, RESTful search and analytics engine
distributed, RESTful search and analytics engine
- FlexSearch - memory-flexible full-text search library
memory-flexible full-text search library
- MeiliSearch - Lightning Fast, Ultra Relevant, and Typo-Tolerant Search Engine
Lightning Fast, Ultra Relevant, and Typo-Tolerant Search Engine
- RiotSearch - distributed, Simple and efficient full text search engine
distributed, Simple and efficient full text search engine
- Sonic - fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM
fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM
Data Processing
- Apache Spark - unified analytics engine for large-scale data processing
unified analytics engine for large-scale data processing
Locking
- Apache Zookeeper - highly reliable distributed coordination
highly reliable distributed coordination
- chubby - A (very simplified) implementation of Chubby, Google's distributed lock service
A (very simplified) implementation of Chubby, Google's distributed lock service
- etcd - distributed reliable key-value store for the most critical data of a distributed system [Raft + gRPC]
distributed reliable key-value store for the most critical data of a distributed system [Raft + gRPC]
Frameworks
- atomix - fully featured framework for building fault-tolerant distributed systems [REST + Raft]
fully featured framework for building fault-tolerant distributed systems [REST + Raft]
- cotyledon - a framework for defining long-running services
a framework for defining long-running services
- Dapr - portable, serverless, event-driven runtime that works as a sidecar and makes it easy for developers to build resilient, stateless and stateful microservices
portable, serverless, event-driven runtime that works as a sidecar and makes it easy for developers to build resilient, stateless and stateful microservices
- ergo - port of Erlang/OTP approaches in Golang
port of Erlang/OTP approaches in Golang
- etcd - framework for distributed systems development
framework for distributed systems development
- gosiris - an actor framework for Golang
an actor framework for Golang
Other Resources
- awesome-database-learning - a list of learning materials to understand databases internals
a list of learning materials to understand databases internals
- awesome-distributed-systems - curated list on awesome material on distributed systems
curated list on awesome material on distributed systems
- awesome-scalability - Reading list for illustrating the patterns of scalable, reliable, and performant large-scale systems
Reading list for illustrating the patterns of scalable, reliable, and performant large-scale systems
- Berkley CS186: Intro into Database Systems
- Build Your Own Redis with C/C++
- MIT 6.830: Database Systems
Showing a sample of 138 resources. View the full list on GitHub →