data-engineering-collection

The Google File System.

MapReduce: Simplied Data Processing on Large Clusters.

Bigtable: A Distributed Storage System for Structured Data.

The Chubby lock service for loosely-coupled distributed systems.

Dynamo: Amazon’s Highly Available Key-value Store.

Chukwa: A large-scale monitoring system.

Platform for Internet of things.

Pub/sub messaging platform for IoT

a programming model and micro-kernel style runtime that can be embedded in gateways and small footprint edge devices enabling local, real-time, analytics on the edge devices.

Cloud-based bi-directional monitoring and messaging hub

an web application for alert management resulting from scheduled searches into Elasticsearch.

Next-generation web analytics processing with Scala, Spark, and Parquet.

a platform that integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis.

open source web crawler.

capturing, processing and sharing of data for NASA's scientific archives.

content analysis toolkit.

ACID-compliant DBMS developed by Pervasive Software, optimized for embedding in applications.

a software library that provides a high-performance embedded database for key/value data.

high performance interactive SQL access to all Hadoop data.

framework that allows efficient translation of queries involving heterogeneous and federated data.

framework for interactive analysis, inspired by Dremel.

table and storage management layer for Hadoop.

SQL-like data warehouse system for Hadoop.

SQL skin over HBase.

column-oriented analytic database.

commercial object-oriented database management systems .

a distributed SQL database with the scalability of a KV store, while keeping the query capabilities of a relational database.

distributed data processing and storage system originally developed at AddThis.

reliable file sharing at memory speed across cluster frameworks.

Amazon's cloud offering, also based on a columnar datastore backend.

NoSQL flash-optimized, in-memory. Open source and "Server code in 'C' (not Java or Erlang) precisely tuned to avoid context switching and memory copies."

an embedded key-value database for Go.

Key Value Database in .Net with Object DB Layer, RPC, dynamic IL and much more

a fast, embeddable, in-memory key/value database for Go with custom indexing and geospatial support.

a new generation multi-model graph database for the modern complex data environment.

implementation of Pregel, based on Hadoop.

implementation of Pregel, part of Spark.