awesome-hadoop

Mozilla's utility library for Hadoop, HBase, Pig, etc.

A collection of libraries for working with large-scale data in Hadoop

Apache Pig

Pig workflow visualization tool.

Open Source Big Data Security Analytics

PigPen is map-reduce for Clojure, or distributed Clojure. It compiles to Apache Pig, but you don't need to know much about Pig to use it.

A big data cluster management tool that creates and manages clusters of different technologies.

Apache Ambari

Apache Bigtop: Packaging and tests of the Apache Hadoop ecosystem

ZooKeeper client wrapper and rich ZooKeeper framework

Apache Zookeeper

The Apache Accumulo™ sorted, distributed key/value store is a robust, scalable, high performance data storage and retrieval system.

Apache HBase

A SQL skin over HBase supporting secondary indices

Haeinsa is linearly scalable multi-row, multi-table transaction library for HBase

Hannibal is tool to help monitor and maintain HBase-Clusters that are configured for manual splitting.

Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines

Data management and processing platform

A dataflow system

Apache Oozie

Python package that helps you build complex pipelines of batch jobs

Enterprise-grade unified stream and batch processing engine.

Apache Flink is a platform for efficient, distributed, general-purpose data processing.

Apache Livy (incubating) is web service that exposes a REST interface for managing long running Apache Spark contexts in your cluster. With Livy, new applications can be built on top of Apache Spark that require fine grained interaction with many Spark contexts.

Cascading is the proven application development platform for building data applications on Hadoop.

Metadata tagging & lineage capture suppoting complex business data taxonomies

A Dynamic Data Management Framework

Kudu provides a combination of fast inserts/updates and efficient columnar scans to enable multiple real-time analytic workloads across a single storage layer, complementing HDFS and Apache HBase.

Schema Registry provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving Avro schemas.

Schema Registry is a framework to build metadata repositories.

Apache Avro is a data serialization system.

Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.

Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application

A web-based notebook that enables interactive data analytics

Native go clients for Apache Hadoop YARN.