Apache Helix: Distributed Systems Get Simpler

me@ryanorban.com (Ryan Orban) — Thu, 05 Sep 2013 00:00:00 +0000

Summary

Apache Helix is a cluster management framework that LinkedIn open-sourced after building it internally to power Espresso (their distributed document store), Databus (change data capture), and other distributed infrastructure. The key insight behind Helix is that many distributed systems share the same coordination problems: partition assignment to nodes, failure detection, rebalancing when nodes join or leave, and state transitions (offline → standby → leader). Helix abstracts all of this over Apache Zookeeper, letting engineers define their system as a state machine and delegating the coordination bookkeeping to the framework.

Get Started: Ambari for Provisioning, Managing and Monitoring Hadoop

me@ryanorban.com (Ryan Orban) — Sat, 04 May 2013 00:00:00 +0000

Summary

Apache Ambari was Hortonworks’ answer to Hadoop’s operational complexity problem. In 2012-2013, setting up and managing a Hadoop cluster required deep knowledge of XML configuration files, SSH scripting, and per-service configuration for HDFS, MapReduce, Hive, HBase, Pig, and a dozen other components. Ambari wrapped all of this in a web UI with guided install wizards, centralized configuration management, and service health dashboards.

Cluster-Management on Ryan Orban

Apache Helix: Distributed Systems Get Simpler

Summary

Get Started: Ambari for Provisioning, Managing and Monitoring Hadoop

Summary