LISA14: Managing Operations in Hadoop Ecosystem

Taming Operations in the Apache Hadoop Ecosystem

Kathleen Ting and Jonathan Hsieh, Cloudera, Inc.

The Apache Hadoop stack includes many distributed storage and processing systems, running on clusters ranging from tens to thousands of nodes. At Cloudera, we’ve been supporting tens of thousands of nodes in hundreds of our customers’ production clusters with diverse use cases. For five years, we have been navigating paths for sys admins to manage, tune, and debug the systems. We'll describe a methodology for debugging and tuning between the different layers (app, hadoop, jvm, kernel, networking). We’ll also talk about new tools and subsystems included in our operational best practices to keep your clusters always up, running, and secure.

View the full LISA14 program at https://www.usenix.org/conference/lisa14/conference-program