Hadoop Ecosystem vs. Cloudera: Big Data Comparison

Matthew Weier O’Phinney (OpenLogic and Zend Senior Product Manager) and Tim Carroll (Perforce Software Director of Product Development) discuss options for big data management, comparing the open source Hadoop ecosystem vs. Cloudera in this webinar. They take an in-depth look at Hadoop big data management and explore how OSS can improve big data infrastructure.

This video is from the OpenLogic webinar, “Is It Time to Open Source Your Big Data Management?” which aired on October 24, 2024.

To watch the full webinar, click here: https://ter.li/gznd4n
Learn more about the Hadoop Service Bundle from OpenLogic: https://ter.li/0ymeyj

About OpenLogic by Perforce:
OpenLogic offers end-to-end enterprise support for organizations using open source software in their infrastructure. With support for over 400 open source packages, guaranteed SLAs, and direct access to highly experienced Enterprise Architects, OpenLogic customers benefit from 24x7 ticket-based technical support, professional services, and training.

Follow OpenLogic on LinkedIn, Twitter, and don’t forget to subscribe to our YouTube channel for more videos on all things open source!

Transcript (Lightly Edited for Clarity)

Matthew: Hadoop has actually been around for nearly two decades, and the ecosystem matured through the adoption and contributions of many Fortune five hundred companies at this point. If it's all built on Hadoop, if your entire solution is built on that, what is the value add from your managed service provider? Tim, why don't you walk us through this?

Tim: To give a little background, Cloudera and then later Hortonworks, which was acquired by Cloudera, are well-established vendors in this space. They've been around since the early days of Hadoop, and they were a catalyst in the widespread adoption of Hadoop in the big data space. But now, as you said, Hadoop's been around for nearly two decades.
The ecosystem's grown. It's matured, through the adoption and contributions of many Fortune five hundred companies and beyond. When you look closely at the Cloudera platform, you see a lot of open source. At the core, you see Apache Hadoop, which immediately creates a one to one mapping for, open source alternatives for data storage analytics, with things like, HDFS, MapReduce, and Yarn.
Really, the key differences that you start to see is in the proprietary pieces of Cloudera, Cloudera Manager primarily, which is used for cluster provisioning, administration, management. You also have Apache Ambari, which is an open source alternative that provides those same functions and has a really strong and healthy community behind it.
Likewise, there's Cloud Air Navigator, which is another proprietary licensed piece of software, that handles data governance and security features, on the Cloud Air platform—and those things can be accomplished through open source solutions. In fact, there, there are some foundations already with Apache Atlas and Apache Century.
As you go further down that rabbit hole, you've got new things that have come out with Apache Ranger that really has a far superior interface to other alternatives for data governance and security with a lot of the auditing capabilities that it provides.
As you look further, all the other aspects of the broader Cloudera solution utilize open source tools. You got, Apache Hive or Hue that provide, SQL interfaces, Apache HBase that provides NoSQL option, Apache Oozie for job scheduling and workflow, and of course Apache Spark for parallel analytics data processing.
Panning out further to that next ring of the concentric circle, whether you're using a vended Hadoop solution like Cloudera or a homespun stack, it's all open source. As you add things like graph database, you'll see JanusGraph allows you to manage data about entities and the relationships among them. You get streaming input from things like Apache Kafka and Apache Spark. You take your job scheduling and execution to a whole another level with the Apache Airflow, and the list goes on.
I think, the key takeaways here is that the old vended solutions really no longer provide a distinguishable software based advantage. The open source equivalents are strong and mature. The open source model puts you more squarely in charge of your environment, where you can adopt additions and improvements on your timeline.