Upcoming Batches: www.edureka.co/big-data-and-hadoop
The question is everyone's mind these days is What is Hadoop?
This 1 hour session introduces participants to Hadoop Architecture. We understand the Hadoop Eco-System and opportunities for professionals in Hadoop.
We then look at the challenges in Hadoop 1.x Architecture and the need for introducing Hadoop 2.0.
Hadoop 2.0 features such as HDFS Federation, Namenode High Availability, YARN, Multi-Tenacy are introduced.
The viewer will be able to get a very good overview of Hadoop Architecture and Hadoop 2.0 architecture advancements.
Agenda:
- Introduction to Big Data
- What is Hadoop?
- What is Hadoop for
Hadoop opportunities for Java Professionals
Hadoop opportunities for ETL and Data Warehousing Professionals
Hadoop opportunities for Administrators and DBAs
Hadoop Testing Opportunities
- Hadoop 1 Architecture
- Challenges in Hadoop 1 architecture
- NameNode being a single point of failure
- Horizontal Scalability issues on scaling beyond 4000 nodes in hadoop 1.0
- Jobtracker becoming overburned since it has to both act as scheduler and has to monitor the jobs running on Task Tracker
- No support for Non-Mapreduce jobs
- No Support for Multi-Tenacy - the ability to run other than map reduce workloads in parallel
- What is Hadoop 2.0 Acrhitecture
- How does NameNode Federation help in scaling beyond 4000 data nodes
- How does the stand by Namenode work and how does failover mechanism work in case of NameNode failure
- What is YARN.
- How does YARN resource management help in reducing the burden on Job Tracker by introducing the components such as Resource Manager, Scheduler, Application Master, Node Manager, Container
- We also talk about how do non-Mapreduce jobs such as Storm (Real Time Big Data Analytics), Spark, Interactive, Online work with YARN
- We also discuss the capacity scheduler and how it helps hadoop acheive multi-tenacy using the concept of dedicated queues for different work loads.
We end with a quiz to reinforce the understanding of participants and finally take up questions from participants.
Happy Learning
Team Edureka