手动安装Hadoop-2.8.0默认含哪些生态组件？是否需单独装ZooKeeper？
Hey there! Great call going with a manual Hadoop 2.8.0 setup instead of CDH given your 8GB RAM—CDH's VM footprint is definitely too heavy for that. Let's break down your questions clearly:
Default Included Components in Hadoop 2.8.0
When you download and unpack the standard hadoop-2.8.0.tar.gz from Apache, it comes with the core Hadoop stack out of the box:
- Hadoop Common: Foundational utilities, libraries, and shared tools that power all other Hadoop modules (like config parsers, filesystem abstractions, and security tools).
- HDFS (Hadoop Distributed File System): The distributed storage layer, including core services like NameNode, DataNode, and SecondaryNameNode.
- YARN (Yet Another Resource Negotiator): The cluster resource management layer, with components like ResourceManager, NodeManager, and ApplicationMaster.
- MapReduce v2: The classic batch processing framework, now built to run on top of YARN for better resource utilization.
All these components are pre-packaged and can be configured to run in standalone, pseudo-distributed, or fully distributed modes directly from the tarball.
Do You Need to Install ZooKeeper Separately?
Short answer: It depends on which parts of the Hadoop ecosystem you want to explore.
- If you're only working with the core Hadoop components (HDFS, YARN, MapReduce) in a non-high-availability (non-HA) setup, you don't need ZooKeeper at all. The basic setup will work perfectly fine without it.
- But if you want to dive into the full ecosystem or enable more robust cluster features, you'll need to install ZooKeeper separately. Specifically, it's required for:
- HDFS/YARN High Availability: To manage quorum and automate failover for critical services like NameNode or ResourceManager (eliminating single points of failure).
- Ecosystem Tools: Most extended Hadoop tools rely on ZooKeeper for distributed coordination, leader election, or state management—think HBase, Hive (for distributed metastore), Kafka, Storm, or Oozie.
Since you mentioned wanting to experience the full Hadoop ecosystem, you’ll almost certainly need to add ZooKeeper to your setup. It’s a foundational tool for most of the ecosystem’s more powerful components.
内容的提问来源于stack exchange，提问作者LearneriOS
