Ambari安装Bigtop编译组件报错：hadoop-hdfs-dfsrouter不被stack-select支持

环境信息 #

• 操作系统：CentOS-7
• 编译使用分支：
  • ambari trunk
  • bigtop branch-3.3
  • ambari-metrics 3.0

报错现象 #

使用Ambari安装Bigtop编译生成的组件时失败，核心错误为：

Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool

详细错误日志 #

stderr：

2024-12-30 13:20:31,798 - 上报组件版本失败
Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 428, in execute
    self.save_component_version_to_structured_out(self.command_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 263, in save_component_version_to_structured_out
    stack_select_package_name = stack_select.get_package_name()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 126, in get_package_name
    package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 265, in get_packages
    f"The package {package} is not supported by this version of the stack-select tool."
resource_management.core.exceptions.**Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool.**
2024-12-30 13:20:36,321 - 上报组件版本失败
Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 428, in execute
    self.save_component_version_to_structured_out(self.command_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 263, in save_component_version_to_structured_out
    stack_select_package_name = stack_select.get_package_name()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 126, in get_package_name
    package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 265, in get_packages
    f"The package {package} is not supported by this version of the stack-select tool."
resource_management.core.exceptions.**Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool.
NoneType: None**

The above exception was the cause of the following exception:

2024-12-30 13:20:36,769 - 上报组件版本失败
Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 413, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 32, in hook
    setup_stack_symlinks(self.stroutfile)
  File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", line 62, in setup_stack_symlinks
    stack_packages = stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 265, in get_packages
    f"The package {package} is not supported by this version of the stack-select tool."
resource_management.core.exceptions.Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 428, in execute
    self.save_component_version_to_structured_out(self.command_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 263, in save_component_version_to_structured_out
    stack_select_package_name = stack_select.get_package_name()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 126, in get_package_name
    package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 265, in get_packages
    f"The package {package} is not supported by this version of the stack-select tool."
resource_management.core.exceptions.Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool.
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 39, in 
    AfterInstallHook().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 413, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 32, in hook
    setup_stack_symlinks(self.stroutfile)
  File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", line 62, in setup_stack_symlinks
    stack_packages = stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 265, in get_packages
    f"The package {package} is not supported by this version of the stack-select tool."
resource_management.core.exceptions.**Fail: The package hadoop-hdfs-dfsrouter is not supported by this version of the stack-select tool.**

stdout：

2024-12-30 13:20:30,932 - Stack Feature Version Info: Cluster Stack=3.3.0, Command Stack=None, Command Version=None -> 3.3.0
2024-12-30 13:20:30,933 - 使用hadoop配置目录: /etc/hadoop/conf

疑问 #

当前是否可以使用新版本？

问题核心是Ambari trunk版本的stack-select工具未适配Bigtop 3.3新增的hadoop-hdfs-dfsrouter包：Bigtop 3.3新增了DFS Router组件对应的包，但Ambari的stack-select规则未添加该包的支持，导致安装时校验失败。

针对该问题，有两种处理方式：

1. 适配Ambari的stack-select配置
   • 找到Ambari中stack-select的包映射配置文件（通常在/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py或对应stack定义目录）
   • 在get_packages函数的包映射规则里添加hadoop-hdfs-dfsrouter条目，将其关联到HDFS服务的对应组件
   • 重启ambari-agent后重新执行安装
2. 调整Bigtop编译选项
   • 若不需要DFS Router组件，可在Bigtop编译时禁用该组件，避免生成hadoop-hdfs-dfsrouter包
   • 修改Bigtop编译配置文件，跳过DFS Router相关模块后重新编译生成安装包

关于是否可以使用新版本：Bigtop 3.3本身可用，解决Ambari与它的适配问题后即可正常使用。

内容的提问来源于stack exchange，提问作者Guang Dong