如何通过CloudFormation为EMR集群添加EC2密钥对以支持SSH登录

修改方法 #

你只需要对现有模板做两处调整即可实现SSH密钥对的配置，对应CLI命令里的--ec2-attributes KeyName=MyKey参数效果：

1. 新增密钥对参数 #

在Parameters代码块中新增EC2密钥对的参数配置：

Parameters:
  # 原有其他参数保持不变
  Ec2KeyName:
    Type: "AWS::EC2::KeyPair::KeyName"
    Description: 用于SSH登录EMR集群节点的EC2密钥对名称
    Default: "MyKey" # 可替换为你常用的密钥对默认名称

2. 关联密钥对到EMR集群配置 #

找到EmrCluster资源下Properties.Instances配置块，新增Ec2KeyName字段引用上面的参数：

Resources:
  EmrCluster:
    Type: AWS::EMR::Cluster
    Properties:
      # 原有其他属性保持不变
      Instances:
        TerminationProtected: false
        Ec2SubnetId: !Ref SubnetId
        Ec2KeyName: !Ref Ec2KeyName # 新增的密钥对关联配置
        MasterInstanceGroup:
          # 原有主节点配置保持不变
        CoreInstanceGroup:
          # 原有核心节点配置保持不变

修改后完整模板 #

AWSTemplateFormatVersion: 2010-09-09

Parameters:
  SubnetId:
    Type: "String"
    Default: "subnet-0caca76de8cc43e70"
  ClusterName:
    Type: "String"
    Default: "Example_Three_Node_Cluster"
  EmrRelease:
    Type: "String"
    Default: "emr-6.2.0"
    AllowedValues:
    - "emr-6.2.0"
    - "emr-5.32.0"
  ClusterInstanceType:
    Type: "String"
    Default: "m5.xlarge"
    AllowedValues:
    - "m5.xlarge"
    - "m5.2xlarge"
  Ec2KeyName:
    Type: "AWS::EC2::KeyPair::KeyName"
    Description: 用于SSH登录EMR集群节点的EC2密钥对名称
    Default: "MyKey"

Resources:
  EmrCluster:
    Type: AWS::EMR::Cluster
    Properties:
      Applications:
      - Name: Spark
      - Name: Livy
      - Name: JupyterEnterpriseGateway
      - Name: Hive
      EbsRootVolumeSize: '10'
      Name: !Ref ClusterName
      JobFlowRole: EMR_EC2_DefaultRole
      ServiceRole: EMR_DefaultRole
      ReleaseLabel: !Ref EmrRelease
      VisibleToAllUsers: true
      LogUri: 
        Fn::Sub: 's3://aws-logs-${AWS::AccountId}-${AWS::Region}/elasticmapreduce/'
      Instances:
        TerminationProtected: false
        Ec2SubnetId: !Ref SubnetId
        Ec2KeyName: !Ref Ec2KeyName
        MasterInstanceGroup:
          InstanceCount: 1
          InstanceType: !Ref ClusterInstanceType
        CoreInstanceGroup:
          InstanceCount: 2
          InstanceType: !Ref ClusterInstanceType
          Market: ON_DEMAND
          Name: Core

Outputs:
  ClusterId:
    Value:
      Ref: EmrCluster
    Description: The ID of the EMR Cluster

注意事项 #

• 你需要提前在对应AWS区域创建好EC2密钥对，部署模板时可以直接在参数下拉列表中选择已有的密钥对
• 集群启动成功后，使用密钥对对应的私钥文件，以hadoop用户身份登录主节点即可，命令示例：ssh -i /path/to/your/key.pem hadoop@<EMR主节点公网IP>
• 请确保集群关联的安全组已经放开22端口的入站规则，否则SSH连接会被拒绝

内容的提问来源于stack exchange，提问作者bruvio