通过CloudFormation部署带VPC CNI插件的EKS 1.24集群时节点NotReady问题求助

我通过CloudFormation创建了一个EKS 1.24集群，没有安装CNI插件时运行正常，但添加vpc-cni插件后就出现问题了，节点状态一直是NotReady。

我的CloudFormation相关配置 #

以下是VPC CNI插件和对应的IAM角色的配置：

AddonCNI:
  Type: 'AWS::EKS::Addon'
  Properties:
    AddonName: vpc-cni
    AddonVersion: v1.12.0-eksbuild.1
    ClusterName: !Ref ControlPlane
    ResolveConflicts: OVERWRITE
    ServiceAccountRoleArn: !GetAtt
      - CNIRole
      - Arn
    Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}/AddonCNI'
  DependsOn:
    - CNIRole

CNIRole:
  Type: 'AWS::IAM::Role'
  Properties:
    AssumeRolePolicyDocument:
      Statement:
        - Action:
            - 'sts:AssumeRole'
          Effect: Allow
          Principal:
            Service:
              - !FindInMap
                - ServicePrincipalPartitionMap
                - !Ref 'AWS::Partition'
                - EKS
      Version: 2012-10-17
    ManagedPolicyArns:
      - !Sub 'arn:${AWS::Partition}:iam::aws:policy/AmazonEKS_CNI_Policy'
      - !Sub 'arn:${AWS::Partition}:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly'
    Tags:
      - Key: Name
        Value: !Sub '${AWS::StackName}/CNIRole'

我也给节点角色添加了AmazonEKS_CNI_Policy策略。

节点状态及日志信息 #

节点状态显示：

container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized

aws-node容器的日志：

{"level":"info","ts":"2023-01-06T16:24:55.411Z","caller":"entrypoint.sh","msg":"Validating env variables ..."}
{"level":"info","ts":"2023-01-06T16:24:55.412Z","caller":"entrypoint.sh","msg":"Install CNI binaries.."}
{"level":"info","ts":"2023-01-06T16:24:55.424Z","caller":"entrypoint.sh","msg":"Starting IPAM daemon in the background ... "}
{"level":"info","ts":"2023-01-06T16:24:55.425Z","caller":"entrypoint.sh","msg":"Checking for IPAM connectivity ... "}
{"level":"info","ts":"2023-01-06T16:24:56.430Z","caller":"entrypoint.sh","msg":"Retrying waiting for IPAM-D"}
{"level":"info","ts":"2023-01-06T16:24:57.435Z","caller":"entrypoint.sh","msg":"Retrying waiting for IPAM-D"}

从节点容器内的/host/var/log/aws-routed-eni/ipamd.log提取的部分日志：

{"level":"info","ts":"2023-01-06T13:19:47.759Z","caller":"logger/logger.go:52","msg":"Constructed new logger instance"}
{"level":"info","ts":"2023-01-06T13:19:47.759Z","caller":"eniconfig/eniconfig.go:61","msg":"Initialized new logger as an existing instance was not found"}
{"level":"info","ts":"2023-01-06T13:19:48.012Z","caller":"aws-k8s-agent/main.go:28","msg":"Starting L-IPAMD   ..."}
{"level":"info","ts":"2023-01-06T13:19:48.020Z","caller":"aws-k8s-agent/main.go:39","msg":"Testing communication with server"}
{"level":"info","ts":"2023-01-06T13:19:48.063Z","caller":"wait/wait.go:211","msg":"Successful communication with the Cluster! Cluster Version is: v1.24+. git version: v1.24.8-eks-ffeb93d. git tree state: clean. commit: abb98ec0631dfe573ec5eae40dc48fd8f2017424. platform: linux/amd64"}
{"level":"warn","ts":"2023-01-06T13:19:48.083Z","caller":"awssession/session.go:64","msg":"HTTP_TIMEOUT env is not set or set to less than 10 seconds, defaulting to httpTimeout to 10sec"}
{"level":"debug","ts":"2023-01-06T13:19:48.085Z","caller":"ipamd/ipamd.go:379","msg":"Discovered region: us-east-1"}
{"level":"info","ts":"2023-01-06T13:19:48.085Z","caller":"ipamd/ipamd.go:379","msg":"Custom networking enabled false"}
{"level":"debug","ts":"2023-01-06T13:19:48.085Z","caller":"awsutils/awsutils.go:415","msg":"Found availability zone: us-east-1c "}
{"level":"debug","ts":"2023-01-06T13:19:48.086Z","caller":"awsutils/awsutils.go:415","msg":"Discovered the instance primary IPv4 address: 10.0.66.216"}
{"level":"debug","ts":"2023-01-06T13:19:48.086Z","caller":"awsutils/awsutils.go:415","msg":"Found instance-id: i-06b7496334df06d96 "}
{"level":"debug","ts":"2023-01-06T13:19:48.087Z","caller":"awsutils/awsutils.go:415","msg":"Found instance-type: c5.xlarge "}
{"level":"debug","ts":"2023-01-06T13:19:48.088Z","caller":"awsutils/awsutils.go:415","msg":"Found primary interface's MAC address: 0a:3f:bd:93:2a:8d"}
{"level":"debug","ts":"2023-01-06T13:19:48.088Z","caller":"awsutils/awsutils.go:415","msg":"eni-05097e0aa87b119d5 is the primary ENI of this instance"}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"awsutils/awsutils.go:415","msg":"Found subnet-id: subnet-0e9870d3f07c0c322 "}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:388","msg":"Using WARM_ENI_TARGET 1"}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:391","msg":"Using WARM_PREFIX_TARGET 1"}
{"level":"info","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:409","msg":"Prefix Delegation enabled false"}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:414","msg":"Start node init"}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:446","msg":"Max ip per ENI 14 and max prefixes per ENI 0"}
{"level":"info","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:456","msg":"Setting up host network... "}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"networkutils/network.go:280","msg":"Trying to find primary interface that has mac : 0a:3f:bd:93:2a:8d"}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"networkutils/network.go:280","msg":"Discovered interface: lo, mac: "}
{"level":"debug","ts":"2023-01-06T13:19:48.089Z","caller":"networkutils/network.go:280","msg":"Discovered interface: eth0, mac: 0a:3f:bd:93:2a:8d"}
{"level":"info","ts":"2023-01-06T13:19:48.089Z","caller":"networkutils/network.go:280","msg":"Discovered primary interface: eth0"}
{"level":"info","ts":"2023-01-06T13:19:48.089Z","caller":"ipamd/ipamd.go:456","msg":"Skip updating RPF for primary interface: net/ipv4/conf/eth0/rp_filter"}
{"level":"info","ts":"2023-01-06T13:19:48.090Z","caller":"awsutils/awsutils.go:1643","msg":"Will attempt to clean up AWS CNI leaked ENIs after waiting 4m41s."}
{"level":"debug","ts":"2023-01-06T13:19:48.090Z","caller":"networkutils/network.go:307","msg":"Found the Link that uses mac address 0a:3f:bd:93:2a:8d and its index is 2 (attempt 1/5)"}
{"level":"debug","ts":"2023-01-06T13:19:48.090Z","caller":"networkutils/network.go:383","msg":"Trying to find primary interface that has mac : 0a:3f:bd:93:2a:8d"}
{"level":"debug","ts":"2023-01-06T13:19:48.090Z","caller":"networkutils/network.go:383","msg":"Discovered interface: lo, mac: "}
{"level":"debug","ts":"2023-01-06T13:19:48.090Z","caller":"networkutils/network.go:383","msg":"Discovered interface: eth0, mac: 0a:3f:bd:93:2a:8d"}
{"level":"info","ts":"2023-01-06T13:19:48.090Z","caller":"networkutils/network.go:383","msg":"Discovered primary interface: eth0"}
{"level":"debug","ts":"2023-01-06T13:19:48.187Z","caller":"networkutils/network.go:403","msg":"Adding 10.0.0.0/16 CIDR to NAT chain"}
{"level":"debug","ts":"2023-01-06T13:19:48.187Z","caller":"networkutils/network.go:403","msg":"Total CIDRs to program - 1"}
{"level":"debug","ts":"2023-01-06T13:19:48.187Z","caller":"networkutils/network.go:403","msg":"Setup Host Network: iptables -N AWS-SNAT-CHAIN-0 -t nat"}
{"level":"debug","ts":"2023-01-06T13:19:48.189Z","caller":"networkutils/network.go:403","msg":"Setup Host Network: iptables -N AWS-SNAT-CHAIN-1 -t nat"}
{"level":"debug","ts":"2023-01-06T13:19:48.190Z","caller":"networkutils/network.go:403","msg":"Setup Host Network: iptables -A POSTROUTING -m comment --comment \"AWS SNAT CHAIN\" -j AWS-SNAT-CHAIN-0"}
{"level":"debug","ts":"2023-01-06T13:19:48.190Z","caller":"networkutils/network.go:403","msg":"Setup Host Network: iptables -A AWS-SNAT-CHAIN-0 ! -d {10.0.0.0/16 %!s(bool=false)} -t nat -j AWS-SNAT-CHAIN-1"}
{"level":"debug","ts":"2023-01-06T13:19:48.190Z","caller":"networkutils/network.go:714","msg":"Setup Host Network: loading existing iptables nat rules with chain prefix AWS-SNAT-CHAIN"}
{"level":"debug","ts":"2023-01-06T13:19:48.237Z","caller":"networkutils/network.go:714","msg":"host network setup: found potentially stale SNAT rule for chain AWS-SNAT-CHAIN-0: [-N AWS-SNAT-CHAIN-0]"}
{"level":"debug","ts":"2023-01-06T13:19:48.238Z","caller":"networkutils/network.go:714","msg":"host network setup: found potentially stale SNAT rule for chain AWS-SNAT-CHAIN-1: [-N AWS-SNAT-CHAIN-1]"}
{"level":"debug","ts":"2023-01-06T13:19:48.238Z","caller":"networkutils/network.go:509","msg":"Setup Host Network: computing stale iptables rules for %s table with chain prefix %s"}
{"level":"debug","ts":"2023-01-06T13:19:48.238Z","caller":"networkutils/network.go:509","msg":"Setup Host Network: active chain found: AWS-SNAT-CHAIN-0"}
{"level":"debug","ts":"2023-01-06T13:19:48.238Z","caller":"networkutils/network.go:509","msg":"Setup Host Network: active chain found: AWS-SNAT-CHAIN-1"}
{"level":"debug","ts":"2023-01-06T13:19:48.238Z","caller":"networkutils/network.go:403","msg":"iptableRules: [nat/POSTROUTING rule first SNAT rules for non-VPC outbound traffic shouldExist true rule [-m comment --comment AWS SNAT CHAIN -j AWS-SNAT-CHAIN-0] nat/AWS-SNAT-CHAIN-0 rule [0] AWS-SNAT-CHAIN shouldExist true rule [! -d 10.0.0.0/16 -m comment --comment AWS SNAT CHAIN -j AWS-SNAT-CHAIN-1] nat/AWS-SNAT-CHAIN-1 rule last SNAT rule for non-VPC outbound traffic shouldExist true rule [! -o vlan+ -m comment --comment AWS, SNAT -m addrtype ! --dst-type LOCAL -j SNAT --to-source 10.0.66.216 --random-fully] mangle/PREROUTING rule connmark for primary ENI shouldExist true rule [-m comment --comment AWS, primary ENI -i eth0 -m addrtype --dst-type LOCAL --limit-iface-in -j CONNMARK --set-mark 0x