Yocto+ATECC608B-TNGTLS HSM搭建Greengrass Core遇PKCS11 MQTT连接错误

我正尝试在基于Yocto的设备上，搭配ATECC608B-TNGTLS HSM搭建AWS Greengrass Core，HSM已通过p11tool验证正常，设备及证书已通过Microchip信任平台自动创建，但启动Greengrass时出现Mqtt connection provider for pkcs11 is unavailable错误，尝试修改PIN、插槽号、运行用户等均无效。

已验证的HSM信息 #

root@blox-rema:~# p11tool --provider=/usr/lib/libcryptoauth.so --list-tokens
Token 0:
URL: pkcs11:model=ATECC608B;manufacturer=Microchip%20Technology%20Inc;serial=ABCDEFGH;token=MCHP
Label: MCHP
Type: Hardware token
Flags: RNG, uPIN uninitialized
Manufacturer: Microchip Technology Inc
Model: ATECC608B
Serial: ABCDEFGH
Module:

root@blox-rema:~# p11tool --provider=/usr/lib/libcryptoauth.so --list-all
Object 0:
URL: pkcs11:model=ATECC608B;manufacturer=Microchip%20Technology%20Inc;serial=ABCDEFGH;token=MCHP;id=ABCDEFGH;object=device;type=private
Type: Private key (EC/ECDSA-SECP256R1)
Label: device
Flags: CKA_PRIVATE; CKA_NEVER_EXTRACTABLE; CKA_SENSITIVE;
ID: ABCDEFGH

Object 1:
URL: pkcs11:model=ATECC608B;manufacturer=Microchip%20Technology%20Inc;serial=ABCDEFGH;token=MCHP;id=ABCDEFGH;object=device;type=public
Type: Public key (EC/ECDSA-SECP256R1)
Label: device
ID: ABCDEFGH

Greengrass Core配置文件 #

---
system:
certificateFilePath: "pkcs11:object=device;type=cert"
privateKeyPath: "pkcs11:object=device;type=private"
rootCaPath: "/greengrass/v2/config/AmazonRootCA1.pem"
rootpath: "/greengrass/v2"
thingName: "ABCDEFGH"
services:
aws.greengrass.crypto.Pkcs11Provider:
configuration:
library: "/usr/lib/libcryptoauth.so"
name: "lybcryptauth_pkcs11"
slot: 0
userPin: 1234
dependencies: []
version: "0.0.0"
aws.greengrass.Nucleus:
componentType: "NUCLEUS"
configuration:
awsRegion: "eu-west-1"
componentStoreMaxSizeBytes: "10000000000"
deploymentPollingFrequencySeconds: "15"
envStage: "prod"
fipsMode: "false"
fleetStatus:
periodicStatusPublishIntervalSeconds: 86400
greengrassDataPlaneEndpoint: ""
greengrassDataPlanePort: "8443"
httpClient: {}
iotCredEndpoint: "ABCDEFGH"
iotDataEndpoint: "ABCDEFGH"
iotRoleAlias: "ABCDEFGH"
jvmOptions: "-Dlog.store=FILE"
logging: {}
mqtt:
spooler: {}
networkProxy:
proxy: {}
platformOverride: {}
runWithDefault:
posixShell: "sh"
posixUser: "ggc_user:ggc_group"
s3EndpointType: "GLOBAL"
telemetry: {}
dependencies: []
lifecycle:
bootstrap:
requiresPrivilege: "true"
script: "\nset -eu\nKERNEL_ROOT="/greengrass/v2"\nUNPACK_DIR="/greengrass/v2/packages/artifacts-unarchived/aws.greengrass.Nucleus/2.13.0/aws.greengrass.nucleus"\nrm -r "$KERNEL_ROOT"/alts/current/*\necho "-Dlog.store=FILE" > "$KERNEL_ROOT/alts/current/launch.params"\nln -sf "$UNPACK_DIR" "$KERNEL_ROOT/alts/current/distro"\nexit 100"
version: "2.12.6"
DeploymentService:
ComponentToGroups: {}
dependencies: []
GroupToRootComponents: {}
version: "0.0.0"
FleetStatusService:
dependencies: []
lastPeriodicUpdateTime: 1731961100156
sequenceNumber: 31
main:
dependencies:
- "aws.greengrass.Nucleus"
lifecycle: {}
TelemetryAgent:
dependencies: []
runtime:
lastPeriodicAggregationMetricsTime: 1730226064766
lastPeriodicPublishMetricsTime: 1731959838274
version: "0.0.0"
UpdateSystemPolicyService:
dependencies: []
version: "0.0.0"

Systemd启动配置 #

[Unit]
Description=Greengrass Core
After=network.target

[Service]
Type=simple
PIDFile=/greengrass/v2/alts/loader.pid
RemainAfterExit=no
Restart=on-failure
RestartSec=10
ExecStart=/bin/sh /greengrass/v2/alts/current/distro/bin/loader
KillMode=mixed

[Install]
WantedBy=multi-user.target

错误日志 #

2024-11-19T16:07:21.668Z [INFO] (main) com.aws.greengrass.util.platforms.Platform: Getting platform instance com.aws.greengrass.util.platforms.unix.linux.LinuxPlatform.. {}
2024-11-19T16:07:21.696Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: No ongoing deployment detected. Proceed as default. {}
2024-11-19T16:07:21.814Z [INFO] (main) com.aws.greengrass.config.Configuration: config-loading. Read configuration from a file path. {path=/greengrass/v2/config/config.tlog}
2024-11-19T16:07:21.896Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: effective-config-dump-complete. {file=/greengrass/v2/config/effectiveConfig.yaml}
2024-11-19T16:07:22.071Z [INFO] (main) com.aws.greengrass.lifecyclemanager.KernelLifecycle: system-start. Launch Nucleus. {configPath=/greengrass/v2/config, rootPath=/greengrass/v2, version=2.12.6}
2024-11-19T16:07:22.288Z [INFO] (main) software.amazon.awssdk.eventstreamrpc.RpcServer: IpcServer started.... {}
2024-11-19T16:07:22.356Z [INFO] (main) com.aws.greengrass.lifecyclemanager.KernelLifecycle: Found Plugin: PKCS11CryptoKeyService. {}
2024-11-19T16:07:22.357Z [INFO] (main) com.aws.greengrass.lifecyclemanager.KernelLifecycle: serviceImplementors: {UpdateSystemPolicyService:class com.aws.greengrass.lifecyclemanager.UpdateSystemPolicyService...}. {}
2024-11-19T16:07:22.369Z [INFO] (main) com.aws.greengrass.security.SecurityService: Register crypto key service provider. {keyType=file}
2024-11-19T16:07:22.370Z [INFO] (main) com.aws.greengrass.security.SecurityService: Register MQTT connection security provider. {keyType=file}
2024-11-19T16:07:22.377Z [INFO] (main) com.aws.greengrass.mqttclient.spool.Spool: Spooler has been configured. {maxSizeInBytes=2621440, storageType=Memory, keepQos0WhenOffline=false}
2024-11-19T16:07:22.378Z [INFO] (main) com.aws.greengrass.mqttclient.spool.Spool: Memory Spooler has been set up. {}
2024-11-19T16:07:22.393Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: Attempt to load service from plugins. {serviceName=aws.greengrass.Nucleus}
2024-11-19T16:07:22.419Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: Attempt to load service from plugins. {serviceName=main}
2024-11-19T16:07:22.433Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: Attempt to load service from plugins. {serviceName=UpdateSystemPolicyService}
2024-11-19T16:07:22.435Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: Attempt to load service from plugins. {serviceName=DeploymentService}
2024-11-19T16:07:22.505Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: Attempt to load service from plugins. {serviceName=TelemetryAgent}
2024-11-19T16:07:22.529Z [INFO] (pool-3-thread-6) com.aws.greengrass.security.SecurityService: get-mqtt-connection-builder. task failed and will be retried. {task-attempt=1}
com.aws.greengrass.security.exceptions.ServiceUnavailableException: Mqtt connection provider for pkcs11 is unavailable
at com.aws.greengrass.security.SecurityService.selectMqttConnectionProvider(SecurityService.java:219)

1. 修复PKCS#11 Provider配置的关键错误 #

• 版本号修正：aws.greengrass.crypto.Pkcs11Provider的version设为0.0.0无效，需指定与Nucleus 2.12.6兼容的版本（例如2.12.6，需匹配Nucleus版本）。
• 名称字段调整：name字段lybcryptauth_pkcs11可能错误，改为空字符串name: ""让Greengrass自动识别PKCS#11库。
• 插槽号验证：执行p11tool --provider=/usr/lib/libcryptoauth.so --list-slots确认实际插槽ID，替换配置中的slot: 0。

2. 确认HSM中的证书对象存在 #

从p11tool输出看，仅存在私钥和公钥，无证书对象，但配置中指定了certificateFilePath: "pkcs11:object=device;type=cert"，导致证书查找失败：

• 执行p11tool --provider=/usr/lib/libcryptoauth.so --list-certs检查证书是否存在。
• 若缺失，使用Microchip工具将设备证书导入HSM，确保证书的label和id与私钥一致（均为device和ABCDEFGH）。

3. 验证运行用户的权限 #

Greengrass以ggc_user:ggc_group运行，需确保该用户具备以下权限：

• 对/usr/lib/libcryptoauth.so的读取权限。
• 对HSM设备节点（如/dev/i2c-*）的读写权限。
• 切换到ggc_user测试HSM访问：su -s /bin/sh ggc_user -c "p11tool --provider=/usr/lib/libcryptoauth.so --list-tokens"，若无法访问，需调整文件/设备权限。

4. 修复Nucleus与PKCS#11 Provider的依赖关系 #

当前Nucleus的dependencies为空，需添加PKCS#11 Provider依赖，确保Nucleus启动时等待其就绪：

aws.greengrass.Nucleus:
  dependencies:
    - "aws.greengrass.crypto.Pkcs11Provider"

5. 调整PIN配置 #

从p11tool输出的Flags: RNG, uPIN uninitialized可知用户PIN未初始化，尝试移除userPin配置项，或设为空字符串userPin: ""，避免PIN验证失败。

6. 启用调试日志排查 #

修改Nucleus配置的logging部分，启用DEBUG级别日志：

aws.greengrass.Nucleus:
  configuration:
    logging:
      level: DEBUG
      format: TEXT

重启后查看详细日志，确认PKCS#11 Provider的加载过程是否有错误。

内容的提问来源于stack exchange，提问作者Jeroen Lardot