FastAPI开发Shopify公开应用时HMAC验证不匹配问题求助

问题描述 #

开发Shopify公开应用时，HMAC验证环节始终失败。已使用partners.shopify.com后台的应用Secret进行计算，但生成的HMAC与请求头中的HMAC完全不匹配，尝试多种组合均无效。

关键信息 #

• 使用的是合作伙伴后台的应用Secret
• 该应用为公开应用，无法使用店铺后台的Webhook密钥，必须使用应用Secret

当前FastAPI实现代码 #

def verify_webhook(data, hmac_header):
    # Compute HMAC using the raw data
    secret = SHOPIFY_SECRET.encode('utf-8')
    digest = hmac.new(secret, data, digestmod=sha256).digest()
    computed_hmac = base64.b64encode(digest).decode('utf-8')

    print("SECRET:", secret)
    print("DATA (raw):", data)
    print("DIGEST (binary):", digest)
    print("DIGEST (hex):", digest.hex())
    print("COMPUTED HMAC (base64):", computed_hmac)
    print("RECEIVED HMAC (header):", hmac_header)

    return hmac.compare_digest(computed_hmac, hmac_header)

@app.post('/webhook/customer/data_request')
async def customer_data_request_webhook(request: Request):
    try:
        data = await request.body()
        headers = dict(request.headers)

        print('RAW DATA:', data)
        print('HEADERS:', headers)

        hmac_header = headers.get('X-Shopify-Hmac-SHA256')
        print("HMAC HEADER:", hmac_header)

        verified = verify_webhook(data, hmac_header)
        if not verified:
            raise HTTPException(status_code=HTTP_401_UNAUTHORIZED, detail="HMAC verification failed")

        print("Received customer data request webhook:")
        print(data)
        return Response(status_code=HTTP_200_OK)

日志信息 #

DATA b'{"shop_id":60225617942,"shop_domain":"app-security.myshopify.com"}'
HEADER {'host': '*******************', 'user-agent': 'Ruby', 'content-length': '66', 'accept': '*/*', 'accept-encoding': 'gzip', 'cdn-loop': 'cloudflare; subreqs=1', 'cf-connecting-ip': '34.23.252.48', 'cf-ew-via': '15', 'cf-ipcountry': 'US', 'cf-ray': '88f83610910badb0-ATL', 'cf-visitor': '{"scheme":"https"}', 'cf-worker': 'onrender.com', 'content-type': 'application/json', 'render-proxy-ttl': '4', 'rndr-id': '4e79bfd2-668d-4233', 'traceparent': '00-61fe6df471212a349f78bfef3aeb53bb-b94c865665aa5ace-01', 'true-client-ip': '34.23.252.48', 'x-cloud-trace-context': '61fe6df471212a349f78bfef3aeb53bb/13352194700877650638;o=1', 'x-forwarded-for': '34.23.252.48, 10.216.35.99', 'x-forwarded-proto': 'https', 'x-request-start': '1717674903202354', 'x-shopify-api-version': '2022-10', 'x-shopify-hmac-sha256': '4Apq2emHXH+wFrY3yTP9u+XUAMrb/pOl/HE0OMmqf1E=', 'x-shopify-shop-domain': 'app-security.myshopify.com', 'x-shopify-test': 'true', 'x-shopify-topic': 'shop/redact', 'x-shopify-trace-context': '61fe6df471212a349f78bfef3aeb53bb/13352194700877650638;o=1', 'x-shopify-trace-hint': 'true', 'x-shopify-triggered-at': '2024-06-06T11:55:03.059362171Z', 'x-shopify-webhook-id': '3c7cc95f-e354-46f4-87b4-8da946b988df'}
HMAC HEADER 4Apq2emHXH+wFrY3yTP9u+XUAMrb/pOl/HE0OMmqf1E=
SECRET: b'********************************'
DATA (raw): b'{"shop_id":60225617942,"shop_domain":"app-security.myshopify.com"}'
DIGEST (binary): b':\x00\xe2\x01\xf5\x88\x8c\xfb,H\\\x88\x9a\xe6\xd7\xaa\x13\x18M\xd2\xcdY\x00\x16L\x12\xda\xbf-\x88N\x16'
COMPUTED HMAC (base64): OgDiAfWIjPssSFyImubXqhMYTdLNWQAWTBLavy2IThY=
RECEIVED HMAC (header): 4Apq2emHXH+wFrY3yTP9u+XUAMrb/pOl/HE0OMmqf1E=
Error processing shop data erasure webhook:
401: HMAC verification failed

问题分析与解决方案 #

核心问题：请求体被HTML转义 #

从日志可以看到，接收的原始数据中双引号被转义为"，但Shopify发送的原始Webhook数据是标准JSON格式（使用双引号），这导致计算HMAC的数据源与Shopify签名时的数据源不一致，直接导致验证失败。

解决步骤 #

1. 排查中间代理/平台的转义行为
   日志显示请求经过Cloudflare和Render的worker，这些服务可能对请求体做了HTML转义。需要在对应平台的配置中关闭自动转义，确保服务能接收到原始未修改的二进制请求体。

2. 临时修复：还原转义的请求体
   如果暂时无法修改平台配置，可以在验证前将转义的请求体还原为原始JSON格式：

   def verify_webhook(data, hmac_header):
       # 还原HTML转义的双引号
       raw_data = data.decode('utf-8').replace('"', '"').encode('utf-8')
       secret = SHOPIFY_SECRET.encode('utf-8')
       digest = hmac.new(secret, raw_data, digestmod=sha256).digest()
       computed_hmac = base64.b64encode(digest).decode('utf-8')
   
       # 保留原有日志打印
       print("SECRET:", secret)
       print("DATA (raw):", data)
       print("DATA (restored):", raw_data)
       print("DIGEST (binary):", digest)
       print("COMPUTED HMAC (base64):", computed_hmac)
       print("RECEIVED HMAC (header):", hmac_header)
   
       return hmac.compare_digest(computed_hmac, hmac_header)
   

3. 验证Secret正确性
   确认SHOPIFY_SECRET完全匹配合作伙伴后台的应用Secret，无多余空格、换行符，区分大小写。

4. 确认数据未被二次处理
   确保FastAPI没有自动解析请求体（比如使用Body()或Pydantic模型），必须使用await request.body()获取原始二进制数据。

验证方法 #

手动计算HMAC验证：

• 将日志中的转义数据还原为原始JSON：{"shop_id":60225617942,"shop_domain":"app-security.myshopify.com"}
• 使用应用Secret对该字符串计算SHA256 HMAC，再进行Base64编码
• 对比结果是否与请求头中的4Apq2emHXH+wFrY3yTP9u+XUAMrb/pOl/HE0OMmqf1E=一致

内容的提问来源于stack exchange，提问作者Kevin Lopez