如何不依赖Aerospike库生成Aerospike Key Edigest

问题背景 #

需要编写Python脚本，根据namespace、set、userkey生成Aerospike对应的Key Digest。已知Aerospike采用RIPEMD-160哈希算法，但手动编写的脚本生成结果与官方库不一致。

官方库生成示例 #

使用Aerospike官方Python库生成Key的代码：

#!/usr/local/bin/python
import sys
import aerospike

config = {
    'hosts': [ ('aerospike', 3000) ],
    'policies': {"write": {"key": aerospike.POLICY_KEY_SEND}}
}

try:
    client = aerospike.client(config).connect()
except:
    print("Failed to connect to the cluster with", config['hosts'])
    sys.exit(1)

key = ('test', 'demo', 'one')
print('key:', key)

try:
    client.put(key, {'bin1': 'value1'})
except Exception as e:
    print("error: {0}".format(e), file=sys.stderr)
    sys.exit(1)

(key, metadata, record) = client.get(key)
print("key from aerospike:", key)

执行输出：

root@d1716cafa3fb:/app# ./a.py
key: ('test', 'demo', 'one')
key from aerospike: ('test', 'demo', None, bytearray(b'\xe4I\x9bJ\xa5r1fL\xe0\x19\xe5\xc2\x9d"Xr\xdb\xbf\x03'))

AQL查询结果 #

aql> set record_print_metadata true
RECORD_PRINT_METADATA = true
aql> select * from test
+-------+----------+--------------------------------+--------+---------+-------+
| PK    | bin1     | {edigest}                      | {set}  | {ttl}   | {gen} |
+-------+----------+--------------------------------+--------+---------+-------+
| "one" | "value1" | "5EmbSqVyMWZM4Bnlwp0iWHLbvwM=" | "demo" | 2591964 | 1     |
+-------+----------+--------------------------------+--------+---------+-------+
1 row in set (0.024 secs)

可知正确的Digest base64值为5EmbSqVyMWZM4Bnlwp0iWHLbvwM=。

原错误脚本 #

#!/usr/local/bin/python
from Crypto.Hash import RIPEMD
import base64

message = "demo one"

hash_obj = RIPEMD.new(data=message.encode('utf-8'))
hash_bytes = hash_obj.digest()
hash_base64 = base64.b64encode(hash_bytes).decode('utf-8')

print(f"RIPEMD-160 hash of '{message}' in base64 is: {hash_base64}")

执行输出：

root@d1716cafa3fb:/app# ./b.py
RIPEMD-160 hash of 'demo one' in base64 is: Lz0mUZgml2l+NLfnhwc+5RwlmEY=

修正方案 #

错误原因 #

原脚本错误地将set和userkey用空格拼接后哈希，而Aerospike的Key Digest生成规则是：

1. 对set名称和userkey分别做UTF-8编码
2. 按4字节大端格式的set长度 + set字节内容 + 4字节大端格式的userkey长度 + userkey字节内容的顺序拼接成字节流
3. 对拼接后的字节流计算RIPEMD-160哈希，再转base64

修正后的脚本 #

#!/usr/local/bin/python
from Crypto.Hash import RIPEMD160
import base64
import struct

def generate_aerospike_digest(set_name, user_key):
    # 编码set和userkey为UTF-8字节
    set_bytes = set_name.encode('utf-8')
    key_bytes = user_key.encode('utf-8')
    
    # 按Aerospike规则拼接字节流
    payload = struct.pack('!I', len(set_bytes)) + set_bytes
    payload += struct.pack('!I', len(key_bytes)) + key_bytes
    
    # 计算RIPEMD-160哈希
    hash_obj = RIPEMD160.new(data=payload)
    hash_bytes = hash_obj.digest()
    
    # 转换为base64字符串
    return base64.b64encode(hash_bytes).decode('utf-8')

# 测试：对应set=demo，userkey=one
digest = generate_aerospike_digest("demo", "one")
print(f"正确的Digest base64值: {digest}")

执行结果 #

运行脚本后会输出：

正确的Digest base64值: 5EmbSqVyMWZM4Bnlwp0iWHLbvwM=

与Aerospike官方生成的结果完全一致。

内容的提问来源于stack exchange，提问作者bessarabov