CICD流水线执行失败：configparser.InterpolationSyntaxError报错排查

问题 #

运行CICD流水线时，从Azure Data Factory获取参数的Python代码触发configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(',报错。尝试value.replace('%','%%')、value=repr(value)等修复方案均未彻底解决：首次添加替换代码后流水线短暂正常，但生成ADF覆盖参数环节失败；产物中参数已替换%为%%，复制到配置文件后重跑仍触发相同报错。

报错指向代码第42行：for key, value in config[env].items()，完整报错栈：

Traceback (most recent call last):
File "/agent/_work/2/s/CICD/pipelines/scripts/adf_overwrite_parameter_check.py", line 42, in
for key, value in config[env].items():
File "/usr/lib/python3.6/_collections_abc.py", line 744, in iter
yield (key, self._mapping[key])
File "/usr/lib/python3.6/configparser.py", line 1234, in getitem
return self._parser.get(self._name, key)
File "/usr/lib/python3.6/configparser.py", line 800, in get
d)
File "/usr/lib/python3.6/configparser.py", line 394, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
File "/usr/lib/python3.6/configparser.py", line 444, in _interpolate_some
"found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%27/sites/U-Uitgevers/Gedeelde%20documenten/Business%20plannen/MJP%20(nieuwe%20fondsstructuur%20en%20uniform%20format)/Power%20BI%20-%20Mapping/mapping%20tabel%20met%20business%20logica.xlsx%27)/$value"'

相关代码片段：

f = open(parametersJson,)
# returns JSON object as dict
data = json.load(f)
# Iterating through the json
parametersJsonDict = {}
for key,value in data['parameters'].items():
    value = value.replace('%','%%')
    parametersJsonDict[key]=value["value"]

f.close()
parametersIniDict = {}
argumentStringDict = {}
for env in envs:
    parametersIniDict[env] = {}
    config = configparser.ConfigParser()
    config.optionxform = str
    config.read(parametersIni)
    argumentStringDict[env] = ""
    for key, value in config[env].items():
        value= repr(value)
        parametersIniDict[env][key]=value
        try:
            value = json.loads(value)
        except:
            pass
        if isinstance(value, Mapping) or isinstance(value, list):
            argumentStringDict[env] = argumentStringDict[env] +" -" + str(key) + " " + json.dumps(value)
        else:
            argumentStringDict[env] = argumentStringDict[env] +" -" + str(key) + ' \"' + str(value) + '\"'


config = configparser.ConfigParser()
config.optionxform = str

failBuildMessage = []
failBuild="false"

for env in envs:
    config[env] = {}
    addIniDict = {}
    removeIniDict = {}
    for key,value in parametersJsonDict.items():
        if key in parametersIniDict[env]:
            config[env][str(key)] = str(parametersIniDict[env][key])
        else:
            addIniDict[key] = str(value)
    for key,value in parametersIniDict[env].items():
        if not key in parametersJsonDict:
            removeIniDict[key] = str(value)
            
    if len(addIniDict)>0:
        failBuild ="true"
        failBuildMessage.append(f"There are some overwrite parameters that need to be added in {env}, check the test artifacts")
        config[env+'_add'] = {}
        for key,value in addIniDict.items():
            config[env+'_add'][str(key)] = f'\"{str(value)}\"'

    if len(removeIniDict)>0:
        failBuild ="true"
        failBuildMessage.append(f"There are some overwrite parameters that need to be removed in {env}, check the test artifacts")
        config[env+'_remove'] = {}
        for key,value in removeIniDict.items():
            config[env+'_remove'][str(key)] = f'\"{str(value)}\"'

解决方案 #

核心原因 #

configparser.ConfigParser默认启用插值语法，当配置值中包含%（比如URL编码后的%20、%27）时，会被解析为插值占位符触发语法错误。之前的替换操作仅处理了ADF参数，但读取本地配置文件时，原文件中的%未被转义，依然会触发报错；同时repr()的使用导致配置格式异常，引发后续环节失败。

修复步骤 #

1. 禁用configparser的插值功能
   初始化ConfigParser时指定interpolation=None（Python 3.2+支持），彻底避免插值解析，代码中两处初始化ConfigParser的地方都需要修改：

   # 替换原初始化代码
   config = configparser.ConfigParser(interpolation=None)
   config.optionxform = str
   

2. 修正ADF参数转义逻辑
   原代码错误地转义了整个字典对象，应该转义具体的参数值：

   # 原代码
   # value = value.replace('%','%%')
   # parametersJsonDict[key]=value["value"]
   # 修改为
   param_value = value["value"]
   param_value = param_value.replace('%', '%%')
   parametersJsonDict[key] = param_value
   

3. 移除不必要的repr()调用
   repr()会给字符串添加额外引号，导致配置格式错误，直接使用原始值：

   # 原代码
   # value= repr(value)
   # parametersIniDict[env][key]=value
   # 修改为
   parametersIniDict[env][key] = value
   

验证修复 #

修改后重新运行流水线：

• 读取配置文件时不会再触发插值语法错误
• 生成的ADF覆盖参数内容格式正确，%被正确转义为%%
• 复制产物中的配置文件重跑，报错不再出现

内容的提问来源于stack exchange，提问作者Greencolor