PMMLPipeline中ContinuousDomain处理浮点值异常排查求助

环境与配置 #

使用sklearn2pmml的ContinuousDomain和sklearn-pandas的DataFrameMapper构建PMMLPipeline，相关配置如下：

{
    'features': [
        (['A'], [ContinuousDomain(dtype=<class 'float'>, high_value=82274.69794050456, low_value=1617.9990694391965, missing_value_replacement=26693.17348049894, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values')]),
        (['B'], [ContinuousDomain(dtype=<class 'int'>, high_value=142, low_value=0, missing_value_replacement=71, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values')]),
        (['C'], [ContinuousDomain(dtype=<class 'int'>, high_value=34, low_value=0, missing_value_replacement=17, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values')]),
        (['D'], [ContinuousDomain(dtype=<class 'int'>, high_value=903, low_value=2, missing_value_replacement=448, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values')]),
        (['E'], [ContinuousDomain(dtype=<class 'int'>, high_value=95, low_value=0, missing_value_replacement=48, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values')])
    ],
    'default': False,
    'built_default': False,
    'sparse': False,
    'df_out': False,
    'input_df': False,
    'drop_cols': [],
    'transformed_names_': ['A', 'B', 'C', 'D', 'E'],
    'built_features': [
        (['A'], TransformerPipeline(steps=[('continuousdomain', ContinuousDomain(dtype=<class 'float'>, high_value=82274.69794050456, low_value=1617.9990694391965, missing_value_replacement=26693.17348049894, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values'))]), {}),
        (['B'], TransformerPipeline(steps=[('continuousdomain', ContinuousDomain(dtype=<class 'int'>, high_value=142, low_value=0, missing_value_replacement=71, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values'))]), {}),
        (['C'], TransformerPipeline(steps=[('continuousdomain', ContinuousDomain(dtype=<class 'int'>, high_value=34, low_value=0, missing_value_replacement=17, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values'))]), {}),
        (['D'], TransformerPipeline(steps=[('continuousdomain', ContinuousDomain(dtype=<class 'int'>, high_value=903, low_value=2, missing_value_replacement=448, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values'))]), {}),
        (['E'], TransformerPipeline(steps=[('continuousdomain', ContinuousDomain(dtype=<class 'int'>, high_value=95, low_value=0, missing_value_replacement=48, missing_value_treatment='as_value', missing_values=[nan, -1], outlier_treatment='as_extreme_values'))]), {})
    ]
}

测试代码 #

构造单行DataFrame进行转换测试：

test_inst = {
    'A': [53.51370],
    'B': [28], 
    'C': [7], 
    'D': [655], 
    'E': [81]
}
test_pd = pd.DataFrame.from_dict(test_inst)
wrap.transform(test_pd)

异常现象 #

• 当A=53.51370时，执行转换报错：ValueError: ['A']: Data contains 1 invalid values
• 当A=53.51371时，转换正常完成

两个值均远低于特征A配置的low_value=1617.9990694391965，按outlier_treatment='as_extreme_values'的配置，应被转换为low_value，而非判定为无效值。

排查方向提示 #

• 浮点数精度问题：检查ContinuousDomain内部数值比较逻辑是否存在浮点数精度误差。53.51370和53.51371的二进制浮点数存储差异，可能导致其中一个被错误判定为超出离群值处理范围。
• 无效值判定逻辑冲突：确认ContinuousDomain中outlier_treatment的触发条件是否与missing_values判定逻辑存在重叠。比如是否因精度问题，将低于low_value的数值误归类到missing_values（当前配置为[nan, -1]）范畴，触发无效值报错。
• Pipeline隐式转换：检查DataFrameMapper的built_features中，TransformerPipeline是否在ContinuousDomain处理前存在其他隐式转换，导致数值被意外修改。
• 版本兼容性：验证sklearn2pmml和sklearn-pandas的版本是否存在已知浮点数处理bug，尝试升级到最新稳定版本复现测试。

内容的提问来源于stack exchange，提问作者Habenzu