C#反序列化Python Pickle文件：numpy类型自定义IObjectConstructor注册

读取Python生成的Pickle文件时，抛出如下运行时错误：

抛出PickleException：
expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct).
This happens when an unsupported/unregistered class is being unpickled that requires construction arguments.
Fix it by registering a custom IObjectConstructor for this class.

我尝试了以下代码，但仍未解决，请问如何在C#中注册自定义IObjectConstructor处理np.core.multiarray类型？

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Razorvine.Pickle;
using NumSharp;
using NumSharp.Extensions;
using Numpy;

// To learn more about cTrader Automate visit our Help Center:
// https://help.ctrader.com/ctrader-automate

// Load the pickle file into a byte array
//byte[] pickleData = File.ReadAllBytes("C:/svm_model.pkl");
        
// Load the model from file
using (FileStream stream = new FileStream("C:/svm_model.pkl", FileMode.Open, FileAccess.Read))
{
    Unpickler unpickler = new Unpickler();
    object obj = unpickler.load(stream);

    // Register custom constructors for NumSharp.NDArray and numpy.core.multiarray._reconstruct
    unpickler.RegisterConstructor("numpy.core.multiarray._reconstruct", new NumpyReconstructConstructor());
    unpickler.RegisterConstructor("numpy.ndarray", new NumpyArrayConstructor());


// obj should be a scikit-learn model instance that you can use for predictions
// For example:
var x = Numpy.np.array(new double[] { 1.0, 2.0, 3.0 });
    var y = (double)obj.GetType().GetMethod("predict").Invoke(obj, new object[] { x });
}
    
public class NumpyArrayConstructor : IObjectConstructor
{
    public object construct(object[] args)
    {
        // Extract the typecode, shape, and dtype from the arguments
        string typecode = (string)args[0];
        int[] shape = (int[])args[1];
        string dtype = (string)args[2];

        // Create a new NumSharp array with the specified typecode, shape, and dtype
        return new NDArray(Array.CreateInstance(Type.GetType(dtype), shape), shape);
    }
}
// Define a custom constructor for the numpy.core.multiarray._reconstruct class
class NumpyReconstructConstructor : IObjectConstructor
{
    public object construct(object[] args)
    {
        // Construct the object using the provided arguments
        return new Numpy.np.core.multiarray._reconstruct((string)args[0], (int)args[1], (string)args[2]);
    }
}
}

1. 纠正构造器注册时机 #

你的代码错误地在unpickler.load()之后才注册构造器，此时反序列化已经启动，注册不会生效。必须先注册构造器，再调用load方法。

2. 实现正确的自定义构造器 #

numpy.core.multiarray._reconstruct是Python中numpy数组的重建逻辑，不需要实例化这个“类”，而是要在C#中生成对应的NDArray对象。以下是修正后的完整实现：

using System;
using System.IO;
using Razorvine.Pickle;
using NumSharp;

class Program
{
    static void Main()
    {
        using (FileStream stream = new FileStream("C:/svm_model.pkl", FileMode.Open, FileAccess.Read))
        {
            Unpickler unpickler = new Unpickler();
            
            // 先注册构造器，再执行反序列化
            unpickler.RegisterConstructor("numpy.core.multiarray._reconstruct", new NumpyReconstructConstructor());
            unpickler.RegisterConstructor("numpy.ndarray", new NumpyArrayConstructor());
            
            object obj = unpickler.load(stream);

            // 测试模型预测（需确保模型实际支持predict方法）
            var x = np.array(new double[] { 1.0, 2.0, 3.0 });
            var predictMethod = obj.GetType().GetMethod("predict");
            if (predictMethod != null)
            {
                var y = predictMethod.Invoke(obj, new object[] { x });
                Console.WriteLine($"预测结果: {y}");
            }
        }
    }
}

public class NumpyArrayConstructor : IObjectConstructor
{
    public object Construct(object[] args)
    {
        // 根据pickle实际参数调整，示例提取形状创建空数组
        if (args.Length >= 2)
        {
            int[] shape = args[1] as int[];
            return np.empty(shape);
        }
        return np.empty(0);
    }
}

public class NumpyReconstructConstructor : IObjectConstructor
{
    public object Construct(object[] args)
    {
        // 解析_reconstruct的参数：数组类型、形状、 dtype信息
        if (args.Length >= 3)
        {
            string dtype = args[2] as string;
            int[] shape = args[1] as int[];
            
            // 根据dtype匹配C#类型
            Type targetType = dtype switch
            {
                "float64" => typeof(double),
                "int32" => typeof(int),
                "float32" => typeof(float),
                _ => typeof(double) // 默认处理未匹配的类型
            };
            
            return np.empty(shape, targetType);
        }
        return np.empty(0);
    }
}

3. 关键注意事项 #

• 参数适配：pickle中numpy类型的反序列化参数会随Python、numpy版本变化，需根据实际调试调整参数解析逻辑。
• 库版本兼容：确保Razorvine.Pickle、NumSharp的版本匹配，避免类型不兼容问题。
• 模型调用验证：通过反射调用predict时，需保证输入数组的类型、形状与模型训练时一致。

内容的提问来源于stack exchange，提问作者jad