单元测试中K8S NewBuilder查询Fake CR资源失败问题咨询

问题背景 #

通过k8s.io/kubectl/pkg/cmd/util/factory.go中的Factory.NewBuilder().Do()查询K8S自定义资源（CR），使用k8s.io/kubectl/pkg/cmd/testing/fake.go中的TestFactory编写单元测试时遇到两个问题：

• 用FakeDynamicClient创建的Fake资源无法被NewBuilder().Do()查询到
• 改用UnstructuredClient伪造响应时，提示error: the server doesn't have a resource type "myCRDresource"

解决方案 #

方案一：正确配置FakeDynamicClient（推荐） #

Kubectl的NewBuilder()处理CR时依赖动态客户端和发现机制，仅设置FakeDynamicClient不够，还需注册CR API到Scheme并配置Fake发现客户端，让Builder识别资源类型。

示例代码：

import (
    "testing"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    cmdtesting "k8s.io/kubectl/pkg/cmd/testing"
    fakedynamic "k8s.io/client-go/dynamic/fake"
    "k8s.io/client-go/kubernetes/scheme"
    fakeopenapi "k8s.io/client-go/openapi/fake"
    "k8s.io/kubectl/pkg/apis/openapi"
    appsv1alpha1 "your/module/api/v1alpha1" // 替换为你的CR API包路径
)

func TestCRQueryWithFakeDynamicClient(t *testing.T) {
    // 1. 注册CR API到全局Scheme
    if err := appsv1alpha1.AddToScheme(scheme.Scheme); err != nil {
        t.Fatalf("注册Scheme失败: %v", err)
    }

    // 2. 创建测试用CR实例
    cv1 := &appsv1alpha1.ClusterVersion{
        ObjectMeta: metav1.ObjectMeta{Name: "cv-1", Namespace: "default"},
        // 按需填充spec/status字段
    }
    cv2 := &appsv1alpha1.ClusterVersion{
        ObjectMeta: metav1.ObjectMeta{Name: "cv-2", Namespace: "default"},
    }

    // 3. 初始化FakeDynamicClient并注入测试资源
    fakeDynamicClient := fakedynamic.NewSimpleDynamicClient(scheme.Scheme, cv1, cv2)

    // 4. 配置TestFactory
    tf := cmdtesting.NewTestFactory()
    defer tf.Cleanup()

    // 配置FakeOpenAPIGetter，让Builder识别CR资源类型
    tf.FakeOpenAPIGetter = &fakeopenapi.FakeOpenAPIGetter{
        Resources: openapi.Resources{
            "clusterversions.apps.yourdomain.com": { // 替换为你的CR的GroupVersionResource标识
                Group:   "apps.yourdomain.com",
                Version: "v1alpha1",
                Kind:    "ClusterVersion",
            },
        },
    }
    tf.FakeDynamicClient = fakeDynamicClient

    // 5. 执行查询逻辑
    builder := tf.NewBuilder().
        WithScheme(scheme.Scheme).
        Namespace("default").
        Resource("clusterversions"). // 替换为你的CR资源名称
        Do()

    // 6. 验证查询结果
    var result appsv1alpha1.ClusterVersionList
    if err := builder.Unstructured().Into(&result); err != nil {
        t.Fatalf("反序列化结果失败: %v", err)
    }
    if len(result.Items) != 2 {
        t.Fatalf("期望2条结果，实际得到%d条", len(result.Items))
    }
}

方案二：修复UnstructuredClient配置 #

若坚持使用UnstructuredClient，需同时配置Fake发现客户端，让Kubectl确认CR资源类型存在：

import (
    "net/http"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/runtime/schema"
    "k8s.io/apimachinery/pkg/runtime/serializer"
    "k8s.io/client-go/discovery/fake"
    "k8s.io/client-go/rest"
    cmdtesting "k8s.io/kubectl/pkg/cmd/testing"
    "k8s.io/kubectl/pkg/resource"
    appsv1alpha1 "your/module/api/v1alpha1"
)

func TestCRQueryWithUnstructuredClient(t *testing.T) {
    // 1. 生成测试CR列表
    cv1 := &appsv1alpha1.ClusterVersion{ObjectMeta: metav1.ObjectMeta{Name: "cv-1", Namespace: "default"}}
    cv2 := &appsv1alpha1.ClusterVersion{ObjectMeta: metav1.ObjectMeta{Name: "cv-2", Namespace: "default"}}
    cvlists := getFakeClusterVersionList(cv1, cv2)

    // 2. 配置FakeDiscoveryClient
    fakeDiscovery := fake.NewSimpleDiscoveryClient()
    fakeDiscovery.Fake.Resources = []*metav1.APIResourceList{
        {
            GroupVersion: appsv1alpha1.SchemeGroupVersion.String(),
            APIResources: []metav1.APIResource{
                {
                    Name:       "clusterversions",
                    Namespaced: true,
                    Kind:       "ClusterVersion",
                    Verbs:      []string{"get", "list"},
                },
            },
        },
    }

    // 3. 初始化TestFactory
    tf := cmdtesting.NewTestFactory()
    defer tf.Cleanup()

    // 关联FakeDiscoveryClient到ClientConfig
    tf.ClientConfigVal = &rest.Config{
        Host:            "http://fake-k8s-server",
        DiscoveryClient: fakeDiscovery,
    }

    // 4. 配置UnstructuredClient
    codec := resource.UnstructuredPlusDefaultContentConfig().NegotiatedSerializer
    tf.UnstructuredClient = &fake.RESTClient{
        NegotiatedSerializer: codec,
        Resp: &http.Response{
            StatusCode: http.StatusOK,
            Header:     cmdtesting.DefaultHeader(),
            Body:       cmdtesting.ObjBody(codec, cvlists),
        },
    }

    // 5. 执行查询并验证结果（逻辑同方案一）
    // ...
}

// 你的CR列表生成函数
func getFakeClusterVersionList(clusterVersions ...*appsv1alpha1.ClusterVersion) *appsv1alpha1.ClusterVersionList {
    res := &appsv1alpha1.ClusterVersionList{
        ListMeta: metav1.ListMeta{ResourceVersion: "1"},
        Items:    make([]appsv1alpha1.ClusterVersion, 0),
    }
    for _, cv := range clusterVersions {
        res.Items = append(res.Items, *cv.DeepCopy())
    }
    return res
}

关键注意事项 #

• 必须注册CR API到Scheme：确保Kubectl的序列化/反序列化机制能识别CR的结构
• 配置Fake发现机制：NewBuilder()会先通过发现客户端验证资源类型，必须伪造CR的API资源信息
• 优先选择FakeDynamicClient：动态客户端的Fake实现更贴近真实场景，无需手动构造HTTP响应，维护成本更低

内容的提问来源于stack exchange，提问作者al al