如何从绑定ExoPlayer的Android SurfaceTexture创建Vulkan VkImage？

核心结论 #

必须使用VK_ANDROID_EXTERNAL_MEMORY_ANDROID_HARDWARE_BUFFER扩展来完成这个过程——因为SurfaceTexture的底层帧数据存储在Android Hardware Buffer（AHB）中，Vulkan需要通过该扩展实现跨API的硬件缓冲区共享与访问。

具体实现步骤 #

1. 获取SurfaceTexture对应的Android Hardware Buffer

在API 29及以上版本中，SurfaceTexture提供了getHardwareBuffer()方法可直接获取底层AHB。注意要在onFrameAvailable回调触发后调用（需先调用updateTexImage()更新帧数据），确保帧已就绪：

// 假设已初始化好SurfaceTexture实例
SurfaceTexture surfaceTexture = ...;
// 等待帧可用后更新纹理
surfaceTexture.updateTexImage();
HardwareBuffer hardwareBuffer = surfaceTexture.getHardwareBuffer();

2. 配置Vulkan环境并启用扩展

• 创建VkInstance和VkDevice时，必须显式声明VK_ANDROID_EXTERNAL_MEMORY_ANDROID_HARDWARE_BUFFER扩展。
• 提前检查物理设备是否支持该扩展：

bool hasAHBExtension = false;
uint32_t extensionCount = 0;
vkEnumerateDeviceExtensionProperties(physicalDevice, nullptr, &extensionCount, nullptr);
std::vector<VkExtensionProperties> extensions(extensionCount);
vkEnumerateDeviceExtensionProperties(physicalDevice, nullptr, &extensionCount, extensions.data());

for (const auto& ext : extensions) {
    if (strcmp(ext.extensionName, VK_ANDROID_EXTERNAL_MEMORY_ANDROID_HARDWARE_BUFFER_EXTENSION_NAME) == 0) {
        hasAHBExtension = true;
        break;
    }
}

3. 从AHB创建VkImage

3.1 获取AHB属性

通过vkGetAndroidHardwareBufferPropertiesANDROID获取缓冲区的格式、尺寸等关键参数，用于配置VkImage：

VkAndroidHardwareBufferPropertiesANDROID ahbProps{};
ahbProps.sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID;
VkResult result = vkGetAndroidHardwareBufferPropertiesANDROID(device, hardwareBuffer, &ahbProps);
if (result != VK_SUCCESS) {
    // 处理错误逻辑
}

3.2 构建VkImage创建信息

填充外部内存创建信息与VkImage基础配置，确保格式、尺寸与AHB完全匹配：

VkExternalMemoryImageCreateInfo externalMemoryInfo{};
externalMemoryInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO;
externalMemoryInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID;

VkImageCreateInfo imageCreateInfo{};
imageCreateInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
imageCreateInfo.pNext = &externalMemoryInfo;
imageCreateInfo.imageType = VK_IMAGE_TYPE_2D;
imageCreateInfo.format = ahbProps.format; // 严格匹配AHB格式
imageCreateInfo.extent.width = ahbProps.width;
imageCreateInfo.extent.height = ahbProps.height;
imageCreateInfo.extent.depth = 1;
imageCreateInfo.mipLevels = 1;
imageCreateInfo.arrayLayers = 1;
imageCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
imageCreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT; // 根据渲染需求调整
imageCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
imageCreateInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;

VkImage vkImage;
result = vkCreateImage(device, &imageCreateInfo, nullptr, &vkImage);

4. 绑定AHB到VkImage

通过vkGetMemoryAndroidHardwareBufferANDROID获取内存要求，直接绑定AHB对应的VkDeviceMemory：

VkMemoryRequirements memReqs{};
vkGetImageMemoryRequirements(device, vkImage, &memReqs);

// 找到匹配的内存类型（需包含VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT）
uint32_t memoryTypeIndex = 0;
for (uint32_t i = 0; i < memoryProperties.memoryTypeCount; i++) {
    if ((memReqs.memoryTypeBits & (1 << i)) && 
        (memoryProperties.memoryTypes[i].propertyFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)) {
        memoryTypeIndex = i;
        break;
    }
}

VkMemoryAllocateInfo memAllocInfo{};
memAllocInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
memAllocInfo.allocationSize = memReqs.size;
memAllocInfo.memoryTypeIndex = memoryTypeIndex;

// 导入AHB作为外部内存
VkImportAndroidHardwareBufferInfoANDROID importInfo{};
importInfo.sType = VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID;
importInfo.hardwareBuffer = hardwareBuffer;
memAllocInfo.pNext = &importInfo;

VkDeviceMemory vkMemory;
result = vkAllocateMemory(device, &memAllocInfo, nullptr, &vkMemory);
if (result != VK_SUCCESS) {
    // 处理错误逻辑
}

// 绑定内存到VkImage
result = vkBindImageMemory(device, vkImage, vkMemory, 0);

关键注意事项 #

• 格式匹配：GL_TEXTURE_EXTERNAL_OES对应的AHB通常是YUV类格式（如HAL_PIXEL_FORMAT_YCbCr_420_888），对应Vulkan格式为VK_FORMAT_G8_B8_R8_3PLANE_420_UNORM，格式不匹配会导致渲染画面异常。
• 同步控制：SurfaceTexture的帧更新与Vulkan的纹理访问需通过VkSemaphore或VkFence同步，避免数据竞争。
• 低版本兼容：若需支持API 29以下版本，可通过EGLImageKHR作为中间层：先从SurfaceTexture创建EGLImage，再通过VK_KHR_external_memory_fd扩展获取文件描述符，最终创建VkImage，但该方式复杂度更高、性能不如AHB直接共享。

内容的提问来源于stack exchange，提问作者zeus