Android NDK/Camera2：从ImageReader Surface创建OpenGL Surface帧处理报错求助

错误原因分析 #

你遇到的already connected错误本质是ImageReader的Surface不能同时被Camera和OpenGL绑定：

• ImageReader默认创建的是CPU侧的缓冲Surface，只能被一个生产者（Camera）和一个消费者（ImageReader的onImageAvailable回调）绑定，无法再作为OpenGL的渲染目标（GPU侧Surface）。
• 你的代码中同时将ImageReader的Surface传给Camera，又试图用它创建EGLWindowSurface，导致Surface被重复绑定，触发相机会话配置失败。

无CPU拷贝的替代实现方案 #

以下两种方案均基于GPU侧处理，完全避免CPU缓存拷贝，性能最优：

方案一：TextureView + OpenGL ES + Skia（兼容性好，API 14+） #

用TextureView替代SurfaceView，利用其SurfaceTexture实现GPU侧的帧流转：

1. Java层配置：

   • 创建TextureView，设置SurfaceTextureListener，在onSurfaceTextureAvailable回调中获取SurfaceTexture。
   • 将此SurfaceTexture作为Camera2 CaptureRequest的目标，让相机直接输出GPU纹理。

   textureView.surfaceTextureListener = object : TextureView.SurfaceTextureListener {
       override fun onSurfaceTextureAvailable(surface: SurfaceTexture, width: Int, height: Int) {
           // 初始化Camera2会话，将surface作为预览目标
           val captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
           val surface = Surface(surfaceTexture)
           captureRequestBuilder.addTarget(surface)
           cameraSession.setRepeatingRequest(captureRequestBuilder.build(), null, cameraHandler)
           // 初始化NDK侧的OpenGL/Skia上下文，传入surfaceTexture
           nativeInitSkiaRendering(surfaceTexture, width, height)
       }
       // 实现其他回调方法...
   }
   

2. NDK侧处理：

   • 用EGL绑定SurfaceTexture为EGLSurface，创建Skia的GrContext和SkSurface。
   • 每一帧到来时，更新SurfaceTexture的纹理，在SkSurface上绘制相机纹理+红框，最后交换缓冲区。

   // 绑定SurfaceTexture到EGL
   EGLSurface eglSurface = eglCreateWindowSurface(eglDisplay, eglConfig, surfaceTexture, nullptr);
   eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext);
   
   // 创建SkSurface
   auto backendInterface = GrGLMakeNativeInterface();
   auto grContext = GrDirectContext::MakeGL(backendInterface);
   GrGLFramebufferInfo fbInfo;
   fbInfo.fFBOID = 0; // SurfaceTexture默认绑定到FBO 0
   fbInfo.fFormat = GR_GL_RGBA8;
   auto renderTarget = GrBackendRenderTarget(width, height, 0, 0, fbInfo);
   auto skSurface = SkSurface::MakeFromBackendRenderTarget(grContext.get(), renderTarget, kBottomLeft_GrSurfaceOrigin, kRGBA_8888_SkColorType, nullptr);
   
   // 帧处理逻辑
   void onFrameAvailable() {
       surfaceTexture->updateTexImage(); // 更新纹理内容
       auto canvas = skSurface->getCanvas();
       // 1. 绘制相机纹理（需要将SurfaceTexture的纹理转为SkImage）
       SkImage* cameraImage = SkImage::MakeFromTexture(grContext.get(), textureId, kTopLeft_GrSurfaceOrigin, kRGBA_8888_SkColorType, kPremul_SkAlphaType, nullptr);
       canvas->drawImage(cameraImage, 0, 0);
       // 2. 绘制红框
       SkPaint paint;
       paint.setColor(SK_ColorRED);
       paint.setStyle(SkPaint::kStroke_Style);
       paint.setStrokeWidth(4);
       canvas->drawRect(SkRect::MakeLTRB(100, 100, 300, 300), paint);
       // 3. 提交渲染
       grContext->flush();
       eglSwapBuffers(eglDisplay, eglSurface);
   }
   

方案二：HardwareBuffer ImageReader + GPU纹理处理（API 26+） #

使用支持HardwareBuffer的ImageReader，直接在GPU侧访问相机帧数据：

1. Java层创建ImageReader：

   • 指定ImageFormat.YUV_420_888，并启用HardwareBuffer支持，避免CPU拷贝。

   val imageReader = ImageReader.newInstance(width, height, ImageFormat.YUV_420_888, 2)
   val hwBufferUsage = HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE or HardwareBuffer.USAGE_GPU_COLOR_OUTPUT
   imageReader.setHardwareBufferUsage(hwBufferUsage)
   imageReader.setOnImageAvailableListener({ reader ->
       val image = reader.acquireLatestImage()
       val hwBuffer = image.hardwareBuffer
       // 将HardwareBuffer传入NDK侧处理
       nativeProcessFrame(hwBuffer, image.width, image.height)
       image.close()
   }, handler)
   // 将imageReader.surface传给Camera2作为目标
   captureRequestBuilder.addTarget(imageReader.surface)
   

2. NDK侧处理：

   • 从HardwareBuffer创建OpenGL纹理，用Skia绘制后输出到预览Surface。

   void nativeProcessFrame(HardwareBuffer* hwBuffer, int width, int height) {
       // 从HardwareBuffer创建OpenGL纹理
       AHardwareBuffer_Desc desc;
       AHardwareBuffer_describe(hwBuffer, &desc);
       GLuint textureId;
       glGenTextures(1, &textureId);
       glBindTexture(GL_TEXTURE_EXTERNAL_OES, textureId);
       AHardwareBuffer_toHardwareBufferDescriptor(hwBuffer, &desc);
       glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, eglImage);
   
       // 创建SkImage并绘制
       auto skImage = SkImage::MakeFromTexture(grContext.get(), textureId, kTopLeft_GrSurfaceOrigin, kRGBA_8888_SkColorType, kPremul_SkAlphaType, nullptr);
       auto canvas = previewSkSurface->getCanvas();
       canvas->drawImage(skImage, 0, 0);
       // 绘制红框...
       grContext->flush();
       // 释放资源
       glDeleteTextures(1, &textureId);
       AHardwareBuffer_release(hwBuffer);
   }
   

内容的提问来源于stack exchange，提问作者mrousavy