Metal 3中2D对象重叠绘制模板测试失效问题求助

我用C++基于Apple Metal 3开发简易2D渲染应用，想通过模板测试实现重叠2D对象绘制，但功能失效。当前代码只能绘制背景（Metal绘制最后一个元素），预期兽人像素模板值为1、背景为0，但实际不符合预期。

现有代码 #

Renderer构造函数 #

Renderer::Renderer( MTL::Device* pDevice, MTK::View* pView )
{
  ...
  auto* pTexDesc = MTL::TextureDescriptor::alloc()->init();
  pTexDesc->setTextureType( MTL::TextureType2D );
  pTexDesc->setWidth( _viewWidth );
  pTexDesc->setHeight( _viewHeight );
  pTexDesc->setPixelFormat( MTL::PixelFormatStencil8 );
  pTexDesc->setStorageMode( MTL::StorageModePrivate );
  pTexDesc->setUsage( MTL::TextureUsageRenderTarget );
  _pStencilTexture = _pDevice->newTexture( pTexDesc );

  _pDepthStencilDesc = MTL::DepthStencilDescriptor::alloc()->init();
  _pStencilState = _pDevice->newDepthStencilState( _pDepthStencilDesc );
  ...
}

每帧update方法 #

void Renderer::update( float delta )
{
  NS::AutoreleasePool* pPool = NS::AutoreleasePool::alloc()->init();
  _pCurrentDrawableView->setClearColor( MTL::ClearColor::Make(1.0f, 1.0f, 1.0f, 1.0f) );

  MTL::CommandBuffer* pCmd = _pCommandQueue->commandBuffer();
  MTL::RenderPassDescriptor* pRPD = _pCurrentDrawableView->currentRenderPassDescriptor();

  auto* pStencilAttach = pRPD->stencilAttachment();
  pStencilAttach->setTexture( _pStencilTexture );
  pStencilAttach->setClearStencil( 0 );
  pStencilAttach->setLoadAction( MTL::LoadActionClear );
  pStencilAttach->setStoreAction( MTL::StoreActionStore );

  MTL::RenderCommandEncoder* pEnc = pCmd->renderCommandEncoder(pRPD);
  
  // ADD WHAT YOU WANT TO DRAW
  pEnc->setDepthStencilState( _pStencilState );

  auto* pStencil = MTL::StencilDescriptor::alloc()->init();
  _pDepthStencilDesc->setFrontFaceStencil( pStencil );
  _pDepthStencilDesc->setBackFaceStencil( pStencil );

  { // draw orc
    pStencil->setStencilCompareFunction( MTL::CompareFunctionAlways ); // glStencilFunc(func, _, _);
    pEnc->setStencilReferenceValue( 1 ); // glStencilFunc(_, ref, _);
    pStencil->setWriteMask( 0xff ); // glStencilFunc(_, _, mask);
    pStencil->setStencilFailureOperation( MTL::StencilOperationReplace ); // glStencilOp( _, _, zpass );
    
    orc->update( delta );
    drawImage( orc->texture(), { 0.0f, 0.0f }, { 100.0f, 100.0f }, orc->frame() );
    drawFrame( pCmd, pEnc );
  }

  { // draw grass backgroud
    pStencil->setStencilCompareFunction( MTL::CompareFunctionEqual ); // glStencilFunc(func, _, _);
    pEnc->setStencilReferenceValue( 0 ); // glStencilFunc(_, ref, _);
    pStencil->setWriteMask( 0x00 ); // glStencilFunc(_, _, mask); 
    pStencil->setStencilFailureOperation( MTL::StencilOperationKeep ); // glStencilOp( _, _, zpass );

    _pCurrentTexture = _textureMap["Grass"]->_pTexture;
    fillRect( { 0.0f, 0.0f, _viewWidth / 2.0f, _viewHeight / 2.0f } );
    drawFrame( pCmd, pEnc );
  }

  pEnc->endEncoding();
  pCmd->presentDrawable(_pCurrentDrawableView->currentDrawable());
  pCmd->commit();

  pStencil->release();
  pPool->release();
}

drawFrame方法 #

void Renderer::drawFrame( MTL::CommandBuffer* pCmd, MTL::RenderCommandEncoder* pEnc )
{
  NS::AutoreleasePool* pPool = NS::AutoreleasePool::alloc()->init();

  pEnc->setRenderPipelineState(_pPSO);

  _pVertexBuffer = _pDevice->newBuffer(_vertices.data(), sizeof(shader_types::VertexData) * _vertices.size(), MTL::StorageModeManaged);
  _pIndexBuffer = _pDevice->newBuffer(_indices.data(), sizeof(uint) * _indices.size(), MTL::StorageModeManaged);

  pEnc->setVertexBuffer(_pVertexBuffer, /* offset */ 0, /* index */ 0);
  pEnc->setFragmentTexture(_pCurrentTexture, 0);

  pEnc->drawIndexedPrimitives(MTL::PrimitiveTypeTriangle,
                            _pIndexBuffer->length(), MTL::IndexTypeUInt16,
                            _pIndexBuffer, 0);

  // flush vectors
  _vertices.clear();
  _indices.clear();
  _indexBufferIndex = 0;

  pPool->release();
}

解决方案与修复点 #

1. 核心问题：Metal模板状态的不可变性 #

Metal的MTLDepthStencilState是不可变对象，一旦创建就无法修改。你在构造函数中提前创建了_pStencilState，之后修改_pDepthStencilDesc不会同步到已创建的状态对象，导致模板规则从未生效。

2. 模板操作映射错误 #

你混淆了OpenGL与Metal的模板操作对应关系：

• OpenGL的glStencilOp(fail, zfail, zpass)对应Metal的三个操作：
  • stencilFailureOperation：模板测试失败时的操作
  • depthFailureOperation：深度测试失败但模板测试通过时的操作
  • depthStencilPassOperation：两者都通过时的操作
• 你的场景没有深度测试，深度测试默认通过，所以需要设置depthStencilPassOperation来控制模板值的更新，而非stencilFailureOperation。

3. 修复后的代码 #

修改构造函数（延迟创建DepthStencilState）

Renderer::Renderer( MTL::Device* pDevice, MTK::View* pView )
{
  ...
  auto* pTexDesc = MTL::TextureDescriptor::alloc()->init();
  pTexDesc->setTextureType( MTL::TextureType2D );
  pTexDesc->setWidth( _viewWidth );
  pTexDesc->setHeight( _viewHeight );
  pTexDesc->setPixelFormat( MTL::PixelFormatStencil8 );
  pTexDesc->setStorageMode( MTL::StorageModePrivate );
  pTexDesc->setUsage( MTL::TextureUsageRenderTarget );
  _pStencilTexture = _pDevice->newTexture( pTexDesc );
  pTexDesc->release(); // 释放描述符避免内存泄漏

  _pDepthStencilDesc = MTL::DepthStencilDescriptor::alloc()->init();
  // 不提前创建_pStencilState，留到每帧按需创建
  ...
}

修改update方法（每次修改规则后重建状态）

void Renderer::update( float delta )
{
  NS::AutoreleasePool* pPool = NS::AutoreleasePool::alloc()->init();
  _pCurrentDrawableView->setClearColor( MTL::ClearColor::Make(1.0f, 1.0f, 1.0f, 1.0f) );

  MTL::CommandBuffer* pCmd = _pCommandQueue->commandBuffer();
  MTL::RenderPassDescriptor* pRPD = _pCurrentDrawableView->currentRenderPassDescriptor();

  auto* pStencilAttach = pRPD->stencilAttachment();
  pStencilAttach->setTexture( _pStencilTexture );
  pStencilAttach->setClearStencil( 0 );
  pStencilAttach->setLoadAction( MTL::LoadActionClear );
  pStencilAttach->setStoreAction( MTL::StoreActionStore );

  MTL::RenderCommandEncoder* pEnc = pCmd->renderCommandEncoder(pRPD);
  
  auto* pStencil = MTL::StencilDescriptor::alloc()->init();
  _pDepthStencilDesc->setFrontFaceStencil( pStencil );
  _pDepthStencilDesc->setBackFaceStencil( pStencil );

  { // draw orc
    // 配置模板规则：总是通过测试，通过时替换模板值为1
    pStencil->setStencilCompareFunction( MTL::CompareFunctionAlways );
    pStencil->setDepthStencilPassOperation( MTL::StencilOperationReplace ); // 关键：设置通过后的操作
    pStencil->setWriteMask( 0xff );
    
    // 重建DepthStencilState并设置到编码器
    if (_pStencilState) _pStencilState->release();
    _pStencilState = _pDevice->newDepthStencilState( _pDepthStencilDesc );
    pEnc->setDepthStencilState( _pStencilState );
    pEnc->setStencilReferenceValue( 1 );

    orc->update( delta );
    drawImage( orc->texture(), { 0.0f, 0.0f }, { 100.0f, 100.0f }, orc->frame() );
    drawFrame( pCmd, pEnc );
  }

  { // draw grass background
    // 配置模板规则：仅当模板值等于0时通过，不修改模板值
    pStencil->setStencilCompareFunction( MTL::CompareFunctionEqual );
    pStencil->setDepthStencilPassOperation( MTL::StencilOperationKeep );
    pStencil->setWriteMask( 0x00 );
    
    // 重建DepthStencilState并设置到编码器
    if (_pStencilState) _pStencilState->release();
    _pStencilState = _pDevice->newDepthStencilState( _pDepthStencilDesc );
    pEnc->setDepthStencilState( _pStencilState );
    pEnc->setStencilReferenceValue( 0 );

    _pCurrentTexture = _textureMap["Grass"]->_pTexture;
    fillRect( { 0.0f, 0.0f, _viewWidth / 2.0f, _viewHeight / 2.0f } );
    drawFrame( pCmd, pEnc );
  }

  pEnc->endEncoding();
  pCmd->presentDrawable(_pCurrentDrawableView->currentDrawable());
  pCmd->commit();

  pStencil->release();
  pPool->release();
}

额外注意事项 #

• 记得释放不再使用的Metal对象（如描述符、状态对象），避免内存泄漏。
• 2D渲染中如果涉及透明通道，需确保模板测试与混合的顺序符合需求。
• Metal的状态管理模式与OpenGL差异较大，所有渲染状态（如DepthStencilState、PipelineState）都是不可变的，修改规则必须重新创建对象。

内容的提问来源于stack exchange，提问作者Chris Redwood