如何在NextJS 13客户端组件中创建图像蒙版实现AI图像编辑？

核心实现思路 #

借助HTML5 Canvas实现画笔绘制蒙版，通过客户端组件处理DOM交互与状态管理，最后将蒙版数据、原始图像及编辑提示词传给Replicate的inpainting模型完成选择性编辑。

步骤1：创建客户端组件 #

NextJS 13中需在组件顶部添加'use client'指令，启用客户端交互能力：

'use client';

import { useState, useRef, useEffect } from 'react';
import replicate from '@/lib/replicate'; // 提前配置好Replicate客户端实例

步骤2：实现Canvas画笔绘制功能 #

通过Canvas元素绑定鼠标/触摸事件，完成蒙版绘制逻辑：

1. 初始化Canvas与原始图像同尺寸，避免拉伸
2. 处理鼠标按下、移动、松开事件，绘制连续路径
3. 支持调整画笔大小，蒙版区域用白色绘制（符合多数inpainting模型要求）

export default function ImageInpainter({ initialImageUrl }: { initialImageUrl: string }) {
  const canvasRef = useRef<HTMLCanvasElement>(null);
  const [isDrawing, setIsDrawing] = useState(false);
  const [brushSize, setBrushSize] = useState(20);
  const [maskBlob, setMaskBlob] = useState<Blob | null>(null);
  const imageRef = useRef<HTMLImageElement>(new Image());

  // 初始化Canvas尺寸与原始图像一致
  useEffect(() => {
    const img = imageRef.current;
    img.src = initialImageUrl;
    img.onload = () => {
      const canvas = canvasRef.current;
      if (canvas) {
        canvas.width = img.width;
        canvas.height = img.height;
        const ctx = canvas.getContext('2d');
        if (ctx) {
          ctx.fillStyle = 'transparent';
          ctx.fillRect(0, 0, canvas.width, canvas.height);
          ctx.strokeStyle = '#ffffff';
          ctx.lineWidth = brushSize;
          ctx.lineCap = 'round';
        }
      }
    };
  }, [initialImageUrl, brushSize]);

  // 鼠标按下开始绘制
  const startDrawing = (e: React.MouseEvent<HTMLCanvasElement>) => {
    const canvas = canvasRef.current;
    if (!canvas) return;
    const ctx = canvas.getContext('2d');
    if (!ctx) return;
    setIsDrawing(true);
    const rect = canvas.getBoundingClientRect();
    const x = e.clientX - rect.left;
    const y = e.clientY - rect.top;
    ctx.beginPath();
    ctx.moveTo(x, y);
  };

  // 鼠标移动绘制路径
  const draw = (e: React.MouseEvent<HTMLCanvasElement>) => {
    if (!isDrawing) return;
    const canvas = canvasRef.current;
    if (!canvas) return;
    const ctx = canvas.getContext('2d');
    if (!ctx) return;
    const rect = canvas.getBoundingClientRect();
    const x = e.clientX - rect.left;
    const y = e.clientY - rect.top;
    ctx.lineTo(x, y);
    ctx.stroke();
  };

  // 鼠标停止绘制，导出蒙版Blob
  const stopDrawing = () => {
    setIsDrawing(false);
    const canvas = canvasRef.current;
    if (!canvas) return;
    canvas.toBlob((blob) => {
      if (blob) setMaskBlob(blob);
    }, 'image/png');
  };

  // 清除蒙版
  const clearMask = () => {
    const canvas = canvasRef.current;
    if (!canvas) return;
    const ctx = canvas.getContext('2d');
    if (!ctx) return;
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    setMaskBlob(null);
  };

步骤3：对接Replicate Inpainting接口 #

将原始图像、蒙版Blob及编辑提示词传给Replicate模型，处理生成结果：

const handleInpaint = async (prompt: string) => {
    if (!maskBlob || !initialImageUrl) return;

    // 将网络图像转为Blob
    const imageResponse = await fetch(initialImageUrl);
    const imageBlob = await imageResponse.blob();

    // 调用Replicate inpainting模型
    const output = await replicate.run(
      'stability-ai/stable-diffusion-inpainting:27c427d44f274f4899763719a467e40046570c7951bd93d5904d647e47065482',
      {
        input: {
          image: imageBlob,
          mask: maskBlob,
          prompt: prompt,
          width: imageRef.current.width,
          height: imageRef.current.height,
        },
      }
    );

    // 处理生成结果，例如更新显示图像
    console.log('编辑结果:', output);
  };

步骤4：渲染UI组件 #

将Canvas、画笔控制、操作按钮及输入框整合到UI中：

return (
    <div className="relative">
      {/* 原始图像作为背景 */}
      <img
        src={initialImageUrl}
        alt="原始图像"
        className="absolute top-0 left-0 w-full h-full object-contain"
      />
      {/* Canvas用于绘制蒙版 */}
      <canvas
        ref={canvasRef}
        className="relative z-10 cursor-crosshair"
        onMouseDown={startDrawing}
        onMouseMove={draw}
        onMouseUp={stopDrawing}
        onMouseLeave={stopDrawing}
      />
      {/* 控制面板 */}
      <div className="mt-4 flex gap-4 items-center">
        <input
          type="range"
          min="5"
          max="50"
          value={brushSize}
          onChange={(e) => setBrushSize(Number(e.target.value))}
          className="w-48"
        />
        <span>画笔大小: {brushSize}</span>
        <button onClick={clearMask} className="px-4 py-2 bg-red-500 text-white">清除蒙版</button>
      </div>
      <div className="mt-4">
        <input
          type="text"
          placeholder="输入编辑提示词..."
          className="w-full p-2 border"
          onKeyDown={(e) => {
            if (e.key === 'Enter') handleInpaint(e.currentTarget.value);
          }}
        />
        <button
          onClick={() => handleInpaint('输入的提示词')}
          className="mt-2 px-6 py-2 bg-blue-500 text-white"
        >
          开始编辑
        </button>
      </div>
    </div>
  );
}

关键注意事项 #

• 确保Canvas与原始图像尺寸完全匹配，避免蒙版错位
• 移动端需额外处理触摸事件（touchstart/touchmove/touchend），转换触摸坐标到Canvas坐标系
• 蒙版需以白色标记编辑区域、透明背景，符合多数inpainting模型的输入要求
• 处理图像跨域问题：若原始图像来自外部域名，需配置CORS或下载到本地转为Blob后再传入模型

内容的提问来源于stack exchange，提问作者HOSENUR