如何消除图像unpatchify后出现的分块边界

将尺寸为1024×1024×3的原始图像通过patchify分割为128×128×3的图像块，完成相关操作后执行unpatchify拼接图像，但拼接后的图像出现明显的分块边界，以下是几种可行的消除方法：

一、重叠分块+加权融合 #

无重叠分块时，模型对每个块的边缘处理易出现不一致，导致拼接边界明显。改用重叠分块并对重叠区域加权融合，能实现边缘平滑过渡。

修改prediction函数如下：

def prediction(img, model):
    img_h, img_w, channel_count = img.shape
    patch_size = 128
    overlap = 32  # 设置重叠像素数，可根据效果调整
    # 重叠分块，步长=块尺寸-重叠尺寸
    img_patches = patchify(img, (patch_size, patch_size, 3), step=patch_size - overlap)
    rows, cols = img_patches.shape[0], img_patches.shape[1]
    
    # 逐个预测图像块
    pred_patches = []
    for i in range(rows):
        for j in range(cols):
            patch = img_patches[i][j][0]
            pred = model.predict(np.expand_dims(patch, axis=0), verbose=0)[0]
            pred_patches.append(pred)
    pred_patches = np.array(pred_patches).reshape(rows, cols, 1, patch_size, patch_size, 3)
    
    # 创建线性权重掩码，边缘区域权重低，中心区域权重高
    def create_weight_mask(size, overlap):
        mask = np.ones((size, size))
        # 左右边缘线性加权
        for i in range(overlap):
            weight = i / overlap
            mask[:, i] = weight
            mask[:, size - 1 - i] = weight
        # 上下边缘线性加权（取与左右边缘的最小权重，避免角落权重过低）
        for i in range(overlap):
            weight = i / overlap
            mask[i, :] = np.minimum(mask[i, :], weight)
            mask[size - 1 - i, :] = np.minimum(mask[size - 1 - i, :], weight)
        return mask
    
    weight_mask = create_weight_mask(patch_size, overlap)
    weight_mask = np.expand_dims(np.expand_dims(weight_mask, axis=-1), axis=0)  # 适配张量维度
    
    # 对预测块应用权重
    weighted_patches = pred_patches * weight_mask
    
    # 拼接加权后的块，并除以总权重得到最终图像
    total_weight = unpatchify(np.ones_like(pred_patches) * weight_mask, img.shape)
    pred_img = unpatchify(weighted_patches, img.shape) / total_weight
    
    return pred_img

二、后处理平滑边界 #

若不想修改分块逻辑，可直接对拼接后的图像做局部边界平滑处理，用高斯模糊或双边滤波弱化边界差异：

def smooth_boundaries(img, patch_size=128, blur_kernel=(5,5)):
    img_h, img_w = img.shape[:2]
    # 处理水平分块边界
    for i in range(1, img_h // patch_size):
        y = i * patch_size
        # 对边界上下各10像素区域做高斯模糊
        img[y-10:y+10, :] = cv2.GaussianBlur(img[y-10:y+10, :], blur_kernel, 0)
    # 处理垂直分块边界
    for j in range(1, img_w // patch_size):
        x = j * patch_size
        # 对边界左右各10像素区域做高斯模糊
        img[:, x-10:x+10] = cv2.GaussianBlur(img[:, x-10:x+10], blur_kernel, 0)
    return img

# 在预测流程中调用
predicted = normalize(p_img)
predicted = smooth_boundaries(predicted)

三、训练阶段加入边缘一致性约束 #

从根源优化模型，在训练时加入边缘一致性损失，让模型学习保持块之间的边缘连续性：

# 定义边缘一致性损失函数
def edge_consistency_loss(y_true_patches, y_pred_patches):
    # 获取每个块的右侧边缘和下侧边缘
    right_edge_true = y_true_patches[:, :, -1:, :]
    right_edge_pred = y_pred_patches[:, :, -1:, :]
    bottom_edge_true = y_true_patches[:, -1:, :, :]
    bottom_edge_pred = y_pred_patches[:, -1:, :, :]
    
    # 计算边缘的均方误差损失
    loss = tf.reduce_mean(tf.square(right_edge_true - right_edge_pred)) + \
           tf.reduce_mean(tf.square(bottom_edge_true - bottom_edge_pred))
    return loss

# 训练时将该损失与主重建损失结合（权重可调整）
total_loss = reconstruction_loss + 0.1 * edge_consistency_loss(y_true, y_pred)

内容的提问来源于stack exchange，提问作者Sumana