如何用OpenCV将带透明背景的Matplotlib图表叠加到图像上？

我尝试使用OpenCV将Matplotlib图表叠加到图像的指定位置，但叠加层始终带有黑色或白色背景，导致叠加后的背景图像比原图偏暗或偏亮。由于需要对视频的每一帧执行此操作，因此解决方案必须具备高效性。

方法一：将图表放在透明图像后叠加到背景 #

import cv2
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure

# 保存带透明背景的图表为PNG
fig = Figure(figsize=(5, 4), dpi=100)
canvas = FigureCanvasAgg(fig)

ax = fig.add_subplot(111)
ax.plot([1, 2, 3])

canvas.draw()
fig.savefig("Test", transparent=True)

graph = cv2.imread('Test.png')

# 为图表添加alpha通道并设为0
graph = cv2.cvtColor(graph, cv2.COLOR_RGB2RGBA)
graph[:, :, 3] = 0

# 读取背景图像
background = cv2.imread('hill-966202_960_720.jpg')
background = cv2.cvtColor(background, cv2.COLOR_RGB2RGBA)

# 创建与背景尺寸相同的透明图像
background_y, background_x, background_channel = background.shape
transparent_image = np.full((background_y, background_x, background_channel), [255, 255, 255, 0])

# 将图表放置到透明图像的指定位置
y, x = 80, 123
overlay_y, overlay_x, _ = graph.shape
transparent_image[y:(y + overlay_y), x:(x + overlay_x)] = graph[:, :, :]

transparent_image = transparent_image.astype(np.uint8)
added_image = cv2.addWeighted(transparent_image, 0.4, background, 0.6, 0)

cv2.imshow("TEST", added_image)
cv2.waitKey(0)

结果：叠加层带有白色背景，与原始背景图像对比差异明显

• 叠加后效果：[白色背景残留的叠加图像]
• 原始背景：[原始背景图像]

方法二：将图表叠加到背景副本后与原图混合 #

import cv2
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure

# 保存带透明背景的图表为PNG
fig = Figure(figsize=(5, 4), dpi=100)
canvas = FigureCanvasAgg(fig)

ax = fig.add_subplot(111)
ax.plot([1, 2, 3])

canvas.draw()
fig.savefig("Test", transparent=True)

graph = cv2.imread('Test.png')

# 为图表添加alpha通道并设为0
graph = cv2.cvtColor(graph, cv2.COLOR_RGB2RGBA)
graph[:, :, 3] = 0

# 读取背景图像
background = cv2.imread('hill-966202_960_720.jpg')
background = cv2.cvtColor(background, cv2.COLOR_RGB2RGBA)
background_copy = background.copy()

# 将图表放置到背景副本的指定位置
y, x = 80, 123
overlay_y, overlay_x, _ = graph.shape
background_copy[y:(y + overlay_y), x:(x + overlay_x)] = graph[:, :, :]

added_image = cv2.addWeighted(background_copy, 0.4, background, 0.6, 0)

cv2.imshow("TEST", added_image)
cv2.waitKey(0)

结果：图表周围仍带有背景残留

• 叠加后效果：[图表周围背景残留的图像]

方法三：逐像素去除图表背景 #

import cv2
import numpy
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure
import time

fig = Figure(figsize=(5, 4), dpi=100)
canvas = FigureCanvasAgg(fig)

ax = fig.add_subplot(111)
ax.plot([1, 2, 3])

canvas.draw()
buf = canvas.buffer_rgba()

graph_as_array = np.asarray(buf)

background = cv2.imread('hill-966202_960_720.jpg')
background = cv2.cvtColor(background, cv2.COLOR_RGB2RGBA)
background_copy = background.copy()

y, x = 80, 123

overlay_y, overlay_x, _ = graph_as_array.shape

background_copy[y:(y + overlay_y), x:(x + overlay_x)] = graph_as_array[:, :, :]

for height in range(y, y + overlay_y):
    for width in range(x, x + overlay_x):
        pixel = background_copy[height][width]
        if pixel[0] == 255 and pixel[1] == 255 and pixel[2] == 255:
            background_copy[height][width] = background[height][width]

cv2.imshow("TEST", background_copy)
cv2.waitKey(0)

结果：叠加效果良好，但逐像素遍历速度过慢，无法满足视频逐帧处理的需求

• 叠加后效果：[无背景残留的图像]

内容的提问来源于stack exchange，提问作者PaulE