为Faster-RCNN生成混淆矩阵时遭遇Shapely维度错误求助

我正尝试为Faster-RCNN目标检测模型生成混淆矩阵，先用简单数组测试时遇到错误，不清楚正确的输入格式。

错误信息 #

Traceback (most recent call last):
  File "c:\Users\lemon\Desktop\ap_py_2\py_deneme.py", line 9, in <module>
    print(evaluation(grounds,preds,0.5))
  File "c:\Users\lemon\Desktop\ap_py_2\confusion_matrix.py", line 30, in evaluation
    f1=shapely.geometry.Polygon(f1)
  File "C:\Users\lemon\miniconda3\envs\cnn-env-03\lib\site-packages\shapely\geometry\polygon.py", line 229, in __new__
    shell = LinearRing(shell)
  File "C:\Users\lemon\miniconda3\envs\cnn-env-03\lib\site-packages\shapely\geometry\polygon.py", line 103, in __new__
    geom = shapely.linearrings(coordinates)
  File "C:\Users\lemon\miniconda3\envs\cnn-env-03\lib\site-packages\shapely\decorators.py", line 77, in wrapped
    return func(*args, **kwargs)
  File "C:\Users\lemon\miniconda3\envs\cnn-env-03\lib\site-packages\shapely\creation.py", line 173, in linearrings
    return lib.linearrings(coords, out=out, **kwargs)
ValueError: The ordinate (last) dimension should be 2 or 3, got 4

测试代码（test.py） #

from confusion_matrix import evaluation
import torch
import numpy as np

pred = [[13,24,25,46], [13,24,25,46], [13,24,25,46]]
ground = [[13,24,25,46],[13,24,25,46],[13,24,25,46]]
preds = np.array(pred)
grounds = np.array(ground)
print(evaluation(grounds,preds,0.5))

混淆矩阵代码（confusion_matrix.py） #

#!/usr/bin/env python
# coding: utf-8

import numpy as np
from shapely.geometry import Polygon,Point
import matplotlib.pyplot as plt
import shapely
import cv2 as cv
import os
import gc


def evaluation(ground,pred,iou_value):
  """
  ground= array of ground-truth contours.
  preds = array of predicted contours.
  iou_value= iou treshold for TP and otherwise.
  """
  truth=np.squeeze(ground)
  preds=np.squeeze(pred)
  #we will use this function to check iou less than threshold
  def CheckLess(list1,val):
    return(all(x<=val for x in list1))

  # Using predicted output as the reference
  prob1=[]
  for i in range(len(preds)):
      f1=np.expand_dims(preds[i], axis=0)
      # define a Shapely polygone for prediction i
      f1=shapely.geometry.Polygon(f1)
      # determine the radius
      f1_radius=np.sqrt((f1.area)/np.pi)
      #buffer the polygon fromt the centroid
      f1_buffered=shapely.geometry.Point(f1.centroid).buffer(f1_radius*500)
      cont=[]
      for i in range(len(truth)):
        ff=shapely.geometry.Polygon(np.squeeze(truth[i]))
        if f1_buffered.contains(ff)== True:
          iou=(ff.intersection(f1).area)/(ff.union(f1).area)  
       
          cont.append((iou))

      prob1.append(cont)

  fp=0

  for t in prob1:
    if CheckLess(t,iou_value)==True:
      fp=fp+1
    
  prob2=[]
  #loop through each groun truth instance 
  for i in range(len(truth)):
      f1=truth[i]
      f1=shapely.geometry.Polygon(f1)
      #find radius
      f1_radius=np.sqrt((f1.area)/np.pi)
      #buffer the polygon from the centroid
      f1_buffered=shapely.geometry.Point(f1.centroid).buffer(f1_radius*500)
      cont=[]
      # merge up the ground truth instance against prediction
      # to determine the IoU
      for i in range(len(preds)):
        ff=shapely.geometry.Polygon(np.squeeze(preds[i]))
        if f1_buffered.contains(ff)== True:
          #calculate IoU
          iou=(ff.intersection(f1).area)/(ff.union(f1).area)
          cont.append((iou))
      # probability of a given prediction to be contained in a
      # ground truth instance
      prob2.append(cont)
  fn=0
  tp=0
  for t in prob2:
    if np.sum(t)==0:
      fn=fn+1
    elif CheckLess(t,iou_value)==False:
      tp=tp+1
  
  #lets add this section just to print the results
  print("TP:",tp,"\t FP:",fp,"\t FN:",fn,"\t GT:",truth.shape[0])
  precision=round(tp/(tp+fp),3) 
  recall=round(tp/(tp+fn),3)
  f1= round(2*((precision*recall)/(precision+recall)),3)
  print("Precall:",precision,"\t Recall:",recall, "\t F1 score:",f1)
  
  return tp,fp,fn,precision,recall,f1

错误根源是传入的边界框格式不符合shapely.Polygon的要求。当前用的[x1,y1,x2,y2]是矩形的对角坐标，但Polygon需要的是多边形顶点的坐标序列，也就是矩形四个角点的(x,y)对。

比如，[13,24,25,46]代表左上角(13,24)、右下角(25,46)的矩形，需要转换成四个顶点的数组：

[[13,24], [25,24], [25,46], [13,46]]

修改后的测试代码如下：

from confusion_matrix import evaluation
import numpy as np

# 将[x1,y1,x2,y2]转换为矩形四个顶点的坐标序列
def bbox_to_polygon(bbox):
    x1, y1, x2, y2 = bbox
    return [[x1, y1], [x2, y1], [x2, y2], [x1, y2]]

pred_bboxes = [[13,24,25,46], [13,24,25,46], [13,24,25,46]]
ground_bboxes = [[13,24,25,46],[13,24,25,46],[13,24,25,46]]

# 转换为Polygon需要的格式
preds = np.array([bbox_to_polygon(bbox) for bbox in pred_bboxes])
grounds = np.array([bbox_to_polygon(bbox) for bbox in ground_bboxes])

print(evaluation(grounds,preds,0.5))

修改后输入符合shapely.Polygon的要求，能正常计算IoU和混淆矩阵相关指标。

补充：该混淆矩阵代码针对任意多边形轮廓设计，而Faster-RCNN输出的是矩形边界框，所以必须做上述转换才能适配。后续处理模型输出时，记得把每个预测/真实的[x1,y1,x2,y2]都转换成四个顶点的坐标序列。

内容的提问来源于stack exchange，提问作者lawlicat17