关于优化ID3多标签决策树MNIST手写数字分类性能的技术问询

Hey there! Let's tackle this ID3 optimization for MNIST together—great job getting that 11% error rate with a pixel threshold of 0, that's a solid starting point. Here are some actionable ideas to refine your feature set and bring that error rate down:

Refine Pixel-Based Attributes #

• Local block statistics instead of global thresholds
  Instead of treating each pixel as a binary attribute (0 vs non-0), split the 28x28 image into smaller blocks (like 4x4 or 7x7) and create attributes based on stats within each block:
  • Average pixel intensity in the block
  • Count of pixels above the block's mean threshold
  • Contrast between the block and its adjacent blocks
    In Java, you can implement this with simple nested loops to iterate over the image array, calculate these stats with basic arithmetic—no fancy libraries required. This captures spatial context that individual pixels miss, helping ID3 distinguish digits like 8 (two enclosed loops) from 0 (one loop).

Implement Shape-Based Features (Curves & Lines) #

You don't need a heavy computer vision library to add shape hints. Here's how to code these in Java:

• Basic edge detection
  Use a 3x3 Sobel filter (easy to code manually). For each pixel, compute gradient values in x and y directions using the Sobel kernels, then check if the gradient magnitude exceeds a threshold (this flags edges). Create attributes like "number of horizontal edges in the top half" or "total edge count in the center region".
• Loop & line detection
  • For looped digits (0,6,8,9), count enclosed regions using a simple flood fill (BFS or DFS) on the thresholded binary image. Subtract the outer background component to get the number of inner loops.
  • For line-focused digits (1,7), count horizontal/vertical runs of foreground pixels longer than a set length (like 5 pixels).
    Both use basic array traversal and queue/recursion logic—totally doable in Java.

Add Projection Profile Features #

These capture the overall shape of digits with minimal code:

• Compute horizontal projection: Sum the number of foreground pixels in each row.
• Compute vertical projection: Sum the number of foreground pixels in each column.
  Turn these into attributes like:
  • "Row with maximum foreground pixels is in the top third of the image"
  • "Number of columns with more than 5 foreground pixels"
  • "Difference between left and right half vertical projection sums"
    For example, this helps ID3 tell apart 1 (most pixels in the middle column) from 7 (strong top-right horizontal projection).

Optimize Attribute Selection #

ID3 can struggle with too many noisy attributes. Try:

• Using information gain ratio instead of raw information gain to prioritize features that are discriminative but not overly specific (avoids overfitting to rare pixel patterns).
• Running a chi-squared test to filter out features that don't correlate strongly with digit labels—this lets ID3 focus on the most impactful attributes.

Start with one or two of these (like projection features and local block stats) for quick wins, then iterate by adding shape-based features once those are working. You’ll likely see a noticeable drop in error rate once you capture more spatial and shape context beyond individual pixels.

内容的提问来源于stack exchange，提问作者Orr Levy