如何去除正态数据中的均匀噪声？平面多二元高斯分布点去噪问询

This is a classic mixed-signal problem where most points form dense clusters (unknown number of bivariate normals) and noise spreads evenly across the bounded rectangle. Below are solutions, starting with those that don’t require estimating noise count:

Preferred: No Noise Count Estimation Needed #

DBSCAN Clustering #

DBSCAN is tailor-made for this scenario—it finds dense clusters and automatically labels sparse points as noise, with no need to know the number of clusters or noise points. Here’s how to use it:

1. Choose two key parameters:
   • eps: The radius of the neighborhood around each point. To pick this, plot the k-distance graph (distance to the k-th nearest neighbor for all points) and select the "elbow" where distances jump sharply (try k=4 for 2D data).
   • min_samples: The minimum number of points needed in the eps neighborhood to form a cluster. A rule of thumb is 2 * dimension (so 4 for 2D) or higher if noise is more prevalent.
2. Run DBSCAN—all points labeled as "noise" by the algorithm are your uniform noise points to remove. This works even if clusters have different sizes or shapes.

2D Kernel Density Estimation (KDE) Thresholding #

1. Compute a 2D KDE of your data. Normal clusters will appear as high-density peaks, while noise forms a low, flat background.
2. Use an automated threshold method like Otsu's threshold (originally for image segmentation) to split density values into high (signal) and low (noise) groups. Alternatively, plot KDE values and pick a threshold that clearly separates peak densities from the background.
3. Discard all points where the KDE value falls below the threshold. This is intuitive and requires no prior counts.

Robust Mixture Model with Uniform Noise Component #

Fit a statistical model combining n bivariate normal components plus a uniform component:

1. Use model selection (like BIC) to find the optimal number of normal clusters (n)—fit models with n=1,2,... and pick the one with the lowest BIC.
2. Once trained, each point gets a probability of belonging to the uniform noise component. Mark points with a noise probability above a threshold (e.g., 0.5) as noise. The model automatically estimates the noise fraction, so no prior count is needed.

When Noise Count is Known #

If you have an estimate of how many noise points exist (say, k):

Top-Density Filtering #

1. Calculate the density for each point (using KDE or k-nearest neighbor density: count of points within a fixed radius).
2. Sort all points by density in descending order. Keep the top total_points - k points—discard the bottom k (lowest density) points, which are almost certainly noise.

GMM with Fixed Noise Weight #

If you know the approximate noise fraction (k/total_points), fix the weight of the uniform component in your mixture model to that value. This stabilizes the fitting process, especially when noise is a small fraction of the data.

内容的提问来源于stack exchange，提问作者Jay