JavaScript实现Gradient Descent线性回归：回归线位置异常问题排查

Looks like you're running into two key issues with your gradient descent implementation that are causing the regression line to misalign and diverge over time. Let's break down the problems and fix them step by step.

1. Critical Gradient Descent Implementation Error #

Your current GradientDescend function is updating theta values incrementally for each training sample, which mixes up batch gradient descent logic (where we accumulate gradients across all samples before updating theta) with incorrect in-place updates. This causes the model to use partially updated theta values during the same iteration, leading to unstable parameter updates and eventual divergence.

The Fix: #

We need to:

• Create a temporary copy of theta to store new values (so we don't overwrite the original theta while calculating gradients)
• Accumulate the gradient sum across all samples for each theta parameter
• Update all theta values at once after processing all samples

Here's the corrected function:

function GradientDescend(X,Y,theta,alpha){
    let m = Y.length;
    // Create a temporary array to hold new theta values
    let newTheta = [...theta];
    // Calculate gradients for each parameter
    for(let j = 0; j < theta.length; j++){
        let gradientSum = 0;
        for(let i = 0; i < m; i++){
            gradientSum += (h(X[i], theta) - Y[i]) * X[i][j];
        }
        // Update theta using the accumulated gradient
        newTheta[j] -= alpha * (1/m) * gradientSum;
    }
    return newTheta;
}

2. Incorrect Regression Line Coordinate Conversion #

Your regLine function is mixing up pixel coordinates with your original data scale, which causes the line to be drawn in the wrong position. Since your training data uses x values between 0 and 11 (from n=11), you need to convert back and forth between pixel positions and your data's scale correctly.

The Fix: #

Update the regLine function to map the data-scale predictions to canvas pixels properly:

function regLine(){
    let theta0 = theta[0];
    let theta1 = theta[1];
    // Get data-scale x values for the start and end of the canvas
    let xDataStart = 0;
    let xDataEnd = n;
    // Calculate predicted y values in data scale
    let yDataStart = theta0 + theta1 * xDataStart;
    let yDataEnd = theta0 + theta1 * xDataEnd;
    // Convert to canvas pixels (y is flipped since canvas y starts at top)
    let x1 = xDataStart * width / n;
    let y1 = height - yDataStart * height / n;
    let x2 = xDataEnd * width / n;
    let y2 = height - yDataEnd * height / n;
    line(x1,y1,x2,y2);
}

3. Optional: Adjust Learning Rate to Prevent Divergence #

Your current alpha=0.01 might be too large for your data scale (x values up to 11). If the line still diverges after fixing the above issues, try reducing the learning rate to 0.001 or even 0.0005. For better stability, you could also normalize your input features (e.g., scale x values to the 0-1 range) which allows using larger learning rates safely.

Feature Normalization Example (add to your click handler): #

document.addEventListener("click",function(e){
    let xx = (e.clientX/width)*n;
    // Normalize x to 0-1 range
    let normalizedX = xx / n;
    let yy = (1-e.clientY/height)*n;
    data.push({x:xx,y:yy});
    // Use normalized x in X matrix
    X.push([1, normalizedX]);
    Y.push(yy);
});

If you use normalization, you'll need to adjust the regLine function to use normalized x values when calculating predictions, then convert back to the original scale for drawing.

After applying these fixes, your regression line should properly converge to fit the data points instead of diverging.

内容的提问来源于stack exchange，提问作者Giuseppe Romeo