函数|x|在0点的次梯度值是多少？Lasso Regression相关问询

Great question—this is a common point of confusion when diving into subgradient methods for L1-regularized models like Lasso! Let’s break this down clearly:

First, remember that for a convex function ( f(x) ), a subgradient ( g ) at a point ( x ) is any value that satisfies the inequality:

( f(y) \geq f(x) + g \cdot (y - x) ) for all ( y ) in the domain of ( f )

For ( f(x) = |x| ), let’s apply this definition at ( x=0 ):

• ( f(0) = 0 ), so the inequality simplifies to ( |y| \geq g \cdot y ) for all real ( y ).

Let’s test different cases for ( y ):

• When ( y > 0 ): ( |y| = y ), so ( y \geq g \cdot y ). Dividing both sides by ( y ) (positive, so inequality direction stays) gives ( g \leq 1 ).
• When ( y < 0 ): ( |y| = -y ), so ( -y \geq g \cdot y ). Dividing both sides by ( y ) (negative, so inequality direction flips) gives ( -1 \leq g ).
• When ( y = 0 ): The inequality becomes ( 0 \geq 0 ), which is always true, no constraint on ( g ) here.

Putting these together, all values ( g ) in the interval ([-1, 1]) are valid subgradients of ( |x| ) at ( x=0 ).

Practical Note for Lasso Regression #

In practice, when implementing subgradient descent for Lasso, you can choose any value within ([-1, 1]) when the weight ( w = 0 ). Most implementations pick ( g = 0 ) for simplicity—it’s a safe choice that works perfectly for convergence, since subgradient methods don’t require a unique subgradient at non-differentiable points.

内容的提问来源于stack exchange，提问作者Surya Prakash Reddy