实践中函数一/二阶导数近似的h值选择及双精度误差估算

Great question—let’s break this down step by step, since choosing the right step size for numerical differentiation boils down to balancing two critical error sources: truncation error (from approximating the Taylor series) and round-off error (from floating-point arithmetic’s limited precision).

1. Do we need to adjust h for second-derivative approximations? #

Absolutely. The h ≈ sqrt(ε) rule works perfectly for first derivatives using the forward difference [f(x+h)-f(x)]/h because it balances the truncation error (O(h)) and round-off error (O(ε/h)). But second-derivative approximations have a different error profile, so the optimal h changes.

Take the standard forward difference formula for the second derivative:

f''(x) ≈ [f(x+2h) - 2f(x+h) + f(x)] / h²

Let’s unpack its error components:

• Truncation error: From the Taylor expansion, the leading error term is proportional to h (specifically f'''(x)*h), so this is O(h).
• Round-off error: Each evaluation of f introduces an error of roughly ε*|f(x)| (assuming f(x) isn’t near zero). The numerator has three terms, so the total round-off error in the numerator is O(ε). Dividing by h² scales this up to O(ε/h²).

To minimize the total error (O(h) + O(ε/h²)), we take the derivative of the error with respect to h and set it to zero. Solving for h gives the optimal step size:

h ≈ ε^(1/3)

This is a big shift from the first-derivative case, so you definitely need to adjust h when computing second derivatives.

2. Estimating approximation error in double-precision #

First, remember that double-precision floating-point has a machine epsilon ε ≈ 2.2e-16 (thanks to its 52-bit mantissa).

Optimal h calculation #

Plug ε into the second-derivative optimal step size formula:

h_opt ≈ (2.2e-16)^(1/3) ≈ 6e-6

(For a more precise calculation: (2.2)^(1/3) ≈ 1.3, (1e-16)^(1/3) ≈ 4.6e-6, so multiplying gives ~6e-6.)

Minimum error estimate #

Substitute h_opt back into the total error expression. The minimum total error scales with ε^(1/3), which for double-precision is:

error_min ≈ (2.2e-16)^(1/3) ≈ 6e-6

Keep in mind this is a ballpark figure—the exact error depends on the third derivative of f (from truncation error) and the magnitude of f (from round-off). In practice, you’ll see errors in the range of 1e-5 to 1e-6 when using the optimal h.

Practical error verification #

If you have an analytical expression for f''(x), you can directly compute the absolute difference between the numerical approximation and the true value. If you don’t, run a convergence test: compute the approximation for several h values around h_opt. You’ll notice:

• When h is larger than h_opt, truncation error dominates—error increases as h grows.
• When h is smaller than h_opt, round-off error dominates—error increases as h shrinks.

内容的提问来源于stack exchange，提问作者oldselflearner1959