Python实现sigmoid函数遇RuntimeWarning溢出错误求助

Got it, let's tackle this sigmoid overflow issue you're facing! That RuntimeWarning: overflow encountered in exp pops up because when your value is a very large negative number (like -1000), -value becomes a huge positive number—and np.exp() can't handle that, since it exceeds the maximum value standard floating-point numbers can represent.

Your attempt to use np.float128 didn't work because even a wider float type can't save you from extremely large exponents forever. And np.float256 doesn't exist in standard NumPy builds, which is why you hit that AttributeError.

Here are two solid, practical solutions to fix this:

1. Use an Equivalent Formula to Avoid Overflow #

The sigmoid function has a mathematical equivalent form that lets us skip calculating those huge exponentials. We can split the logic based on whether value is positive or negative:

• When value >= 0: Stick with your original formula—np.exp(-value) will be a small number, no overflow risk.
• When value < 0: Use the alternative form np.exp(value)/(1 + np.exp(value))—here, np.exp(value) is a tiny decimal (since value is negative), so no overflow at all.

Here's the updated code that works for both single values and NumPy arrays:

import numpy as np

def sigmoid(self, value):
    if isinstance(value, np.ndarray):
        result = np.empty_like(value, dtype=np.float64)
        # Handle positive values with original formula
        positive_mask = value >= 0
        result[positive_mask] = 1.0 / (1.0 + np.exp(-value[positive_mask]))
        # Handle negative values with the safe alternative
        negative_mask = value < 0
        exp_val = np.exp(value[negative_mask])
        result[negative_mask] = exp_val / (1.0 + exp_val)
        return result
    else:
        # For single scalar values
        if value >= 0:
            return 1.0 / (1.0 + np.exp(-value))
        else:
            exp_val = np.exp(value)
            return exp_val / (1.0 + exp_val)

This approach eliminates the overflow warning entirely because we never ask np.exp() to compute an astronomically large number.

2. Use a Pre-built Optimized Function (If SciPy Is an Option) #

If you're okay with adding SciPy as a dependency, the scipy.special.expit function is a pre-built sigmoid that already handles overflow and underflow cases internally. It's tested, optimized, and saves you from writing custom logic:

from scipy.special import expit

def sigmoid(self, value):
    return expit(value)

This will give you correct sigmoid values without any warnings.

Why Your Previous Fixes Didn't Work #

• np.float128 only extends the range of floating-point numbers slightly—np.exp(1000) is still way beyond what even float128 can handle (it would take an exponent of ~16384 to overflow float128). This doesn't fix the root problem.
• np.float256 isn't supported in standard NumPy distributions, which is why you got the AttributeError.

内容的提问来源于stack exchange，提问作者user9155773