基于KM方法的时序数据脉冲型人工异常值生成技术咨询

Hey Mario, let's work through your problem—this is such a common challenge in anomaly detection when you don't have access to the original paper's dataset, so I'm glad you're thinking through these approaches. Let's break down solutions for each part of your question:

1. Integrating Your Generated Impulse Sequence into Time Series Data #

Your current code generates clean unit impulses, but the key is to merge these with your existing (or ideal) time series. There are two straightforward methods, depending on the type of impulse anomaly you need:

Method 1: Direct Addition (Best for Instant Up/Down Pulses) #

This is the simplest and most common approach for sharp, instantaneous pulse outliers—perfect for matching the "up/down pulse" requirement from the paper. Here's how to adapt your code:

import scipy.signal as signal
import matplotlib.pyplot as plt
import numpy as np

# Step 1: Generate your base time series (replace this with your actual data)
# Example: Simulate a periodic signal with minor noise to mimic real data
time_steps = np.arange(200)
base_signal = np.sin(2 * np.pi * time_steps / 20) + np.random.normal(0, 0.1, 200)  # 10-cycle period

# Step 2: Generate impulse outliers (adjust amplitude for up/down pulses)
# Positive impulses = upward anomalies, negative values = downward anomalies
impulse_amplitude = 2.5  # Tweak this to match the paper's plot scale
imp = signal.unit_impulse(200, [10, 50, 60]) * impulse_amplitude
# Uncomment below to add downward pulses if needed:
# down_imp = signal.unit_impulse(200, [30, 70]) * (-impulse_amplitude)
# imp += down_imp

# Step 3: Merge impulses with the base signal
anomalous_signal = base_signal + imp

# Visualize the results
fig, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(8, 10))
ax1.plot(time_steps, base_signal)
ax1.set_title('Original Normal Time Series')
ax2.plot(time_steps, imp)
ax2.set_title('Impulse Outliers (Upward)')
ax3.plot(time_steps, anomalous_signal)
ax3.scatter([10,50,60], anomalous_signal[[10,50,60]], color='red', label='Anomalies')
ax3.set_title('Time Series with Impulse Anomalies')
ax3.set_xlabel('Cycles')
ax3.legend()
plt.tight_layout()
plt.show()

This method injects sharp, discrete anomalies exactly where you want them—just adjust impulse_amplitude until the anomaly size matches what’s shown in the paper.

Method 2: Convolution (For Diffused/Extended Pulses) #

You asked if convolving the pulse as noise works—this is useful only if the paper's anomalies aren't perfectly instantaneous (e.g., a pulse that lingers for 2-3 time steps). For example:

# Create a wider pulse kernel to soften the impulse
pulse_kernel = np.array([0.3, 1, 0.3])  # Spreads the anomaly over 3 time steps
extended_impulse = np.convolve(imp, pulse_kernel, mode='same')
anomalous_signal_conv = base_signal + extended_impulse

Important note: If the paper shows sharp, single-time-step pulses, direct addition is better—convolution will blur the anomaly, making it less true to the paper's examples.

2. Strategy for Your Existing Normal Time Series #

You mentioned identifying high/low states first—here's how to turn that idea into actionable code for realistic anomaly injection:

• Step 1: Detect state boundaries: Use a simple threshold or rolling window to label segments of your time series as "high state" or "low state". For example:

  threshold = np.mean(base_signal)
  state_labels = np.where(base_signal > threshold, 'high', 'low')
  

• Step 2: Targeted anomaly injection: Add upward pulses only in low states, or downward pulses only in high states to mimic context-dependent real-world anomalies:

  # Example: Add upward anomalies only in low state regions
  low_state_indices = np.where(state_labels == 'low')[0]
  anomaly_indices = np.random.choice(low_state_indices, 3, replace=False)
  targeted_imp = signal.unit_impulse(200, anomaly_indices) * impulse_amplitude
  

3. Worst-Case Scenario: Ideal Signals as a Baseline #

If your existing data is too messy, start with ideal signals to validate your anomaly generation first, then transfer the logic to real data:

• Use a perfect periodic pulse sequence: ideal_pulse = signal.square(2 * np.pi * time_steps / 20)
• Or a clean sine wave (as in the example above)
• Generate anomalies on this ideal signal, then apply the same amplitude/location rules to your real data once you’ve matched the paper's plot.

4. Additional Tips for Matching the Paper's Results #

• Amplitude calibration: Plot the paper's anomaly figure alongside your generated signal, and adjust impulse_amplitude until the anomaly magnitude matches exactly.
• Add small noise: Real-world data isn’t perfect—add a tiny amount of Gaussian noise to your anomalies if the paper's plots look slightly noisy.
• Validate with your KM method: After generating anomalies, run your KM detection algorithm to ensure it picks up the anomalies as expected. This confirms your generation logic aligns with the method’s sensitivity.

Hope this helps you get your experiments up and running smoothly!

内容的提问来源于stack exchange，提问作者Mario