如何使用Python补全缺失的2019-2020年月度及逐日气象数据

Hey there! Let's walk through how you can fill in those missing 2019 and 2020 meteorological data points using Python, building on your existing 1981-2018 daily and monthly records. I'll break this down into practical, actionable steps that match the periodic nature of weather data.

First, you'll need to get your data into a clean, time-aware format using Pandas:

• Load your dataset and convert the date column to a datetime type (critical for time-based operations)
• Identify which variables (e.g., temperature, precipitation, wind speed) have missing values, and where those gaps occur

import pandas as pd

# Load your data (adjust file path/format as needed)
df = pd.read_csv('meteorological_records.csv')

# Convert date column to datetime
df['date'] = pd.to_datetime(df['date'])

# Check for missing values
print("Missing values per column:\n", df.isnull().sum())

# Sort data by date to ensure chronological order
df = df.sort_values('date')

Weather data has strong periodic patterns (daily, monthly, seasonal), so we'll prioritize methods that leverage this history. Here are the most effective approaches:

Method 1: Historical Same-Period Mean/Median (Most Common) #

For missing values on a specific day (e.g., Jan 1st), use the average or median value from that same day across all your 1981-2018 records. This works great for stable variables like temperature.

# Create a "month-day" key to group same calendar days
df['month_day'] = df['date'].dt.strftime('%m-%d')

# Calculate historical stats (1981-2018) for each month-day
historical_summary = df[df['date'].dt.year <= 2018].groupby('month_day').agg(
    temp_mean=('temperature', 'mean'),
    precip_median=('precipitation', 'median')  # Median is better for precipitation (avoids extreme rain skew)
).reset_index()

# Merge historical stats back to the original data and fill gaps
df = df.merge(historical_summary, on='month_day', how='left')
df['temperature'] = df['temperature'].fillna(df['temp_mean'])
df['precipitation'] = df['precipitation'].fillna(df['precip_median'])

Note: For monthly data, just group by month instead of month_day to calculate historical monthly averages.

Method 2: Time-Series Interpolation (For Continuous Gaps) #

If you have consecutive missing days (e.g., 3 days of missing temperature), interpolation creates a smooth, trend-aligned fill. Linear interpolation works for short gaps, while spline interpolation is better for longer gaps with clear trends.

# Set date as index for easier time-based interpolation
df = df.set_index('date')

# Linear interpolation (best for short, consecutive gaps)
df['temperature'] = df['temperature'].interpolate(method='linear')

# Spline interpolation (smoother, ideal for longer gaps)
df['wind_speed'] = df['wind_speed'].interpolate(method='spline', order=3)

# Reset index to get date column back
df = df.reset_index()

Method 3: Machine Learning Prediction (For Complex Gaps) #

If you have large gaps or variables that correlate strongly (e.g., humidity and temperature), use a regression model to predict missing values. We'll use a Random Forest here, which handles non-linear relationships well.

from sklearn.ensemble import RandomForestRegressor

# Add time-based features for the model
df['year'] = df['date'].dt.year
df['month'] = df['date'].dt.month
df['day'] = df['date'].dt.day
df['day_of_week'] = df['date'].dt.dayofweek

# Train on complete 1981-2018 data
train_data = df[(df['year'] <= 2018) & (df['temperature'].notnull())]
X_train = train_data[['month', 'day', 'day_of_week', 'humidity', 'wind_speed']]
y_train = train_data['temperature']

# Fit the model
model = RandomForestRegressor(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# Predict missing temperature values in 2019-2020
missing_mask = df['temperature'].isnull()
X_test = df[missing_mask][['month', 'day', 'day_of_week', 'humidity', 'wind_speed']]
df.loc[missing_mask, 'temperature'] = model.predict(X_test)

Don't skip this! Make sure your filled data makes sense:

• Compare 2019-2020 stats (mean, min/max) to your 1981-2018 baseline with df.describe()
• Plot time series for key variables to check for anomalies:

  df.plot(x='date', y='temperature', figsize=(12, 6), title='Temperature Over Time')
  

• Ensure extreme values (e.g., a record high in 2019) aren't overwritten by historical averages.

内容的提问来源于stack exchange，提问作者Desta Werash