基于TensorFlow/Keras的Transformer气象数据任务求解咨询

1. 数据预处理 #

先处理两类特征，统一特征格式适配Transformer输入：

• 数值特征（气压、气温等）：用标准化消除量级差异
• 类别特征（风向）：用独热编码转换为数值型特征

import pandas as pd
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer

# 替换为你的数据集列名
numeric_features = ['气压', '气温', '相对湿度']
categorical_features = ['风向']

# 构建预处理管道
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features),
        ('cat', OneHotEncoder(sparse_output=False), categorical_features)
    ])

# 加载数据集并预处理（假设数据集已加载为df）
processed_data = preprocessor.fit_transform(df)

2. 构造序列数据 #

构造输入序列（过去N小时气象数据）与目标值（未来6小时气温），示例用过去24小时数据预测：

import numpy as np

def create_sequences(data, temp_col_idx, seq_len=24, pred_len=6):
    X, y = [], []
    # temp_col_idx是气温在预处理后数据中的索引
    for i in range(len(data) - seq_len - pred_len + 1):
        X.append(data[i:i+seq_len])
        y.append(data[i+seq_len:i+seq_len+pred_len, temp_col_idx])
    return np.array(X), np.array(y)

# 替换为气温在预处理后数据中的实际索引
X, y = create_sequences(processed_data, temp_col_idx=1)

3. 整合Transformer编码器与位置嵌入 #

基于你提供的Keras实现，搭建完整预测模型：

假设你已有的模块示例（若你的代码不同，替换对应层即可） #

import tensorflow as tf
from tensorflow.keras.layers import Layer, Dense, MultiHeadAttention, LayerNormalization, Dropout

class PositionalEmbedding(Layer):
    def __init__(self, sequence_length, output_dim, **kwargs):
        super().__init__(**kwargs)
        self.pos_emb = tf.keras.layers.Embedding(input_dim=sequence_length, output_dim=output_dim)
    
    def call(self, inputs):
        position_indices = tf.range(tf.shape(inputs)[1])
        positions = self.pos_emb(position_indices)
        return inputs + positions

class TransformerEncoderBlock(Layer):
    def __init__(self, embed_dim, num_heads, ff_dim, rate=0.1):
        super().__init__()
        self.att = MultiHeadAttention(num_heads=num_heads, key_dim=embed_dim)
        self.ffn = tf.keras.Sequential(
            [Dense(ff_dim, activation="relu"), Dense(embed_dim)]
        )
        self.layernorm1 = LayerNormalization(epsilon=1e-6)
        self.layernorm2 = LayerNormalization(epsilon=1e-6)
        self.dropout1 = Dropout(rate)
        self.dropout2 = Dropout(rate)

    def call(self, inputs, training=False):
        attn_output = self.att(inputs, inputs)
        attn_output = self.dropout1(attn_output, training=training)
        out1 = self.layernorm1(inputs + attn_output)
        ffn_output = self.ffn(out1)
        ffn_output = self.dropout2(ffn_output, training=training)
        return self.layernorm2(out1 + ffn_output)

搭建预测模型 #

# 模型参数
seq_len = 24
embed_dim = 64
num_heads = 2
ff_dim = 128
num_features = processed_data.shape[1]
pred_len = 6

# 输入层
inputs = tf.keras.Input(shape=(seq_len, num_features))
# 将原始特征映射到Transformer所需的嵌入维度
x = Dense(embed_dim)(inputs)
# 添加位置嵌入
x = PositionalEmbedding(sequence_length=seq_len, output_dim=embed_dim)(x)
# 加入Transformer编码器（可堆叠多个，示例用1层）
x = TransformerEncoderBlock(embed_dim, num_heads, ff_dim)(x)
# 序列全局池化压缩为向量
x = tf.keras.layers.GlobalAveragePooling1D()(x)
# 输出未来6小时气温
outputs = Dense(pred_len)(x)

model = tf.keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer="adam", loss="mse")  # 回归任务用均方误差

4. 训练与预测 #

from sklearn.model_selection import train_test_split

# 时间序列数据不打乱顺序拆分
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)

# 训练模型
history = model.fit(X_train, y_train, epochs=20, batch_size=32, validation_split=0.1)

# 预测未来6小时气温
y_pred = model.predict(X_test)

常见问题排查 #

• 模型不收敛：检查序列构造是否匹配“过去-未来”逻辑、特征预处理是否遗漏、Transformer参数（嵌入维度、头数）是否适配数据规模
• 类别特征错误：若风向类别过多，可替换独热编码为Embedding层
• 位置嵌入报错：确保位置嵌入的sequence_length与输入序列长度seq_len完全一致

内容的提问来源于stack exchange，提问作者tomasb21