Keras实现PilotNet时如何输入3张相机图像得到单个转向角输出

你当前使用的Sequential序列模型仅支持单输入场景，要实现左、中、右三路相机图像同时输入，需要改用Keras函数式API实现。核心思路是让三路输入共享卷积层权重提取图像特征，拼接融合三路特征后接入全连接层，最终输出单个转向角结果。

调整后的实现代码 #

from keras.models import Model
from keras.layers import Input, Convolution2D, Flatten, Dense, concatenate
from keras.optimizers import Adam

def create_multi_input_model():
    # 定义共享的卷积特征提取器，和原PilotNet卷积层结构完全一致
    def build_shared_feature_extractor(input_shape):
        inputs = Input(shape=input_shape)
        x = Convolution2D(24, (5, 5), strides=(2, 2), activation='relu')(inputs)
        x = Convolution2D(36, (5, 5), strides=(2, 2), activation='relu')(x)
        x = Convolution2D(48, (5, 5), strides=(2, 2), activation='relu')(x)
        x = Convolution2D(64, (3, 3), activation='relu')(x)
        x = Convolution2D(64, (3, 3), activation='relu')(x)
        x = Flatten()(x)
        return Model(inputs=inputs, outputs=x)
    
    # 单张图像输入尺寸和原模型保持一致
    input_shape = (66, 200, 3)
    shared_extractor = build_shared_feature_extractor(input_shape)
    
    # 定义三个输入分支：左、中、右相机图像
    left_input = Input(shape=input_shape, name='left_camera')
    center_input = Input(shape=input_shape, name='center_camera')
    right_input = Input(shape=input_shape, name='right_camera')
    
    # 三个输入分别经过共享特征提取器得到对应特征
    left_features = shared_extractor(left_input)
    center_features = shared_extractor(center_input)
    right_features = shared_extractor(right_input)
    
    # 拼接三路特征进行融合
    merged_features = concatenate([left_features, center_features, right_features])
    
    # 全连接层结构和原模型对齐
    x = Dense(100, activation='relu')(merged_features)
    x = Dense(50, activation='relu')(x)
    x = Dense(10, activation='relu')(x)
    output = Dense(1, activation='linear')(x)
    
    # 构建完整模型并编译
    model = Model(inputs=[left_input, center_input, right_input], outputs=output)
    model.compile(optimizer=Adam(learning_rate=0.0001), loss='mse')
    return model

关键说明 #

• 卷积层采用权重共享机制：三路输入复用同一套卷积参数，既和原PilotNet的特征提取逻辑保持一致，也不会大幅增加参数量，降低过拟合风险
• 训练输入需要适配：训练时输入数据调整为[左相机图像数组, 中相机图像数组, 右相机图像数组]的格式，标签仍为对应的转向角数值
• 推理时也需要同时传入三路相机图像，即可得到单个转向角输出

内容的提问来源于stack exchange，提问作者RektAngle