如何在Keras的Flatten层输出后添加新特征用于图像分类

Got it, let's break this down! Your current code builds a standard CNN pipeline, but to add age, gender, and other metadata, you'll need to use Keras' Functional API (since the Sequential API can't handle multiple input streams or feature merging easily). Here's exactly how to modify your code:

Step 1: Define Input Layers for Metadata #

First, create separate Input layers for your extra features. Adjust the shapes based on how you're representing each feature:

• Age: A single normalized numerical value (e.g., scaled to [0,1])
• Gender: A binary value (0/1 for male/female) or a one-hot encoded vector (e.g., [1,0] / [0,1])

from keras.layers import Input, Dense, Flatten, MaxPooling2D, Conv2D, Concatenate
from keras.models import Model

# Define input for your images (adjust shape to match your image dimensions)
image_input = Input(shape=(224, 224, 3), name='image_input')

# Input for age (single numeric feature)
age_input = Input(shape=(1,), name='age_input')

# Input for gender (binary feature; use shape=(2,) if using one-hot encoding)
gender_input = Input(shape=(1,), name='gender_input')

Step 2: Keep Your Existing CNN Pipeline #

Continue building your CNN as you already have, ending with the Flatten layer:

# Your existing CNN layers (replace conv_final with your actual final convolutional layer)
conv_final = Conv2D(64, (3,3), activation='relu')(image_input)  # Example conv layer
max_pool_final = MaxPooling2D(pool_size=(2,2))(conv_final)
flat = Flatten()(max_pool_final)

Step 3: Merge Visual Features with Metadata #

Use the Concatenate layer to combine the flattened CNN features with your age/gender inputs:

# Combine all features into a single tensor
merged_features = Concatenate()([flat, age_input, gender_input])

Step 4: Feed Merged Features to Dense Layers #

Now pass the merged features into your Dense layer (and any subsequent layers for your classification task):

# Your Dense layer, now using the merged feature set
dense = Dense(128, activation='relu')(merged_features)

# Add your output layer (adjust num_classes to match your task)
output = Dense(num_classes, activation='softmax')(dense)

Step 5: Build and Compile the Model #

Finally, create the Model object by specifying all inputs and the output:

# Define the full model with multiple inputs and one output
model = Model(inputs=[image_input, age_input, gender_input], outputs=output)

# Compile the model as you normally would
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

How to Train the Model #

When training, pass a list of input arrays instead of a single array—make sure your data is aligned (same number of samples across all inputs):

# Assume you have:
# - X_images: Array of shape (num_samples, 224, 224, 3)
# - X_age: Array of shape (num_samples, 1)
# - X_gender: Array of shape (num_samples, 1)
# - y_labels: Array of shape (num_samples, num_classes)

model.fit(
    [X_images, X_age, X_gender],
    y_labels,
    epochs=10,
    batch_size=32,
    validation_split=0.2
)

Key Tips #

• Preprocess Metadata: Normalize numerical features like age (scale to 0-1) and encode categorical features like gender correctly (binary or one-hot) to ensure they play well with CNN features.
• Flexibility: You can add more metadata features (e.g., ethnicity) just by adding another Input layer and including it in the Concatenate list.
• Batch Alignment: Double-check that all input arrays have the same batch size—this is critical for the concatenation step to work.

内容的提问来源于stack exchange，提问作者Nikhil Mishra