如何为MNIST数据集添加输入特征并基于TensorFlow利用左右手书写特征

Hey there! Let's tackle your two questions about working with extended MNIST data in TensorFlow—super practical stuff when you want to add domain-specific features to classic datasets.

Adding extra inputs to MNIST boils down to two key steps: preparing your augmented dataset, and structuring your model to accept multiple input streams. Here's how to approach it:

• Step 1: Generate or load your extra feature data
  If you don't have real "left/right hand" labels, you can simulate them for testing (e.g., assign 50% of samples to left, 50% to right). If you have real annotated data, load it alongside the standard MNIST images and labels.
  For example, using TensorFlow's built-in MNIST loader, you can create a parallel array for the hand feature:

  import tensorflow as tf
  from tensorflow.keras.datasets import mnist
  
  (x_train, y_train), (x_test, y_test) = mnist.load_data()
  # Normalize image data
  x_train = x_train / 255.0
  x_test = x_test / 255.0
  
  # Simulate left/right hand feature (0 = left, 1 = right)
  hand_train = tf.random.uniform(shape=(len(y_train),), minval=0, maxval=2, dtype=tf.int32)
  hand_test = tf.random.uniform(shape=(len(y_test),), minval=0, maxval=2, dtype=tf.int32)
  

• Step 2: Package data for training
  You'll need to pass both the image data and the extra feature as separate inputs to your model. Use TensorFlow's Dataset API to bundle them together:

  train_dataset = tf.data.Dataset.from_tensor_slices(((x_train, hand_train), y_train)).batch(32)
  test_dataset = tf.data.Dataset.from_tensor_slices(((x_test, hand_test), y_test)).batch(32)
  

The best way to leverage this categorical feature is to build a multi-input model using TensorFlow's Functional API. This lets you process the image data and the hand feature through separate branches, then combine their outputs for final classification. Here's a step-by-step implementation:

• Step 1: Define the input layers
  Create two distinct input layers—one for the 28x28 MNIST images, and one for the hand feature (a single integer):

  # Image input branch
  image_input = tf.keras.Input(shape=(28, 28), name="image_input")
  # Reshape for convolution (add channel dimension)
  x = tf.keras.layers.Reshape((28, 28, 1))(image_input)
  x = tf.keras.layers.Conv2D(32, (3,3), activation='relu')(x)
  x = tf.keras.layers.MaxPooling2D((2,2))(x)
  x = tf.keras.layers.Flatten()(x)
  image_features = tf.keras.layers.Dense(64, activation='relu')(x)
  
  # Hand feature branch
  hand_input = tf.keras.Input(shape=(1,), name="hand_input")
  # Encode the categorical feature (since it's binary, we can use a simple dense layer or embedding)
  hand_features = tf.keras.layers.Dense(16, activation='relu')(hand_input)
  

• Step 2: Combine the feature branches
  Merge the outputs from both branches using a concatenation layer, then add final dense layers for classification:

  # Combine features
  combined = tf.keras.layers.concatenate([image_features, hand_features])
  combined = tf.keras.layers.Dense(32, activation='relu')(combined)
  # Final output layer (10 classes for MNIST digits)
  output = tf.keras.layers.Dense(10, activation='softmax')(combined)
  

• Step 3: Build and train the model
  Assemble the model with both inputs, compile it, and train using your augmented dataset:

  model = tf.keras.Model(inputs=[image_input, hand_input], outputs=output)
  model.compile(optimizer='adam',
                loss='sparse_categorical_crossentropy',
                metrics=['accuracy'])
  
  # Train the model
  model.fit(train_dataset, epochs=5, validation_data=test_dataset)
  

• Why this works
  By processing the image and hand feature separately, you let the model learn patterns specific to each input: the convolutional branch extracts visual features from the digits, while the hand feature branch learns how left/right hand writing correlates with digit classes (if at all). Combining them gives the model a more complete signal for prediction.

内容的提问来源于stack exchange，提问作者Hirotaka Nakagame