如何将分类CVAE适配为回归任务？图像+连续值数据集改造咨询

Great question! Adapting a classification CVAE to handle continuous conditional variables for regression-based image generation is totally doable, but there are a few key adjustments you need to make to the original code. Let's walk through this step by step.

Key Modifications for Continuous Conditional Variables #

• Data Preprocessing: Unlike discrete class labels that get one-hot encoded, your single continuous value just needs proper normalization (e.g., scaling to 0-1 to match your image pixel range). This ensures the model treats both image and conditional features equally.
• Input Layer Adjustments: Replace the one-hot label input (shape (n_y,)) with a single-value input (shape (1,)).
• Concatenation Logic: Your initial thought of concatenating images with the continuous value is valid—but only if the continuous value is normalized. Without normalization, a large-scale continuous variable could dominate the model's learning over the image pixels.
• Generation Workflow: When generating images, you'll input your target continuous value (normalized) alongside latent space samples, instead of one-hot vectors.

Modified CVAE Code (For Continuous Regression Task) #

import warnings
import numpy as np
from keras.layers import Input, Dense, Lambda
from keras.layers.merge import concatenate as concat
from keras.models import Model
from keras import backend as K
from keras.datasets import mnist  # Replace with your custom dataset
from keras.callbacks import EarlyStopping
from keras.optimizers import Adam
import matplotlib.pyplot as plt

warnings.filterwarnings('ignore')
%pylab inline

# -------------------------- Data Preprocessing --------------------------
# Example: Simulate regression task with MNIST (replace with your real data)
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
X_train = X_train.astype('float32') / 255.
X_test = X_test.astype('float32') / 255.
n_pixels = np.prod(X_train.shape[1:])
X_train = X_train.reshape((len(X_train), n_pixels))
X_test = X_test.reshape((len(X_test), n_pixels))

# Generate simulated continuous labels (replace with your real continuous values)
y_train = np.mean(X_train, axis=1, keepdims=True)
y_test = np.mean(X_test, axis=1, keepdims=True)
# Normalize continuous values to 0-1 range
y_train = (y_train - y_train.min()) / (y_train.max() - y_train.min())
y_test = (y_test - y_test.min()) / (y_test.max() - y_test.min())

# -------------------------- Model Parameters --------------------------
n_z = 2  # Latent space dimension (increase for better image quality)
encoder_dim1 = 512
decoder_dim = 512
decoder_out_dim = 784  # Match your image pixel count
activ = 'relu'
optim = Adam(lr=0.001)
n_epoch = 50
batch_size = 128

n_x = X_train.shape[1]
n_y = y_train.shape[1]  # Now equals 1 (single continuous value)

# -------------------------- Encoder --------------------------
X = Input(shape=(n_x,))
cond_var = Input(shape=(n_y,))  # Continuous conditional variable input
inputs = concat([X, cond_var])

encoder_h = Dense(encoder_dim1, activation=activ)(inputs)
mu = Dense(n_z, activation='linear')(encoder_h)
l_sigma = Dense(n_z, activation='linear')(encoder_h)

def sample_z(args):
    mu, l_sigma = args
    eps = K.random_normal(shape=(K.shape(mu)[0], n_z), mean=0., stddev=1.)
    return mu + K.exp(l_sigma / 2) * eps

# Sample latent space
z = Lambda(sample_z, output_shape=(n_z,))([mu, l_sigma])
# Merge latent space with continuous condition
zc = concat([z, cond_var])

# -------------------------- Decoder --------------------------
decoder_hidden = Dense(decoder_dim, activation=activ)
decoder_out = Dense(decoder_out_dim, activation='sigmoid')

h_p = decoder_hidden(zc)
outputs = decoder_out(h_p)

# -------------------------- Loss Functions --------------------------
def vae_loss(y_true, y_pred):
    recon = K.sum(K.binary_crossentropy(y_true, y_pred), axis=-1)
    kl = 0.5 * K.sum(K.exp(l_sigma) + K.square(mu) - 1. - l_sigma, axis=-1)
    return recon + kl

def KL_loss(y_true, y_pred):
    return 0.5 * K.sum(K.exp(l_sigma) + K.square(mu) - 1. - l_sigma, axis=1)

def recon_loss(y_true, y_pred):
    return K.sum(K.binary_crossentropy(y_true, y_pred), axis=-1)

# -------------------------- Model Compilation & Training --------------------------
cvae = Model([X, cond_var], outputs)
encoder = Model([X, cond_var], mu)

# Decoder for generation: input = latent z + continuous condition
d_in = Input(shape=(n_z + n_y,))
d_h = decoder_hidden(d_in)
d_out = decoder_out(d_h)
decoder = Model(d_in, d_out)

cvae.compile(optimizer=optim, loss=vae_loss, metrics=[KL_loss, recon_loss])

cvae_hist = cvae.fit(
    [X_train, y_train], X_train,
    verbose=1,
    batch_size=batch_size,
    epochs=n_epoch,
    validation_data=([X_test, y_test], X_test),
    callbacks=[EarlyStopping(patience=5)]
)

# -------------------------- Generate Images from Continuous Values --------------------------
def construct_cond_vec(continuous_val, z=None):
    """Build decoder input: latent z + normalized continuous value"""
    out = np.zeros((1, n_z + n_y))
    out[:, n_z:] = continuous_val
    if z is not None:
        for i in range(len(z)):
            out[:, i] = z[i]
    return out

# Example: Generate image for normalized continuous value = 0.5
sample_cond = construct_cond_vec(0.5)
generated_img = decoder.predict(sample_cond)
plt.figure(figsize=(3,3))
plt.imshow(generated_img.reshape(28,28), cmap=plt.cm.gray), axis('off')
plt.title(f'Generated Image (Cond: 0.5)')
plt.show()

# Generate grid of images with varying continuous values
sides = 5
max_cond = 1.0
min_cond = 0.0
img_it = 0
plt.figure(figsize=(10,10))
for i in range(sides):
    cond_val = min_cond + (i/(sides-1))*(max_cond - min_cond)
    for j in range(sides):
        z_rand = np.random.normal(0, 1, size=n_z)
        vec = construct_cond_vec(cond_val, z_rand)
        decoded = decoder.predict(vec)
        plt.subplot(sides, sides, 1 + img_it)
        img_it +=1
        plt.imshow(decoded.reshape(28,28), cmap=plt.cm.gray), axis('off')
        plt.title(f'Cond: {cond_val:.2f}')
plt.subplots_adjust(left=0, bottom=0, right=1, top=1, wspace=0.3, hspace=0.3)
plt.show()

Additional Notes #

• Dataset Replacement: Swap the MNIST simulated continuous labels with your real dataset. Use MinMaxScaler from scikit-learn if you need to normalize your continuous values.
• Latent Space Size: n_z=2 is great for visualization, but try increasing it to 8 or 16 if you want higher-quality generated images.
• Loss Function: Binary crossentropy works here because images are scaled to 0-1. If your images use a different range (e.g., 0-255), adjust the loss function accordingly.
• Exploration: Fix the latent vector z and vary the continuous value to see how images change, or fix the continuous value and sample different z vectors to get diverse outputs for the same condition.

内容的提问来源于stack exchange，提问作者David