TensorFlow中构建200组正态分布并适配样本数据的实现问询

The key here is leveraging broadcasting in TensorFlow to align the dimensions of your distribution parameters with your data. Here's how to fix your code step by step:

Step 1: Convert Data to TensorFlow Tensor #

First, convert your pandas Series of heights to a TensorFlow tensor to ensure dtype compatibility and proper broadcasting:

df_height = tf.convert_to_tensor(df.height, dtype=tf.float32)

Step 2: Expand Dimensions of mu and sigma #

Your mu and sigma tensors are currently shape (200,). To compute log probabilities for each (mu, sigma) pair against all 352 data points, we need to expand their dimensions to (200, 1). This tells TensorFlow to broadcast each parameter across all data points:

mu_expanded = mu[:, tf.newaxis]  # Shape becomes (200, 1)
sigma_expanded = sigma[:, tf.newaxis]  # Shape becomes (200, 1)

Step 3: Compute Log Probabilities #

Now create the Normal distribution with the expanded parameters, then compute the log probability for each data point. The result will be a (200, 352) tensor where each row corresponds to a (mu, sigma) pair, and each column corresponds to a height data point:

log_probs = tfd.Normal(loc=mu_expanded, scale=sigma_expanded).log_prob(df_height)

Full Updated Code #

Putting it all together:

import pandas as pd
import numpy as np
import tensorflow as tf
import tensorflow_probability as tfp
from tensorflow_probability import distributions as tfd

_BASE_URL = "https://raw.githubusercontent.com/rmcelreath/rethinking/Experimental/data"
HOWELL_DATASET_PATH = f"{_BASE_URL}/Howell1.csv"
df = pd.read_csv(HOWELL_DATASET_PATH, sep=';')
df = df[df['age'] >= 18]

# Create parameter grids
mu = tf.linspace(start=140.0, stop=160.0, num=200)
sigma = tf.linspace(start=4.0, stop=9.0, num=200)

# Prepare data tensor
df_height = tf.convert_to_tensor(df.height, dtype=tf.float32)

# Expand parameters for broadcasting
mu_expanded = mu[:, tf.newaxis]
sigma_expanded = sigma[:, tf.newaxis]

# Compute (200, 352) log probability tensor
log_probs = tfd.Normal(loc=mu_expanded, scale=sigma_expanded).log_prob(df_height)

# Verify shape
print(log_probs.shape)  # Should output (200, 352)

Why This Works #

TensorFlow uses broadcasting to match tensor dimensions:

• The expanded mu_expanded and sigma_expanded have shape (200, 1)
• The df_height tensor has shape (352,)
• When computing log_prob, TensorFlow broadcasts df_height to (1, 352), then computes the log probability for each combination of (mu, sigma) and height, resulting in the (200, 352) tensor you need.

If you want to compute the total log probability for each (mu, sigma) pair (summing across all data points), you can add:

total_log_probs = tf.reduce_sum(log_probs, axis=1)

This will give you a (200,) tensor where each element is the sum of log probabilities for that parameter pair.

内容的提问来源于stack exchange，提问作者Tom Kealy