Let’s walk through your loss functions, loss weights, and metrics to assess their reasonableness, plus share some practical tweaks to consider.

Loss Functions: Mostly On Target #

Your loss function selections align well with each task’s type, which is a great start:

• EMOTIONS (multi-class multi-label classification): Using binary_crossentropy is exactly right here. Since each emotion label is independent (e.g., a sample can be both "happy" and "surprised"), binary crossentropy treats each label as a separate binary classification problem—perfect for multi-label scenarios.
• VALENCE/AROUSAL/DOMINANCE (regression): mse (Mean Squared Error) is a standard choice for regression tasks. It penalizes larger errors more heavily, which is reasonable if you want to prioritize minimizing big deviations in these continuous values. If you later find it’s too sensitive to outliers, you could experiment with mae (Mean Absolute Error), but mse is a solid default.
• AGE (multi-class classification): categorical_crossentropy works assuming your age labels are one-hot encoded. If you’re using integer labels (e.g., 0 for 10-20, 1 for 20-30), you’d want to switch to sparse_categorical_crossentropy to avoid unnecessary one-hot encoding, but otherwise this is correct.

Loss Weights: Needs Tuning Based on Loss Scales #

Your current weight assignment (EMOTIONS:1.0, VALENCE/AROUSAL/DOMINANCE:0.025, AGE:0.45) is a starting point, but it’s critical to validate it against the actual magnitude of each loss during training. Here’s why:

• Loss functions operate on different scales. For example, binary_crossentropy and categorical_crossentropy typically range between 0 and ~1 (when predictions are reasonably aligned with labels), while mse depends entirely on your regression targets’ range. If valence/arousal/dominance are normalized to 0-1, their mse might hover around 0.1-0.2 initially—multiplying by 0.025 would make their total contribution to the combined loss negligible (0.0025-0.005), meaning the model will prioritize optimizing emotions and age almost exclusively.
• The fix: Train for a few epochs, then check the raw loss values for each head (before applying weights). Adjust the weights so that each loss * weight is in a similar order of magnitude. For example, if your raw mse for valence is 10 (if targets are 0-10), multiplying by 0.025 gives 0.25, which is closer to the emotions loss (say 0.7 1=0.7) and age loss (0.50.45=0.225). If the mse is smaller, you might need to increase the weight (e.g., to 0.1 or 0.2) to give those regression tasks more influence.

Metrics: Room for Improvement #

Your metrics are functional, but you can make them more informative for each task:

• EMOTIONS: Using crossentropy as a metric is okay, but it’s more of a loss proxy than an intuitive performance measure. For multi-label classification, better metrics include:
  • precision/recall (per-label or macro-averaged)
  • f1_score (balances precision and recall)
  • hamming_loss (measures the fraction of misclassified labels)
    These will give you a clearer sense of how well the model is actually predicting individual emotion labels, not just the crossentropy value.
• VALENCE/AROUSAL/DOMINANCE: mse as a metric is fine, but adding mae can help you understand the average absolute error (which is more interpretable than squared error for real-world context).
• AGE: categorical_accuracy is the right default. If you have many age categories, you might also want to add top_k_categorical_accuracy (e.g., top-3 accuracy) to see if the model is at least predicting a plausible age range even if it misses the exact category.

Final Takeaway #

Your core setup is logically sound—you’ve matched loss functions to task types correctly. The main areas to iterate on are:

1. Tuning loss weights based on observed raw loss magnitudes during early training
2. Swapping out the crossentropy metric for multi-label-specific metrics for the emotion task

内容的提问来源于stack exchange，提问作者Grigorios Kalliatakis