如何从随机种群获取最优种群(gStar)并基于FPA优化及打印权重

Hey there! Let's work through your problem step by step. I'll start by filling in the gaps in your JST model, then show you how to find the optimal initial population (gStar), print its weights, and implement the Flower Pollination Algorithm to optimize it.

Step 1: Define a Fitness Function & Complete the JST Model #

First, we need a way to evaluate each population member's performance. For regression tasks, Mean Squared Error (MSE) between predictions and true labels is a standard fitness metric. We'll also complete your JST model definition with forward pass logic.

import numpy as np
from sklearn.metrics import mean_squared_error

# Your provided parameters
num_train = len(X_trainNorm)  # Assume X_trainNorm is your normalized training data
input_dim = windowSize        # Replace with your actual window size
output_dim = 1
popSize = 5

# Complete JST Model with forward pass
def buildModel(hidden_dim):
    np.random.seed(7)
    pop = []
    for i in range(popSize):
        # Initialize weights with He initialization (reduces vanishing gradients)
        W1 = np.random.randn(input_dim, hidden_dim) / np.sqrt(input_dim)
        b1 = np.zeros((1, hidden_dim))
        W2 = np.random.randn(hidden_dim, output_dim) / np.sqrt(hidden_dim)
        b2 = np.zeros((1, output_dim))
        # Store each individual's weights as a dictionary for easy access
        pop.append({"W1": W1, "b1": b1, "W2": W2, "b2": b2})
    return pop

# Fitness function: Lower MSE = better performance (we'll minimize this)
def calculate_fitness(individual, X, y):
    # Forward pass through the JST model
    z1 = np.dot(X, individual["W1"]) + individual["b1"]
    a1 = np.tanh(z1)  # Using tanh activation; swap with ReLU if your model uses it
    z2 = np.dot(a1, individual["W2"]) + individual["b2"]
    y_pred = z2
    # Return MSE as the fitness score
    return mean_squared_error(y, y_pred)

Step 2: Find the Optimal Initial Population (gStar) #

Now we'll generate the random population, calculate each member's fitness, and select the one with the lowest MSE as gStar. We'll also print its weights for inspection.

# Set your desired hidden layer size (adjust based on your task)
hidden_dim = 16
# Generate initial random population
population = buildModel(hidden_dim)

# Assume you have your training features X_trainNorm and labels y_train
# Calculate fitness for every individual in the population
fitness_scores = [calculate_fitness(ind, X_trainNorm, y_train) for ind in population]

# Find gStar: the individual with the smallest fitness (lowest MSE)
gStar_idx = np.argmin(fitness_scores)
gStar = population[gStar_idx]
best_initial_fitness = fitness_scores[gStar_idx]

print(f"Optimal initial population (gStar) found with MSE: {best_initial_fitness:.4f}")
print("\nWeights of gStar:")
print("W1:\n", gStar["W1"])
print("\nb1:\n", gStar["b1"])
print("\nW2:\n", gStar["W2"])
print("\nb2:\n", gStar["b2"])

Step 3: Optimize gStar with Flower Pollination Algorithm (FPA) #

FPA mimics natural flower pollination: global pollination (via Levy flight) for exploring new solutions, local pollination for refining existing ones, and a switch probability to balance both. Here's how to implement it for your model:

import random

def fpa_optimize(initial_gStar, X, y, hidden_dim, max_iter=100, p_switch=0.8, step_size=0.1):
    current_best = initial_gStar.copy()
    current_best_fitness = calculate_fitness(current_best, X, y)
    
    for iter in range(max_iter):
        for _ in range(popSize):
            # Switch between global and local pollination
            if random.random() < p_switch:
                # Global pollination: Use Levy flight for long-distance exploration
                levy_step_W1 = np.random.standard_cauchy(size=initial_gStar["W1"].shape) * step_size
                new_W1 = current_best["W1"] + levy_step_W1
                
                levy_step_b1 = np.random.standard_cauchy(size=initial_gStar["b1"].shape) * step_size
                new_b1 = current_best["b1"] + levy_step_b1
                
                levy_step_W2 = np.random.standard_cauchy(size=initial_gStar["W2"].shape) * step_size
                new_W2 = current_best["W2"] + levy_step_W2
                
                levy_step_b2 = np.random.standard_cauchy(size=initial_gStar["b2"].shape) * step_size
                new_b2 = current_best["b2"] + levy_step_b2
            else:
                # Local pollination: Cross two random individuals from the initial population
                ind1, ind2 = random.sample(population, 2)
                new_W1 = ind1["W1"] + random.random() * (ind2["W1"] - ind1["W1"])
                new_b1 = ind1["b1"] + random.random() * (ind2["b1"] - ind1["b1"])
                new_W2 = ind1["W2"] + random.random() * (ind2["W2"] - ind1["W2"])
                new_b2 = ind1["b2"] + random.random() * (ind2["b2"] - ind1["b2"])
            
            # Create new candidate individual
            new_ind = {"W1": new_W1, "b1": new_b1, "W2": new_W2, "b2": new_b2}
            new_fitness = calculate_fitness(new_ind, X, y)
            
            # Update the best solution if the new candidate is better
            if new_fitness < current_best_fitness:
                current_best = new_ind.copy()
                current_best_fitness = new_fitness
        
        # Print progress every 10 iterations to track convergence
        if (iter + 1) % 10 == 0:
            print(f"Iteration {iter+1}/{max_iter}, Best MSE: {current_best_fitness:.4f}")
    
    return current_best, current_best_fitness

# Run FPA optimization on gStar
optimized_gStar, optimized_fitness = fpa_optimize(gStar, X_trainNorm, y_train, hidden_dim)

print("\nOptimized gStar (after FPA):")
print(f"Best MSE after optimization: {optimized_fitness:.4f}")
print("\nOptimized Weights:")
print("W1:\n", optimized_gStar["W1"])
print("\nb1:\n", optimized_gStar["b1"])
print("\nW2:\n", optimized_gStar["W2"])
print("\nb2:\n", optimized_gStar["b2"])

Quick Tips for You: #

• Adjust Activation: If your JST model uses ReLU instead of tanh, replace np.tanh(z1) with np.maximum(0, z1).
• Tune FPA Params: Tweak max_iter (number of iterations), p_switch (balance between global/local pollination), and step_size (size of weight updates) if convergence is slow or unstable.
• Data Shape: Make sure X_trainNorm is shaped as (number_of_samples, input_dim) and y_train matches the model's output dimension (1 in your case).

内容的提问来源于stack exchange，提问作者Hana