自动化生成含数学常数与未知量的表达式及面板数据转移矩阵ML估计问题

Let's tackle your two requirements step by step:

If you want to generate symbolic expressions with mathematical constants (like π, e) and variables (like x, y), using a symbolic computation library is the way to go. Here's a flexible implementation using Python's sympy library—you can customize the constants, variables, and operations as needed:

import sympy as sp
import random

def generate_expression(constants=None, variables=None, operation_types=None):
    # Set default parameters if not provided
    constants = constants or ['pi', 'e']
    variables = variables or ['x', 'y']
    operation_types = operation_types or ['add', 'mul', 'pow']
    
    # Convert string names to sympy objects
    sym_constants = [getattr(sp, c) for c in constants]
    sym_vars = [sp.symbols(v) for v in variables]
    
    # Map operation names to sympy functions
    op_map = {
        'add': sp.Add,
        'mul': sp.Mul,
        'pow': sp.Pow
    }
    
    # Generate a simple expression (easily extendable to nested structures)
    elements = sym_constants + sym_vars
    selected_op = random.choice(operation_types)
    left = random.choice(elements)
    right = random.choice(elements)
    
    return op_map[selected_op](left, right)

# Example usage
print(generate_expression())  # Might output: pi*y
print(generate_expression(variables=['z'], operation_types=['pow']))  # Might output: e**z

This function leverages sympy's symbolic capabilities. You can expand it to create more complex nested expressions, fix specific operation rules, or return string-formatted expressions instead of sympy objects based on your needs.

From your description, this is a logit-based state transition problem where an individual's log-likelihood is the sum of log-probabilities for all transitions in their path. Here's a step-by-step implementation using Python's pandas and numpy:

Assumptions About Your DataFrame #

Let’s assume your data frame df has these columns:

• individual_id: Unique identifier for each individual
• current_state: Current state (e.g., 'a' or 'b')
• next_state: Next period's state (e.g., 'a' or 'b')
• Y_ab: Linear predictor for transitions from a to b (i.e., log(p_ab/(1-p_ab)) in the logit model)
• Y_ba: Linear predictor for transitions from b to a
• time: Time index (to ensure observations are ordered correctly)

Step 1: Define a Function for Single Transition Log-Probability #

To avoid numerical overflow (when Y is large, exp(Y) can exceed float limits), we simplify log(exp(Y)/(1+exp(Y))) to Y - np.log(1 + np.exp(Y)):

import numpy as np
import pandas as pd

def transition_log_prob(row):
    # Handle a→b transition
    if row['current_state'] == 'a' and row['next_state'] == 'b':
        y = row['Y_ab']
    # Handle b→a transition
    elif row['current_state'] == 'b' and row['next_state'] == 'a':
        y = row['Y_ba']
    # Handle self-transitions (adjust based on your model's Y values)
    elif row['current_state'] == row['next_state']:
        # Assume you have columns like Y_aa/Y_bb for self-transition predictors
        y = row[f'Y_{row["current_state"]}{row["next_state"]}']
    else:
        # Return NaN for invalid state combinations
        return np.nan
    
    return y - np.log(1 + np.exp(y))

Step 2: Calculate Total Log-Likelihood per Individual #

Use pandas grouping to sum log-probabilities for each individual's path:

# Critical: Sort data by individual and time to preserve transition order
df = df.sort_values(['individual_id', 'time'])

# Compute log-likelihood for each individual
individual_log_likelihood = df.groupby('individual_id').apply(
    lambda group: group.apply(transition_log_prob, axis=1).sum()
).reset_index(name='log_likelihood')

# Example: Check log-likelihood for individual 120421006
print(individual_log_likelihood[individual_log_likelihood['individual_id'] == 120421006])

Key Notes #

• Data Order: Always sort your data by individual and time—incorrect order will break the transition logic.
• Self-Transitions: If your model includes self-transitions (a→a or b→b), make sure to add corresponding Y columns (like Y_aa) and adjust the function to handle those cases.
• Numerical Stability: The simplified formula Y - np.log(1 + np.exp(Y)) avoids overflow issues that can occur with direct computation of log(exp(Y)/(1+exp(Y))).

内容的提问来源于stack exchange，提问作者Arrebimbomalho