如何自动检测并将Pandas DataFrame中的numpy.ndarray列转为列表列

Got it, let's solve this problem where you need to auto-detect columns storing numpy.ndarray values (even though they show up as object in .dtypes) and convert them to regular Python lists—no manual column name required.

Here's a straightforward approach: #

We'll first target all columns marked as object (since that's where your ndarrays are hiding), then check the actual type of elements in each column to confirm if they're ndarrays, and convert them if so.

import numpy as np

# Step 1: Grab all columns with dtype 'object'
object_columns = results_df.select_dtypes(include=["object"]).columns

# Step 2: Iterate through each object column and convert ndarrays to lists
for col in object_columns:
    # Skip empty columns to avoid errors
    if results_df[col].dropna().empty:
        continue
    # Check the type of the first non-null element to determine column content
    sample_element = results_df[col].dropna().iloc[0]
    if isinstance(sample_element, np.ndarray):
        # Convert every element in the column to a list (handle potential NaNs too)
        results_df[col] = results_df[col].apply(
            lambda x: list(x) if isinstance(x, np.ndarray) else x
        )

How this works: #

1. Filter object columns: We start by narrowing down to columns that Pandas labels as object, since both strings and ndarrays fall into this category.
2. Check column content: For each object column, we look at the first non-null element to check if it's a numpy.ndarray (assuming all non-null elements in the column are the same type, which is standard practice).
3. Convert to lists: If the column contains ndarrays, we use apply() to turn each ndarray into a Python list. The lambda also handles any stray non-ndarray values (like NaNs) to avoid breaking the conversion.

This way, you don't have to hardcode column names like competitionIds—the code will automatically pick up any ndarray-storing columns in your DataFrame.

内容的提问来源于stack exchange，提问作者Canovice