Pandas按日期统计列表类型技术字段频次报错问题

Hey there! I totally get where you're coming from—trying to group by date and count tech occurrences, but hitting that annoying unhashable list error. Let's break down why this happens and fix it step by step.

Why the Error Happens #

Pandas groupby() relies on hashable data types (like strings, integers) to group rows. Lists are mutable (you can change their contents) and can't be hashed, so trying to group directly on a list column throws that TypeError.

The Solution: Explode the List Column First #

The easiest way around this is to "unpack" each list into individual rows (one tech per row) using Pandas' explode() method, then group and count normally. Here's how to do it with a concrete example:

Step 1: Sample Data (to mirror your setup)

Let's start with a DataFrame that matches your structure—dates paired with lists of technologies:

import pandas as pd

data = {
    "date": ["2023-10-01", "2023-10-01", "2023-10-02"],
    "techs": [["Python", "SQL"], ["Python"], ["Java", "SQL", "Python"]]
}
df = pd.DataFrame(data)

Step 2: Explode the List Column

Use explode() to turn each element in the techs list into its own row, keeping the corresponding date:

df_exploded = df.explode("techs")

This transforms your DataFrame into something like this:

date techs
0 2023-10-01 Python
0 2023-10-01 SQL
1 2023-10-01 Python
2 2023-10-02 Java
2 2023-10-02 SQL
2 2023-10-02 Python

Step 3: Group & Count Frequencies

Now you can safely group by date and techs to count occurrences. Two simple ways to do this:

Method 1: groupby() + size()

freq_stats = df_exploded.groupby(["date", "techs"]).size().reset_index(name="count")

Method 2: value_counts() (more concise)

freq_stats = df_exploded.value_counts(["date", "techs"]).reset_index(name="count")

Either way, you'll get your desired frequency table:

date techs count
0 2023-10-01 Python 2
1 2023-10-01 SQL 1
2 2023-10-02 Python 1
3 2023-10-02 SQL 1
4 2023-10-02 Java 1

Bonus: Pivot to Tech-as-Rows Format

If you want technologies as rows and dates as columns (like a cross-tab), use pivot_table():

tech_date_pivot = freq_stats.pivot(
    index="techs", 
    columns="date", 
    values="count"
).fillna(0).astype(int)

Result:

date 2023-10-01 2023-10-02
techs
Java 0 1
Python 2 1
SQL 1 1

Quick Tip: Handle Empty/Null Lists #

If your techs column has empty lists or None values, add dropna() to clean up after exploding:

df_exploded = df.explode("techs").dropna(subset=["techs"])

That's it! This approach gets rid of the unhashable list error and gives you the clean frequency stats you need.

内容的提问来源于stack exchange，提问作者kaksat