同期群分析中，分组后用agg方法如何包含含NaN值的所有行？

Hey there! Let's figure out how to get your cohort analysis grouping to count all rows—including those with NaN values—like you want.

First, let's break down the core issue: when you use agg() with functions like count(), it automatically excludes rows with missing values (NaN) for the column you're aggregating. But since you need to include every user (even those with no status_id/status_period), we can use a couple of simple workarounds:

1. Use size() instead of count() for total row counts #

The size() method counts the total number of rows in each group, regardless of NaN values. Since your id column has no missing values (looking at your sample DataFrame), we can use that to get the full count of users per cohort-period group:

# 统计每个cohort-group + status_period组的总用户数（含NaN行）
cohort_stats = df.groupby(['cohort_group', 'status_period']).agg(
    total_users=('id', 'size')
)

This will include the rows where status_period is NaN as a separate group for each cohort—perfect for tracking users who didn't have any activity in a period.

2. Combine total and active user counts (for cohort retention) #

If you want to calculate both the total cohort size and the number of active users (those with non-NaN status_id), you can mix size() and count() in the same agg() call:

cohort_stats = df.groupby(['cohort_group', 'status_period']).agg(
    total_users=('id', 'size'),       # 所有用户（含无活动的）
    active_users=('status_id', 'count')  # 有活动记录的用户
)

This gives you exactly what you need for cohort retention analysis: you can later calculate retention rates as active_users / total_users for each group.

Why this works #

• size(): Returns the total number of observations in the group, ignoring no values—this is key for including users with NaN in status_period.
• count(): Only counts non-missing values, which is ideal for tracking active users who have a valid status_id.

For example, in your sample DataFrame, the 2017-12 cohort has one user with NaN in status_period—using size() will count that user, while count() on status_period would skip them.

内容的提问来源于stack exchange，提问作者Alexandre Lara