Pandas 0.21.0 groupby返回不同索引异常问题咨询

Background & Version Info #

I’m addressing this issue using your provided context and environment details:

• Pandas 0.21.0
• Python 2.7.12

You noticed that when using groupby with sort_values on different DataFrames, the resulting index structure is inconsistent—specifically, df1's descending sort returns a multi-index with the group reference time, while other cases don’t. Let’s break this down step by step.

Reproducible Code #

import pandas as pd
from datetime import datetime
df1 = pd.DataFrame({'dtime': [datetime(2017,1,1,1,5), datetime(2017,1,1,1,20)], 'val1': [11, None], 'val2': [None, 31] })
df2 = pd.DataFrame({'dtime': [datetime(2017,1,1,1,5), datetime(2017,1,1,1,20)], 'val1': [11, None], 'val2': [31, None] })
df1b = df1.melt("dtime").dropna().set_index("dtime")
df2b = df2.melt("dtime").dropna().set_index("dtime")
r1a = df1b.groupby(pd.Grouper(freq="1h")).value.apply(lambda x: x.sort_values(ascending=True))
r1b = df1b.groupby(pd.Grouper(freq="1h")).value.apply(lambda x: x.sort_values(ascending=False))
r2a = df2b.groupby(pd.Grouper(freq="1h")).value.apply(lambda x: x.sort_values(ascending=True))
r2b = df2b.groupby(pd.Grouper(freq="1h")).value.apply(lambda x: x.sort_values(ascending=False))
print "\n--- df1 ascending sort ---------------\n", r1a
print "\n--- df1 descending sort SHOULD IT ALWAYS BE LIKE THIS? --------------\n", r1b
print "\n--- df2 ascending sort ---------------\n", r2a
print "\n--- df2 descending sort --------------\n", r2b

Observed Output #

--- df1 ascending sort ---------------
dtime
2017-01-01 01:05:00    11.0
2017-01-01 01:20:00    31.0
Name: value, dtype: float64
--- df1 descending sort SHOULD IT ALWAYS BE LIKE THIS? --------------
dtime                  dtime
2017-01-01 01:00:00  2017-01-01 01:20:00    31.0
                      2017-01-01 01:05:00    11.0
Name: value, dtype: float64
--- df2 ascending sort ---------------
dtime
2017-01-01 01:05:00    11.0
2017-01-01 01:05:00    31.0
Name: value, dtype: float64
--- df2 descending sort --------------
dtime
2017-01-01 01:05:00    31.0
2017-01-01 01:05:00    11.0
Name: value, dtype: float64

Why This Inconsistency Happens #

This is a bug specific to older Pandas versions (pre-0.24+) tied to how the library handles index alignment during groupby.apply operations:

• For df1b, after melting and dropping NaNs, your index has unique timestamps (01:05 and 01:20). When you sort in descending order, Pandas 0.21.0 fails to properly flatten the sorted series back to the expected structure, resulting in the group’s reference time (01:00:00) being added as a top-level index.
• For df2b, the melted data has duplicate timestamps (both entries are at 01:05). In this case, the sorting operation retains the original flat index instead of promoting the group key to a multi-index—creating the mismatch you observed.

Your expectation of consistent multi-index output is totally valid: groupby operations should uniformly include the group key in the result index, regardless of the underlying data’s index uniqueness or sort direction.

Fixes & Workarounds #

1. Upgrade Pandas: The cleanest solution is to move to a newer Pandas version (1.x or later). Index handling in groupby operations was completely overhauled in post-0.24 releases, which resolves this inconsistency entirely.
2. Explicit Index Construction (for Pandas 0.21.0): If you can’t upgrade, you can force a consistent multi-index by modifying your apply function to preserve the group key:

   # Force multi-index output for df1b's descending sort
   r1b_fixed = df1b.groupby(pd.Grouper(freq="1h")).value.apply(
       lambda x: x.sort_values(ascending=False).reset_index(level=0, drop=False)
   ).set_index(["dtime", "dtime"])
   

   This explicitly retains the group key as part of the index, matching the structure you expected.

内容的提问来源于stack exchange，提问作者Alex Poca