绘制分组时间序列折线图时维度不匹配错误的排查求助

问题 #

尝试为包含event_date、key、count字段的数据集绘制时间序列折线图，7个唯一key各对应一条折线，X轴为event_date，Y轴为count。循环遍历每个唯一key绘图时出现维度不匹配错误：ValueError: x and y must have same first dimension, but have shapes (21,) and (1,)。

代码 #

import matplotlib.pyplot as plt
import pandas as pd

df_pandas = df.toPandas()

df_pandas.event_date = pd.to_datetime(df_pandas.event_date) # converting object to pandas datetime

colors = ['r', 'g', 'b', 'y', 'k', 'c', 'm']
for i, k in enumerate(df_pandas.key.unique()):
    plt.plot(df_pandas[df_pandas.key == k].event_date, df_pandas[df_pandas.key == k].count, '-o', label=k, c=colors[i])
plt.gcf().autofmt_xdate() ## Rotate X-axis so you can see dates clearly without overlap
plt.legend() ## Show legend

报错信息 #

Traceback (most recent call last):
  File "/tmp/1697561012681-0/zeppelin_python.py", line 153, in <module>
    exec(code, _zcUserQueryNameSpace)
  File "<stdin>", line 13, in <module>
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/pyplot.py", line 2813, in plot
    is not None else {}), **kwargs)
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/__init__.py", line 1805, in inner
    return func(ax, *args, **kwargs)
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/axes/_axes.py", line 1603, in plot
    for line in self._get_lines(*args, **kwargs):
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/axes/_base.py", line 393, in _grab_next_args
    yield from self._plot_args(this, kwargs)
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/axes/_base.py", line 370, in _plot_args
    x, y = self._xy_from_xy(x, y)
  File "/usr/local/lib64/python3.7/site-packages/matplotlib/axes/_base.py", line 231, in _xy_from_xy
    "have shapes {} and {}".format(x.shape, y.shape))
ValueError: x and y must have same first dimension, but have shapes (21,) and (1,)

数据集样本 #

event_date  key count
7/23/23 0:00    389628-135052   74858
7/28/23 0:00    389628-135052   75139
7/12/23 0:00    389631-135055   60910
...
7/23/23 0:00    389629-135053   68272

解决方法 #

定位与修复步骤 #

1. 检查异常key的数据维度
   在循环中加入维度检查，快速定位出问题的key：

   for i, k in enumerate(df_pandas.key.unique()):
       subset = df_pandas[df_pandas.key == k]
       print(f"Key {k}: event_date长度 {len(subset.event_date)}, count长度 {len(subset['count'])}")
       plt.plot(subset.event_date, subset['count'], '-o', label=k, c=colors[i])
   

   根据输出找到维度不匹配的key，针对性检查该key对应的数据是否存在缺失、聚合错误等问题。

2. 避免字段名与内置方法冲突
   原代码中df_pandas.count容易和pandas内置的DataFrame.count()方法混淆，导致获取的不是列数据。改用方括号明确指定列名：

   # 把 df_pandas[df_pandas.key == k].count 替换为
   df_pandas[df_pandas.key == k]['count']
   

3. 处理数据缺失或索引问题
   检查子集是否有缺失值：

   subset = df_pandas[df_pandas.key == k]
   print(subset.isnull().sum())
   

   若存在缺失，可根据业务需求填充或删除；如果有重复索引，重置索引解决：

   subset = subset.reset_index(drop=True)
   

4. 更稳定的分组绘图方案
   直接使用pandas分组功能绘图，避免手动筛选的潜在问题，同时保证数据按日期排序：

   import matplotlib.pyplot as plt
   import pandas as pd
   
   df_pandas = df.toPandas()
   df_pandas['event_date'] = pd.to_datetime(df_pandas['event_date'])
   df_pandas = df_pandas.sort_values('event_date') # 按日期排序，折线顺序更合理
   
   colors = ['r', 'g', 'b', 'y', 'k', 'c', 'm']
   for (key, group), color in zip(df_pandas.groupby('key'), colors):
       group.plot(x='event_date', y='count', kind='line', marker='o', label=key, color=color, ax=plt.gca())
   
   plt.gcf().autofmt_xdate()
   plt.legend()
   plt.show()
   

内容的提问来源于stack exchange，提问作者Explorer