如何在Pandas中根据另一列表的值生成不连续序列？

How to efficiently add a sequential ID column to a Pandas DataFrame based on row values? #

Problem Description

I'm trying to add a sequential Ida column to my Pandas DataFrame, and I expect the following result:

My initial code works but runs extremely slowly:

Address['Ida'] = ''
Ida = 1
Address['Ida'][0] = Ida
for x in range(len(Address)-1):
    if str(Address['Unit'][x+1]) == ('Parcel 1' or ''):
        Ida = Ida + 1
        Address['Ida'][x+1] = Ida
    else:
        Address['Ida'][x+1] = Ida

Is there a more efficient way to achieve this in Pandas?

Answer

First, let's break down why your original code is so slow:

• Row-wise loops are anti-patterns in Pandas: Pandas is built around vectorized operations that leverage C-level optimizations. Iterating through each row with a Python for loop completely bypasses these optimizations, leading to terrible performance—especially with large DataFrames.
• Chained indexing is risky: Using Address['Ida'][x+1] to modify values can lead to unintended behavior (like modifying a copy instead of the original DataFrame) on top of being slow.

Here's a much faster, vectorized approach that aligns with Pandas' best practices:

Step 1: Understand the grouping logic #

Looking at your desired output, the Ida value increments whenever we hit a "group start" row. These start rows are:

1. The very first row of the DataFrame
2. Any row where Unit is Parcel 1 (the start of a parcel group)
3. Any row where Unit does not start with Parcel (like the numeric values 1, 3, 4)

Step 2: Vectorized implementation #

We can create a boolean mask to mark these group start rows, then use cumsum() to compute the sequential Ida values in one go:

import pandas as pd

# Example DataFrame (replace with your actual data)
data = {'Unit': ['1', 'Parcel 1', 'Parcel 2', 'Parcel 3', '3', '4', 'Parcel 1', 'Parcel 2']}
Address = pd.DataFrame(data)

# Create a mask for group start rows
start_mask = (Address['Unit'] == 'Parcel 1') | (~Address['Unit'].str.startswith('Parcel'))
# Ensure the first row is marked as a start (since it's the initial group)
start_mask.iloc[0] = True

# Compute the sequential Ida column using cumulative sum
Address['Ida'] = start_mask.cumsum()

Running this code will produce exactly the output you want, and it will handle even large DataFrames in milliseconds instead of minutes.

Fixing the original condition issue #

Note that your original code's condition str(Address['Unit'][x+1]) == ('Parcel 1' or '') doesn't actually check for empty strings—('Parcel 1' or '') evaluates to just 'Parcel 1' in Python. If you do need to handle empty Unit values as group starts, adjust the mask like this:

start_mask = (Address['Unit'] == 'Parcel 1') | (~Address['Unit'].str.startswith('Parcel')) | (Address['Unit'] == '')
start_mask.iloc[0] = True

内容的提问来源于stack exchange，提问作者yuchen huang