如何删除Pandas DataFrame中含单/双/三字符重复序列的行

实现思路 #

• 通过正则表达式匹配三类任意字母的重复字符序列：
  • 单字符连续重复3次及以上：([a-zA-Z])\1{2,}，可匹配uuuu、zzzz这类场景
  • 双字符组连续重复2次及以上：([a-zA-Z]{2})\1{1,}，可匹配eded、rsrsrs这类场景
  • 三字符组连续重复2次及以上：([a-zA-Z]{3})\1{1,}，可匹配xyzxyz这类场景
• 逐行校验DataFrame所有列，只要任意单元格命中匹配规则就删除整行
• 清洗完成后重置行索引

完整可运行代码 #

import pandas as pd
import numpy as np
import re

data = {'Address1': ['High Street', 'wssssss', 'qfdgfdgdg', 'Green Lane', 'Kingsway', 'Church Street', 'qaaaaass'],
        'Address2': ['Park Avenue', 'The Crescent', 'dddfffff', 'Highfield Road', 'Stanley Road', 'New Street', 'mjkhjk'],
        'Address3': ['St. John’s Road', 'tyutyut', 'qdffgfdgfggfbvbvbv', 'Springfield Road', 'George Street', 'Queensway', 'chfghfghh'],
        'Address4': ['The Grove', 'Mill Road', 'sefsdfdyuytutu', 'School Lane', 'Albert Road', 'Broadway', 'fghfhfh']}

address_details = pd.DataFrame(data)

# 匹配重复序列的正则规则
repeat_pattern = r'([a-zA-Z])\1{2,}|([a-zA-Z]{2})\1{1,}|([a-zA-Z]{3})\1{1,}'
# 标记所有包含重复序列的行
bad_row_mask = address_details.apply(lambda col: col.str.contains(repeat_pattern, regex=True)).any(axis=1)
# 过滤掉不符合要求的行并重置索引
address_details = address_details[~bad_row_mask].reset_index(drop=True)

print(address_details)

输出结果 #

Address1         Address2          Address3     Address4
0   High Street     Park Avenue   St. John’s Road    The Grove
1    Green Lane  Highfield Road  Springfield Road  School Lane
2      Kingsway    Stanley Road     George Street  Albert Road
3 Church Street      New Street         Queensway     Broadway

内容的提问来源于stack exchange，提问作者hello