如何在Python DataFrame列中精准去除字符串冗余部分？

我是Python新手，基于Reddit用户u/commandlineluser的脚本修改，想要移除DataFrame列中各URL对应的文件名里的可变冗余后缀，但用字典+str.replace的方式处理后，部分结果不符合预期。

尝试的代码 #

import pandas as pd
import re
data = {"full_url": ['https://meps.ahrq.gov/data_files/pufs/h225/h225dat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h51bdat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h47f1dat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h225/h225ssp.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220i/h220if1dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220h/h220hv9.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220e/h220exlsx.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h224/h224xlsx.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h036brr/h36brr20dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h036/h36u20dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h197i/h197if1dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h197i/h197if2dta.zip']}
df = pd.DataFrame(data)
extensions = ["dat", "ssp", "dta", "20dta", "u20dta", "f1dta", "f2dta", "v9", "xlsx"]
replacements = dict.fromkeys((f"{ext}[.]zip$" for ext in extensions), "")
df["file_id"] = df["full_url"].str.split("/").str[-1].replace(replacements, regex=True)
print(df["file_id"])

当前输出与期望对比 #

0         h225 (符合预期)
1         h51b (符合预期)
2        h47f1 (期望: h47)
3         h225 (符合预期)
4      h220if1 (期望: h220i)
5        h220h  (符合预期)
6        h220e  (符合预期)
7         h224 (符合预期)
8     h36brr20 (期望: h36brr)
9       h36u20 (期望: h36)
10     h197if1 (期望: h197i)
11     h197if2 (期望: h197i)

问题原因 #

问题出在正则匹配的优先级上：extensions列表里短后缀（比如dta）排在长后缀（比如f1dta）前面，pandas的str.replace会按字典插入顺序匹配正则，导致短后缀先被匹配。例如h220if1dta.zip会先匹配到dta.zip，替换后得到h220if1，而非匹配更长的f1dta.zip得到期望的h220i。

解决方案 #

把extensions按字符串长度从长到短排序，让长后缀的正则先被匹配，就能优先替换完整的冗余部分：

import pandas as pd
import re
data = {"full_url": ['https://meps.ahrq.gov/data_files/pufs/h225/h225dat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h51bdat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h47f1dat.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h225/h225ssp.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220i/h220if1dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220h/h220hv9.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h220e/h220exlsx.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h224/h224xlsx.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h036brr/h36brr20dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h036/h36u20dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h197i/h197if1dta.zip',
                     'https://meps.ahrq.gov/data_files/pufs/h197i/h197if2dta.zip']}
df = pd.DataFrame(data)
extensions = ["dat", "ssp", "dta", "20dta", "u20dta", "f1dta", "f2dta", "v9", "xlsx"]
# 按字符串长度降序排序，长后缀优先匹配
extensions_sorted = sorted(extensions, key=lambda x: -len(x))
replacements = dict.fromkeys((f"{ext}[.]zip$" for ext in extensions_sorted), "")
df["file_id"] = df["full_url"].str.split("/").str[-1].replace(replacements, regex=True)
print(df["file_id"])

修改后输出 #

0         h225
1         h51b
2          h47
3         h225
4        h220i
5        h220h
6        h220e
7         h224
8       h36brr
9         h36
10       h197i
11       h197i

内容的提问来源于stack exchange，提问作者user1849365