调整关键词列表顺序后Python程序返回结果异常,求解答
关键词统计程序异常:仅首位关键词统计正确,其余关键词返回0
我正在提升大数据分析能力,编写了一款程序:通过导入关键词CSV文件,统计这些关键词在另一份数据CSV中的出现次数。但遇到异常情况:当"matlab"位于关键词列表首位时,返回结果97(正确);但将其他词放在首位时,返回结果为0。我原本预期程序会遍历每个关键词,再遍历数据集CSV进行匹配检查,但实际执行结果不符合预期。已在第一层循环后添加打印语句,确认程序确实在遍历每个关键词,但不清楚后续逻辑为何出错。
初始代码
import csv from pandas import * import pandas as pd from array import array import csv keywords = read_csv("Book1.csv") with open('ss.csv','r') as csvfile: reader = csv.DictReader(csvfile) list=["matlab","souren","deez"] Attended=0 no_show=0 Registered=0 word = 'matlab' nuts=[] for x in list: for row in reader: if x in row['one']: Registered=Registered+1 print(Registered)
修改后的代码
import csv import pandas as pd from array import array import csv keywords = pd.read_csv("Book2.csv") biomedical = keywords['Biomedical'].tolist() Registered=0 counts = dict.fromkeys(biomedical, 0) with open('ss.csv','r') as csvfile: reader = csv.DictReader(csvfile) lines=list(reader) pd.set_option("display.max_rows", None) df = pd.read_csv('ss.csv') ss=df.stack().value_counts() print(ss) ##print(ss) #for x in biomedical: # print(x)
部分数据示例
solidworks, microsoft office, lua","Python, MATLAB, C++, HTML, CSS, Javascript, C, SOLIDWORKS, Microsoft Office, LUA","I would like to further my experience in SOLIDWORKS, my extra personal skills, and develop my coding skills.",Female,,Yes,No,Yes,No,"Mississauga, Canada",No,No,Terese Kattar,Student was approved to submit Fall 2022 application,Yes,,,10/31/2022,11/14/2022,,,,,,, Mechanical Engineering,2022 - Fall,Application Accepted (Final Status),11/14/2022,BaoAnh Le,yes,10/31/2022,Lauren,Sena,Shirley,Dacanay,,,Melissa,tkattar@ryerson.ca,(647) 518-3977,,Mechanical Engineering,,,Mississauga,L5M 6N3,Female,Canadian,In Canada,N/A,false,Active,Uploaded,2.7,No,,No,,No,No,"Accommodation and food services, Administrative and support, waste management and remediation services, Construction, Financial services and insurance, Health care and social assistance, Information and cultural industries, Management of companies and enterprises, Manufacturing, Mining, quarrying, and oil and gas extraction, Other services (except public administration), Professional, scientific and technical services, Public administration, Real estate and rental and leasing, Retail Trade, Transportation and warehousing, Wholesale trade, Educational Services",Yes,Yes,"expert in c and javascript. can proficiently use matlab. skillful in microsoft word, office and excel. knows the basic of vue and vuetify. expert in html and css. expert in google docs, slides, sheets. skillful in cad software, such as: fusion 360 and solidworks..
问题根源
核心问题是**csv.DictReader返回的是一次性迭代器**:第一次循环(第一个关键词)遍历reader时,已经把CSV里的所有行都读取完毕了。后续的关键词循环再去遍历reader时,迭代器已经没有剩余数据,自然统计不到任何结果,返回0。
解决方案
1. 修复初始代码
先把CSV的所有行读取到列表中,这样每次遍历关键词时都能重新访问所有行:
import csv import pandas as pd # 示例关键词列表,实际可替换为CSV读取的内容 keyword_list = ["matlab","souren","deez"] Registered = 0 # 一次性读取数据集所有行到内存,避免迭代器耗尽 with open('ss.csv','r') as csvfile: reader = csv.DictReader(csvfile) rows = list(reader) # 将迭代器转为列表,可重复遍历 # 遍历每个关键词,再遍历所有行统计 for x in keyword_list: for row in rows: # 统一转小写,避免大小写导致的匹配遗漏 if x.lower() in row['one'].lower(): Registered += 1 print(Registered)
2. 更高效的Pandas实现
用Pandas可以更简洁地完成统计,同时处理大小写、空值等问题:
import pandas as pd # 读取关键词CSV,提取目标列转为列表 keywords_df = pd.read_csv("Book2.csv") biomedical_keywords = keywords_df['Biomedical'].tolist() # 读取数据集CSV,处理空值 df = pd.read_csv('ss.csv') target_column = df['one'].dropna().astype(str) # 假设目标列为'one',根据实际调整 # 初始化统计字典 counts = {kw: 0 for kw in biomedical_keywords} # 遍历每行文本,统计关键词出现次数 for text in target_column: lower_text = text.lower() for kw in biomedical_keywords: if kw.lower() in lower_text: counts[kw] += 1 # 输出每个关键词的统计结果 for keyword, count in counts.items(): print(f"{keyword}: {count}")
额外优化建议
- 大小写统一:数据中常出现
MATLAB、Matlab等不同写法,统一转小写后匹配能避免漏统计 - 重复计数控制:如果同一行包含多个关键词,当前逻辑会分别给每个关键词计数;若只需统计包含任意关键词的行数,可调整为每行只计数一次
- 大文件性能优化:若数据集极大,可将关键词组合成正则表达式,批量匹配文本,提升运行效率
内容的提问来源于stack exchange,提问作者souren.p
相关产品推荐
相关产品推荐

