如何用Python从Etsy HTML提取星级评分？代码修复咨询

问题背景 #

现有Python代码可从本地存储的Etsy网页源码文本文件中提取商品描述、价格、评论数并存入DataFrame，但星级评分列始终为空。以下是原代码、对应HTML片段及输出情况：

原代码 #

import os
import pandas as pd
import re

directory_path = r"C:\Users\..."

descriptions = []
prices = []
review_numbers = []
file_names = []
stars = []

for file_name in os.listdir(directory_path):
    if file_name.endswith(".txt"):
        file_path = os.path.join(directory_path, file_name)
        
        with open(file_path, "r", encoding="utf-8") as file:
            lines = file.readlines()
            description = ""
            price = ""
            review_number = ""
            star_rating = ""
            for line in lines:
                if line.startswith('----------------------------------------'):
                    if description != "":
                        descriptions.append(description.strip())
                        prices.append(price)
                        review_numbers.append(review_number)
                        file_names.append(file_name)
                        stars.append(star_rating)

                    description = ""
                    price = ""
                    review_number = ""
                    star_rating = ""
                elif re.match(r'\$\d+(?:\.\d+)?', line):
                    price = line.strip()
                elif re.search(r'\(\d[\d,]*\)', line):
                    review_number = re.search(r'\(\d[\d,]*\)', line).group()
                else:
                    description += line.strip() + " "

                star_match = re.search(r'class="wt-align-items-center wt-max-height-full wt-display-flex-xs flex-direction-row-xs wt-text-title-small wt-no-wrap">([\d.]+)', line)
                if star_match:
                    star_rating = star_match.group(1)

                if 'v2-listing-card_title' in line:
                    description = line.split('title="', 1)[1].split('"', 1)[0]

            if description != "":
                descriptions.append(description.strip())
                prices.append(price)
                review_numbers.append(review_number)
                file_names.append(file_name)
                stars.append(star_rating)

data = {'Description': descriptions, 'Price': prices, 'Review_Number': review_numbers, 'File_Name': file_names, 'Stars': stars}
df = pd.DataFrame(data)

包含星级的HTML片段 #

<div class="v2-listing-card_info">
<h3 class="wt-text-caption v2-listing-card_title wt-text-truncate" title="Luna moth cape dance wings costume adult Spanish luna moth cape chiffon belly halloween burning man" id="ad-listing-title-280310434"></h3>
<div class="streamline-spacing-shop-rating">
<div class="shop-name-with-rating wt-flex-wrap wt-display-flex-xs flex-direction-row-xs wt-align-items-center">
<span class="wt-display-inline-flex-xs wt-flex-wrap wt-align-items-center larger_review_stars">
<div role="img" aria-label="4.9 star rating with 15.4k reviews" class="wt-align-items-center wt-max-height-full wt-display-flex-xs flex-direction-row-xs wt-text-title-small wt-no-wrap">
"4.9"
</div>
</span>
</div>
</div>

当前输出情况 #

修复方案 #

原代码无法提取星级的核心问题：

1. 正则表达式要求class属性和星级数字在同一行，但实际源码中两者被换行拆分
2. 星级数字可能包裹在引号中，且存在两种提取路径（aria-label属性或直接文本）

修改后的代码如下：

import os
import pandas as pd
import re

directory_path = r"C:\Users\..."

descriptions = []
prices = []
review_numbers = []
file_names = []
stars = []

for file_name in os.listdir(directory_path):
    if file_name.endswith(".txt"):
        file_path = os.path.join(directory_path, file_name)
        
        # 读取整个文件内容（而非按行读），解决跨换行匹配问题
        with open(file_path, "r", encoding="utf-8") as file:
            content = file.read()
            # 按分隔符分割每个商品区块
            product_blocks = re.split(r'----------------------------------------', content)
            
            for block in product_blocks:
                if not block.strip():
                    continue
                    
                description = ""
                price = ""
                review_number = ""
                star_rating = ""
                
                # 提取商品标题
                title_match = re.search(r'v2-listing-card_title.*?title="(.*?)"', block, re.DOTALL)
                if title_match:
                    description = title_match.group(1).strip()
                
                # 提取价格
                price_match = re.search(r'\$\d+(?:\.\d+)?', block)
                if price_match:
                    price = price_match.group().strip()
                
                # 提取评论数
                review_match = re.search(r'\(\d[\d,]*\)', block)
                if review_match:
                    review_number = review_match.group().strip()
                
                # 提取星级：优先从aria-label取，再匹配直接文本
                star_match = re.search(r'aria-label="([\d.]+) star rating', block)
                if not star_match:
                    star_match = re.search(r'wt-no-wrap.*?([\d.]+)', block, re.DOTALL)
                if star_match:
                    star_rating = star_match.group(1).strip().strip('"')
                
                descriptions.append(description)
                prices.append(price)
                review_numbers.append(review_number)
                file_names.append(file_name)
                stars.append(star_rating)

data = {'Description': descriptions, 'Price': prices, 'Review_Number': review_numbers, 'File_Name': file_names, 'Stars': stars}
df = pd.DataFrame(data)

关键修改点 #

1. 读取整个文件内容：不再按行读取，而是一次性读取全部内容，按分隔符分割商品区块，避免跨行匹配的问题
2. 优化星级提取逻辑：
   • 优先从aria-label属性中提取星级（格式为"X.X star rating..."），这是最可靠的来源
   • 如果aria-label匹配失败，再匹配目标class附近的数字文本
   • 处理数字周围的引号，确保提取到干净的星级值
3. 区块化处理商品：每个商品对应一个分割后的区块，逻辑更清晰，避免状态重置错误

内容的提问来源于stack exchange，提问作者Santiago V