将soup元素转为str传入read_html出现FutureWarning的解决咨询

当把BeautifulSoup元素转为字符串后传入pandas的read_html方法时，会出现如下警告：

"FutureWarning: Passing literal html to 'read_html' is deprecated and will be removed in a future version. To read from a literal string, wrap it in a 'StringIO' object"

目前解析HTML表格能得到正确结果，但需要消除这个警告。

解决步骤 #

1. 导入标准库io中的StringIO模块；
2. 将原本直接传入的str(soup)替换为StringIO(str(soup))，把HTML字符串包装成类文件对象。

修改后的完整代码 #

from io import StringIO
import pandas as pd
from bs4 import BeautifulSoup

def parse_html(box_scores):
    with open(box_scores) as f: 
        html = f.read()
    
    soup = BeautifulSoup(html, features="lxml")
    [s.decompose() for s in soup.select("tr.over_header")]
    [s.decompose() for s in soup.select("tr.theader")]
    return soup


def read_line_score(soup):
    # 用StringIO包装HTML字符串
    line_score = pd.read_html(StringIO(str(soup)), attrs = {'id': 'line_score'})[0]
    cols = list(line_score.columns)
    cols[0] = "team"
    cols[-1] = "total"
    line_score.columns = cols
    
    line_score = line_score[["team", "total"]]
    
    return line_score

def read_stats(soup, team, stat):
    # 用StringIO包装HTML字符串
    df = pd.read_html(StringIO(str(soup)), attrs={"id": f"box-{team}-game-{stat}"}, index_col=0)[0]
    df = df.apply(pd.to_numeric, errors="coerce")
    return df

说明 #

pandas官方弃用直接传字符串的方式，是因为read_html的设计初衷是读取文件路径或类文件对象（比如StringIO、文件句柄）。用StringIO包装字符串后，既符合方法的设计规范，也能避免未来版本的兼容性问题。

内容的提问来源于stack exchange，提问作者Cayp