求助：使用Pandas生成CSV仅输出一行数据的问题排查

核心原因：缩进错误 #

你的代码里，遍历URL的for循环结束后才执行结果存入列表的操作，导致只有最后一个URL的分析结果被添加到列表中，最终生成的CSV自然只有1行。

具体来说，sentiment = doc._.blob.polarity开始到total_neg.append(', '.join(set(negative_words)))的所有代码，都应该缩进在for count, x in enumerate(urls):的循环体内部，这样每个URL处理完后，都会把结果存入对应的列表。

另外还有个小问题：处理HTML文本时，page_text和doc的生成被放在了遍历soup(["script", ...])的循环里，这会导致每删除一个标签就重新处理一次文本，完全没必要，应该把这部分代码移到删除标签的循环外面。

修正后的代码 #

import spacy
from spacytextblob.spacytextblob import SpacyTextBlob
import pandas as pd
from bs4 import BeautifulSoup
import requests

nlp = spacy.load('en_core_web_sm')
nlp.add_pipe('spacytextblob')
df = pd.read_csv("urls2.csv")
urls = df["Address"].tolist()

url_sent_score = []
url_sent_label = []
total_pos = []
total_neg = []

for count, x in enumerate(urls):
    url = x
    headers = {'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'}
    res = requests.get(url, headers=headers)
    html_page = res.text

    soup = BeautifulSoup(html_page, 'html.parser')
    # 移除不需要的标签，仅负责删除操作
    for script in soup(["script", "style","meta","label","header","footer"]):
        script.decompose()
    # 文本处理与doc生成移到循环外，仅执行一次
    page_text = (soup.get_text()).lower()
    page_text = page_text.strip().replace("  ","")
    page_text = "".join([s for s in page_text.splitlines(True) if s.strip("\r\n")])
    doc = nlp(page_text)

    # 以下代码缩进在for循环内，每个URL处理后立即存入列表
    sentiment = doc._.blob.polarity
    sentiment = round(sentiment, 2)

    if sentiment > 0:
        sent_label = "Positive"
    else:
        sent_label = "Negative"

    url_sent_label.append(sent_label)
    url_sent_score.append(sentiment)
    positive_words = []
    negative_words = []

    for x in doc._.blob.sentiment_assessments.assessments:
        if x[1] > 0:
            positive_words.append(x[0][0])
        elif x[1] < 0:
            negative_words.append(x[0][0])
        else:
            pass

    total_pos.append(', '.join(set(positive_words)))
    total_neg.append(', '.join(set(negative_words)))

# 生成结果文件
data = {
      'Sentiment Score':url_sent_score,
      'Sentiment Label':url_sent_label,
      'Positive Words':total_pos,
      'Negative Words':total_neg}

df=pd.DataFrame(data)
df.to_csv("sentiment.csv")
df.to_json("sentiment.json",orient="split")

内容的提问来源于stack exchange，提问作者Paul Chavaux