基于机器学习的积极文本二分类（积极/非常积极）Python实现方案咨询

Absolutely, splitting positive texts into "positive" and "very positive" is totally doable with machine learning in Python—here's a practical, step-by-step approach you can follow:

1. 数据准备：打好分类基础 #

细粒度的情感分类最核心的是高质量标注数据：

• 标注数据获取：如果没有现成数据集，要么手动标注（可以用label-studio这类Python可集成的标注工具提高效率），要么从公开情感数据集（比如Amazon Reviews、IMDB）中筛选出原标记为"positive"的文本，再做二次细分标注。
• 数据预处理：清洗文本（去除特殊字符、冗余空格）、小写化、分词，用nltk或spaCy过滤停用词，比如：

  import nltk
  from nltk.corpus import stopwords
  nltk.download('stopwords')
  stop_words = set(stopwords.words('english'))
  
  def preprocess(text):
      text = text.lower().strip()
      tokens = text.split()
      return " ".join([t for t in tokens if t not in stop_words])
  

2. 特征工程：把文本转成模型能理解的数值 #

两种主流方案，按需选择：

• 传统TF-IDF特征：适合快速验证思路，用sklearn实现，还可以加入n-gram捕捉"best ever"这类强积极短语：

  from sklearn.feature_extraction.text import TfidfVectorizer
  tfidf = TfidfVectorizer(ngram_range=(1,2), max_features=5000)
  X = tfidf.fit_transform(processed_texts)
  

• 预训练语言模型嵌入：效果更优，能理解上下文语义，用transformers库提取BERT/RoBERTa的文本嵌入：

  from transformers import AutoTokenizer, AutoModel
  import torch
  
  tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
  model = AutoModel.from_pretrained("bert-base-uncased")
  
  def get_text_embedding(text):
      inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
      with torch.no_grad():
          outputs = model(**inputs)
      return outputs.last_hidden_state[:, 0, :].numpy()  # 取<[BOS_never_used_51bce0c785ca2f68081bfa7d91973934]>token作为文本嵌入
  

3. 模型训练：二分类任务的两种路线 #

这是典型的二分类任务（positive vs very positive），有两种成熟方案：

• 传统机器学习模型：快速迭代，适合数据量不大的场景，用sklearn就能搞定：

  from sklearn.model_selection import train_test_split
  from sklearn.linear_model import LogisticRegression
  from sklearn.metrics import classification_report
  
  # X是特征矩阵，y是标注标签（比如0=positive，1=very positive）
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
  clf = LogisticRegression(max_iter=1000)
  clf.fit(X_train, y_train)
  # 评估模型
  print(classification_report(y_test, clf.predict(X_test)))
  

• 微调预训练语言模型：效果天花板，适合数据量充足的场景，用transformers的Trainer API：

  from transformers import AutoModelForSequenceClassification, Trainer, TrainingArguments
  import torch
  
  # 加载分类模型（num_labels=2对应二分类）
  model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
  training_args = TrainingArguments(
      output_dir="./fine_tuned_model",
      per_device_train_batch_size=8,
      num_train_epochs=3,
      evaluation_strategy="epoch",
      logging_dir="./logs"
  )
  
  # 自定义数据集类
  class SentimentDataset(torch.utils.data.Dataset):
      def __init__(self, texts, labels, tokenizer):
          self.encodings = tokenizer(texts, truncation=True, padding=True)
          self.labels = labels
      def __getitem__(self, idx):
          item = {k: torch.tensor(v[idx]) for k, v in self.encodings.items()}
          item["labels"] = torch.tensor(self.labels[idx])
          return item
      def __len__(self):
          return len(self.labels)
  
  train_dataset = SentimentDataset(train_texts, train_labels, tokenizer)
  test_dataset = SentimentDataset(test_texts, test_labels, tokenizer)
  
  trainer = Trainer(
      model=model,
      args=training_args,
      train_dataset=train_dataset,
      eval_dataset=test_dataset
  )
  
  trainer.train()
  trainer.evaluate()
  

4. 模型优化与评估 #

• 重点关注F1-score、召回率（避免把"very positive"误判成普通positive），用混淆矩阵分析错误案例，针对性补充难区分的样本。
• 可以尝试模型融合（比如把TF-IDF+逻辑回归和BERT的预测结果加权融合）提升效果。

5. 部署推理 #

把训练好的模型封装成API，用FastAPI快速实现：

from fastapi import FastAPI
from transformers import pipeline

app = FastAPI()
# 加载微调后的模型
classifier = pipeline("text-classification", model="./fine_tuned_model")
label_map = {"LABEL_0": "positive", "LABEL_1": "very positive"}

@app.post("/classify_sentiment")
def classify_text(text: str):
    result = classifier(text)[0]
    return {
        "input_text": text,
        "category": label_map[result["label"]],
        "confidence": round(result["score"], 4)
    }

内容的提问来源于stack exchange，提问作者Leaf Leafword