You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

虚假新闻检测训练报错:已建logs目录仍触发FailedPreconditionError

虚假新闻检测训练报错:./logs is not a directory 问题排查

问题描述

开发虚假新闻检测项目,完成数据预处理后执行模型训练时出现如下错误,且已在项目目录下创建logs文件夹但问题仍存在:

Traceback (most recent call last):
File "c:\Users\asus\Masaüstü\BitirmeProjesi\training.py", line 123, in 
trainer.train()
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\trainer.py", line 1591, in train
return inner_training_loop(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\trainer.py", line 1826, in _inner_training_loop
self.control = self.callback_handler.on_train_begin(args, self.state, self.control)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\trainer_callback.py", line 362, in on_train_begin
return self.call_event("on_train_begin", args, state, control)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\trainer_callback.py", line 406, in call_event
result = getattr(callback, event)(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\integrations\integration_utils.py", line 628, in on_train_begin
self._init_summary_writer(args, log_dir)
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\integrations\integration_utils.py", line 614, in _init_summary_writer
self.tb_writer = self._SummaryWriter(log_dir=log_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\tensorboard\writer.py", line 243, in __init__
self._get_file_writer()
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\tensorboard\writer.py", line 273, in _get_file_writer
self.file_writer = FileWriter(
^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\tensorboard\writer.py", line 72, in __init__
self.event_writer = EventFileWriter(
^^^^^^^^^^^^^^^^
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\tensorboard\summary\writer\event_file_writer.py", line 72, in __init__
tf.io.gfile.makedirs(logdir)
File "C:\Users\asus\AppData\Local\Programs\Python\Python311\Lib\site-packages\tensorflow\python\lib\io\file_io.py", line 513, in recursive_create_dir_v2
_pywrap_file_io.RecursivelyCreateDir(compat.path_to_bytes(path))
tensorflow.python.framework.errors_impl.FailedPreconditionError: ./logs is not a directory

训练代码

import torch
import pandas as pd
from transformers.file_utils import is_tf_available, is_torch_available, is_torch_tpu_available
from transformers import DistilBertForSequenceClassification, DistilBertTokenizerFast
from transformers import Trainer, TrainingArguments
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

import random

news_d=pd.read_csv('C:\\Users\\asus\\Masaüstü\\BitirmeProjesi\\preprocessed_data.csv', encoding='utf-8')

def set_seed(seed: int):
    random.seed(seed)
    np.random.seed(seed)
    if is_torch_available():
        torch.manual_seed(seed)
        torch.cuda.manual_seed_all(seed)
        # ^^ safe to call this function even if cuda is not available
    if is_tf_available():
        import tensorflow as tf

        tf.random.set_seed(seed)

set_seed(1)

model_name = "distilbert-base-uncased"
# max sequence length for each document/sentence sample
max_length = 300

# load the tokenizer
tokenizer = DistilBertTokenizerFast.from_pretrained(model_name, do_lower_case=True)

news_df = news_d[news_d['text'].notna()]
news_df = news_df[news_df["title"].notna()]

def prepare_data(df, test_size=0.2, include_title=True):
  texts = []
  labels = []
  for i in range(len(df)):
    text = df["text"].iloc[i]
    label = df["label"].iloc[i]
    if include_title:
      text = df["title"].iloc[i] + " - " + text
    if text and label in [0, 1]:
      texts.append(text)
      labels.append(label)
  return train_test_split(texts, labels, test_size=test_size)

train_texts, valid_texts, train_labels, valid_labels = prepare_data(news_df)

print(len(train_texts), len(train_labels))
print(len(valid_texts), len(valid_labels))

# tokenize the dataset, truncate when passed `max_length`, 
# and pad with 0's when less than `max_length`
train_encodings = tokenizer(train_texts, truncation=True, padding=True, max_length=max_length)
valid_encodings = tokenizer(valid_texts, truncation=True, padding=True, max_length=max_length)

class NewsGroupsDataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {k: torch.tensor(v[idx]) for k, v in self.encodings.items()}
        item["labels"] = torch.nn.functional.one_hot(torch.tensor([self.labels[idx]]), num_classes=2).squeeze(0)
        return item

    def __len__(self):
        return len(self.labels)

# convert our tokenized data into a torch Dataset
train_dataset = NewsGroupsDataset(train_encodings, train_labels)
valid_dataset = NewsGroupsDataset(valid_encodings, valid_labels)

# load the model
model = DistilBertForSequenceClassification.from_pretrained(model_name, num_labels=2)

def compute_metrics(pred):
  labels = pred.label_ids
  preds = pred.predictions.argmax(-1)
  # calculate accuracy using sklearn's function
  acc = accuracy_score(labels, preds)
  return {
      'accuracy': acc,
  }

# Training configuration
training_args = TrainingArguments(
    output_dir='./results',
    num_train_epochs=1,
    per_device_train_batch_size=10,
    per_device_eval_batch_size=20,
    warmup_steps=100,
    logging_dir='./logs', 
    load_best_model_at_end=False,
    logging_steps=200,
    save_steps=200,
    evaluation_strategy="steps",
)

trainer = Trainer(
    model=model,                         # the instantiated Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=train_dataset,         # training dataset
    eval_dataset=valid_dataset,          # evaluation dataset
    compute_metrics=compute_metrics,     # the callback that computes metrics of interest
)

# train the model
trainer.train()

原因分析与解决方案

可能原因

  • 工作目录不匹配:代码中使用相对路径./logs,但运行脚本时的终端工作目录并非项目根目录,导致程序找不到你创建的logs文件夹
  • 权限不足:虽然创建了logs文件夹,但Python进程没有对该文件夹的读写权限
  • 同名文件存在:项目目录下可能存在名为logs的文件(而非文件夹),导致TensorFlow无法将其识别为目录
  • TensorFlow路径解析问题:Windows系统下,TensorFlow的tf.io.gfile对相对路径的解析逻辑和Python原生存在差异,或路径存在编码问题

对应解决方案

  1. 验证并修正工作目录
    在代码开头添加以下代码,打印当前工作目录:

    import os
    print("当前工作目录:", os.getcwd())
    

    如果输出不是C:\Users\asus\Masaüstü\BitirmeProjesi,可以:

    • 切换终端到项目根目录后再运行脚本
    • 或修改TrainingArguments中的logging_dir为绝对路径:
      logging_dir='C:\\Users\\asus\\Masaüstü\\BitirmeProjesi\\logs'
      
  2. 检查文件夹权限
    右键项目目录下的logs文件夹,选择「属性」→「安全」,确认当前用户拥有「读取」「写入」权限,若没有则添加对应权限。

  3. 检查是否存在同名文件
    打开项目目录,确认logs是文件夹而非文件。如果是文件,删除后重新创建logs文件夹。

  4. 临时禁用TensorBoard日志(可选)
    如果不需要使用TensorBoard查看训练日志,可以在TrainingArguments中添加report_to='none',跳过日志文件的创建:

    training_args = TrainingArguments(
        # 其他参数保持不变
        report_to='none'
    )
    

内容的提问来源于stack exchange,提问作者Hilal Derya Eryılmaz

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.03 18:53:17