Parallel.ForEach与foreach统计目录数据结果不一致的原因排查

问题背景 #

我正在完成一个C#课程小项目，需分别使用foreach和Parallel.ForEach统计指定目录的文件夹总数、文件总数及文件总大小。

最初我采用递归返回函数实现，但因基准情况需覆盖所有返回代码路径而遇到问题，于是改用ref参数传递统计值。以下是当前的方法代码：

/*
* Calculate total directories, count of files, and size of all files from
* a given path using a singular foreach. Update passed reference parameters.
*/
static void singRecurse(DirectoryInfo di, ref int countFolder, ref int countFile,
    ref long countByte)
{
    try{
        DirectoryInfo[] directories = di.GetDirectories();
        foreach(DirectoryInfo d in directories){
            countFolder += 1;
            foreach(FileInfo f in d.GetFiles()){
                countFile += 1;
                countByte += f.Length;
            }
            singRecurse(d, ref countFolder, ref countFile, ref countByte);
        }
    } catch (UnauthorizedAccessException){
        Console.WriteLine("You do not have access to this directory");
    }
}

/*
* Calculate total directories, count of files, and size of all files from
* a given path using a parallel foreach. Update passed reference parameters.
*/
static void parRecurse(DirectoryInfo di, ref int countFolder, ref int countFile,
    ref long countByte)
{
    int countFolderinLambda = countFolder;
    int countFileinLambda = countFile;
    long countByteinLambda = countByte;

    try{
        DirectoryInfo[] directories = di.GetDirectories();
        Parallel.ForEach(directories, d => {
            countFolderinLambda += 1;
            foreach(FileInfo f in d.GetFiles()){
                countFileinLambda += 1;
                countByteinLambda += f.Length;
            }
            parRecurse(d, ref countFolderinLambda, ref countFileinLambda,
                ref countByteinLambda);
        });
    } catch (UnauthorizedAccessException){
        Console.WriteLine("You do not have access to this directory");
    }

    countFolder = countFolderinLambda;
    countFile = countFileinLambda;
    countByte = countByteinLambda;
}

运行输出 #

Parallel calculated in 44ms
6 folders, 20 files, 250498 bytes
    
Single calculated in 11ms
8 folders, 25 files, 405153 bytes

问题原因 #

核心问题是并行环境下的线程安全冲突：

• parRecurse里的countFolderinLambda、countFileinLambda、countByteinLambda是普通变量，Parallel.ForEach会启动多个线程同时执行lambda逻辑，多个线程会并发执行+=操作。
• +=并非原子操作，它包含「读取当前值→计算新值→写入新值」三个步骤。当多个线程同时操作时，会出现值覆盖：比如线程A和B同时读取count为2，各自加1后都写入3，最终结果只增加了1而非2，导致统计值被少算。
• 单线程的foreach不存在并发冲突，所有操作按顺序执行，因此统计结果准确。

另外还有一个潜在问题：Parallel.ForEach中单个线程抛出的异常会被包装成AggregateException，当前的try-catch无法捕获所有线程的异常，可能导致部分目录的统计被遗漏，进一步加剧结果偏差。

修复方案 #

方案1：使用原子操作类保证线程安全 #

利用Interlocked类提供的原子操作方法，确保变量的增减操作不会被并发打断：

static void parRecurse(DirectoryInfo di, ref int countFolder, ref int countFile, ref long countByte)
{
    try
    {
        DirectoryInfo[] directories = di.GetDirectories();
        Parallel.ForEach(directories, d => {
            // 原子递增文件夹计数
            Interlocked.Increment(ref countFolder);
            foreach(FileInfo f in d.GetFiles()){
                // 原子递增文件计数
                Interlocked.Increment(ref countFile);
                // 原子累加文件大小
                Interlocked.Add(ref countByte, f.Length);
            }
            parRecurse(d, ref countFolder, ref countFile, ref countByte);
        });
    }
    catch (AggregateException ae)
    {
        // 处理并行执行中的所有异常
        foreach (var ex in ae.InnerExceptions)
        {
            if (ex is UnauthorizedAccessException)
                Console.WriteLine("You do not have access to this directory");
        }
    }
    catch (UnauthorizedAccessException)
    {
        Console.WriteLine("You do not have access to this directory");
    }
}

方案2：线程内独立统计再合并 #

每个线程单独统计自己负责的目录数据，最后合并结果，避免共享变量的并发冲突：

// 定义统计结果类
class DirectoryStats
{
    public int FolderCount { get; set; }
    public int FileCount { get; set; }
    public long TotalFileSize { get; set; }
}

static DirectoryStats parRecurse(DirectoryInfo di)
{
    var currentStats = new DirectoryStats();
    try
    {
        DirectoryInfo[] directories = di.GetDirectories();
        Parallel.ForEach(directories, d => {
            // 递归统计子目录数据
            var subStats = parRecurse(d);
            // 当前目录的统计：自身1个文件夹 + 子目录的文件夹数
            currentStats.FolderCount += 1 + subStats.FolderCount;
            // 当前目录的文件数 + 子目录的文件数
            var currentFiles = d.GetFiles();
            currentStats.FileCount += currentFiles.Length + subStats.FileCount;
            // 当前目录的文件总大小 + 子目录的文件总大小
            currentStats.TotalFileSize += currentFiles.Sum(f => f.Length) + subStats.TotalFileSize;
        });
    }
    catch (AggregateException ae)
    {
        foreach (var ex in ae.InnerExceptions)
        {
            if (ex is UnauthorizedAccessException)
                Console.WriteLine("You do not have access to this directory");
        }
    }
    catch (UnauthorizedAccessException)
    {
        Console.WriteLine("You do not have access to this directory");
    }
    return currentStats;
}

内容的提问来源于stack exchange，提问作者Aidan Mellin