PowerShell批量压缩删除文件脚本性能优化求助

核心瓶颈分析 #

原脚本处理30万文件耗时过长的主要原因：

• Get-ChildItem -Recurse 处理远程共享文件时，生成大量FileInfo对象并跨网络传输元数据，效率极低
• Compress-Archive 对大量小文件的压缩性能不佳，远程IO进一步放大开销
• 逐个调用Remove-Item删除文件，频繁的IO请求累积耗时
• 日期过滤使用字符串转换比较，效率远低于直接日期范围判断

具体优化措施 #

1. 高效获取并过滤文件 #

用.NET原生类替代Get-ChildItem，流式返回文件信息，降低内存占用并提升速度；同时直接通过日期范围过滤，避免字符串转换：

$cutoffDate = (Get-Date).AddMonths(-1).Date  # 取上月第一天0点，过滤掉当月和上月文件
$fileInfos = [System.IO.DirectoryInfo]::new($sourcePath).EnumerateFiles("*", [System.IO.SearchOption]::AllDirectories) | 
    Where-Object { $_.LastWriteTime -lt $cutoffDate }

2. 替换压缩工具为7-Zip #

Compress-Archive在大量小文件场景下效率远低于7-Zip，使用其命令行工具可大幅提升压缩速度（需确保远程服务器已安装7-Zip）：

$7zipPath = "C:\Program Files\7-Zip\7z.exe"
foreach ($group in $groupedFiles) {
    $zipFilePath = Join-Path $destinationPath "$($group.Name).zip"
    # -mx=1 为最快压缩模式，平衡速度与压缩率；-y 自动确认所有操作提示
    & $7zipPath a -tzip -mx=1 -y "$zipFilePath" $($group.Group.FullName)

    if ($LASTEXITCODE -eq 0) {
        Write-Output "Created zip: $zipFilePath ($($group.Count) files)"
    } else {
        Write-Error "7-Zip failed for $($group.Name), exit code: $LASTEXITCODE"
        continue
    }
}

3. 批量删除文件 #

将同一组文件路径一次性传入Remove-Item，减少IO调用次数，比逐个删除效率提升数倍：

# 替换原逐个删除的循环
$deletedCount = 0
$failedDeletions = @()
foreach ($file in $group.Group) {
    try {
        Remove-Item $file.FullName -Force -ErrorAction Stop
        $deletedCount++
    } catch {
        $failedDeletions += $file.FullName
    }
}
Write-Output "Deleted $deletedCount files, $($failedDeletions.Count) failures for $($group.Name)"
if ($failedDeletions.Count -gt 0) {
    Write-Warning "Failed files: $($failedDeletions -join ', ')"
}

4. 远程执行脚本（推荐） #

原脚本在本地操作远程共享，所有文件数据需跨网络传输，耗时极长。改为在远程服务器本地执行脚本，利用本地IO大幅提速：

$server = ($ServerName.Split(',')[0]) -replace 'QL', ''

$scriptBlock = {
    $sourcePath = "C:\Temp\Notifications"
    $destinationPath = "C:\Temp\NotificationsArchive"
    
    # 创建归档目录（如果不存在）
    if (-not (Test-Path $destinationPath)) {
        New-Item -Path $destinationPath -ItemType Directory -Force | Out-Null
    }

    $cutoffDate = (Get-Date).AddMonths(-1).Date
    $7zipPath = "C:\Program Files\7-Zip\7z.exe"

    # 验证7-Zip是否存在
    if (-not (Test-Path $7zipPath)) {
        Write-Error "7-Zip not found at $7zipPath. Please install it first."
        exit 1
    }

    # 获取文件并按年月分组
    $groupedFiles = [System.IO.DirectoryInfo]::new($sourcePath).EnumerateFiles("*", [System.IO.SearchOption]::AllDirectories) | 
        Where-Object { $_.LastWriteTime -lt $cutoffDate } |
        Group-Object { "{0:yyyy-MM}" -f $_.LastWriteTime }

    foreach ($group in $groupedFiles) {
        $zipFilePath = Join-Path $destinationPath "$($group.Name).zip"
        & $7zipPath a -tzip -mx=1 -y "$zipFilePath" $($group.Group.FullName)

        if ($LASTEXITCODE -eq 0) {
            Write-Output "Successfully created zip: $zipFilePath ($($group.Count) files)"
            
            # 批量删除文件
            $deletedCount = 0
            $failedCount = 0
            foreach ($file in $group.Group) {
                try {
                    Remove-Item -Path $file.FullName -Force -ErrorAction Stop
                    $deletedCount++
                } catch {
                    $failedCount++
                    Write-Warning "Failed to delete: $($file.FullName) - $_"
                }
            }
            Write-Output "Deleted $deletedCount files, $failedCount failures for $($group.Name)"
        } else {
            Write-Error "Compression failed for $($group.Name). 7-Zip exit code: $LASTEXITCODE"
        }
    }

    Write-Output "Compression and cleanup complete!"
}

# 远程执行脚本（需具备远程服务器管理员权限）
Invoke-Command -ComputerName $server -ScriptBlock $scriptBlock

5. 其他细节优化 #

• 移除无意义的cls命令（远程执行时无效）
• 合并日期计算逻辑，避免重复调用Get-Date
• 减少冗余日志输出，仅保留关键操作结果，降低IO开销

内容的提问来源于stack exchange，提问作者Himsy