运行Get-ADUser处理大数据集时遇System.OutOfMemoryException的解决办法

我需要处理约50000个AD用户，之前先获取所有用户集合再逐个调用Get-ADUser提取信息的方式耗时长达16小时。改成单次调用Get-ADUser批量获取的方案后，处理速度大幅提升（11分钟处理20000个用户），但后续突然变慢并抛出System.OutOfMemoryException异常。

原脚本 #

$csv = "Computers-{0:dd-MM-yyyy_HHmm}.csv" -f (get-date)
$UsrProps = "SamAccountName",
"AccountExpirationDate",
"accountExpires",
"AccountLockoutTime",
"BadLogonCount",
"badPwdCount",
"badPasswordTime",
"SamAccountName"


Get-ADUser -Filter *  -Properties $UsrProps -server $domain |
ForEach-Object {

    $hash = @{
        AccountExpirationDate = $_.AccountExpirationDate
        AccountLockoutTime    = $_.AccountLockoutTime
        accountExpires        = $_.accountExpires
        BadLogonCount         = $_.BadLogonCount
        badPwdCount           = $_.badPwdCount
        badPasswordTime       = $_.badPasswordTime
        SamAccountName        = $_.SamAccountName
    }
       
    $PSCustObj = [pscustomobject]$hash
    $results = $PSCustObj
    $results |
    select-object @{ l = "SamAccountName"; e = { [string]$_.SamAccountName } },
    @{ l = "AccountExpirationDate"; e = { [string]$_.AccountExpirationDate } },
    @{ l = "AccountLockoutTime"; e = { [string]$_.AccountLockoutTime } },
    @{ l = "BadLogonCount"; e = { [string]$_.BadLogonCount } },
    @{ l = "badPwdCount"; e = { [string]$_.badPwdCount } },
    @{ N = 'badPasswordTime'; E = { [DateTime]::FromFileTime($_.badPasswordTime) } }  | 
    export-csv "$PWD\Logs\$domain\Users\$csv" -noTypeInformation -Append
    
}

原错误信息 #

Get-ADUser : Exception of type 'System.OutOfMemoryException' was thrown. At line:231 char:5
+     Get-ADUser -Filter *  -Properties $UsrProps -server $domain |
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-ADUser], OutOfMemoryException
    + FullyQualifiedErrorId : ActiveDirectoryCmdlet:System.OutOfMemoryException,Microsoft.ActiveDirectory.Management.Commands.GetADUser

问题根源 #

1. 内存过载：单次Get-ADUser -Filter *会尝试把所有5万AD用户对象一次性加载到内存，随着数据量累积，内存占用超出上限触发溢出。
2. 低效IO：每次循环调用Export-Csv -Append，频繁打开/关闭文件，后期数据量增大后拖慢处理速度。
3. 冗余操作：脚本中重复定义SamAccountName、多次转换对象，额外消耗内存和CPU。

解决方案 #

1. 分页加载AD用户（核心修复） #

利用PageSize和ResultSetSize参数分批获取用户，避免一次性加载全部数据。AD默认最大页面大小为1000，设置该值即可分批处理：

# 修正CSV文件名（原脚本写的Computers，应该是Users）
$csvPath = "$PWD\Logs\$domain\Users\Users-{0:dd-MM-yyyy_HHmm}.csv" -f (Get-Date)
# 去掉重复属性，SamAccountName默认已返回，无需加入Properties
$UsrProps = "AccountExpirationDate", "accountExpires", 
            "AccountLockoutTime", "BadLogonCount", "badPwdCount", "badPasswordTime"

Get-ADUser -Filter * -Properties $UsrProps -Server $domain -PageSize 1000 -ResultSetSize $null |
    ForEach-Object {
        # 直接构造最终输出对象，跳过冗余步骤
        [PSCustomObject]@{
            SamAccountName        = [string]$_.SamAccountName
            AccountExpirationDate = [string]$_.AccountExpirationDate
            AccountLockoutTime    = [string]$_.AccountLockoutTime
            BadLogonCount         = [string]$_.BadLogonCount
            badPwdCount           = [string]$_.badPwdCount
            # 处理badPasswordTime为0的情况，避免无效日期
            badPasswordTime       = if ($_.badPasswordTime -ne 0) { [DateTime]::FromFileTime($_.badPasswordTime) } else { $null }
        }
    } | Export-Csv -Path $csvPath -NoTypeInformation

2. 优化IO操作 #

把处理后的对象通过管道直接传递给Export-Csv，一次性完成写入（或分批但大幅减少IO次数），彻底解决频繁Append的低效问题。

3. 移除冗余代码 #

• 删除$UsrProps中重复的SamAccountName
• 跳过中间的$hash、$PSCustObj、$results变量，直接构造最终需要的自定义对象，减少内存开销。

4. 可选：强制垃圾回收（极端场景） #

如果仍存在内存压力，可在每处理一定数量用户后强制回收内存，但不建议频繁调用：

$csvPath = "$PWD\Logs\$domain\Users\Users-{0:dd-MM-yyyy_HHmm}.csv" -f (Get-Date)
$UsrProps = "AccountExpirationDate", "accountExpires", 
            "AccountLockoutTime", "BadLogonCount", "badPwdCount", "badPasswordTime"
$counter = 0

Get-ADUser -Filter * -Properties $UsrProps -Server $domain -PageSize 1000 -ResultSetSize $null |
    ForEach-Object {
        $counter++
        [PSCustomObject]@{
            SamAccountName        = [string]$_.SamAccountName
            AccountExpirationDate = [string]$_.AccountExpirationDate
            AccountLockoutTime    = [string]$_.AccountLockoutTime
            BadLogonCount         = [string]$_.BadLogonCount
            badPwdCount           = [string]$_.badPwdCount
            badPasswordTime       = if ($_.badPasswordTime -ne 0) { [DateTime]::FromFileTime($_.badPasswordTime) } else { $null }
        }
        # 每处理1000个用户回收一次内存
        if ($counter % 1000 -eq 0) {
            [GC]::Collect()
            [GC]::WaitForPendingFinalizers()
        }
    } | Export-Csv -Path $csvPath -NoTypeInformation

内容的提问来源于stack exchange，提问作者Itchydon