为何我的PowerShell Runspace池执行仍呈串行状态?
Word文档关键词批量搜索工具优化方案
问题背景
我开发了一款辅助工作的网络关键词搜索工具,功能是打开Word文档并检查内容中的关键词。传统foreach遍历文件的搜索方式耗时太长,所以我用Runspace池实现了多线程执行,但实际运行时还是串行状态,推测是Runspace池的构建有问题,希望得到改进这个函数的反馈。
原代码
Function getWordMatches ($wordFiles, $TermArray) { #Synchronized Hashtable is need for variable control during runspace execution $dataSync = [hashtable]::Synchronized(@{}) $dataSync.returns = New-Object -TypeName "System.Collections.ArrayList" # Create an empty arraylist that we'll use later $Jobs = New-Object -TypeName "System.Collections.ArrayList" $maxThreads = Get-WmiObject –class Win32_processor | Select-Object -ExpandProperty NumberOfLogicalProcessors # Create a Runspace Pool with a minimum and maximum number of run spaces. (http://msdn.microsoft.com/en-us/library/windows/desktop/dd324626(v=vs.85).aspx) $RunspacePool = [RunspaceFactory]::CreateRunspacePool(1,$maxThreads) # Open the RunspacePool so we can use it $RunspacePool.Open() # Define a script block to actually do the work $WordScriptBlock = { Param($file, $TermArray, $dataSync) #Establish Word variables for searching $matchCase = $false $matchWholeWord = $true $matchWildCards = $false $matchSoundsLike = $false $matchAllWordForms = $false $forward = $true $wrap = 1 #The below lines open new instances of the MS Word and Excel applications that will be used. $check = Test-Path -Path $file -PathType Leaf if (-Not $check){return $false} $Word = New-Object -ComObject Word.Application $Word.visible = $False #the visible value is set to false so as not to interfere with the user during execution. $hash = Get-FileHash $file -Algorithm SHA256 | Select-Object Hash try{ #I needed to add a default password 'ttt' due some password protected files that stopped the script from running $document = $Word.documents.open($file.FullName, 'ReadOnly:=$true', 'AddToRecentFiles:=$false', "PasswordDocument:='ttt'", 'Visible:=$false') $range = $document.content }catch{ $Word.Quit() } Foreach ($term In $TermArray) { $wordFound = $range.find.execute($term,$matchCase,$matchWholeWord,$matchWildCards, $matchSoundsLike,$matchAllWordForms,$forward,$wrap) If($wordFound) { #Hashtable to store the data for writing the csv output. Stored the data $properties = @{File = $file.FullName; Term_Match = $term; Hash = $hash.Hash} $result = New-Object -TypeName PsCustomObject -Property $properties $dataSync.returns += $result Break } } $document.close($false) $Word.Quit() } #/WordScriptBlock # Loop through all word files Foreach ($file In $wordFiles) { $Powershell = [Powershell]::Create().AddScript($WordScriptBlock) $Powershell.RunspacePool = $RunspacePool $Powershell.AddArgument($file).AddArgument($TermArray).addArgument($dataSync) | Out-Null $JobObj = New-Object -TypeName PSObject -Property @{ Runspace = $Powershell.BeginInvoke() Powershell = $Powershell } #/New-Object $Jobs.Add($JobObj) | Out-Null #added to a runspace arraylist } #cycles through the jobs and will check to see if they are complete. If it is then it will remove the job from the job list. $i = 0 $total = $Jobs.Count while($Jobs){ Write-Progress -Activity "Processing Word jobs" -Status "$($Jobs.Count) Jobs Remaining" -PercentComplete ($i/$total * 100) foreach ($Runspace in $Jobs.ToArray()){ If ($Runspace.Runspace.IsCompleted) { $Runspace.Powershell.Dispose() $Jobs.Remove($Runspace) $i++ } } } $wordResults = $dataSync.returns return $wordResults }
核心问题与优化方案
1. 解决Runspace池串行执行的关键问题
Word的COM对象必须在**单线程公寓(STA)**模式下运行,原代码创建Runspace池时未指定公寓模式,默认是多线程公寓(MTA),导致COM对象强制串行执行。修改如下:
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $maxThreads) $RunspacePool.ApartmentState = [System.Threading.ApartmentState]::STA # 新增此行,指定STA模式 $RunspacePool.Open()
2. 替换过时的处理器核心数获取方式
Get-WmiObject已被弃用,改用Get-CimInstance获取逻辑核心数:
$maxThreads = (Get-CimInstance -ClassName Win32_Processor).NumberOfLogicalProcessors | Measure-Object -Sum | Select-Object -ExpandProperty Sum
3. 修复Word文档打开的参数错误
Documents.Open的参数不能用字符串形式的'ReadOnly:=$true',需按官方参数顺序传递布尔值,同时调整Visible属性的设置位置:
$Word = New-Object -ComObject Word.Application $Word.Visible = $false # 直接设置Word应用可见性,无需在Open参数中传递 try { $document = $Word.Documents.Open( $file.FullName, $false, # ConfirmConversions $true, # ReadOnly $false, # AddToRecentFiles 'ttt' # PasswordDocument ) $range = $document.Content } catch { # 错误处理逻辑 }
4. 线程安全的结果存储
原代码使用的ArrayList并非线程安全,即使放在同步哈希表中,+=操作仍可能引发竞争条件。改用ConcurrentBag实现线程安全的结果存储:
# 初始化同步哈希表时替换为ConcurrentBag $dataSync = [hashtable]::Synchronized(@{}) $dataSync.returns = New-Object -TypeName "System.Collections.Concurrent.ConcurrentBag[PSObject]" # 在脚本块中添加结果时使用Add方法 $dataSync.returns.Add($result)
5. 完善COM对象清理逻辑
确保无论操作成功与否,都能正确释放Word进程,避免残留后台进程:
try { # 文档操作代码 } catch { Write-Error "处理文件 $($file.FullName) 时出错: $_" } finally { # 确保文档关闭 if ($document) { $document.Close($false) } # 确保Word进程退出并释放COM对象 if ($Word) { $Word.Quit() [System.Runtime.Interopservices.Marshal]::ReleaseComObject($document) | Out-Null [System.Runtime.Interopservices.Marshal]::ReleaseComObject($Word) | Out-Null [System.GC]::Collect() [System.GC]::WaitForPendingFinalizers() } }
6. 优化进度条计算逻辑
原代码中$i的递增逻辑不准确,改为通过总任务数减去剩余任务数计算已完成数量:
$total = $Jobs.Count while($Jobs.Count -gt 0){ $completedCount = $total - $Jobs.Count Write-Progress -Activity "Processing Word jobs" -Status "$($Jobs.Count) Jobs Remaining" -PercentComplete (($completedCount / $total) * 100) foreach ($Runspace in $Jobs.ToArray()){ If ($Runspace.Runspace.IsCompleted) { $Runspace.Powershell.Dispose() $Jobs.Remove($Runspace) } } Start-Sleep -Milliseconds 100 # 添加短暂延迟,降低CPU占用 }
完整优化后的函数代码
Function getWordMatches ($wordFiles, $TermArray) { # 同步哈希表,用于跨Runspace共享数据 $dataSync = [hashtable]::Synchronized(@{}) # 使用线程安全的ConcurrentBag存储结果 $dataSync.returns = New-Object -TypeName "System.Collections.Concurrent.ConcurrentBag[PSObject]" # 存储Runspace任务对象 $Jobs = New-Object -TypeName "System.Collections.ArrayList" # 获取逻辑核心数,作为最大线程数 $maxThreads = (Get-CimInstance -ClassName Win32_Processor).NumberOfLogicalProcessors | Measure-Object -Sum | Select-Object -ExpandProperty Sum # 创建STA模式的Runspace池 $RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $maxThreads) $RunspacePool.ApartmentState = [System.Threading.ApartmentState]::STA $RunspacePool.Open() # 定义执行Word文档处理的脚本块 $WordScriptBlock = { Param($file, $TermArray, $dataSync) # 搜索参数配置 $matchCase = $false $matchWholeWord = $true $matchWildCards = $false $matchSoundsLike = $false $matchAllWordForms = $false $forward = $true $wrap = 1 # 检查文件是否存在 if (-not (Test-Path -Path $file -PathType Leaf)) { return } $Word = $null $document = $null $hash = Get-FileHash $file -Algorithm SHA256 | Select-Object -ExpandProperty Hash try { # 创建Word应用实例 $Word = New-Object -ComObject Word.Application $Word.Visible = $false # 打开文档(只读模式,指定默认密码) $document = $Word.Documents.Open( $file.FullName, $false, # ConfirmConversions $true, # ReadOnly $false, # AddToRecentFiles 'ttt' # PasswordDocument ) $range = $document.Content # 遍历关键词搜索 foreach ($term in $TermArray) { $wordFound = $range.Find.Execute( $term, $matchCase, $matchWholeWord, $matchWildCards, $matchSoundsLike, $matchAllWordForms, $forward, $wrap ) if ($wordFound) { # 生成结果对象并添加到线程安全集合 $result = [PSCustomObject]@{ File = $file.FullName Term_Match = $term Hash = $hash } $dataSync.returns.Add($result) break # 找到第一个匹配项后退出循环 } } } catch { Write-Error "处理文件 $($file.FullName) 时出错: $_" } finally { # 清理COM对象,避免残留进程 if ($document) { $document.Close($false) [System.Runtime.Interopservices.Marshal]::ReleaseComObject($document) | Out-Null } if ($Word) { $Word.Quit() [System.Runtime.Interopservices.Marshal]::ReleaseComObject($Word) | Out-Null } [System.GC]::Collect() [System.GC]::WaitForPendingFinalizers() } } # 为每个文件创建Runspace任务 foreach ($file in $wordFiles) { $psInstance = [Powershell]::Create().AddScript($WordScriptBlock) $psInstance.RunspacePool = $RunspacePool $psInstance.AddArgument($file).AddArgument($TermArray).AddArgument($dataSync) | Out-Null $jobObj = [PSCustomObject]@{ Runspace = $psInstance.BeginInvoke() Powershell = $psInstance } $Jobs.Add($jobObj) | Out-Null } # 等待所有任务完成并清理 $total = $Jobs.Count while ($Jobs.Count -gt 0) { $completedCount = $total - $Jobs.Count Write-Progress -Activity "处理Word文档" -Status "$($Jobs.Count) 个任务剩余" -PercentComplete (($completedCount / $total) * 100) foreach ($runspaceJob in $Jobs.ToArray()) { if ($runspaceJob.Runspace.IsCompleted) { $runspaceJob.Powershell.EndInvoke($runspaceJob.Runspace) | Out-Null $runspaceJob.Powershell.Dispose() $Jobs.Remove($runspaceJob) } } Start-Sleep -Milliseconds 100 } # 返回结果 return $dataSync.returns }
内容的提问来源于stack exchange,提问作者Trevor Winge
相关产品推荐
相关产品推荐

