PowerShell: speed up Get-ChildItem to scan files in a large file system

I wrote a PowerShell script for scanning PST files in the file system.

The part of script likes the below code.

$stamp = Get-Date

$location = "\\deptshare\shares"
$csvfile  = "C:\temp\pstfiles\" + $(Get-Date -Format yyyyMMddHmm)+ "deptshare-pst.csv"
$pstarray = @()


ForEach ($share in Get-ChildItem $location){
        Write-Host "Scanning $share..."  
        Get-ChildItem  $share.FullName  -recurse -file -force -ErrorAction SilentlyContinue | Where-Object{$_.extension -eq ".pst"} | ForEach-Object{
				$_                
                $pstobj = New-Object PSObject
                $pstobj | Add-Member NoteProperty Directory $_.DirectoryName
                $pstobj | Add-Member NoteProperty Name $_.Name
                $pstobj | Add-Member NoteProperty Owner ((Get-ACL $_.FullName).Owner)
                $pstobj | Add-Member NoteProperty CreationTime $_.CreationTime
                $pstobj | Add-Member NoteProperty LastWriteTime $_.LastWriteTime
                $pstobj | Add-Member NoteProperty Length $_.Length
                $pstarray += $pstobj                
        }

    }  

However, it takes very long time to complete the scan.

It took about 19-21 hours to scan PST files in the 28 TB file system.

I found the key point to slow down the scan is the line 10 “Get-ChildItem” cmdlet.

So, I did a Measure-Command experiment in 60 GB scan.

In the old way, it took 307 seconds to complete the task. I noticed that the script stuck on scanning the files.

Get-ChildItem  $share.FullName  -recurse -file -force -ErrorAction SilentlyContinue | Where-Object{$_.extension -eq ".pst"} 

Here is the new way to do it, I used -Include *.pst instead of where-object{$_.extension -eq “.pst”}. It took only 19 seconds to complete the task!

Get-ChildItem  $share.FullName  -recurse -file -force -Include *.pst -ErrorAction SilentlyContinue 

After I applied the change in the production, the scan time reduced to 14 hours from 21 hours.

Massive improvement!