PowerShell - fast remove a directory with 10,000+ files

Question

I am sick of File Explorer's super slow deletion speed, so I tried to write a PowerShell script to make deletion faster, and while it does its job, its speed isn't as high as what I intended it to be.

I have written this:

Function Fast-Delete {
 Param(
 [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory0
 )
 Write-Warning "This Process Will Delete Everything In The Target Directory: $Directory0, Do You Want To Confirm Deletion?" -Warningaction Inquire
 $Directory=$Directory0
 While ((Get-Childitem -Path $Directory0 -Depth 0 -Force).Count -Ne 0) {
 If ((Get-Childitem -Path $Directory -Directory -Force -Depth 0).Count -Ne 0) {
 $Directory=(Get-Childitem -Path $Directory -Directory -Force -Depth 0).Fullname | Select-Object -Index 0
 }
 If ((Get-Childitem -Path $Directory -File -Force).Count -Ne 0) {
 (Get-Childitem -Path $Directory -File -Recurse -Force).Fullname | Foreach {Remove-Item -Path $_}
 }
 $Directory1=$Directory
 $Directory=$Directory | Split-Path -Parent
 Remove-Item -Path $Directory1
 }
 Remove-Item -Path $Directory0
}

It is significantly faster than explorer, but still isn't ideal, I have tested it, I used it to delete 208,000 files in 1,000 folders , and the folders disappear at speed of 1 per second, so it's about 208 files/s, now the next challenge should be parallelization, But this is currently really way above me, but it shouldn't be too hard, I am just not experienced enough.

Update2

I have managed to make it run in parallel with this script:

function Parallel-Delete {
 param(
 [Parameter(Valuefrompipeline=$true, Mandatory=$true, Position=0)] [array]$filelist,
 [Parameter(Valuefrompipeline=$true, Mandatory=$true, Position=1)] [int]$number
 )
 0..($filelist.count-1) | Where-Object {$_ % 16 -eq $number} | foreach {Remove-Item -Path $filelist[$_]}
}
Function Fast-Delete {
 Param(
 [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory0
 )
 Write-Warning "This Process Will Delete Everything In The Target Directory: $Directory0, Do You Want To Confirm Deletion?" -Warningaction Inquire
 $Directory=$Directory0
 While ((Get-Childitem -Path $Directory0 -Depth 0 -Force).Count -Ne 0) {
 If ((Get-Childitem -Path $Directory -Directory -Force -Depth 0).Count -Ne 0) {
 $Directory=(Get-Childitem -Path $Directory -Directory -Force -Depth 0).Fullname | Select-Object -Index 0
 }
 If ((Get-Childitem -Path $Directory -File -Force).Count -Ne 0) {
 If ((Get-Childitem -Path $Directory -File -Force).Count -Ge 128) {
 [array]$filelist=(Get-Childitem -Path $Directory -File -Force).Fullname
 0..15 | foreach-object {Invoke-Command -ScriptBlock { Parallel-Delete $filelist $_}}
 } else {
 (Get-Childitem -Path $Directory -File -Force).Fullname | Foreach {Remove-Item -Path $_}
 }
 }
 $Directory1=$Directory
 $Directory=$Directory | Split-Path -Parent
 Remove-Item -Path $Directory1
 }
 Remove-Item -Path $Directory0 -erroraction silentlycontinue
}
$Directory0=Read-Host "Please input target directory to be deleted"
Fast-Delete $Directory0

But it still isn't ideal, it isn't 15 times faster as expected, what did I miss?

Edit: simplified the creation of parallel processes.

• Lee_Dailey: the fastest way to delete an entire dir tree is to use robocopy. you mirror a blank "source" dir to the one you want removed ... and the "destination" gets emptied quickly. [grin]
• Ξένη Γήινος: @Lee_Dailey, in fact I wanted to write a new script, not to use existing things, and the method you mentioned sounds like a bug...
• Lee_Dailey: i understand your desire. [grin] as for a bug ... it is exactly what the options mean - mirror empty source to full destination gives empty source and empty destination. so ... have fun playing with a script ...but PoSh will always be slower than any well written utility.
• Esperento57: why you dont remove the dire complete and recreate him after ?
• Cyril: So... I've been doing some testing, as I'm in a similar boat. A 2 GB folder, containing 10,000 items, is stored on a network drive (I had ~200 when I started removing directories). An individual directory click-delete via explorer has been deleting 3.1 files/sec (includes analyze and deletiion time); a single line remove-item I average 2.4 files/sec; a power shell script for robocopy /purge with a blank directory averages 2.4 files/sec (this still analyzes each individual file in the parent directory)... i am about to try robocopy /mir to see if it's any faster.

Answer 1

As pointed out in a comment from @Lee_Daily, reinventing the wheel from an interpreted language like PowerShell will be slower than a native implementation. If your primary concern is speed, use the correct tool for the job. We have a native implementation in robocopy.

It has an option /mir

Mirrors a directory tree (equivalent to /e plus /purge). Using this option with the /e option and a destination directory, overwrites the destination directory security settings.

Below is the robocopy purge method using your framework:

Param(
 [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory
)
Function Fast-Delete {
 Param(
 [Parameter(Valuefrompipeline=$True, Mandatory=$True)] [String]$Directory
 )
 Write-Warning "This Process Will Delete Everything In The Target Directory: $Directory, Do You Want To Confirm Deletion?" -Warningaction Inquire
 $emptyDir = GetRandomTempPath
 mkdir $emptyDir
 robocopy $emptyDir $Directory /mir
 Remove-Item -Path $Directory -erroraction silentlycontinue
 Remove-Item -Path $emptyDir -erroraction silentlycontinue
}
Function GetRandomTempPath {
 Do {
 $fullPath = Join-Path -Path $env:TEMP -ChildPath [System.IO.Path]::GetRandomFileName()
 } While (Test-Path $fullPath);
 Return $fullPath;
} 
Fast-Delete $Directory

Save as: Fast-Delete.ps1

Comparison of delete methods (2 GB, 10,000 files in directory):

• remove-item: 2.4 files/sec average (n=10)
• /purge : 2.4 files/sec average (n=3)
• /purge /mt: 2.5 files/sec average (n=3)
• explorer-based click-delete: 3.1 files/sec (n=30)
• /mir: 5.5 files/sec (n = 100)

• Toby Speight: I don't use PowerShell myself (I'm more of a POSIX shell and Bash user), but it looks to my untrained eye that your suggested code assumes that there's no existing non-empty $TEMP/uniqueemptydirname, without checking. Is there no mktemp available from this shell? Also, I'd guess that the most useful amount of parallelism would depend more on the storage throughput than on the number of CPU threads here.
• HackSlash: You are 100% correct in that assumption. I added a function for GetRandomTempPath in PowerShell. It doesn't come with a method like this that I could find.

Answer 2

The foreach-object is not run in parallel by default. You need specify the -Parallel parameter. And the version of Powershell need be greater than 7.0

See Official Blob and Official document