Search files content fast in powershell

Question

I need to search a large repository of files for a specific resource, and return it only if it is used or not.

It is working as expected, but it is very time consuming. To search for 50 resource names takes around 200 seconds. I have many more than 50.

I found this question that helped a bit, but I was wondering if there is anything else I can do to speed it up.

$searchList = Get-ChildItem -r *.cs -OutBuffer 1000000
$usedIcons = @()
foreach ($name in $testNames) {
 $pattern = "Resources."+$name
 $found = $searchList | Select-String -Pattern $pattern -SimpleMatch -Quiet
 if($found -eq $true){
 Write-Host $name 
 $usedIcons += $name
 }
}

• Oh My Goodness: I would try a parallel tool such as the silver searcher. To find a servies of names preceded by "Resources.", the command will be ag --csharp "Resources\.(name1|name2|name3|...)"

Answer 1

I also suggest to use a better RegEx but stay with PowerShell.

Without exactly knowing your $testnames you can manually or automatically build a RegEx with a lookbehind and alternations and read the *.cs files only once.

## build some sample files 
'foo','bar','baz'|ForEach-Object{"Resources.$_" > "$_.cs"}
$testnames = [RegEx]"(?<=Resources\.)(foo|bar|baz)"
# (zero length look behind assertion)(alternation)
$Result = Get-ChildItem *.cs -Recurse -File | 
 Select-String -Pattern $testnames|
 Select-Object @{n='UsedIcons';e={$_.Matches.Groups[0].Value}} -Unique
$Result

Sample output:

UsedIcons
---------
bar
baz
foo

• Roman: I didn't need regex, actually. it was a simple string match. Thanks for the answer :)

Answer 2

Thanks for the suggestions, I found the most increase in performance came from loading the text of each file into a variable and then doing -match of all needed names in that file. That gave me over 10x improvement in runtime!

Here is final code:

$usedIcons = @()
$searchList = Get-ChildItem -r *.cs
$searchList | ForEach-Object{
 $file = Get-Content $_ -raw
 foreach ($name in $uniqueNames){
 $pattern = "Resources."+$name
 #$pattern = $name
 $found = $file -match $pattern 
 if($found -eq $true){
 Write-Host $name - $_.Name
 $usedIcons += $name
 $uniqueNames = $uniqueNames | Where-Object { $_ -ne $name}
 }
 }
}