How do I assign values to variables in PowerShell ThreadJobs?

Question

ThreadJobs has access to the same environment as it was started in.
But normally, PowerShell will respond with a syntax error when trying to change variables from the parent level.

The documentation MS Learn - about_Thread_Jobs has som insights but nothing I could find useful.

The example below illustrates the issue when trying to use plain PowerShell variables.

[Array]$Numbers = @()
foreach ($i in 0..11) {
 $Jobs = Start-ThreadJob {
 $Using:Numbers += $using:i
 }
}
$Jobs | Wait-Job | Remove-Job
$Numbers
ParserError: 
Line |
 6 | $Using:Numbers += $using:i
 | ~~~~~~~~~~~~~~~
 | The assignment expression is not valid. The input to an assignment operator must 
 | be an object that is able to accept assignments, such as a variable or a property.

Answer 1

As the threads are run in parallell, there has to be some kind of method that prevents the parent object from getting corrupted or a ThreadJob from failing if two or more threads tries to perform operations at the exact same time on the object.

After wrestling several days with the concept of thread safe execution (and getting great help with the patience of Santiago Squarzon and others, my own conclusion is:

All the operation in the thread has to be made thread safe (hence the name).

1. Don't try setting values with Using: if the objects are "plain" unless you can guarantee the value has been locked
2. Even if using thread safe objects, don't try to both read and write of the value unless you can guarantee the value has been locked in between the two operations
3. Only use the provided thread safe methods for manipulating data in the thread safe objects unless you can guarantee the value has been locked

In .Net, I found two thread safe classes you can work with

• System.Collections.Concurrent.ConcurrentBag<T> for working with unordered objects
• System.Collections.Concurrent.ConcurrentDictionary<TKey,TValue> for working with hashtables

(There is one class name available per T-reference.)

None of the classes has a thread safe method for incrementing values.

So a thread safe ThreadJob in PowerShell 7.x, only adding new items to parent objects, might look like this

$SafeNumbers = [System.Collections.Concurrent.ConcurrentBag[object]]::new()
foreach ($i in 0..11) {
 $Thread = Start-ThreadJob {
 ($Using:SafeNumbers).Add($Using:i)
 }
}
Get-Job | Wait-Job | Remove-Job
$SafeNumbers.ToArray()
11
10
9
8
7
6
5
4
3
2
1
0

The order of the output is of course not guaranteed.

• Santiago Squarzon: Now this is the right thing to do :) but now comes the time when you need to ask yourself: "Do I really need to add items from the threads themselves? Wouldn't it be easier to just produce the output from the threads and capture that output synchronously via Receive-Job"? That's exactly what the last example of my answer I linked you before was trying to explain ;)
• Dennis: Exactly :) I tried being clever and save me some job, which is never a good thing if Murphy has he's say.
• Santiago Squarzon: Since you seem to be keen on learning multithreading in powershell and you also seem to be stuck on windows powershell 5.1 (as I see no mention of ForEach-Object -Parallel) this might be helpful to you: stackoverflow.com/questions/74257556/… (the code in the GitHub repo is far better than the one showed in that answer if you decide to invest some time researching it)
• Dennis: Start-ThreadJob is native only to PowerShell 7.x + (you can of course install some modules for that in PoSH 5.1 as well). Pipes get's too hard to read when they get too big, so I try to stay away if I really don't need them for big objects. I try to focus on maintainable code :)
• Dennis: I just discovered that the Start-ThreadJob in PowerShell 7.x is actually done by exactly the same module ThreadJob that is available for (but not delivered with) PowerShell 5.x. ForEach-Object -Parallel, however, is unique for PowerShell 7.x

Answer 2

After some searching, I found a work around where variable referrals apparently works...

But it will fail sooner or later, so DON ́T use the example.

[Array]$Numbers = @()
$refNumbersVar = Get-Variable Numbers
foreach ($i in 0..11) {
 $Jobs = Start-ThreadJob {
 ($Using:refNumbersVar).Value += $using:i
 }
}
$Jobs | Wait-Job | Remove-Job
$Numbers

Edit: Warned all to not use this example.

• Santiago Squarzon: This is not thread safe fyi
• Dennis: I was suspecting something like that, but I'm only adding values. Not changing or replacing. So that would still make it safe, right?
• Dennis: Replaced [array] with a thread safe example.
• Santiago Squarzon: ConcurrentDictionary doesn't ensure thread safety for updating a single Value either. If you want thread safety for a single value you must use a locking mechanism like the one demonstrated here: stackoverflow.com/a/75252238/15339544
• Santiago Squarzon: github.com/MicrosoftDocs/PowerShell-Docs/issues/… that comment demos how easy it is to make a sync hash or a concurrent dict fail while updating a single value.