Can I (safely) write to an Array in a Parallel.For loop construct?

Question

Will the below code snippet write correctly to results for all indexes of the range of iteration?

While there are multiple threads accessing the same object simultaneously; each thread is writing to a unique location in memory due to the indexing.

let results:NewType[] = Array.zeroCreate temp.Length
let mutable data = Unchecked.defaultof<OldType>
let loopResult = 
 System.Threading.Tasks.Parallel.For(
 0, 
 temp.Length, 
 (fun i -> 
 data <- temp.[i]
 results.[i] <- NewType(data.X, data.Y) 
 )
 )

• Just another metaprogrammer: As already pointed out this is not thread-safe but it is not performant as well. Because the different cores writes & reads to the same memory area (data) there migh be lot of "chatting" between the cores to ensure cache coherence (Architecture dependent). In order to achieve good parallel performance you want to make sure that each core can execute independently from eachother (ie don't share mutable data). A tip is to always compare single-core performance with multi-core performance. Suprisingly often one see slow-down instead of speed-up. Most easily done by changing process affinity

Answer 1

The way that you have written this code, it will not behave correctly across a parallel iteration. You have added a mutable temporary object which will destroy your ability to parallelise this code safely. Writing to results is safe because you know that each thread will be accessing a different element of the array but writing to data is unsafe because many threads can be assigning to that mutable object at the same time.

If you reconstructed your code as such:

let results:NewType[] = Array.zeroCreate temp.Length
let loopResult = 
 System.Threading.Tasks.Parallel.For(
 0, 
 temp.Length, 
 (fun i -> 
 let data = temp.[i]
 results.[i] <- NewType(data.X, data.Y) 
 )
 )

the behaviour would then be safe.

This code, however, is just straightforward parallel map so it would be more idiomatic to write it that way:

let loopResult = temp |> Array.Parallel.map (fun data -> NewType(data.X, data.Y))

• Sam: Thanks. Should I expect performance differences between Array.Parallel.map versus System.Threading.Tasks.Parallel.For?
• TheInnerLight: I can't say I've extensively profiled but a quick test in FSI suggests that they both run at approximately the same speed which is what I would intuitively expect. It is trivial to implement a function which is equivalent to Array.Parallel.map using System.Threading.Tasks.Parallel.For so I can't see a reason for a performance difference.

Answer 2

Since the type of F# arrays is usual .NET System.Array simultaneous writing to the different parts of array will be thread safe. That was already discussed HERE and HERE