'MaxDegreeOfParallelism' for Array.Parallel?

Question

Is it possible to set 'MaxDegreeOfParallelism' (that is maximum number of threads to use) for Array.Parallel module since under the hood it uses Parallel.For?

Answer 1

According to this post, it seems that there is no way to limit the number of threads globally in the final version of Parallel Extensions. An alternative to what brian suggests would be to use PLINQ (which works with parallel sequences) instead of functions that work with arrays.

This can be done using the PSeq module from F# PowerPack. It provides functions such as PSeq.map, PSeq.filter and many other that work with parallel sequences (which can be also nicely composed using pipelining). For parallel sequences, you can use the WithDegreeOfParallelism extension method to specify the behavior.

(削除) You could implement a wrapper function for it: (削除ここまで)
[EDIT: It is already there!]

let withDegreeOfParallelism n (pq:ParallelQuery<_>) = 
 pq.WithDegreeOfParallelsm(n)

And then write:

let res = 
 data |> PSeq.map (fun n -> ...)
 |> PSeq.withDegreeOfParallelism ParallelOptions.MaxDegreeOfParallelism
 |> Array.ofSeq

This may have different perfromance, because it is implemented differently than functions in the Array.Parallel module, but this certainly depends on your scenario.

• sthlm58: Thanks for an answer. That wrapper is already a part of PSeq module (as PSeq.withDegreeOfParallelism), for what I know. For me, the only problem with ParallelQuery<T> is it's deferred execution.
• Tomas Petricek: You're right, it's already there - I corrected my post. The fact that it is deferred means that if you sequence multiple operations using |>, the framework can run them at once (with some smart optimizations), while Array.Parallel runs each operation separately (so it may create a larger number of tasks - which may, or may not be an issue...).

Answer 2

Assuming I want say at most 10 threads I've been replacing:

myArray 
|> Array.Parallel.iter (fun item -> doWork item)

with

let maxPara = 10
myArray
|> Array.splitInto maxPara
|> Array.Parallel.iter (fun items -> items |> List.iter (fun item -> doWork item))

Answer 3

No, I don't think so.

You can always create your own versions of any of the methods in the Array.Parallel module, using the source code from array.fs (in the CTP release) as a starter.

• sthlm58: That's what I thought so. Nevertheless, thanks for assuring me. Sources form CTP release are certainly going to be valuable.