2
\$\begingroup\$

Here is my code which helps me to contol the maximum amount of goroutines run simultaneously.
In example below maximum is 5.

var wg sync.WaitGroup
// Create bufferized channel with size 5
goroutines := make(chan struct{}, 5)
// Read data from input channel
for data := range input {
 // 1 struct{}{} - 1 goroutine
 goroutines <- struct{}{}
 wg.Add(1)
 go func(data string, goroutines <-chan struct{}, results chan<- result, wg *sync.WaitGroup) {
 // Process data
 results <- process(data)
 // Read from "goroutines" channel to free space for new goroutine
 <-goroutines
 wg.Done()
 }(data, goroutines, results, &wg)
}
wg.Wait()
close(goroutines)

Is there a better way to control the maximum amount of goroutines?
How can I improve my code?

200_success
145k22 gold badges190 silver badges478 bronze badges
asked Jul 27, 2017 at 12:49
\$\endgroup\$
2
  • \$\begingroup\$ this is correct. but inefficient because your code is performing O(2n) synchronizations that are not required if you would declare 5 permanent workers upfront. The runtime is built to handle millions routines, having 5 permanent workers, even sparingly consumed, is easy to handle for it and will prevent useless cpu usage. but again nothing wrong in your code. \$\endgroup\$ Commented Jan 13, 2020 at 19:02
  • \$\begingroup\$ I think your method the Best. Thanks. \$\endgroup\$ Commented Dec 2, 2021 at 11:28

1 Answer 1

1
\$\begingroup\$

You can use the Golang Concurrency Manager goccm to limit the maximum number of goroutines to run concurrently:

Example:

package main
import (
 "fmt"
 "goccm"
 "time"
)
func main() {
 // Limit 3 goroutines to run concurrently.
 c := goccm.New(3)
 for i := 1; i <= 10; i++ {
 // This function have to call before any goroutine
 c.Wait()
 go func(i int) {
 fmt.Printf("Job %d is running\n", i)
 time.Sleep(2 * time.Second)
 // This function have to when a goroutine has finished
 // Or you can use `defer c.Done()` at the top of goroutine.
 c.Done()
 }(i)
 }
 // This function have to call to ensure all goroutines have finished 
 // after close the main program.
 c.WaitAllDone()
}
Sᴀᴍ Onᴇᴌᴀ
29.5k16 gold badges45 silver badges201 bronze badges
answered Aug 16, 2019 at 15:50
\$\endgroup\$

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.