-
Notifications
You must be signed in to change notification settings - Fork 924
Fixing a hang when upserting 90,000 rows with :batchexec #2029
Unanswered
austincollinpena
asked this question in
Issue Triage
-
Maybe this will help someone in the future!
I dealt with a situation, of unknown cause, where a :batchexec would cause the program to stall. I don't know why it would happen, but I found a fix.
I chunked the bulk insert arguments (https://stackoverflow.com/a/72408490) and ran them concurrently inside of an errGroup.
Fixing the root cause is better but this works for now!
chunks := chunkBy[db_access.AdGroupPerformanceUpdateParams](p, 1000) errGroup, ctx := errgroup.WithContext(ctx) errGroup.SetLimit(maxConcurrency) for _, chunk := range chunks { chunkCopy := chunk errGroup.Go(func() error { batch := q.AdGroupPerformanceUpdate(ctx, chunkCopy) var outsideErr error batch.Exec(func(i int, err error) { if err != nil { outsideErr = err batch.Close() } }) return outsideErr }) }
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment