Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Fixing a hang when upserting 90,000 rows with :batchexec #2029

Unanswered
austincollinpena asked this question in Issue Triage
Discussion options

Maybe this will help someone in the future!

I dealt with a situation, of unknown cause, where a :batchexec would cause the program to stall. I don't know why it would happen, but I found a fix.

I chunked the bulk insert arguments (https://stackoverflow.com/a/72408490) and ran them concurrently inside of an errGroup.

Fixing the root cause is better but this works for now!

	chunks := chunkBy[db_access.AdGroupPerformanceUpdateParams](p, 1000)
	errGroup, ctx := errgroup.WithContext(ctx)
	errGroup.SetLimit(maxConcurrency)
	for _, chunk := range chunks {
		chunkCopy := chunk
		errGroup.Go(func() error {
			batch := q.AdGroupPerformanceUpdate(ctx, chunkCopy)
			var outsideErr error
			batch.Exec(func(i int, err error) {
				if err != nil {
					outsideErr = err
					batch.Close()
				}
			})
			return outsideErr
		})
	}
You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /