Skip to content

Navigation Menu

Sign in
Sign up

Speed up GitLab plugin by fetching pages in parallel - #467

Open
thrix wants to merge 1 commit into
psss:main from
thrix:speed-up-gitlab
Open

Speed up GitLab plugin by fetching pages in parallel #467
thrix wants to merge 1 commit into
psss:main from
thrix:speed-up-gitlab

Conversation

@thrix

@thrix thrix commented Apr 8, 2026

Copy link
Copy Markdown

Use ThreadPoolExecutor to fetch paginated GitLab API results concurrently instead of sequentially. Parse the last page link to determine total pages upfront, then fetch all in parallel. Fall back to sequential next-link following when last header is absent. Filter results by since date per-page.

Add unit tests for page ordering, error handling, next-link fallback, and missing MR/Note lookups.

Assisted-by: Claude Code

Use `ThreadPoolExecutor` to fetch paginated GitLab API results
concurrently instead of sequentially. Parse the `last` page link
to determine total pages upfront, then fetch all in parallel.
Fall back to sequential `next`-link following when `last` header
is absent. Filter results by `since` date per-page.
Handle HTTP 429 rate limiting with retry using `Retry-After`
header. Use `tenacity` for connection retry (consistent with
the GitHub plugin).
Add unit tests for page ordering, error handling, next-link
fallback, and missing MR/Note lookups.
Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /