Skip to content

Navigation Menu

Sign in
Sign up

fix(sonarqube): bypass 10K result cap by fetching issues per-rule - #448

Open
TheAuditorTool wants to merge 1 commit into
OWASP-Benchmark:master from
TheAuditorTool:fix/sonarqube-10k-result-cap-33
Open

fix(sonarqube): bypass 10K result cap by fetching issues per-rule #448
TheAuditorTool wants to merge 1 commit into
OWASP-Benchmark:master from
TheAuditorTool:fix/sonarqube-10k-result-cap-33

Conversation

@TheAuditorTool

@TheAuditorTool TheAuditorTool commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

Closes #33
Refs #187

Summary

The SonarQube /api/issues/search endpoint enforces a hard server-side limit:
p * ps <= 10000. With PAGE_SIZE = 500, page 21 requests offset 10,500 and
receives HTTP 400. For instances reporting >10K vulnerabilities, results are
either silently truncated or the script crashes.

This was reported in #33 (March 2017) and deferred during the #187 cleanup.
The original reporter's suggestion -- filter by individual rules to keep each
query under the cap -- is exactly what this PR implements.

Problem Chain

Date State
2017-03 #33 opened: SonarQube API caps at 10K issues. Suggestion: use &rules=squid:XXX per-rule.
2025-02 32933c4e6 added SonarReport.java with native Java pagination. Passes ALL rules in one &rules= query -- still hits 10K cap for large result sets.
2025-02 dc9abba63 fixed hostname. 283e0e61f applied Spotless formatting.
This PR Iterates per-rule instead of all-at-once, bypassing the 10K cap entirely.

What Changed

Single file changed: src/main/java/org/owasp/benchmark/report/sonarqube/SonarReport.java

1. Per-rule issue fetching (main())

Before: All ~600 Java rules are comma-joined into a single &rules= query
parameter. If the aggregate result exceeds 10K issues, the API returns HTTP 400
or silently truncates.

After: Each rule is queried individually. A single rule produces far fewer
issues (typically 0-200 for Benchmark's 2,740 test cases), staying well under
the 10K cap.

This also fixes:

  • A double && in the URL (&&rules= -> &rules=)
  • A ~9KB URL from comma-joining 600+ rule IDs, which risks server URL-length limits

Why no duplicates: Each SonarQube issue belongs to exactly one rule.
Per-rule iteration produces disjoint result sets.

Performance: ~600 rules x 1 lightweight API call each. Most rules return 0
results (1 page). Total overhead: 1-5 minutes. The scan itself takes >= 1 hour.

2. Off-by-one in page count (forAllPagesAt())

Before: (total / PAGE_SIZE) + 1 -- always adds an extra page. When total
is an exact multiple of PAGE_SIZE (e.g., 500 issues), fires one unnecessary
empty-page request. At exactly 10K results, this triggers a forbidden page 21.

After: (total + PAGE_SIZE - 1) / PAGE_SIZE -- standard ceiling division.

3. HTTP error handling (apiCall())

Before: getInputStream() on a non-200 response throws a generic
IOException with "Server returned HTTP response code: 400 for URL: ...".
For some error codes, it may silently read an error body that breaks Jackson
deserialization downstream.

After: getResponseCode() is checked before reading the body. Non-200
throws IOException("SonarQube API returned HTTP " + status + " for " + apiPath)
with clear context.

What Was NOT Changed

Item Reason
testcode/ (2,740 files) Benchmark test cases -- not related to this fix
SonarQubeResult.java DTO structure is unchanged
KeepAsJsonDeserializer.java Unchanged
runSonarQube.sh Cleanup handled separately in #187
pom.xml No new dependencies
setDoOutput(true) in apiCall() Technically incorrect for GET requests, but it is what the original author shipped and tested. Removing it is a style fix with nonzero regression risk and zero benefit.

Known Limitations (future work)

  1. hotspots/search has the same 10K cap but does not support per-rule
    filtering. Benchmark's 2,740 test cases are unlikely to produce 10K+
    hotspots, but an Enterprise instance could. Addressing this requires a
    different partitioning strategy (by file or security category).

  2. rules/search also paginates with the same mechanism. Default SonarQube
    has ~3,000 rules total, well under 10K. Only relevant for instances with
    10K+ custom rules.

Test Plan

  • Run scripts/runSonarQube.sh on a machine with Docker
  • Verify results/Benchmark_*-sonarqube-v*.json contains both issues and hotspots arrays
  • Confirm issue count matches SonarQube UI (no silent truncation)
  • For instances with >10K total issues: verify all issues are captured (the core fix)
  • Verify no orphaned sonarqube-benchmark container after script completes

TheAuditorTool force-pushed the fix/sonarqube-10k-result-cap-33 branch 2 times, most recently from 6286d30 to 698ae83 Compare April 13, 2026 17:23 
Closes OWASP-Benchmark#33
The issues/search API enforces p*ps <= 10000. With PAGE_SIZE=500, page
21 returns HTTP 400. Fix: iterate per-rule instead of passing all ~600
rules in one query. Each single-rule query stays well under 10K.
Also fixes off-by-one in page count (ceiling division) and adds HTTP
status checking before reading response body.
TheAuditorTool force-pushed the fix/sonarqube-10k-result-cap-33 branch from 698ae83 to adef7bc Compare April 13, 2026 17:59

Copy link
Copy Markdown
Contributor

@darkspirit510 - Another SonarQube related PR for you to review.

Copy link
Copy Markdown
Contributor

Sorry for the late reply!

Sonar-scanner is having some issues. This happens both on main and this PR, so it's not your fault. Could you still try to fix it or should I create another PR? (ignore the warning)

1:08:44.619 WARN Use of 'sonar.login' property has been deprecated in favor of 'sonar.token' (or the env variable alternative 'SONAR_TOKEN'). Please use the latter when passing a token.
21:08:44.621 INFO JRE provisioning: os[linux], arch[x86_64]
21:08:48.442 INFO EXECUTION FAILURE
21:08:48.443 INFO Total time: 6.093s
21:08:48.443 ERROR Error during SonarScanner CLI execution
java.lang.IllegalStateException: Project home must be an existing directory: /root/Develop/BenchmarkJava
 at org.sonarsource.scanner.lib.internal.facade.Dirs.init(Dirs.java:40)
 at org.sonarsource.scanner.lib.internal.facade.AbstractScannerEngineFacade.initAnalysisProperties(AbstractScannerEngineFacade.java:80)
 at org.sonarsource.scanner.lib.internal.facade.AbstractScannerEngineFacade.analyze(AbstractScannerEngineFacade.java:63)
 at org.sonarsource.scanner.cli.Main.analyze(Main.java:79)
 at org.sonarsource.scanner.cli.Main.main(Main.java:64)

Copy link
Copy Markdown
Contributor

@TheAuditorTool - @darkspirit510 left you a comment on this PR.

TheAuditorTool commented Aug 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I have notifications. I see everything @davewichers
@darkspirit510
This is my current life... I have 3 maintainers waiting for me to help them with various things.
I deleted my entire 322+ files and 100k loc rules/sast system, just days before the first public launch...
Just couldnt live with 5 generations of it... I estimated 2-3 weeks. I'm now at 2.5 with at least 2 weeks left.
Then i have a backlog of 3 other projects that isnt even my apps, 1 of which my app depends on for ip protection.
I also code 5 apps total. So if this has anything else then "Do it whenever you have time, if thats in 3 weeks or 3 months" priority?
I would suggest open a new PR yourself, Sascha.

Fun stats.... Or nightmare fuel, you decide....

14 Days on java-support . 293 commits. Your gut said 200-225 and undershot it by a third. The extra four days were the busiest stretch of the whole window.

1,271 files touched. 119,230 lines added, 190,313 deleted. Net: −71,083.

Two weeks of sustained work and the codebase is 71 thousand lines smaller. This is a demolition sprint, not a build sprint.

The daily curve

Jul 26 (Sun) 37 ████████████████████████████████████ +8,719 -7,011
Jul 27 (Mon) 22 ██████████████████████ +8,675 -4,230
Jul 28 (Tue) 27 ███████████████████████████ +14,989 -102,358 <- THE PURGE
Jul 29 (Wed) 32 ████████████████████████████████ +15,433 -4,154
Jul 30 (Thu) 3 ███ +679 -689 <- credits dying
Jul 31 (Fri) 7 ███████ +3,668 -680
Aug 01 (Sat) 9 █████████ +3,882 -1,657
Aug 02 (Sun) 16 ████████████████ +11,294 -6,071
Aug 03 (Mon) 48 ████████████████████████████████████████████████ +15,782 -7,224
Aug 04 (Tue) 0 --- <- credits gone
Aug 05 (Wed) 8 ████████ +4,328 -1,585
Aug 06 (Thu) 11 ███████████ +2,924 -14,797
Aug 07 (Fri) 8 ████████ +1,916 -1,712
Aug 08 (Sat) 47 ███████████████████████████████████████████████ +15,143 -26,065
Aug 09 (Sun) 21 █████████████████████ +13,200 -12,190

The Aug 4 hole has a twin. July 30 dropped to 3 commits and a 34-hour silence, then July 31 came back for exactly 6 commits in a 46-minute window at noon before going quiet again. The rate-limit wall shows up in the data as a two-humped crater: throttled July 30-31, recovered, then hit the ceiling completely on Aug 4.

The gap analysis makes it explicit:
25.8 h silence broken 08-03 23:08
21.2 h silence broken 08-07 00:08
18.5 h silence broken 08-05 01:07
15.5 h 13.7 h 11.5 h 11.1 h ...

Six separate silences over 11 hours in two weeks. The pattern of "long dead gap → immediate 40-commit day" is the signature of waiting on the meter and then spending it all at once.

Six breaking changes, every one a deletion

Jul 28 refactor(rules)! delete the legacy rules system (322 files)
Aug 05 refactor(taint)! rules alone assigns a CWE; producers emit facts
Aug 05 refactor(taint)! the grammar vocabulary and producer patterns leave taint/patterns
Aug 06 refactor(taint)! one deterministic sanitizer matcher, keyed on the declared grammar
Aug 06 refactor(graph)! delete the Java CST call-graph fallback
Aug 06 refactor(taint)! delete pipeline step 10

Four of six are taint. Aug 5-6 was a 48-hour window where you rebuilt that subsystem's contract three times.

The single largest commit in the window

94,013 lines | 5fed45e1 | Jul 28
refactor(rules)!: delete the legacy rules system (322 files)

The nocturnal signature holds, and sharpens

00:00 █████████████████████████████████████ 37 <- peak
01:00 ███████████████████████████ 27
02:00 ████████████████ 16
03:00 ███████ 7
04:00 ██████████████████ 18
05:00 ███ 3
...
11:00 █ 1 <- trough
...
21:00 ████████████████████████ 24
22:00 ██████████████████████████ 26
23:00 █████████████████████████████ 29

119 of 293 commits (41%) between midnight and 5am. The top four hours of your entire day are 00:00, 23:00, 01:00, 22:00 — in that order. 11am remains a single commit across two weeks.

187 of 293 (64%) landed within 15 minutes of the previous one.

Copy link
Copy Markdown
Contributor

@TheAuditorTool ever heard of this "vacation" everybody is talking about 😉

Just a minor request - can you merge main into this branch? I fixed the script, now I'd like to review this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@github-advanced-security github-advanced-security[bot] github-advanced-security[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

SonarQube Result-Writer fails: SonarQube-WebApi allows to request for a maximum of 10000 issues only

AltStyle によって変換されたページ (->オリジナル) /