I have a solution with many projects. When I run dotnet test, it selects all my xUnit projects and seems to correctly execute all of them. But at the last line of what dotnet test prints, the number of test cases is inconsistent. Sometimes it shows 8, sometimes it shows 9.
The output when it shows 9:
C:...\SiLho\MREDotnetTest>dotnet test --logger html
Restore complete (0.6s)
SharedLibraries succeeded (0.1s) → SharedLibraries\bin\Debug\net9.0\win-x64\SharedLibraries.dll
TestProject1 succeeded (0.1s) → TestProject1\bin\Debug\net9.0\TestProject1.dll
Connectors.Xlsx succeeded (0.1s) → Connectors\Xlsx\bin\Debug\net9.0\win-x64\Connectors.Xlsx.dll
Connectors.Fusion succeeded (0.2s) → Connectors\Fusion\bin\Debug\net9.0\win-x64\Connectors.Fusion.dll
VersionTracker succeeded (0.2s) → VersionTracker\bin\Debug\net9.0\win-x64\VersionTracker.dll
Core succeeded (0.2s) → Core\bin\Debug\net9.0\win-x64\Core.dll
Thisisalibrary succeeded (0.2s) → Thisisalibrary\bin\Debug\net9.0\win-x64\Thisisalibrary.dll
VersionTracker.Tests succeeded (0.2s) → VersionTracker.Tests\bin\Debug\net9.0\win-x64\VersionTracker.Tests.dll
Thisisalibrarytest succeeded (0.2s) → Thisisalibrarytest\bin\Debug\net9.0\win-x64\Thisisalibrarytest.dll
Connectors.Tests.Fusion.Tests succeeded (0.2s) → Connectors.Tests\Fusion.Tests\bin\Debug\net9.0\win-x64\Connectors.Tests.Fusion.Tests.dll
Mylibrary succeeded (0.3s) → Mylibrary\bin\Debug\net9.0\win-x64\Mylibrary.dll
SharedLibraries.Tests succeeded (0.2s) → SharedLibraries.Tests\bin\Debug\net9.0\win-x64\SharedLibraries.Tests.dll
Connectors.Tests.Xlsx.Tests succeeded (0.2s) → Connectors.Tests\Xlsx.Tests\bin\Debug\net9.0\win-x64\Connectors.Tests.Xlsx.Tests.dll
Mylibrarytest succeeded (0.2s) → Mylibrarytest\bin\Debug\net9.0\win-x64\Mylibrarytest.dll
Core.Tests succeeded (0.2s) → Core.Tests\bin\Debug\net9.0\win-x64\Core.Tests.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.10] Discovering: TestProject1
[xUnit.net 00:00:00.14] Discovered: TestProject1
[xUnit.net 00:00:00.16] Starting: TestProject1
[xUnit.net 00:00:00.20] Finished: TestProject1
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.10] Discovering: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.10] Discovering: Thisisalibrarytest
[xUnit.net 00:00:00.10] Discovering: VersionTracker.Tests
[xUnit.net 00:00:00.13] Discovered: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.14] Discovered: Thisisalibrarytest
[xUnit.net 00:00:00.14] Discovered: VersionTracker.Tests
[xUnit.net 00:00:00.15] Starting: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.16] Starting: Thisisalibrarytest
[xUnit.net 00:00:00.16] Starting: VersionTracker.Tests
[xUnit.net 00:00:00.25] Finished: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.26] Finished: Thisisalibrarytest
[xUnit.net 00:00:00.25] Finished: VersionTracker.Tests
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
Html test results file : C:\Users\...\MREDotnetTest\TestProject1\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.14] Discovering: Core.Tests
[xUnit.net 00:00:00.12] Discovering: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.12] Discovering: Mylibrarytest
[xUnit.net 00:00:00.11] Discovering: SharedLibraries.Tests
[xUnit.net 00:00:00.18] Discovered: Core.Tests
[xUnit.net 00:00:00.16] Discovered: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.16] Discovered: Mylibrarytest
TestProject1 test succeeded (1.5s)
[xUnit.net 00:00:00.20] Starting: Core.Tests
[xUnit.net 00:00:00.15] Discovered: SharedLibraries.Tests
[xUnit.net 00:00:00.20] Starting: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.20] Starting: Mylibrarytest
[xUnit.net 00:00:00.19] Starting: SharedLibraries.Tests
[xUnit.net 00:00:00.31] Finished: Core.Tests
[xUnit.net 00:00:00.31] Finished: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.31] Finished: Mylibrarytest
[xUnit.net 00:00:00.29] Finished: SharedLibraries.Tests
Html test results file : C:\Users\...\MREDotnetTest\Thisisalibrarytest\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Html test results file : C:\Users\...\MREDotnetTest\VersionTracker.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Html test results file : C:\Users\...\MREDotnetTest\Connectors.Tests\Fusion.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Thisisalibrarytest test succeeded (1.7s)
VersionTracker.Tests test succeeded (1.7s)
Connectors.Tests.Fusion.Tests test succeeded (1.7s)
ConsoleApp succeeded (0.3s) → ConsoleApp\bin\Debug\net9.0\win-x64\ConsoleApp.dll
Html test results file : C:\Users\...\MREDotnetTest\Core.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Core.Tests test succeeded (1.5s)
Html test results file : C:\Users\...\MREDotnetTest\SharedLibraries.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Html test results file : C:\Users\...\MREDotnetTest\Connectors.Tests\Xlsx.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Html test results file : C:\Users\...\MREDotnetTest\Mylibrarytest\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160659.html
Connectors.Tests.Xlsx.Tests test succeeded (1.6s)
SharedLibraries.Tests test succeeded (1.6s)
Mylibrarytest test succeeded (1.6s)
ConsoleApp.Tests succeeded (0.2s) → ConsoleApp.Tests\bin\Debug\net9.0\win-x64\ConsoleApp.Tests.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.07] Discovering: ConsoleApp.Tests
[xUnit.net 00:00:00.11] Discovered: ConsoleApp.Tests
[xUnit.net 00:00:00.13] Starting: ConsoleApp.Tests
[xUnit.net 00:00:00.16] Finished: ConsoleApp.Tests
Html test results file : C:\Users\...\MREDotnetTest\ConsoleApp.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160700.html
ConsoleApp.Tests test succeeded (1.0s)
Test summary: total: 9, failed: 0, succeeded: 9, skipped: 0, duration: 3.1s
Build succeeded in 4.1s
The output when it shows 8:
C:\...\MREDotnetTest>dotnet test --logger html
Restore complete (0.6s)
SharedLibraries succeeded (0.1s) → SharedLibraries\bin\Debug\net9.0\win-x64\SharedLibraries.dll
TestProject1 succeeded (0.1s) → TestProject1\bin\Debug\net9.0\TestProject1.dll
VersionTracker succeeded (0.1s) → VersionTracker\bin\Debug\net9.0\win-x64\VersionTracker.dll
Thisisalibrary succeeded (0.1s) → Thisisalibrary\bin\Debug\net9.0\win-x64\Thisisalibrary.dll
Core succeeded (0.1s) → Core\bin\Debug\net9.0\win-x64\Core.dll
Connectors.Xlsx succeeded (0.1s) → Connectors\Xlsx\bin\Debug\net9.0\win-x64\Connectors.Xlsx.dll
VersionTracker.Tests succeeded (0.1s) → VersionTracker.Tests\bin\Debug\net9.0\win-x64\VersionTracker.Tests.dll
Thisisalibrarytest succeeded (0.1s) → Thisisalibrarytest\bin\Debug\net9.0\win-x64\Thisisalibrarytest.dll
Mylibrary succeeded (0.3s) → Mylibrary\bin\Debug\net9.0\win-x64\Mylibrary.dll
Mylibrarytest succeeded (0.1s) → Mylibrarytest\bin\Debug\net9.0\win-x64\Mylibrarytest.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.07] Discovering: TestProject1
[xUnit.net 00:00:00.10] Discovered: TestProject1
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.11] Starting: TestProject1
[xUnit.net 00:00:00.15] Finished: TestProject1
[xUnit.net 00:00:00.06] Discovering: VersionTracker.Tests
[xUnit.net 00:00:00.09] Discovered: VersionTracker.Tests
[xUnit.net 00:00:00.11] Starting: VersionTracker.Tests
[xUnit.net 00:00:00.15] Finished: VersionTracker.Tests
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
Html test results file : C:\Users\...\MREDotnetTest\TestProject1\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160749.html
TestProject1 test succeeded (1.0s)
[xUnit.net 00:00:00.07] Discovering: Mylibrarytest
[xUnit.net 00:00:00.10] Discovered: Mylibrarytest
[xUnit.net 00:00:00.12] Starting: Mylibrarytest
Html test results file : C:\Users\...\MREDotnetTest\VersionTracker.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160749.html
Connectors.Fusion succeeded (0.1s) → Connectors\Fusion\bin\Debug\net9.0\win-x64\Connectors.Fusion.dll
VersionTracker.Tests test succeeded (1.0s)
Thisisalibrarytest test succeeded (1.0s)
[xUnit.net 00:00:00.16] Finished: Mylibrarytest
ConsoleApp succeeded (0.1s) → ConsoleApp\bin\Debug\net9.0\win-x64\ConsoleApp.dll
SharedLibraries.Tests succeeded (0.1s) → SharedLibraries.Tests\bin\Debug\net9.0\win-x64\SharedLibraries.Tests.dll
Connectors.Tests.Xlsx.Tests succeeded (0.1s) → Connectors.Tests\Xlsx.Tests\bin\Debug\net9.0\win-x64\Connectors.Tests.Xlsx.Tests.dll
Core.Tests succeeded (0.1s) → Core.Tests\bin\Debug\net9.0\win-x64\Core.Tests.dll
Connectors.Tests.Fusion.Tests succeeded (0.1s) → Connectors.Tests\Fusion.Tests\bin\Debug\net9.0\win-x64\Connectors.Tests.Fusion.Tests.dll
ConsoleApp.Tests succeeded (0.1s) → ConsoleApp.Tests\bin\Debug\net9.0\win-x64\ConsoleApp.Tests.dll
Html test results file : C:\Users\...\MREDotnetTest\Mylibrarytest\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
Mylibrarytest test succeeded (1.0s)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.07] Discovering: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.06] Discovering: SharedLibraries.Tests
[xUnit.net 00:00:00.07] Discovering: Core.Tests
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)
[xUnit.net 00:00:00.07] Discovering: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.10] Discovered: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.10] Discovered: SharedLibraries.Tests
[xUnit.net 00:00:00.10] Discovered: Core.Tests
[xUnit.net 00:00:00.12] Starting: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.11] Starting: SharedLibraries.Tests
[xUnit.net 00:00:00.10] Discovered: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.12] Starting: Core.Tests
[xUnit.net 00:00:00.12] Starting: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.07] Discovering: ConsoleApp.Tests
[xUnit.net 00:00:00.15] Finished: Connectors.Tests.Xlsx.Tests
[xUnit.net 00:00:00.15] Finished: SharedLibraries.Tests
[xUnit.net 00:00:00.16] Finished: Core.Tests
[xUnit.net 00:00:00.15] Finished: Connectors.Tests.Fusion.Tests
[xUnit.net 00:00:00.11] Discovered: ConsoleApp.Tests
[xUnit.net 00:00:00.13] Starting: ConsoleApp.Tests
[xUnit.net 00:00:00.17] Finished: ConsoleApp.Tests
Html test results file : C:\Users\...\MREDotnetTest\SharedLibraries.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
Html test results file : C:\Users\...\MREDotnetTest\Connectors.Tests\Xlsx.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
Html test results file : C:\Users\...\MREDotnetTest\Core.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
SharedLibraries.Tests test succeeded (1.0s)
Connectors.Tests.Xlsx.Tests test succeeded (1.0s)
Core.Tests test succeeded (1.0s)
Html test results file : C:\Users\...\MREDotnetTest\Connectors.Tests\Fusion.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
Connectors.Tests.Fusion.Tests test succeeded (1.0s)
Html test results file : C:\Users\...\MREDotnetTest\ConsoleApp.Tests\TestResults\TestResult_lhott_IPHONEDEROMAIN_20251024_160750.html
ConsoleApp.Tests test succeeded (1.0s)
Test summary: total: 8, failed: 0, succeeded: 8, skipped: 0, duration: 2.2s
Build succeeded in 3.1s
I need to have around a dozen projects in the solution for the bug to trigger, I cannot make a MRE with less projects. These projects can be completely empty (with empty test cases for the xUnit projects). They need to "call" each other in the .csproj (even though all the files are empty so technically they don't actually call each other).
I cannot reproduce on another computer that has everything set up the same except it has 64 GB RAM and a Ryzen 9.
For the record, I use --logger html but I don't think that has anything to do with the problem. What should I investigate? How can I get to the bottom of this?
Already checked:
- xUnit is up to date.
- I have the latest .NET version (
9.0.306) - Windows is up to date
Specs:
- AMD Ryzen 7 5800H with Radeon Graphics
- 32.0 GB RAM
- Windows 11, Version 10.0.26200 Build 26200
- BIOS: LENOVO HHCN24WW, 24/11/2021
MRE to reproduce (really, I apologize I can't make it smaller, the bug disappears when there are less files ... - it's a lot of files but they are all basically empty): https://github.com/JasonMendoza2008/MREDotnetTest
Proof:
Maybe to investigate?
- Why is there
xUnit.net VSTest Adapter v3.1.5+1b188a7b0a (64-bit .NET 9.0.10)in the output? I only have9.0.306:
PS C:\...> dotnet --list-sdks
9.0.306 [C:\Program Files\dotnet\sdk]
- I cannot reproduce the bug with:
-m:1but it is not really a solution .. using multiple cores speeds up the process so much (doesn't matter for the MRE, matters for a real monolith solution).