-
-
Notifications
You must be signed in to change notification settings - Fork 74
Adds .NET distributed tracing instrumentation & metrics. #1200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Included the final portion of #1196 (metrics) here, as it requires the same infrastructure as traces.
|
Here is before and after comparison of Perf benchmark results on my machine. BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.26120.2130) AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores .NET SDK=9.0.100-rc.2.24474.11 [Host] : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2 NuGet : .NET 6.0.35 (6.0.3524.45918), X64 RyuJIT AVX2 Project : .NET 6.0.35 (6.0.3524.45918), X64 RyuJIT AVX2 Jit=RyuJit Platform=X64 Toolchain=.NET 6.0 WarmupCount=3 Perf.CommandBenchmarkFirebirdSQL:master (Old) <-> fdcastel:add-telemetry-trace (New)
|
Thanks @willibrandon. This surely can be improved.
Could you kindly share the steps/configuration you used for the above benchmark results?
Mine here produces a comparison between Release (built from local sources) and ReleaseNuget (NuGet package for FirebirdClient 8.5.4).
@willibrandon I pushed a new PR in #1203 (it is based on this PR).
Could you please clone it and run ./run-benchmark.ps1 in your system?
The script will compare the current project (built from sources with .NET8) with the latest NuGet release (10.3.1).
BenchmarkDotNet v0.14.0, Windows 11 (10.0.22621.4317/22H2/2022Update/SunValley2)
13th Gen Intel Core i7-13700T, 1 CPU, 24 logical and 16 physical cores
.NET SDK 9.0.100-rc.2.24474.11
[Host] : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2
Core80 : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2
NuGet80 : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2
Jit=RyuJit Platform=X64 Toolchain=.NET 8.0
WarmupCount=3
| Method | Job | BuildConfiguration | DataType | Count | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|-------- |-------- |------------------- |--------------------- |------ |------------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| Execute | Core80 | Release | BIGINT | 100 | 16,219.7 us | 140.65 us | 131.57 us | 1.00 | 0.01 | - | 306.16 KB | 1.01 |
| Execute | NuGet80 | ReleaseNuGet | BIGINT | 100 | 16,283.9 us | 29.35 us | 26.01 us | 1.00 | 0.00 | - | 301.65 KB | 1.00 |
| | | | | | | | | | | | | |
| Fetch | Core80 | Release | BIGINT | 100 | 372.1 us | 2.91 us | 2.43 us | 1.00 | 0.01 | 2.9297 | 55.66 KB | 1.09 |
| Fetch | NuGet80 | ReleaseNuGet | BIGINT | 100 | 371.3 us | 2.47 us | 2.19 us | 1.00 | 0.01 | 2.9297 | 51.15 KB | 1.00 |
| | | | | | | | | | | | | |
| Execute | Core80 | Release | VARCH(...) UTF8 [30] | 100 | 18,021.3 us | 352.58 us | 482.62 us | 1.00 | 0.03 | - | 310.1 KB | 1.01 |
| Execute | NuGet80 | ReleaseNuGet | VARCH(...) UTF8 [30] | 100 | 18,071.4 us | 352.48 us | 377.14 us | 1.00 | 0.03 | - | 305.64 KB | 1.00 |
| | | | | | | | | | | | | |
| Fetch | Core80 | Release | VARCH(...) UTF8 [30] | 100 | 416.7 us | 6.50 us | 5.43 us | 1.01 | 0.03 | 2.9297 | 59.54 KB | 1.08 |
| Fetch | NuGet80 | ReleaseNuGet | VARCH(...) UTF8 [30] | 100 | 413.6 us | 8.09 us | 10.52 us | 1.00 | 0.03 | 2.9297 | 55.02 KB | 1.00 |
// * Hints *
Outliers
CommandBenchmark.Execute: NuGet80 -> 1 outlier was removed (17.09 ms)
CommandBenchmark.Fetch: Core80 -> 2 outliers were removed (383.60 us, 403.48 us)
CommandBenchmark.Fetch: NuGet80 -> 1 outlier was removed (389.54 us)
CommandBenchmark.Execute: Core80 -> 1 outlier was removed, 3 outliers were detected (16.46 ms, 16.87 ms, 19.28 ms)
CommandBenchmark.Fetch: Core80 -> 2 outliers were removed (442.98 us, 447.90 us)
// * Legends *
DataType : Value of the 'DataType' parameter
Count : Value of the 'Count' parameter
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Ratio : Mean of the ratio distribution ([Current]/[Baseline])
RatioSD : Standard deviation of the ratio distribution ([Current]/[Baseline])
Gen0 : GC Generation 0 collects per 1000 operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
Alloc Ratio : Allocated memory ratio distribution ([Current]/[Baseline])
1 us : 1 Microsecond (0.000001 sec)
// * Diagnostic Output - MemoryDiagnoser *
// ***** BenchmarkRunner: End *****
Run time: 00:01:41 (101.22 sec), executed benchmarks: 8
Global total time: 00:01:51 (111.73 sec), executed benchmarks: 8
@fdcastel - Interesting, and thanks! Taking a look now.
I had the same thought the Perf project toolchain was out of date.
Okay, I see how the performance project works now with the conditional project/package reference, and with the latest Nuget release being the baseline. That is way better approach than the way I was attempting to compare the the results historically.
@fdcastel - Thank you very much for showing me how the performance project works. I just want try and confirm we haven't regressed in someway with this change.
47a5b9b to
fe2b942
Compare
Rebased with latest master.
Initial work for #1196.