-
-
Notifications
You must be signed in to change notification settings - Fork 0
OpenTelemetry sampler is not configurable #184
Description
OpenTelemetryInstrument.bootstrap() builds the provider with no sampler:
tracer_provider = TracerProvider(resource=resource)
which means the SDK default, parentbased_always_on. There is no configuration surface for a sampler anywhere in OpenTelemetryConfig, so a service cannot head-sample its own traces at all - every request is recorded, serialised and shipped to the collector. sentry_traces_sample_rate has an equivalent for Sentry; OpenTelemetry has nothing.
What it costs
Benchmark: async def endpoint returning {"ok": True}, FastAPIBootstrapper with only the OpenTelemetry instrument on, OTLP/HTTP to a local stub sink that returns 200. In-process ASGI driving so only library cost is measured. Apple M2, CPython 3.14.7, opentelemetry-sdk 1.44.0, opentelemetry-instrumentation-fastapi 0.65b0.
| scenario | RPS | μs/req |
|---|---|---|
| as configured today (always-on) | 7375 | 135.6 |
sampler=ParentBased(TraceIdRatioBased(0.01)) |
12484 | 80.1 |
55 μs/request, on an endpoint whose bare cost is 16 μs. For context, in the same harness OpenTelemetry is the most expensive instrument in the stack by a wide margin:
| instrument alone | +μs/req |
|---|---|
LoggingInstrument (no logs emitted) |
+0.1 |
PrometheusInstrument |
+17.5 |
SentryInstrument |
+58.5 |
OpenTelemetryInstrument |
+120.1 |
Proposal
Add a sampler field to OpenTelemetryConfig and pass it through:
opentelemetry_sampler: "Sampler | None" = None ... tracer_provider = TracerProvider(resource=resource, sampler=self.bootstrap_config.opentelemetry_sampler)
TracerProvider(sampler=None) is not the same as omitting it, so the pass-through needs a conditional or sampler=... or _DEFAULT.
The default should not change - always-on is the right default and the sample rate is a user decision. The gap is that it cannot be expressed today without bypassing the instrument.
Open question: a bare Sampler field is the most flexible, but it puts an opentelemetry.sdk type in the config signature for a package where the OTel extra is optional. An alternative is opentelemetry_traces_sample_rate: float | None = None mapped internally to ParentBased(TraceIdRatioBased(...)), which mirrors sentry_traces_sample_rate and keeps the config dependency-free at the cost of flexibility. I lean towards the float.
This applies to every bootstrapper, since they share OpenTelemetryInstrument.
Found while benchmarking the observability stack; see also the sibling issue about exclude_spans.