Add sdk-instrumentation-otel: wire the observability seams to an OpenTelemetry backend

[OmarAlJarrah]

Summary

sdk-core defines the full observability seam surface — a span-level Tracer/Span, the richer HTTP-shaped HttpTracer/HttpTracerFactory event vocabulary, and a metrics Meter/LongCounter/DoubleHistogram SPI — but ships only no-op implementations of every one. The default instrumentation step already registers OpenTelemetry-named instruments and opens a span per request against these seams, so the plumbing to a real backend is a small, well-scoped mapping. What is missing is the module that provides it.

This issue proposes a new sdk-instrumentation-otel adapter module that maps the core seams onto OpenTelemetry spans and meters, mirroring the existing async/transport adapter pattern: one public entry point, the third-party runtime confined to the adapter, sdk-core left seam-only.

Problem

Consumers of this toolkit are external — generated service clients, downstream SDKs, and application code built on top of the pipeline. For observability, sdk-core gives them contracts and no-op defaults, and nothing else:

• The metrics seam ships NoopMeter (sdk-core/src/main/kotlin/org/dexpace/sdk/core/instrumentation/metrics/NoopMeter.kt), which returns shared NoopCounter/NoopHistogram singletons that discard every measurement. NoopTracer (.../instrumentation/NoopTracer.kt) returns Span.NOOP for every startSpan. NoopHttpTracerFactory (.../instrumentation/HttpTracerFactory.kt) returns the shared NoopHttpTracer.
• HttpInstrumentationOptions defaults tracer = NoopTracer and meter = NoopMeter (.../http/pipeline/steps/HttpInstrumentationOptions.kt:70-71), and InstrumentationContext.httpTracerFactory defaults to NoopHttpTracerFactory (.../instrumentation/InstrumentationContext.kt:59-60).
• The default step is already wired to emit against these seams. DefaultInstrumentationStep registers a request counter http.client.request.count (unit {request}) and a latency histogram http.client.request.duration (unit ms) (.../http/pipeline/steps/DefaultInstrumentationStep.kt:79-92), and starts a span http <METHOD> per request (DefaultInstrumentationStep.kt:100-104); the metrics are tagged with http.request.method plus http.response.status_code on success or error.type on failure (DefaultInstrumentationStep.kt:300-319). DefaultAsyncInstrumentationStep mirrors the same registration.

So with the toolkit alone, every metric and span is discarded. A consumer who wants real telemetry has to hand-write a Tracer, a Meter (with a LongCounter and DoubleHistogram), and — if they want the retry/attempt event stream — an HttpTracerFactory/HttpTracer, then install them onto HttpInstrumentationOptions. That mapping is non-trivial (attribute conversion, instrument caching, the must-not-throw contract) and it is identical for every consumer. The predictable outcome is that most consumers write none of it and get no telemetry from an SDK whose instrumentation is otherwise fully built. This is platform surface that is unused inside this repo yet essential to the SDK's users; the Meter KDoc and docs/refs-comparison.md already name the missing piece as sdk-instrumentation-otel.

There is currently no observability module in settings.gradle.kts, and no OpenTelemetry or Micrometer coordinates in gradle/libs.versions.toml.

Proposed approach

Add a published adapter module, sdk-instrumentation-otel, that depends on sdk-core and the OpenTelemetry Java API, and provides implementations of the core seams that forward to an OpenTelemetry backend.

• Single public entry point, per the adapter-module convention (only one public type, like OkioIoProvider in sdk-io-okio3). A factory that takes an io.opentelemetry.api.OpenTelemetry (or its Tracer/Meter) and produces the core seam implementations a consumer installs onto HttpInstrumentationOptions:
  • a Tracer that maps startSpan(name, attributes) onto an OTel SpanBuilder, converting the Map<String, Any> attributes onto OTel Attributes, and returns a Span wrapper whose end()/scope/recording semantics satisfy OBS-21/OBS-22/OBS-23;
  • a Meter whose counter(...)/histogram(...) build OTel LongCounter/DoubleHistogram instruments (caching by (name, description, unit) as the seam encourages, or relying on OTel's register-by-name idempotency), so the already-registered http.client.request.count / http.client.request.duration instruments flow through with their existing attribute sets;
  • optionally an HttpTracerFactory/HttpTracer that maps the richer event vocabulary (operationStarted, attemptStarted/attemptFailed/attemptRetriesExhausted, transport milestones) onto span events/attributes. Note the HTTP-tracer vocabulary is defined and carried on InstrumentationContext but its pipeline wiring is still a follow-up (see the HttpTracer KDoc and OBS-29); the adapter should provide the mapping so it is ready when that wiring lands.
• Core stays seam-only. No OpenTelemetry (or any third-party runtime) dependency is added to sdk-core; the near-zero-dep core (Kotlin stdlib + compile-only SLF4J) is untouched. The OTel runtime is a normal (non-compile-only) dependency of the adapter module only, added to gradle/libs.versions.toml.
• Opt-in wiring, no auto-install. Unlike IoProvider, the tracer/meter seams are installed explicitly through HttpInstrumentationOptions.Builder.tracer(...) / .meter(...). The adapter is wired by the consumer at that call site; it must not ServiceLoader-register itself or otherwise activate implicitly.
• Module wiring. Apply id("dexpace.published-module"), add include("sdk-instrumentation-otel") to settings.gradle.kts, MIT header on every file. Target Java 8 (the OpenTelemetry API supports Java 8), keeping it consistent with the other Java-8 adapters; use ReentrantLock (not synchronized) for any instrument-cache guard.

An sdk-metrics-micrometer adapter for consumers standardized on Micrometer is a reasonable follow-up built on the same Meter seam, but it is out of scope here (see non-goals).

Scope and non-goals

In scope:

• New sdk-instrumentation-otel module mapping Tracer/Span, Meter/LongCounter/DoubleHistogram, and (optionally) HttpTracer/HttpTracerFactory onto OpenTelemetry.
• Version-catalog entry for the OpenTelemetry API/SDK, scoped to that module.
• Tests proving the mapping and the must-not-throw / non-finite contracts.

Non-goals:

• No dependency, seam reshaping, or behavior change in sdk-core; the seams keep their current shapes. If the HttpTracer event vocabulary needs pipeline wiring to fire, that is a separate follow-up.
• Not the sdk-metrics-micrometer module — a separate issue.
• No ServiceLoader auto-discovery/auto-install of the tracer or meter; wiring stays explicit via HttpInstrumentationOptions.
• No exporter/collector configuration opinions baked into the adapter — the consumer supplies a configured OpenTelemetry instance.
• No codegen changes.

Acceptance criteria

• sdk-instrumentation-otel exists in settings.gradle.kts, applies dexpace.published-module, and has exactly one public entry point.
• sdk-core gains no new runtime dependency: its resolved runtime classpath contains no OpenTelemetry (or other third-party) coordinate, and the OTel runtime appears only on sdk-instrumentation-otel's classpath (OBS-31). sdk-core's public API is unchanged, so its .api snapshot needs no update.
• Installing the adapter's Tracer and Meter on HttpInstrumentationOptions and driving one success and one failure through the pipeline produces two instruments named http.client.request.count (unit {request}) and http.client.request.duration (unit ms), tagged with http.request.method plus http.response.status_code (success) / error.type (failure) (OBS-32), and one span per request via the installed tracer.
• The adapter Meter returns the same instrument instance for repeated identical (name, description, unit) calls; any cache the adapter maintains is guarded with ReentrantLock (not synchronized).
• The counter and histogram tolerate any input without throwing; recording non-finite values (NaN/Infinity) into the histogram is handled by the adapter and does not propagate (OBS-33, which delegates non-finite handling to concrete adapters).
• No tracer, meter, or HTTP-tracer callback throws under concurrent invocation from transport threads (OBS-30) — the runtime does not defensively wrap these calls (OBS-20), so the adapter must honor the never-throw contract itself.
• Span recording flag, idempotent end(), and scope restore-on-close (including on throw) behave per OBS-21/OBS-22; log-correlation activation pushes/restores trace.id/span.id per OBS-23.
• The module builds under the full gated build (ktlint, detekt, allWarningsAsErrors, explicit-API strict, apiCheck, coverage floor), with tests covering the mapping (an in-memory/SDK-testing OTel exporter is sufficient; no real sockets).

References

• Spec (docs/product-spec.md): OBS-31 (metrics SPI + no-op default + core pulls no metrics runtime), OBS-32 (OTel-convention names/units and the two default HTTP instruments with method + status/error tags), OBS-33 (counter non-negative-only, histogram tolerates any input, non-finite delegated to concrete adapters), OBS-30 (tracer/meter callbacks concurrent-safe and never throw), OBS-20 (runtime does not defensively wrap tracer/meter), OBS-25 (allocation-free no-op tracing defaults), OBS-21/OBS-22/OBS-23 (span recording/scope/log-correlation), OBS-28/OBS-29 (HTTP-tracer vocabulary and lifecycle ordering; wiring is a follow-up), OBS-26/OBS-27 (W3C identifiers and id generation).
• docs/refs-comparison.md: "An sdk-instrumentation-otel adapter wiring our events to OpenTelemetry spans + metrics is not yet built" (line 278); listed under Remaining work, item 2 (line 346).
• Core seams: .../instrumentation/metrics/Meter.kt, NoopMeter.kt, .../instrumentation/Tracer.kt, NoopTracer.kt, HttpTracer.kt, HttpTracerFactory.kt, InstrumentationContext.kt.
• Default wiring: .../http/pipeline/steps/DefaultInstrumentationStep.kt:79-92 (instrument registration), :100-104 (span start), :300-319 (metric attributes + emission); DefaultAsyncInstrumentationStep.kt (async mirror); HttpInstrumentationOptions.kt:70-71 (tracer/meter defaults), :116,119 (install points).