Tracing: W3C Trace Context Support

[danielkhan]

While we are moving to a world where we may process, propagate, and ingest pure OpenTelemetry (OTLP) data, we have to reconsider our trace context propagation schema and adopt W3C Trace Context fully.

Project Board

About W3C Trace Context

W3C Trace Context is a standard for trace context propagation.
It defines three headers.

• traceparent (W3C): to capture the relationship between spans and propagate sampling decisions
• tracestate (W3C): A vendor-prefixed key-value list to hold vendor-specific (meta) data, like tenant IDs or vendor-local trace IDs
• baggage (W3C): A userland read- and writeable key-value list of properties that can propagate arbitrary information—E.g., for a/b tests or for propagating a customer loyalty status.

Sentry has a very similar trace propagation concept.
It defines

• sentry-trace, which is almost similar to trace-parent, except missing some flags W3C introduced
• baggage where we follow the spec, yet we are using it to propagate essential sentry-specific metadata like the dynamic sampling context (DSC). This can have unexpected consequences as the user can remove entries or create a new Baggage object altogether (see Baggage interface in OTel).

We are not using a concept like trace-state.

Problem Description

The problem arises in mixed scenarios where one tier might be instrumented purely with OpenTelemetry.

In this example, the web frontend sends sentry trace headers to the Node middleware where no Sentry SDK or propagator is present, which leads to context loss ( a broken trace).
There is also a Python backend tier with a Sentry SDK that receives a trace-parent and needs to continue this trace.

Out Of Scope

While we are currently not using the baggage header to the spec, this problem is out-of-scope for this project.

Next steps

• We must find a migration path that lets us first emit sentry-trace and trace-parent in parallel. This can't be turned on by default because of CORS, though
• We also need to ensure that we can pick up an incoming trace-parent header
• Later, we should come to a world where we fully support Trace Context and deprecate sentry-trace

A quick TL;DR: How to Implement

Add support continuing a trace from the traceparent header.
If both sentry-trace and traceparent headers are present, sentry-trace takes precedence.
On outgoing HTTP requests, attach a traceparent header next to the sentry-trace header by default. Users can opt out by setting trace_propagation_tragets to [].
The W3C traceparent header only has two sampling states; hence, deferred or not-to-sample decisions are both represented with -00.
Add a new global getW3CTraceparent() function as well as Span::toW3CTraceparent().

Get inspired by the PHP SDK PR #1 & #2 and Laravel SDK PR.

Time constraints

We will soon be able to ingest OTel traces and this turns out to be rather critical. So we should have a plan by end of Q4 and implement this in Q1.

Stakeholder(s)

• @stephanie-anderson
• @smeubank
• @AbhiPrasad
• @phacops
• @jjbayer

Team(s)

Web-Backend, Mobile, Web-Frontend

### SDK Tasks (General)
- [ ] Develop https://github.com/getsentry/develop/pull/1136

### SDK Tasks (Mobile)
- [ ] https://github.com/getsentry/sentry-cocoa/issues/3598
- [ ] https://github.com/getsentry/sentry-java/issues/3160
- [ ] https://github.com/getsentry/sentry-react-native/issues/3564
- [ ] https://github.com/getsentry/sentry-dart/issues/1831

### SDK Tasks (Web Backend)
- [x] https://github.com/getsentry/sentry-php/pull/1680
- [x] https://github.com/getsentry/sentry-laravel/pull/834
- [ ] https://github.com/getsentry/sentry-go/issues/796
- [ ] https://github.com/getsentry/sentry-java/issues/3160
- [ ] https://github.com/getsentry/sentry-dotnet/issues/3069
- [ ] https://github.com/getsentry/sentry-python/issues/2684
- [ ] https://github.com/getsentry/sentry-ruby/issues/2276
- [ ] https://github.com/getsentry/sentry-elixir/issues/702

### SDK Tasks (Web Frontend)
- [ ] https://github.com/getsentry/sentry-javascript/issues/11171
- [ ] https://github.com/getsentry/sentry-electron/issues/850

[cleptric]

We must find a migration path that lets us first emit sentry-trace and trace-parent in parallel. This can't be turned on by default because of CORS, though

This is only a concern in FE JS, all other platforms can in theory send both headers today.

We also need to ensure that we can pick up an incoming trace-parent header

The question will be what will take precedence for the moment, likely sentry-trace.

[danielkhan]

The question will be what will take precedence for the moment, likely sentry-trace

Yes, sentry-trace it is.