add OpenTelemetry backend for work unit reporting by tdyas · Pull Request #23284 · pantsbuild/pants

tdyas · 2026-04-22T14:46:47Z

Overview

Add a new pants.backend.observability.opentelemetry backend to report work unit tracing to OpenTelemetry. The backend is based on shoalsoft-pants-opentelemetry-plugin with unnecessary compatibility code and "shoalsoft" branding removed.

Notes:

This backend only reports Pants engine work units to OpenTelemetry; it does not report tracing data for Pants rule code or Rust code.
This backend does not support gRPC export due to fork safety issues with the gRPC C library and Python. See gRPC exporter core dumping when invoked within GitHub Actions shoalsoft/shoalsoft-pants-opentelemetry-plugin#84 and https://github.com/grpc/grpc/blob/master/doc/fork_support.md for additional details.

Lockfile

    Lockfile diff: 3rdparty/python/user_reqs.lock [python-default]

    ==                    Upgraded dependencies                     ==

      anyio                          4.12.1       -->   4.13.0
      certifi                        2026.1.4     -->   2026.4.22
      charset-normalizer             3.4.4        -->   3.4.7
      click                          8.3.1        -->   8.3.2
      cross-web                      0.4.1        -->   0.6.0
      cryptography                   46.0.5       -->   46.0.7
      graphql-core                   3.2.7        -->   3.2.8
      idna                           3.11         -->   3.12
      librt                          0.8.1        -->   0.9.0
      pydantic                       2.12.5       -->   2.13.3
      pydantic-core                  2.41.5       -->   2.46.3
      pygments                       2.19.2       -->   2.20.0
      pyjwt                          2.11.0       -->   2.12.1
      python-dotenv                  1.2.1        -->   1.2.2
      python-multipart               0.0.22       -->   0.0.26
      ujson                          5.11.0       -->   5.12.0

    ==                      Added dependencies                      ==

      googleapis-common-protos       1.74.0
      importlib-metadata             8.7.1
      opentelemetry-api              1.41.0
      opentelemetry-exporter-otlp-proto-common 1.41.0
      opentelemetry-exporter-otlp-proto-http 1.41.0
      opentelemetry-proto            1.41.0
      opentelemetry-sdk              1.41.0
      opentelemetry-semantic-conventions 0.62b0
      protobuf                       6.33.6
      zipp                           3.23.1

``` Lockfile diff: 3rdparty/python/user_reqs.lock [python-default] == Upgraded dependencies == anyio 4.12.1 --> 4.13.0 certifi 2026.1.4 --> 2026.4.22 charset-normalizer 3.4.4 --> 3.4.7 click 8.3.1 --> 8.3.2 cross-web 0.4.1 --> 0.6.0 cryptography 46.0.5 --> 46.0.7 graphql-core 3.2.7 --> 3.2.8 idna 3.11 --> 3.12 librt 0.8.1 --> 0.9.0 pydantic 2.12.5 --> 2.13.3 pydantic-core 2.41.5 --> 2.46.3 pygments 2.19.2 --> 2.20.0 pyjwt 2.11.0 --> 2.12.1 python-dotenv 1.2.1 --> 1.2.2 python-multipart 0.0.22 --> 0.0.26 ujson 5.11.0 --> 5.12.0 == Added dependencies == googleapis-common-protos 1.74.0 importlib-metadata 8.7.1 opentelemetry-api 1.41.0 opentelemetry-exporter-otlp-proto-common 1.41.0 opentelemetry-exporter-otlp-proto-http 1.41.0 opentelemetry-proto 1.41.0 opentelemetry-sdk 1.41.0 opentelemetry-semantic-conventions 0.62b0 protobuf 6.33.6 zipp 3.23.1 ```

tdyas · 2026-04-22T14:47:43Z

AI disclosure: AI had been in used parts of the original shoalsoft-pants-opentelemetry-plugin, but no AI was used in the migration of the existing sources into Pants in this PR.

tdyas · 2026-04-22T14:53:46Z

cc @wisechengyi

tdyas · 2026-04-22T15:50:57Z

src/python/pants/init/plugin_resolver_test.py is failing even though this PR does nothing to change the plugin resolution system due to a mismatch on the requests version.. https://github.com/pantsbuild/pants/actions/runs/24785703939/job/72530152547#step:12:461

tdyas · 2026-04-22T15:53:45Z

Lint failure where ./pants fix and ./pants fmt make no changes despite there being an import sorting issue highlighted. https://github.com/pantsbuild/pants/actions/runs/24785703939/job/72530152642?pr=23284#step:7:482

benjyw

I mostly skimmed this, as it's pretty isolated from the rest of the codebase and we know it works.

I wish we didn't have to have Pants depend on the opentelemetry requirements unless you opt in to the plugin, but that's for another day. At least those requirements are robust and well-maintained, so I'm not more concerned about them than I am about other 3rd-party deps, although every external requirement is a potential supply chain attack vector.

@sureshjoshi , I know you have opinions here.

benjyw · 2026-04-22T17:00:21Z

The one dep I am leery of is zipp. I'm sad that otel relies on it.

tdyas · 2026-04-22T17:16:24Z

The one dep I am leery of is zipp. I'm sad that otel relies on it.

Seems to be a test requirement based on these search results: https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-python%20zipp&type=code

Maybe we should just exclude it from the Pants resolve?

sureshjoshi · 2026-04-22T20:13:52Z

@sureshjoshi , I know you have opinions here.

Yes. [...rage builds...]

So, I just glanced over the code - I'm trusting it's just a copy/paste basically of the existing plugin, so whatever, it's already been running for quite some time.

Kinda sucks to have requirements be a dumping ground, whether or not a backend is in use. Would this plugin still work if we just vendored it as a pantsbuild/...plugin..?

Essentially, I'm just wondering what "opt-in" we have. As a practical usage, I run pants on one of my older/slower computers (which does a pretty good job emulating a github action), and the network side is already pretty slow.

Also, just a note. I ran into this, and was literally looking at a GH ticket earlier this week, but pants bootstrapping bypasses lockfiles, so I'd argue we have a higher than normal supply chain risk. That's not directly related to this PR - just to the supply chain comment.

tdyas · 2026-04-22T20:26:15Z

Kinda sucks to have requirements be a dumping ground, whether or not a backend is in use. Would this plugin still work if we just vendored it as a pantsbuild/...plugin..?

I believe built-in backends can have a requirements.txt which is applied when plugin resolution occurs?

tdyas · 2026-04-22T20:28:01Z

So, I just glanced over the code - I'm trusting it's just a copy/paste basically of the existing plugin, so whatever, it's already been running for quite some time.

It is mostly a straight copy/paste. The main changes are removing compatibility code needed to have this plugin work across a range of Pants versions, simplifying the integration test to not use my fork of the integration test helper (which was done to support testing against multiple Pants versions), and renaming the package given its new location.

tdyas · 2026-04-22T23:14:09Z

@sureshjoshi: You can also review the individual commits to see the changes. The original plugin is copied as-is in the first commit, then edited by the remaining commits.

sureshjoshi · 2026-04-23T02:18:10Z

@sureshjoshi: You can also review the individual commits to see the changes. The original plugin is copied as-is in the first commit, then edited by the remaining commits.

Yeah, I might do that, but 🤷🏽 - more interested in the surface area of dependencies, in general anyways.

sureshjoshi · 2026-04-23T15:14:13Z

Is this PR alone intended to remove PANTS_SHOALSOFT_OPENTELEMETRY_ENABLED from CI, or subsequent PR?

tdyas · 2026-04-23T16:00:30Z

Is this PR alone intended to remove PANTS_SHOALSOFT_OPENTELEMETRY_ENABLED from CI, or subsequent PR?

This PR doesn't touch the CI configuration to send work units to Honeycomb. That'll be a subsequent PR since messing with CI configuration should be done in a distinct change.

cburroughs

A few very minor comments that can be followup if you are trying to minimize drift on initial import.

Thank you for contributing this. I think it is going to be a great capability. Is the spooky lint failure the last blocker before merging?

cburroughs · 2026-04-27T17:52:14Z

+            (_MessageType.FINISH, _FinishDetails(timeout=timeout, context=context), context)
+        )
+        if not self._finish_completed_event.wait(
+            timeout=timeout.total_seconds() * 1000.0 if timeout is not None else None


I'm not sure I'm following the conversion correctly. Doesn't https://docs.python.org/3/library/threading.html#threading.Event.wait take seconds already? (Why the *1000)

cburroughs · 2026-04-27T17:53:05Z

+            """
+            The key-value pairs to be used as headers for spans associated with gRPC or HTTP requests.
+
+            This option is consumed by both the `{TracingExporterId.HTTP.value}` and `{TracingExporterId.GRPC.value}`


Is this supposed to be an f string for the {} formatting?

tdyas added 12 commits April 22, 2026 17:08

copy in shoalsoft-pants-opentelemetry-plugin

add7dff

rename packages from shoalsoft + license comments + fmt

d37f5bf

checkpoint

5149aa4

fix SpanProcessor import

b6b469a

add intermediate __init__.py

8718811

remove shoalsoft name from subsystem

611a254

get tests working

28b8116

remove moot comment

0d09b45

delete leftover file

f54e867

remove compatibility shim

f08d26f

release notes

57391e5

tdyas requested review from benjyw, cburroughs and sureshjoshi April 22, 2026 14:46

tdyas added the category:new feature label Apr 22, 2026

tdyas mentioned this pull request Apr 22, 2026

Python 3.14 support shoalsoft/shoalsoft-pants-opentelemetry-plugin#121

Closed

fix & fmt

0025e12

benjyw approved these changes Apr 22, 2026

View reviewed changes

Comment thread docs/notes/2.33.x.md Outdated

Comment thread src/python/pants/backend/observability/opentelemetry/BUILD Outdated

review comments

7fdb442

tdyas added 4 commits April 27, 2026 09:36

Merge branch 'main' into tdyas/observability/opentelemetry

7ad72f7

disable failing plugin resolver test

df106c2

remove future import and newline for block sorting

91fabbc

fmt

5d229fe

cburroughs approved these changes Apr 28, 2026

View reviewed changes

Uh oh!

Conversation

tdyas commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Lockfile

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

benjyw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

benjyw commented Apr 22, 2026

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

sureshjoshi commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

tdyas commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdyas commented Apr 22, 2026

Uh oh!

sureshjoshi commented Apr 23, 2026

Uh oh!

sureshjoshi commented Apr 23, 2026

Uh oh!

tdyas commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cburroughs left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cburroughs Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

cburroughs Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tdyas commented Apr 22, 2026 •

edited

Loading

sureshjoshi commented Apr 22, 2026 •

edited

Loading

tdyas commented Apr 22, 2026 •

edited

Loading

tdyas commented Apr 23, 2026 •

edited

Loading

cburroughs left a comment •

edited

Loading