Skip to content

Update JUnit4 instrumentation to fire test suite events for Bazel#11322

Open
daniel-mohedano wants to merge 4 commits into
masterfrom
daniel.mohedano/bazel-run-notifier-instrumentation
Open

Update JUnit4 instrumentation to fire test suite events for Bazel#11322
daniel-mohedano wants to merge 4 commits into
masterfrom
daniel.mohedano/bazel-run-notifier-instrumentation

Conversation

@daniel-mohedano
Copy link
Copy Markdown
Contributor

@daniel-mohedano daniel-mohedano commented May 8, 2026

What Does This Do

  • Introduces proper handling for Bazel's JUnit tests runner suite events by:
    • Removing the previous temporary solution of lazy starting suites.
    • Introduces a new instrumentation for fireTestSuiteStarted and fireTestSuiteFinished methods to unwrap the notifiers and propagate the firing to our tracing listener.

Motivation

Bazel's default JUnit4 configuration for running test suites introduces changes that mess with our instrumentation:

  1. Bazel's JUnit entry-point is through BazelTestRunner.main(), which builds a JUnit4Runner (does not inherit org.junit.runner.Runner) and calls run().
  2. JUnit4Runner.run() instantiates JUnitCore, registers its run listeners and calls core.run(cancellableRequest).
  3. JUnitCore creates a fresh RunNotifier (from now on A) and registers the listeners on it.
  4. The request is run with a custom CancellableRunner (extends org.junit.runner.Runner). Its run(A) does currentNotifier = new ThreadSafeRunNotifier(A); delegate.run(currentNotifier);.
  5. ThreadSafeRunNotifier extends RunNotifierWrapper extends RunNotifier, so currentNotifier (from now on B) has its own private inherited listeners field.
  6. The real runner.run(B) executes.
  7. Our instrumentation modifies the call to do B.addListener(tracingListener), but Bs implementation overrides the method to call delegate.addListener(tracingListener); so our listener lands on A.listeners.
  8. The real runner fires events such as test suite starts with B.fireTestSuiteStarted(...).
  9. The wrapper does not override test suite methods, so it is resolved as RunNotifier.fireTestSuiteStarted(...) invoked with this = B. This iterates over B.listeners, which is empty.
  10. The wrapper does forward regular test events with delegate.fireTest...(...) by overriding the methods

All this caused test suite events to be dropped.

Additional Notes

test-environment-trigger: skip

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@daniel-mohedano daniel-mohedano added type: bug Bug report and fix comp: ci visibility Continuous Integration Visibility labels May 8, 2026
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sbt-scalatest

Job Status: success

Scenario Overhead (%)
agent 56.98
agentEvpProxy 56.37

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - nebula-release-plugin

Job Status: success

Scenario Overhead (%)
agent 37.94
agentless 36.40
agentlessCodeCoverage 43.61
agentlessLineCoverage 76.60

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - netflix-zuul

Job Status: success

Scenario Overhead (%)
agent 88.39
agentless 81.20
agentlessCodeCoverage 95.83
agentlessLineCoverage 112.65

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - pass4s

Job Status: failed

Scenario Overhead (%)

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - reactive-streams-jvm

Job Status: success

Scenario Overhead (%)
agent 21.87
agentless 18.54
agentlessCodeCoverage 19.48
agentlessLineCoverage 29.68

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sonar-kotlin

Job Status: success

Scenario Overhead (%)
agent 13.86
agentless 12.42
agentlessCodeCoverage 15.76
agentlessLineCoverage 18.84

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - jolokia

Job Status: success

Scenario Overhead (%)
agent 94.94
agentless 90.16
agentlessCodeCoverage 97.43
agentlessLineCoverage 99.99

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - okhttp

Job Status: success

Scenario Overhead (%)
agent 20.51
agentless 19.78
agentlessCodeCoverage 23.38
agentlessLineCoverage 45.82

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - spring_boot

Job Status: success

Scenario Overhead (%)
agent 16.25
agentless 9.68
agentlessCodeCoverage 12.93
agentlessLineCoverage 33.86

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sonar-java

Job Status: success

Scenario Overhead (%)
agent 13.06
agentless 14.07
agentlessCodeCoverage 76.97
agentlessLineCoverage 176.85

@daniel-mohedano daniel-mohedano changed the title fix: proper handling of bazel test runner suite events Update JUnit4 instrumentation to fire test suite events for Bazel May 11, 2026
@daniel-mohedano daniel-mohedano added type: enhancement Enhancements and improvements and removed type: bug Bug report and fix labels May 11, 2026
@daniel-mohedano daniel-mohedano marked this pull request as ready for review May 12, 2026 06:32
@daniel-mohedano daniel-mohedano requested a review from a team as a code owner May 12, 2026 06:32
@daniel-mohedano
Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 12, 2026

View all feedbacks in Devflow UI.

2026-05-12 07:20:26 UTC ℹ️ Start processing command /merge


2026-05-12 07:20:36 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2026-05-12 07:49:22 UTC ⚠️ MergeQueue: This merge request was unqueued

daniel.mohedano@datadoghq.com unqueued this merge request

@daniel-mohedano
Copy link
Copy Markdown
Contributor Author

/merge -c

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented May 12, 2026

View all feedbacks in Devflow UI.

2026-05-12 07:49:16 UTC ℹ️ Start processing command /merge -c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants