Update Protobuf Bazel Workspace by psamanoelton · Pull Request #7094 · tensorflow/tensorboard

psamanoelton · 2026-04-09T21:46:45Z

Summary

This PR updates TensorBoard’s local Bazel/protobuf build stack to stay compatible with the TensorFlow 2.21 ecosystem changes that this branch needs:

Bazel 6.5.0 -> 7.7.0
protobuf build/runtime alignment to 6.31.1
related Bazel repo-loading, packaging, and test fixes needed for local source builds and CI

The goal of this PR is to keep the smallest working set of changes needed to make the branch build, test, and package correctly with the upgraded toolchain.

What changed

Bazel / workspace updates

update .bazelversion and Bazel version guards to 7.7.0
update workspace dependencies and repository wiring needed for Bazel 7
keep existing repo setup model (WORKSPACE) rather than switching this PR to bzlmod

Protobuf 6.31.1 alignment

update the Bazel-side protobuf dependency to 6.31.1
add the small compatibility shims/patches needed for protobuf’s Bazel build in this repo
regenerate/update checked-in generated artifacts where required

rules_closure / Soy compatibility

keep TensorBoard’s existing Closure/Soy toolchain working with protobuf 6
vendor third_party/safe_html_types as a build-time Java dependency override for the existing Soy toolchain
explicitly prevent rules_closure_dependencies(...) from re-introducing conflicting older transitive copies

Packaging and test fixes

keep tensorboard.compat package exports working correctly in Bazel runfiles and wheel/runtime smoke tests
keep test_pip_package and local Bazel test flows working under the upgraded toolchain
reduce PR-specific drift where possible by reverting summary modules back toward upstream import structure

Vendored code

This PR vendors third_party/safe_html_types, which is the main true vendored code in this branch.

Why it is needed:

TensorBoard still uses the Closure/Soy Java toolchain during Bazel builds
the older transitive safe-html-types classes pulled by that toolchain were not working for this protobuf 6.31.1 upgrade path
this vendored copy is used as a build-time dependency override so the branch can stay on the upgraded Bazel/protobuf stack

Why this is safe:

it is build-time Java tooling support, not TensorBoard Python runtime logic
it is not intended to change TensorBoard user-facing behavior
it is scoped to keeping the existing Soy/Closure path working with protobuf 6

We investigated avoiding vendoring here, but the straightforward http_archive alternatives we tried did not work for this branch’s current dependency combination.

Review-driven cleanup in this PR

In response to review, this PR now also:

adds clearer comments for patches and Bazel helper/shim files
documents the vendored safe_html_types dependency more explicitly
keeps optional cleanup/refactor work out of scope where possible
simplifies some summary-module changes back toward upstream shape

Validation

Validated through the local/container workflow used for this branch, including:

Bazel repo loading / fetch
focused and full Bazel test runs
pip package build
test_pip_package smoke validation
TensorBoard import/runtime verification against TensorFlow 2.21

Follow-up work

The following are better handled in follow-up PRs rather than expanding this one further:

CI/setup-python cleanup and container-specific Python setup simplification
any future move to bzlmod
further dependency-management cleanup in third_party/repo.bzl
splitting this large change into smaller independently reviewable PRs once the full working baseline is settled

…into protobuf

arcra

As discussed offline, I'd like some optional and somewhat independent changes to be done in separate PRs, ideally.

Also, not sure what issues you were finding with the urllib and six packages, but there were a couple related PRs: #7016 and #7013 (we vendor the urllib3 package, IIRC, and according to that second PR, the six library shouldn't be necessary anymore).

arcra · 2026-04-22T23:08:27Z

          sudo apt-get update
-          sudo apt-get install -y libgbm-dev libxss1 libasound2
+          sudo apt-get install -y "${py_abi}-dev" libgbm-dev libxss1 libasound2
+          # Bazel's system_python repository resolves headers relative to the


Is this a problem that started from upgrading to bazel version 7.7? Or from running in our self-hosted container? Or with the new protobuf version?

Perhaps this comment explains it... although it's not super clear to me either. Or maybe an incompatibility of the setup-python action in a self-hosted runner, as described here.

Generally, this seems hacky for something that I expect should be easily supported (setting up python).

Having said this, I think the environment is fairly complex, so if this works, I guess I'm ok with it, but it will just be complex to maintain. If we could make it work in a simpler way, that would be my preference.

Perhaps we can try either:

Using the env var mentioned in the "advance usage" documentation of the setup-python action (linked above)

Using a newer setup-python that might handle this better

Remove the setup-python action, and just installing the specific python version via shell commands directly in the container.

We can leave this as a follow-up, but if possible, I'd like this change to be submitted as a standalone PR.

This is primarily a CI environment issue:

Bazel's system_python repo resolves headers relative to the interpreter exposed by setup-python

In the self-hosted container, that include path does not already contain Python.h

The fix is needed for Bazel repo loading/builds in that environment

This is not caused by protobuf logic directly.

It is hacky and reasonable to split later with:

newer setup-python

shell-installed Python in container

alternate Bazel python toolchain wiring

Sg, we can try to clean up this setup later. Although, if possible, it would still be nice if this could be merged separately from other changes.

I expect that "update the way we setup python for CI" can be independent of everything else, or at least (if necessary) as part of a "bazel 7 upgrade" PR, but without the changes for the protobuf dependencies upgrade.

I know you've mentioned it's been hard to get a stable version to work, but I don't know if you've already attempted to do the bazel upgrade without doing the protobuf upgrade. If that's possible, that could be a cleaner way of merging these changes.

arcra · 2026-04-22T23:10:07Z

  # in our WORKSPACE file.
-  BAZEL_VERSION: '6.5.0'
-  BAZEL_SHA256SUM: 'a40ac69263440761199fcb8da47ad4e3f328cbe79ffbf4ecc14e5ba252857307'
+  BAZEL_VERSION: '7.7.0'


FWIW, my suggestion to upgrade Bazel version to 7.7 was to use bazel modules, which should ideally help with dependency management.

Are we getting any benefit from doing this upgrade? Does it help updating some dependencies or something?

TensorFlow 2.21 uses Bazel 7.7.0

The upgrade unblocked protobuf 6.31.1 and related dependency alignment

This branch already relies on several custom repository overrides, patches, and local repos

Moving to bzlmod would be a larger dependency-management refactor than the minimum needed to align with TF 2.21, but this could be considered in a future split.

The version of Bazel that TF uses should not have any effect on what we use.

I don't really see how this helps at this point, but not opposed to it. We will want to update to use Bazel modules eventually (and I had hoped that could have helped with this protobuf upgrade), but since this is working as is. I'm ok with moving forward with this.

arcra · 2026-04-23T00:41:16Z

+ def protobuf_java_export(**kwargs):
+     java_export(
+         javacopts = JAVA_RELEASE_OPTS,
+-        # https://github.com/bazelbuild/rules_jvm_external/issues/1245


Can we add comments for why we are patching these changes? (all of these files)

Adding comments to this files

arcra · 2026-04-23T18:31:55Z

@@ -2,6 +2,11 @@

 We use [patch-package](https://www.npmjs.com/package/patch-package) to apply


Is this still accurate? (i.e. are we still using patch-package somewhere?) Or should we remove this comment?

It's unclear with the current phrasing. If this is still used, can you rephrase to make it clear that both methods are used for different libraries? (and on what this depends or why we're using two methods?)

patch-package is still the authoring/generation mechanism for npm patches
this branch applies the generated patch artifacts through Bazel yarn_install(post_install_patches=...)
that avoids the less reliable install-time invocation inside the repository rule

When you say "this branch", are you referring to a git branch?

This PR shows to me as attempting to merge into master. Did you intend it to go into the 2.21 branch?

Can we update the comment to say that the patches are generated using patch-package, but they are applied using yarn_install(post_install_patches=...) with a reason for why we do this? What is "less reliable" about the other method?

I refer to the branch used in this PR "psamanoelton:protobuf", the merge can be either master or 2.21 branch.

Taking note to update with a comment

arcra · 2026-04-23T18:58:35Z

+    srcs = ["__init__.py"],
    srcs_version = "PY3",
+    # Keep the package's real __init__.py on this target instead of depending
+    # on :compat so Bazel 7 test runfiles expose the lazy tf/tf2 exports from


Is this bazel-7-specific? We might forget to update this if we update the project to use a new bazel version. Could we just say "bazel", without the specific version?

This is a real fix, not cosmetic:

Bazel test runfiles were exposing a synthesized/incorrect package layout

that broke from tensorboard.compat import tf / tf2

the failure reproduced in both Bazel tests and wheel/runtime smoke tests

This comment specifically was about saying "bazel 7" in the comment.

I guess the point was to communicate that "this started breaking with bazel 7". Consider rephrasing, but not a big deal.

Like I've mentioned before, I'd like it if we could make this change separately either "in preparation" of upgrading to Bazel 7, or perhaps as a "Bazel 7 upgrade" without upgrading the protobuf dependencies and any changes necessary for that (assuming that is possible, not sure if you attempted this already).

I can use part of this PR to create a smaller one that only focus in Bazel 7

arcra · 2026-04-24T04:30:28Z

+    path = "third_party/safe_html_types",
+)
+
+# rules_closure's Soy toolchain still expects safe-html-types classes that are


Can this comment be above the previous lines where the local repository is defined?

arcra · 2026-04-24T04:32:45Z

@@ -53,19 +84,108 @@ py_repositories()

 http_archive(


I thought you'd use the tb_http_archive rule here. Why is it only used in some places?

arcra · 2026-04-24T04:33:33Z

+)
+
+java_import_external(
+    name = "com_google_template_soy",


Can you add comments for why we added all of these dependencies? What needs this template soy package? And why are we adding the extra_build_file_content?

arcra · 2026-04-24T04:34:40Z

+    package_json_remove = ["scripts.postinstall"],
+    patch_args = ["-p1"],
+    # Under Bazel 7.7.0 on this stack, invoking patch-package from the
+    # repository rule is fragile. Apply the existing git-format patches


What does "fragile" mean in this context? It would sometimes not work correctly?

arcra · 2026-04-24T04:55:33Z

+    omit_com_google_common_html_types = True,
    omit_com_google_protobuf = True,
    omit_com_google_protobuf_js = True,
+    omit_com_google_template_soy = True,


Does this mean the rules_closure_dependencies are the ones that require the new dependencies above?

I will answer all the WORKSPACE comments in this one...

safe_html_types is the main true vendored code in this PR

it is a build-time Java dependency for the existing Closure/Soy toolchain, not TensorBoard runtime code

the local copy is used because this branch needs a protobuf-6-compatible adjusted version of those classes, not just any upstream release

the omit_* entries are exactly what stop rules_closure_dependencies from re-introducing older/conflicting transitive versions of the same Java deps

tb_http_archive is used where we needed common mirror/patch/link behavior; plain http_archive remains where that helper was not yet applied or where we stayed closer to upstream declarations

Suggested actions:

Keep vendoring for now unless we can prove an exact upstream archive works unchanged with protobuf 6.31.1.

Improve comments in WORKSPACE:

move the vendor rationale above local_repository

add one comment before com_google_template_soy

add one comment before rules_closure_dependencies(...) explaining the omit_* linkage

Expand third_party/safe_html_types/README.md:

what is vendored

why it is needed

why it is safe

why it is local_repository instead of a plain archive in this branch

I'd like us to try to find a version that works for this, for which we can just use an http_archive, instead of vendoring it. If we can't find one, then it's fine to proceed as is.

psamanoelton added 22 commits April 3, 2026 13:41

Updated bazel version to 7.7.0

a8d769f

update on protobuf 60 tests remaining to fix

865671f

Solving import issue

c38a446

Updating builds for import issues

4660b76

update on build files

3157d7e

adding incompatible_default_to_explicit_init_py

9791333

cleaning up

74e926f

adding readme for safe_html_types

e95ba80

update on ci to support Bazel upgrade and python 310

d31d912

solving ci

999853b

fixing ci

37c2763

updating ci

7e640ab

adding prints for debugging ci

ddb7811

updating ci

99abfbb

updating proto file for update_protos_test

76a439a

upgrading test_pip_package

71907b4

Fix ci stuck in tests

0c6bae3

upgrading dev stack for the legacy aws packages

6a771f6

Solving issue in req_dev.txt

9e460bf

Merge branch 'master' into protobuf

fbd76bc

Updating files to match latest merge

f3502f3

Merge branch 'protobuf' of https://github.com/psamanoelton/tensorboard …

b53795d

…into protobuf

arcra reviewed Apr 24, 2026

View reviewed changes

psamanoelton added 2 commits May 11, 2026 14:31

low-risk comment/doc improvements

5f6275d

Clean up summary imports and restore writer stub dependency

cdd1811

		@@ -2,6 +2,11 @@

		We use [patch-package](https://www.npmjs.com/package/patch-package) to apply

Conversation

psamanoelton commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Bazel / workspace updates

Protobuf 6.31.1 alignment

rules_closure / Soy compatibility

Packaging and test fixes

Vendored code

Review-driven cleanup in this PR

Validation

Follow-up work

Uh oh!

arcra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

psamanoelton commented Apr 9, 2026 •

edited

Loading