Skip to content

Speed up workspace compilation in Eclipse by avoiding unnecessary rebuild cascades - #7209

Open
chrisrueger with Copilot wants to merge 1 commit into
masterfrom
copilot/optimize-bnd-workspace-builds
Open

Speed up workspace compilation in Eclipse by avoiding unnecessary rebuild cascades#7209
chrisrueger with Copilot wants to merge 1 commit into
masterfrom
copilot/optimize-bnd-workspace-builds

Conversation

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces a plugin-based architecture for intercepting the JAR artifact lifecycle, enabling build optimizations like rebuild trigger policies without modifying core bnd code.

Overview

The core contribution is a new JarLifecycleListener plugin interface that allows tools to observe and participate in the JAR write lifecycle (before and after a JAR is written to disk). This enables:

  • Rebuild optimization (primary use case) – Avoid cascade rebuilds when only non-API changes occur
  • Artifact metadata – Compute and store digests, checksums, or signatures as sidecar files
  • JAR signing – Post-write signing and verification
  • Build artifact indexing – Track or report built artifacts
  • Bytecode enhancement – Transform JAR contents before write

How It Works

1. Core Plugin Interface

A new JarLifecycleListener interface in the core allows plugins to hook into the JAR write lifecycle:

public interface JarLifecycleListener {
    default void beforeJarWrite(Project project, Jar jar, File outputFile, Map<String, Object> context)
        throws Exception {}
    
    default void afterJarWrite(Project project, File outputFile, Jar jar, Map<String, Object> context)
        throws Exception {}
}

Listeners receive a shared context map that persists between before/after phases, enabling stateless, thread-safe data exchange without storing state in the listener itself.

2. Bndtools Plugin Implementation

The rebuild trigger policy optimization (previously hardcoded in Project.java) is now implemented as an optional bndtools plugin that:

  • Before write: Computes content and API digests of the JAR
  • After write: Stores digests in sidecar files (.digest, .api-digest) and restores the old timestamp if content is unchanged

This keeps the optimization completely out of core bnd and makes it a bndtools-specific feature.

3. Workspace-Level Configuration

The rebuild trigger policy is configured at the workspace level via:

workspace.setRebuildTriggerPolicy("api");  // Enable optimization
workspace.getRebuildTriggerPolicy();       // Get current policy

Bndtools Eclipse IDE reads user preferences and applies the policy before building.

Rebuild Trigger Policies

Two policies are supported:

Policy Behavior
always (default) Every rebuild updates the JAR timestamp, triggering rebuilds in all downstream projects
api Preserves JAR timestamp when content is byte-identical OR exported API is unchanged, preventing cascade rebuilds

Files Changed

Core changes (minimal, plugin-focused):

  • Project.java – Added JarLifecycleListener plugin calls in saveBuildWithoutClose() and artifact cleanup hooks in buildLocal()
  • Workspace.java – Added getRebuildTriggerPolicy() and setRebuildTriggerPolicy() methods

Bndtools changes (implements the optimization):

  • New RebuildTriggerPolicyListener.java – Plugin implementation for rebuild optimization
  • BndPreferences.java – Eclipse preferences for rebuild trigger policy
  • BndBuildPreferencePage.java – UI for rebuild policy selection
  • BndtoolsExplorer.java – Toolbar indicator showing current rebuild policy

Tests:

  • Core tests verify the JarLifecycleListener plugin interface works correctly
  • Bndtools tests verify the rebuild optimization implementation

Benefits

  1. Non-intrusive – Core logic unaffected; optimization lives in bndtools plugin
  2. Extensible – Other build tools (Maven, Gradle, Bazel) can implement their own lifecycle listeners
  3. Thread-safe – Fresh context per JAR write; no cross-build interference
  4. Voluntary – If no listener is registered, behavior is unchanged
  5. Backward compatible – Existing code paths unchanged; plugin "sits on the shelf" if unused

Eclipse Integration

Users can enable the optimization via:
Preferences → Bndtools → Build → Rebuild Trigger Policy

The toolbar of Bndtools Explorer displays the current status:

  • "Rebuild: Always" – default behavior
  • "Rebuild: Optimized" – API-based optimization enabled
image

Also show the current status in the toolbar of Bndtools Explorer:

image

Future Extensibility

This design allows other tools to adopt similar optimizations:

  • Maven Plugin – Read policy from pom.xml, register plugin, call workspace.setRebuildTriggerPolicy()
  • Gradle Plugin – Read policy from build.gradle, register plugin
  • CI Builds – Set via environment variable or system property

Each tool controls when and how the plugin is applied; the core mechanism remains unified.

Limitations

  • Timestamps are preserved by directly setting the file's lastModified() value. This works well in most incremental build scenarios but may not perfectly reflect semantic changes in all edge cases.
  • The preserved timestamp approach is pragmatic: attempting to perfectly distinguish between "this JAR actually changed" vs. "this build cycle recomputed the same thing" across multiple asynchronous build tool invocations is complex and fragile.
  • Not recommended for: highly dynamic build environments, complex classpath interdependencies, or scenarios requiring perfect timestamp accuracy for external build tools.

In practice, this conservative approach still provides substantial benefits by preventing most unnecessary rebuilds while maintaining simplicity and reliability.

Copilot AI requested a review from chrisrueger April 18, 2026 08:49
@chrisrueger chrisrueger changed the title Add content-hash optimization to avoid unnecessary rebuild cascades Add content-hash optimization to avoid unnecessary rebuild cascades (Opus 4.6) Apr 18, 2026
@chrisrueger

Copy link
Copy Markdown
Contributor

First experiments in my larger project look very promising. Simple java changes which do not change public API or the manifest do not cause a downstream rebuild cascade. Only e.g. changes to interfaces , method signatures etc. would cause a downstream rebuild of dependant projects like before.

This makes development nicer since it cuts down waiting time a lot.

@chrisrueger chrisrueger changed the title Add content-hash optimization to avoid unnecessary rebuild cascades (Opus 4.6) New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades (Opus 4.6) Apr 18, 2026
@chrisrueger
chrisrueger marked this pull request as ready for review April 18, 2026 19:49
@chrisrueger

Copy link
Copy Markdown
Contributor

@peterkir FYI Let's discuss next week.

@chrisrueger chrisrueger changed the title New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades (Opus 4.6) New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades Apr 18, 2026
@chrisrueger chrisrueger changed the title New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades Speed up workspace compilation e.g. in Eclipse - New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades Apr 29, 2026
@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from bb790c6 to 8ec68dc Compare June 24, 2026 12:30
Comment thread biz.aQute.bndlib/src/aQute/bnd/build/Project.java Outdated
@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from b33fb78 to 3b3afe9 Compare July 18, 2026 15:25
@chrisrueger

Copy link
Copy Markdown
Contributor

@peterkir I now added a status indicator in the Toolbar as we discussed some weeks ago.
image

So let's discuss this what we do with the feature once I'm back. BTW: I am using the feature already in my local bndtools which I used for developing the last part of this PR :)

@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from c450c92 to 1c3f985 Compare July 20, 2026 20:51
@kriegfrj

Copy link
Copy Markdown
Contributor

This is a promising development.

I just wanted to remind us collectively of #3175 as well for consideration. It had a similar aim but which would also provide some additional benefits. In summary: Eclipse's built-in project dependency checking already does this kind of more granular dependency checking, but only with project dependencies - jar dependencies are treated coarsely. More precisely: if you have a project dependency A, and a class file changes in project A that your project B doesn't use directly, then it will not rebuild your project B. This also applies (from memory) if (eg) you change a comment in the source file - the generated class file doesn't change, so your dependent project will not rebuild. With a jar dependency, if the timestamp on the jar is updated, then it will rebuild your project even if none of the classes in the jar have actually changed.

A Bndtools workspace basically forces the coarse dependency behaviour because BndContainer adds every project dependency as a project and as a jar. We do this because Bnd can generate jars in complex ways that pull in other classes that are not in the project and so it is gives greater fidelity; however my initial investigations found that Eclipse should support emulating the same dependency behaviour to the same level of fidelity using dependency rules and allowing a project to propagate transitive dependencies.

Fixing it this way would also fix a number of other problems that are caused by having both project and jar on the classpath (eg, source lookup F3 sometimes takes you to the Jar rather than original source file; duplicate search results, etc).

It's been a long time, but last time I looked at it I nearly had it working but I think there was a bug somewhere in Eclipse that was preventing the dependency propagation from working as advertised, and so I had to abandon it. I was also trying to build a regression test suite in preparation for this change, which again was almost working but I was having niggling flakiness caused by race conditions.

@chrisrueger

Copy link
Copy Markdown
Contributor

Thanks @kriegfrj for your comment. For completeness I want to emphasize that the approach in this PR is purely inside bnd core and not IDE/Eclipse specific. The part which touches bndtools (Eclipse) is mainly for display purposes and to configure the behaviour from bndtools (Eclipse bndtools preferences).

With a jar dependency, if the timestamp on the jar is updated, then it will rebuild your project even if none of the classes in the jar have actually changed.

This "timestamp of the jar" is the main idea of this PR.

In

private String calcApiDigest(Jar jar) {

and

private void digestTree(MessageDigest md, Tree tree) {

I detect the kind of change of the bundle (was it an API change or change affecting consumers or purely internal)

But of course maybe it could be combined with #3175. But I don't know about Eclipse's dependency rules and how they work.

@chrisrueger

Copy link
Copy Markdown
Contributor

I'd like to propose reworking this PR to use a workspace-only configuration instead of the current instruction-based approach. Here's the plan:

Rationale

The current design exposes -rebuildtriggerpolicy as a bnd instruction, which allows it to be set at both workspace (cnf/build.bnd) and project level (project/bnd.bnd). However:

  1. Project-level overrides are not useful — this is fundamentally a workspace-wide build system optimization that should be consistent across all projects
  2. Instructions imply permanence in source control — but this feature is primarily useful for Eclipse IDE and should not pollute CI builds
  3. Better extensibility — making it a Workspace property (not an instruction) allows Maven and Gradle plugins to set it directly in the future without instruction parsing

Proposed Changes

  • Remove the instruction from Constants.java, Syntax.java, and Syntax.md
  • Add a RebuildTriggerPolicy field directly to Workspace.java with getter/setter
  • Keep the Eclipse preference (same as now) — it just calls ws.setRebuildTriggerPolicy(policy) instead of writing to an instruction
  • Update Project.getRebuildTriggerPolicy() to fetch from workspace.getRebuildTriggerPolicy() instead of parsing the instruction

This keeps the feature working in Eclipse immediately and sets up the infrastructure for Maven/Gradle plugins to use the same API in the future.

Impact

  • Cleaner separation of concerns (workspace settings vs. bundle directives)
  • Eliminates project-level confusion
  • Scales to other build tools naturally
  • Documentation becomes simpler: "this is a workspace build optimization, not a bundle configuration"

@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from 6e67b78 to 4767395 Compare July 23, 2026 19:50
@chrisrueger

chrisrueger commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

In 4767395 I implement my earlier comment #7209 (comment)

The rebuildtrigger policy is no longer an instruction but now a setting on the Workspace.java object which can be set by build tools.
Currently bndtools Eclipse plugin is the only caller which sets it. But theoretically other IDE plugins / maven / gradle are possible if needed. Of course this is mostly useful in continuous / incremental build looks like in Eclipse IDE where you like to rebuilt only when really needed and avoid huge downstream rebuilts when just changing javadoc or the code of a method without changing its API.

I updated the PR description accordingly.

@kriegfrj

Copy link
Copy Markdown
Contributor

Thanks @kriegfrj for your comment. For completeness I want to emphasize that the approach in this PR is purely inside bnd core and not IDE/Eclipse specific. The part which touches bndtools (Eclipse) is mainly for display purposes and to configure the behaviour from bndtools (Eclipse bndtools preferences).

With a jar dependency, if the timestamp on the jar is updated, then it will rebuild your project even if none of the classes in the jar have actually changed.

This "timestamp of the jar" is the main idea of this PR.

Actually, from what you've described below, this PR is trying to do something better than merely "timestamp of the jar", because it is calculating a digest of the public API. This is good, because there are plenty of cases where the timestamp of the jar may change but the public API does not (eg, internal code only, or

In

private String calcApiDigest(Jar jar) {

and

private void digestTree(MessageDigest md, Tree tree) {

I detect the kind of change of the bundle (was it an API change or change affecting consumers or purely internal)

But of course maybe it could be combined with #3175. But I don't know about Eclipse's dependency rules and how they work.

Ok, I had a closer look. Forgive me if I have missed anything, but...

It works by "cheating" and rewinding the clock on the Jar update if it detects that the API wasn't changed by the update.

This seems to me a little bit hacky. In effect, you're lying to downstream tools about whether or not the jar has actually changed. Not every downstream dependency of a Jar is another Java build. What happens if you have a final build step that creates a "fat jar" from the dependent jars? Or if you have a "deploy" phase in your build? They may not run as they will conclude that there is no need as there have been no upstream changes.

Another potential problem: If you have a jar dependency on the path that you are pulling classes into your bundle via (eg) -includepackage or -conditionalpackage, these need to be rebuilt even if the public API of the dependency hasn't changed, because they are supposed to pull in the actual implementation rather than link to the API. This approach to dependency checking may circumvent automatic building.

A less important issue: there is also a small window where the timestamp on disk will be the "real" value before it gets rewound to the "old" value, which could lead to race conditions if the build is multi-threaded.

The advantage of this approach vs #3175 is its portability - it can assist in other build drivers. However, it may also cause issues because downstream builds expect the timestamp to update, and not every downstream dependency is a Java build (eg, "deploy"). However, I think that #3175 is conceptually cleaner.

Having said all of that, given the pain of cascading builds, it might be worth all of the complications this change could introduce in some circumstances, especially if you can disable it.

@chrisrueger

chrisrueger commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Thanks @kriegfrj for your input.

It works by "cheating" and rewinding the clock on the Jar update if it detects that the API wasn't changed by the update.

Exactly. Yes I cheat with the timestamp as a vehicle to avoid downstream rebuilts.

This seems to me a little bit hacky.

It is :)

But maybe this can help us: Now we know the exact place where the timestamp comparison happens to do that "has the bundle changed so that I have to rebuild" decision.

public boolean hasOutOfDateTarget(Project model, long lastModified) throws Exception {
//
// $/buildfiles must exists
//
File[] buildFiles = model.getBuildFiles(false);
if (buildFiles == null)
return true;
File file = IO.getFile(model.getTarget(), Constants.BUILDFILES);
if (!file.isFile() || (file.lastModified() < lastModified))
return true;

I wonder if we could find a way to replace or extend the timestamp check with another kind of "check" so that we must not "cheat" with the timestamp. We could reuse all the other things which are now done in this PR (calculating a digest of the public API and content).

I will comment on the other issues:

In effect, you're lying to downstream tools about whether or not the jar has actually changed. Not every downstream dependency of a Jar is another Java build. What happens if you have a final build step that creates a "fat jar" from the dependent jars? Or if you have a "deploy" phase in your build? They may not run as they will conclude that there is no need as there have been no upstream changes.

Another potential problem: If you have a jar dependency on the path that you are pulling classes into your bundle via (eg) -includepackage or -conditionalpackage, these need to be rebuilt even if the public API of the dependency hasn't changed, because they are supposed to pull in the actual implementation rather than link to the API. This approach to dependency checking may circumvent automatic building.

hmm... valid cases. I wonder what would be the required check (in words) to build that bundle anyway? Something like:
"if I am building a "fat jar" or "I contain -includepackage or -conditionalpackage) then rebuild myself always."
This seems to be a decision the downstream bundle / project needs to decide.

My current timestamp "cheat" is just about the upstream bundle "knowing" that its API and content have not changed and the timestamp is currently my "hack" to transport that knowledge to consumers (since this check is already there and used). But we know that the timestamp alone is not enough information and also cheating.

To summarize: there are two parties having to make a decision:

  • a) upstream bundles (telling if their public API and or content has changed)
  • b) downstream consumer bundles (decide if a) is enough information to decide if they skip rebuilding or if they need to rebuild regardless)

A less important issue: there is also a small window where the timestamp on disk will be the "real" value before it gets rewound to the "old" value, which could lead to race conditions if the build is multi-threaded.

true.

The advantage of this approach vs #3175 is its portability - it can assist in other build drivers.

Yes this is something I value, since Eclipse IDE usage is declining and it would be great to find something which could help other tooling in the future too - although I myself and our company are using Eclipse IDE and those long builds are our main pain I try to address here.

However, I think that #3175 is conceptually cleaner.

I need to read into #3175 more.

Having said all of that, given the pain of cascading builds, it might be worth all of the complications this change could introduce in some circumstances, especially if you can disable it.

Yes. That is why I have made that property rebuildTriggerPolicy a String field (currently always (default) and api (the new optimization we are talking here), which allows us to add maybe other policies in the future.
At least we now know that this timestamp-cheat-approach solves like 80%.

UPDATE:

I noticed that BndtoolsBuilder.java already has some checks which seem to be more than just timestamp. It looks a little bit to me that those checks are like your "fat jar" or "include resource" examples:

if (!postponed && (delta.havePropertiesChanged(model) || delta.hasChangedSubbundles())) {
buildLog.basic("project was dirty from changed bnd files postponed = " + postponed);
model.forceRefresh();
setupChanged = true;

@kriegfrj

Copy link
Copy Markdown
Contributor

Thanks @kriegfrj for your input.

👍 Thank you for creating something for me to comment on!

It works by "cheating" and rewinding the clock on the Jar update if it detects that the API wasn't changed by the update.

Exactly. Yes I cheat with the timestamp as a vehicle to avoid downstream rebuilts.

This seems to me a little bit hacky.

It is :)

😆

But maybe this can help us: Now we know the exact place where the timestamp comparison happens to do that "has the bundle changed so that I have to rebuild" decision.

public boolean hasOutOfDateTarget(Project model, long lastModified) throws Exception {
//
// $/buildfiles must exists
//
File[] buildFiles = model.getBuildFiles(false);
if (buildFiles == null)
return true;
File file = IO.getFile(model.getTarget(), Constants.BUILDFILES);
if (!file.isFile() || (file.lastModified() < lastModified))
return true;

I wonder if we could find a way to replace or extend the timestamp check with another kind of "check" so that we must not "cheat" with the timestamp. We could reuse all the other things which are now done in this PR (calculating a digest of the public API and content).

We definitely can; however the BndToolsBuilder is making a decision about whether or not to assemble the jar - not

I will comment on the other issues:

In effect, you're lying to downstream tools about whether or not the jar has actually changed. Not every downstream dependency of a Jar is another Java build. What happens if you have a final build step that creates a "fat jar" from the dependent jars? Or if you have a "deploy" phase in your build? They may not run as they will conclude that there is no need as there have been no upstream changes.
Another potential problem: If you have a jar dependency on the path that you are pulling classes into your bundle via (eg) -includepackage or -conditionalpackage, these need to be rebuilt even if the public API of the dependency hasn't changed, because they are supposed to pull in the actual implementation rather than link to the API. This approach to dependency checking may circumvent automatic building.

hmm... valid cases. I wonder what would be the required check (in words) to build that bundle anyway? Something like: "if I am building a "fat jar" or "I contain -includepackage or -conditionalpackage) then rebuild myself always." This seems to be a decision the downstream bundle / project needs to decide.

My current timestamp "cheat" is just about the upstream bundle "knowing" that its API and content have not changed and the timestamp is currently my "hack" to transport that knowledge to consumers (since this check is already there and used). But we know that the timestamp alone is not enough information and also cheating.

To summarize: there are two parties having to make a decision:

  • a) upstream bundles (telling if their public API and or content has changed)
  • b) downstream consumer bundles (decide if a) is enough information to decide if they skip rebuilding or if they need to rebuild regardless)

This is a good summary. I think for build dependencies to work properly (ideally), it should be the upstream builders' job to advertise its current state accurately, and the downstream builders' job to decide whether or not it needs to rebuild. In the general case, only the downstream builder can make the decision because only it knows what it is trying to build.

A less important issue: there is also a small window where the timestamp on disk will be the "real" value before it gets rewound to the "old" value, which could lead to race conditions if the build is multi-threaded.

true.

The advantage of this approach vs #3175 is its portability - it can assist in other build drivers.

Yes this is something I value, since Eclipse IDE usage is declining and it would be great to find something which could help other tooling in the future too - although I myself and our company are using Eclipse IDE and those long builds are our main pain I try to address here.

The thing is that this optimisation is most helpful for incremental building, not so much for CI.

Another way I thought of that could work and would be portable is for the driver to explode the Jar and then have the depend on the classes rather than on the Jar. I didn't look at this in more detail as it seemed more complicated than the approach I was looking at.

Another thought: I didn't look into it in detail, but from memory Gradle does have sophisticated enough dependency checking mechanism that you could achieve the same fine-grained dependency checking as Eclipse can do. You could store the API hash in the dependency object.

However, I think that #3175 is conceptually cleaner.

I need to read into #3175 more.

Me too...

UPDATE:

I noticed that BndtoolsBuilder.java already has some checks which seem to be more than just timestamp. It looks a little bit to me that those checks are like your "fat jar" or "include resource" examples:

if (!postponed && (delta.havePropertiesChanged(model) || delta.hasChangedSubbundles())) {
buildLog.basic("project was dirty from changed bnd files postponed = " + postponed);
model.forceRefresh();
setupChanged = true;

This is true, but BndToolsBuilder's responsibility only covers part of the build process - it works together with JavaBuilder. Its job is to re-assemble the bundle if the bundle's source files (eg, the .class files, and any jars on the build path that contain classes that have been included by -conditionalpackage, any resources included by -includeresource, projects referenced by -dependson, etc) have changed. I think it already does this pretty well. However, Eclipse's JavaBuilder is still responsible for determining if/when the .class files in the project will be built, and if the dependency is a jar it still relies on the timestamp of the jar. And if the JavaBuilder re-runs, it will update the .class files, which will then trigger the BndToolsBuilder to re-assemble the output bundle, and then you're back to the cascading build, in spite of BndToolsBuilder's best efforts to be smart about its dependency checking.

@chrisrueger

chrisrueger commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

The thing is that this optimisation is most helpful for incremental building, not so much for CI.

Definitly: This was one reason I decided against a instruction and just made it a programmatic option at the Workspace.
I mainly need it for Eclipse and not for CI. Other build tools would only make sense if they are used in a incremental build scenario with a texteditor or IDE.

However, Eclipse's JavaBuilder is still responsible for determining if/when the .class files in the project will be built, and if the dependency is a jar it still relies on the timestamp of the jar. And if the JavaBuilder re-runs, it will update the .class files, which will then trigger the BndToolsBuilder to re-assemble the output bundle, and then you're back to the cascading build, in spite of BndToolsBuilder's best efforts to be smart about its dependency checking.

I just spent a couple of hours learning the above the hard way.
The interdepency of JavaBuilder and BndtoolsBuilder makes it pretty hard. I was at all the places you mentioned like BndContainerInitializer, BndContainer, BndContainerCompilationParticipant.aboutToBuild(IJavaProject)

But in the end JavaBuilder

still relies on the timestamp

I give up for now.
I leave the PR in its current state. I am already using it locally with the given limitations which I added to the PR description and in the code. For me it currently works quite well, but of course you need to know about the limitations about this "cheat".

I will think about it a bit more, and maybe I have a better idea in the future.

@chrisrueger chrisrueger changed the title Speed up workspace compilation e.g. in Eclipse - New -rebuildtriggerpolicy instruction to avoid unnecessary rebuild cascades Speed up workspace compilation in Eclipse by avoiding unnecessary rebuild cascades Jul 24, 2026
@chrisrueger
chrisrueger marked this pull request as draft July 25, 2026 08:14
@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch 3 times, most recently from f27ea1c to 639d7fe Compare July 25, 2026 15:57
@chrisrueger

Copy link
Copy Markdown
Contributor

Update:
I refactored this PR and introduced a new JarLifecycleListener plugin interface instead of hardcoding the rebuild optimization into Project.java. This keeps the optimization out of core bnd and makes it a voluntary, composable feature which currently lives in bndtools Eclipse Plugin:

  • Core change: New JarLifecycleListener plugin interface for JAR write lifecycle hooks (before/after write)
  • Eclipse Bndtools implementation: Rebuild trigger policy moved to an optional bndtools plugin that implements the interface
  • Benefit: minimal bnd core change (just calling the plugin in project.saveJar(...). We can exchange it, improve it, throw it away without touching bnd core.

@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from 639d7fe to 26dee45 Compare July 25, 2026 16:52
@chrisrueger
chrisrueger requested a review from Copilot July 25, 2026 18:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new core JarLifecycleListener plugin extension point in bndlib to hook into the JAR write lifecycle, and implements an Eclipse Bndtools “rebuild trigger policy” on top of it to avoid unnecessary downstream rebuild cascades by preserving output JAR timestamps when content/API is unchanged.

Changes:

  • Add aQute.bnd.service.JarLifecycleListener and invoke it from Project.saveBuildWithoutClose() (before/after JAR write), bumping the aQute.bnd.service package version.
  • Add Bndtools rebuild trigger policy implementation (digest sidecar files + timestamp preservation) plus preference UI and toolbar indicator.
  • Add/adjust docs and introduce tests/testdata for the new lifecycle hook and rebuild policy behavior.

Reviewed changes

Copilot reviewed 23 out of 24 changed files in this pull request and generated 16 comments.

Show a summary per file
File Description
docs/_plugins/jar-lifecycle.md New docs page describing the JarLifecycleListener plugin interface.
docs/_plugins/bndtools-rebuild-policy-plugin.md New docs page describing the Bndtools rebuild trigger policy plugin behavior.
docs/_plugins/00-overview.md Plugin docs overview adjusted; adds registration/error-handling guidance and repositions repository tagging section.
docs/_chapters/150-build.md Minor formatting/indentation fixes in build chapter.
bndtools.core/testdata/ws/p-stale/bnd.bnd New test workspace project for rebuild trigger tests.
bndtools.core/testdata/ws/p-stale/.gitignore Ignores generated output for test workspace project.
bndtools.core/testdata/ws/p-stale-dep/bnd.bnd New dependent test project definition.
bndtools.core/testdata/ws/p-stale-dep/.gitignore Ignores generated output for dependent test project.
bndtools.core/testdata/ws/cnf/build.bnd New cnf build config for the test workspace.
bndtools.core/test/bndtools/tasks/RebuildTriggerTest.java New tests validating digest sidecar behavior and timestamp preservation semantics.
bndtools.core/src/bndtools/preferences/ui/BndPreferencePage.java Adds build preferences page ID constant.
bndtools.core/src/bndtools/preferences/ui/BndBuildPreferencePage.java Adds rebuild trigger policy radio-button UI and persists preference.
bndtools.core/src/bndtools/preferences/BndPreferences.java Stores new rebuild trigger policy preference with defaults and accessors.
bndtools.core/src/bndtools/explorer/BndtoolsExplorer.java Adds toolbar indicator for rebuild trigger policy and opens preferences on click.
bndtools.core/src/bndtools/central/RebuildTriggerPolicyPlugin.java New JarLifecycleListener implementation wiring rebuild policy into the JAR lifecycle.
bndtools.core/src/bndtools/central/RebuildTriggerPolicy.java New digest/timestamp preservation policy implementation (content + API digest).
bndtools.core/src/bndtools/central/Central.java Applies rebuild trigger policy preference to workspaces and re-applies after refresh.
bndtools.core/bnd.bnd Adds biz.aQute.bnd.test to the testpath.
bndtools.builder/src/org/bndtools/builder/CnfWatcher.java Re-applies rebuild policy after workspace refresh.
bndtools.builder/src/org/bndtools/builder/BndtoolsBuilder.java Re-applies rebuild policy after cnf-triggered refresh.
biz.aQute.bndlib/src/aQute/bnd/service/package-info.java Bumps package version due to new public service interface.
biz.aQute.bndlib/src/aQute/bnd/service/JarLifecycleListener.java New core plugin interface for before/after JAR write hooks.
biz.aQute.bndlib/src/aQute/bnd/build/Project.java Calls JarLifecycleListener hooks around JAR write.
biz.aQute.bndlib.tests/test/test/ProjectTest.java Adds tests asserting listener callbacks and context map behavior.
Comments suppressed due to low confidence (6)

bndtools.core/test/bndtools/tasks/RebuildTriggerTest.java:180

  • This block says it "writes a fake API digest sidecar", but no file is written and the variable is unused. The comments should match what the test actually does to avoid future confusion.
			// Write a fake API digest sidecar. The next build will produce
			// the same resource-only bundle (no Export-Package), so
			// calcApiDigest() returns null. However, when we change the
			// project content the content digest will differ. To simulate
			// the API-digest-unchanged path, we need a non-null API digest.
			// We'll change the content, then verify the mechanism by
			// checking the timestamp. Since calcApiDigest returns null for
			// resource-only bundles, the API digest path won't engage here,
			// but we can verify the infrastructure is correctly wired:

bndtools.core/test/bndtools/tasks/RebuildTriggerTest.java:199

  • newDigest is computed but never used. Either assert on it (e.g., changed from previous digest) or remove the unused variable.
			File jarFile2 = secondBuild[0];
			// Content changed so the digest should differ
			String newDigest = IO.collect(digestFile).trim();
			// The JAR should have a new timestamp since both content and

biz.aQute.bndlib/src/aQute/bnd/build/Project.java:2166

  • The debug log message refers to "JarArtifactLifecycleListener.afterJarWrite", but the actual callback is JarLifecycleListener.afterWrite. Aligning the message with the real interface improves diagnosability.
			try {
				listener.afterWrite(project, outputFile, jar, context);
			} catch (Exception e) {
				logger.debug("Exception in JarArtifactLifecycleListener.afterJarWrite", e);
			}

biz.aQute.bndlib/src/aQute/bnd/service/JarLifecycleListener.java:160

  • Several Javadoc link labels/reference names still mention beforeJarWrite/afterJarWrite even though the API uses beforeWrite/afterWrite. This makes the rendered docs inconsistent and harder to follow.
	 * The listener can store computed values in the {@code context} map for use
	 * in the {@link #afterWrite afterJarWrite} phase.
	 * <p>
	 *
	 * @param project the project being built
	 * @param jar the in-memory JAR object, before being written to disk
	 * @param outputFile the file where the JAR will be written
	 * @param context a mutable map for sharing data between before and after
	 *            phases; persists across both phases for a single JAR write
	 * @throws Exception if an error occurs; the exception is logged but does
	 *             not prevent other listeners or the JAR write from proceeding
	 */
	default void beforeWrite(Project project, Jar jar, File outputFile, Map<String, Object> context)
		throws Exception {}

	/**
	 * Invoked after the JAR has been successfully written to the output file.
	 * <p>
	 * This phase is suitable for:
	 * <ul>
	 * <li>Storing metadata (digests, signatures) in sidecar files.</li>
	 * <li>Modifying file properties (timestamps, permissions).</li>
	 * <li>Triggering post-write notifications or side-effects.</li>
	 * <li>Validating the written file.</li>
	 * </ul>
	 * <p>
	 * The listener can retrieve data computed in the {@link #beforeWrite
	 * beforeJarWrite} phase from the {@code context} map.
	 * <p>

bndtools.core/test/bndtools/tasks/RebuildTriggerTest.java:184

  • apiDigestFile is declared but never used (and no API digest is written), which adds confusion without asserting anything. Remove the unused variable or actually write/assert against the sidecar file in a test that can compute an API digest.
			File apiDigestFile = new File(jarFile.getParentFile(), jarFile.getName() + ".api-digest");

biz.aQute.bndlib/src/aQute/bnd/service/JarLifecycleListener.java:173

  • The afterWrite() Javadoc still refers to "beforeJarWrite" in link labels/text, but the API method is named beforeWrite. This makes generated Javadoc inconsistent.
	 * The listener can retrieve data computed in the {@link #beforeWrite
	 * beforeJarWrite} phase from the {@code context} map.
	 * <p>
	 * <b>Note:</b> The JAR object may be in a different state after writing
	 * (e.g., resources may have been closed). Listeners should not rely on the
	 * JAR being fully functional; use the {@code outputFile} and the
	 * {@code context} map instead.
	 * <p>
	 *
	 * @param project the project that was built
	 * @param outputFile the file to which the JAR was written; the file is
	 *            guaranteed to exist on the filesystem
	 * @param jar the JAR object that was written (state may have changed)
	 * @param context the same mutable map passed to {@link #beforeWrite
	 *            beforeJarWrite}, allowing access to data computed in that
	 *            phase

Comment thread docs/_plugins/00-overview.md
Comment thread docs/_plugins/jar-lifecycle.md Outdated
Comment thread bndtools.core/src/bndtools/central/RebuildTriggerPolicy.java
Comment thread bndtools.core/src/bndtools/central/RebuildTriggerPolicy.java
Comment thread bndtools.core/src/bndtools/central/RebuildTriggerPolicy.java
Comment thread biz.aQute.bndlib/src/aQute/bnd/service/JarLifecycleListener.java
Comment thread biz.aQute.bndlib.tests/test/test/ProjectTest.java
Comment thread biz.aQute.bndlib.tests/test/test/ProjectTest.java
Comment thread biz.aQute.bndlib.tests/test/test/ProjectTest.java Outdated
Comment thread bndtools.core/src/bndtools/preferences/ui/BndBuildPreferencePage.java Outdated
@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from 5fa0a3b to bd9ac00 Compare July 25, 2026 19:43
@chrisrueger
chrisrueger marked this pull request as ready for review July 26, 2026 09:23
After building a project's JAR, compute a timeless content digest
and compare with the previously stored digest. If the content is
unchanged, preserve the old JAR's timestamp. This prevents
dependent projects from seeing a newer timestamp and triggering
unnecessary rebuilds.

The optimization uses Jar.getTimelessDigest() which ignores
build-time-specific data (BND_LASTMODIFIED, version qualifier)
to determine if the meaningful content has changed. A .digest
sidecar file stores the hex digest alongside each build output.

Agent-Logs-Url: https://github.com/bndtools/bnd/sessions/662b413c-3754-4104-a50f-cb8503721220

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>

Add Eclipse preference UI for rebuild trigger policy setting

Adds a new 'Rebuild Trigger Policy' dropdown to the Bnd Build preferences
page, letting users choose between 'Always rebuild' and
'API-based (skip if API unchanged)' without editing cnf/build.bnd.

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>

Add rebuild trigger policy indicator to explorer toolbar

Adds a visual indicator in the Bndtools Explorer toolbar displaying the current rebuild trigger policy. The indicator updates when preferences change and opens the build preferences page when clicked.

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>

Make rebuild trigger policy workspace-level, not project-scoped

Refactor rebuild trigger policy from a project-level build.bnd property to a workspace-level programmatic setting. This allows build tools like Eclipse IDE to control the policy for the entire workspace via the new Workspace.setRebuildTriggerPolicy() API rather than users configuring it in build.bnd. Simplify Eclipse preferences by removing the 'Default' option—the workspace now always has a policy value (defaults to 'always'). Move documentation from instruction file to build chapter and update for programmatic-only configuration. Bump package version to 4.8.0.

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>

Refactor rebuild policy into lifecycle plugin

Introduces a new `JarLifecycleListener` service hook in bndlib and wires `Project` to call listeners before/after JAR writes, replacing the in-core rebuild trigger policy logic. The old `Workspace` rebuild policy state and related `Constants` entries are removed, and the service package version is bumped to 4.11.0.

Bndtools now owns rebuild-trigger behavior via `RebuildTriggerPolicy` and `RebuildTriggerPolicyPlugin`, which are registered from `Central.applyRebuildTriggerPolicy()` based on preferences. Preference/UI/explorer code is updated to use the new policy constants, and tests that depended on `Workspace#setRebuildTriggerPolicy` are temporarily commented with TODO markers.

Add rebuild trigger policy plugin tests

Move content-hash/API-digest rebuild behavior tests out of `ProjectTest` into a new `bndtools.core` test suite that exercises `RebuildTriggerPolicyPlugin` with dedicated workspace testdata. Add `ProjectTest` coverage for `JarLifecycleListener` integration, verifying `beforeWrite`/`afterWrite` invocation and shared context data flow. Also wire `biz.aQute.bnd.test` into `bndtools.core` testpath and remove stale commented cleanup lines.

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>

Document JAR lifecycle and rebuild policy plugins

Add new plugin documentation for `JarLifecycleListener` and the Bndtools rebuild trigger policy, including lifecycle phases, context usage, registration, sidecar digests, and policy behavior (`always` vs `api`). Update the plugin overview with generic registration/error-handling guidance and retain repository tagging details in the index page.

Signed-off-by: Christoph Rueger <chrisrueger@gmail.com>
Co-Authored-By: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-Authored-By: chrisrueger <188422+chrisrueger@users.noreply.github.com>
@chrisrueger
chrisrueger force-pushed the copilot/optimize-bnd-workspace-builds branch from bd9ac00 to 3f3a2cf Compare July 26, 2026 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants