Skip to content

Added new base path and cleanup logic to fix tasty determinism - #115

Merged
jjudd merged 1 commit into
lucidsoftware:lucid-masterfrom
Garrett-Thomas:gthomas-deterministic-tmp-file
Jul 29, 2026
Merged

jjudd merged 1 commit into
lucidsoftware:lucid-masterfrom
Garrett-Thomas:gthomas-deterministic-tmp-file

Conversation

@Garrett-Thomas

@Garrett-Thomas Garrett-Thomas commented Jul 28, 2026 •

Copy link
Copy Markdown

What:

  • sourceJars were extracted into random tmp directories. The tasty files then contained these random file-paths in their metadata.
  • The final jar that was produced was non deterministic because of this behavior.
  • Source extraction now happens in the parent directory of the output jar path in a folder /tmp. e.g. /bazel-out/x86/target/tmp/<source>.
  • That directory is checked and deleted before building. The main logic in work is wrapped inside a try finally so that the folder is always deleted.

Why:

  • By removing this source of non-determinism, we increase the reproducibility and the cache-ability of our codebase.

Testing:

  • There was already a non-determinism test, but it tested only on a single .scala file. I zipped this file into a sourceJar, making it so that source extraction happens before compilation. The test already performs a byte-for-byte comparison on the compiled output.

Comment on lines +20 to +29
import sbt.internal.inc.{CompileOutput, PlainVirtualFile, PlainVirtualFileConverter, ZincUtil}
import sbt.internal.inc.classfile.analyzeJavaClasses
import sbt.internal.inc.classpath.ClassLoaderCache
import sbt.internal.inc.javac.{DiagnosticsReporter, DirectoryClassFinder}
import sbt.internal.inc.{CompileOutput, PlainVirtualFile, PlainVirtualFileConverter, ZincUtil}
import sbt.internal.util.LoggerWriter
import scala.collection.View
import scala.jdk.CollectionConverters.*
import scala.util.control.NonFatal
import xsbti.compile.{DependencyChanges, ScalaInstance}
import xsbti.{AnalysisCallback, AnalysisCallback3, CompileFailed, Logger, Reporter, VirtualFile, VirtualFileRef}
import xsbti.compile.{DependencyChanges, ScalaInstance}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatter

@jjudd
jjudd self-requested a review July 29, 2026 04:35

@jjudd jjudd left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this and for making the directory deletion more robust. I spent some time poking at this PR and it LGTM.

I'll merge this and include it in the release I intend to cut tomorrow after merging my PR. I'll get this to the monorepo tomorrow with the PR I've got open over there.

@jjudd
jjudd merged commit 9b9afc4 into lucidsoftware:lucid-master Jul 29, 2026
1 check passed
@jadenPete

Copy link
Copy Markdown

Thanks for doing this!

I probably would've left a more detailed comment explaining why the temporary directory needs to be where it is to prevent the bug we were seeing, but otherwise, this looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants