Update benchmarks to OrdinaryDiffEq v7 / SciMLBase v3 / RecursiveArrayTools v4 stack#1551
Merged
ChrisRackauckas merged 10 commits intoSciML:masterfrom May 5, 2026
Merged
Conversation
…4 stack
Mechanical compat-section bumps across all benchmark Project.tomls for the
v7 release of the OrdinaryDiffEq solver stack:
OrdinaryDiffEq "6" -> "7"
OrdinaryDiffEqCore "1"/"3" -> "5"
OrdinaryDiffEqDifferentiation -> "3"
OrdinaryDiffEqLowStorageRK -> "3"
OrdinaryDiffEq{Default,Tsit5,Verner,Rosenbrock,SDIRK,BDF,FIRK,
NonlinearSolve,Explicit/ImplicitTableaus,LowOrderRK,
HighOrderRK,Extrapolation,ExponentialRK,StabilizedRK,
StabilizedIRK,Linear,SSPRK,AdamsBashforthMoulton,QPRK,
IMEXMultistep,PDIRK,Feagin,Nordsieck,SymplecticRK,RKN,
PRK,TaylorSeries,FunctionMap,SIMDRK} -> "2"
DiffEqBase "6.x" -> "7"
DiffEqDevTools "2.x" -> "4"
StochasticDiffEq "6.x" -> "7"
DelayDiffEq "5.x" -> "6"
RecursiveArrayTools "3" -> "4"
SciMLBase -> "3"
DifferentialEquations "7" -> "8"
NonlinearSolveFirstOrder "1" -> "2"
Only [compat] entries are touched; [deps] UUIDs and other sections are
preserved exactly. See OrdinaryDiffEq.jl NEWS.md for the full v7 migration
guide. Manifest regeneration and benchmark-script API updates follow in
subsequent commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Mechanical updates across .jmd files for the v7 type-stability changes: autodiff = false -> autodiff = AutoFiniteDiff() autodiff = true -> autodiff = AutoForwardDiff() verbose = false -> verbose = SciMLLogging.None() verbose = true -> verbose = SciMLLogging.Standard() sol.destats -> sol.stats has_destats(...) -> has_stats(...) `AutoForwardDiff` / `AutoFiniteDiff` are re-exported by OrdinaryDiffEq v7, so no extra `using ADTypes` is needed. `SciMLLogging.None()` and `SciMLLogging.Standard()` require `using SciMLLogging`, which is added to the topmost `using ...` block in each affected script (handling multi-line trailing-comma `using` statements correctly), and `SciMLLogging` is added to [deps] / [compat = "2"] of the affected benchmark Project.tomls. 48 .jmd files updated across 16 benchmark folders. Per OrdinaryDiffEq.jl NEWS.md, passing a `Bool` for these kwargs now throws ArgumentError — this commit eliminates that breakage class. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
…s kwarg
Three classes of v7 breakage handled:
(1) RecursiveArrayTools v4 makes AbstractVectorOfArray <: AbstractArray, so
sol[i] now returns the i-th scalar element (column-major) instead of
the i-th timestep
length(sol) now returns prod(size(sol)) instead of nsteps
Migration uses the forward-compatible sol.u[i] / length(sol.t) form,
which works under both v3 and v4. Updated:
benchmarks/AutomaticDifferentiation/SimpleODEAD.jmd (length(sol))
benchmarks/NonStiffBVP/orbital.jmd (sol[1][i] / sol[end][i])
benchmarks/NonStiffODE/ThreeBody_wpd.jmd (sol[1] / sol[end])
(2) Ensemble RNG redesign: prob_func/output_func signatures changed from
(prob, i, repeat) / (sol, i) to taking a single EnsembleContext arg:
function prob_func(prob, ctx) ... ctx.i ctx.repeat
output_func = (sol, ctx) -> ...
sol[end][1] in output_func bodies updated to sol.u[end][1]. Updated:
benchmarks/NonStiffSDE/HighOrderWeakSDEWorkPrecision.jmd
benchmarks/StiffSDE/Oval2Timings.jmd
(3) `precs` kwarg removed from implicit/Rosenbrock/BDF/SDIRK solver
constructors; LinearSolve.jl's KrylovJL_* now owns the preconditioner
interface. Migration: move `precs = X` from the alg constructor into the
KrylovJL_GMRES(; precs = X) call. 40 occurrences across:
benchmarks/Bio/BCR.jmd
benchmarks/Bio/fceri_gamma2.jmd
benchmarks/StiffODE/Bruss.jmd
benchmarks/SimpleHandwrittenPDE/allen_cahn_fdm_wpd.jmd
benchmarks/SimpleHandwrittenPDE/burgers_fdm_wpd.jmd
Surviving `length(sol)` instances are all in commented-out lines and are
left alone. `concrete_jac = true` remains on the solver constructor — it
was not removed in v7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
In v7, \`using OrdinaryDiffEq\` only loads the default solver set (DefaultODEAlgorithm, Tsit5, AutoTsit5, Vern6-9, AutoVern6-9, Rosenbrock23, Rodas5P, FBDF). Every other solver lives in a per-family sublibrary (OrdinaryDiffEqSDIRK, OrdinaryDiffEqBDF, OrdinaryDiffEqRosenbrock, OrdinaryDiffEqFIRK, OrdinaryDiffEqLowOrderRK, OrdinaryDiffEqHighOrderRK, OrdinaryDiffEqExtrapolation, OrdinaryDiffEqExponentialRK, OrdinaryDiffEqLowStorageRK, OrdinaryDiffEqSSPRK, OrdinaryDiffEqIMEXMultistep, OrdinaryDiffEqAdamsBashforthMoulton, OrdinaryDiffEqStabilizedRK, OrdinaryDiffEqSymplecticRK, OrdinaryDiffEqRKN, OrdinaryDiffEqTaylorSeries, OrdinaryDiffEqVerner, OrdinaryDiffEqPDIRK, OrdinaryDiffEqFeagin, OrdinaryDiffEqNordsieck, OrdinaryDiffEqLinear, OrdinaryDiffEqQPRK, ...). Each .jmd is scanned for solver-name occurrences and missing sublibraries are added as a single \`using OrdinaryDiffEqXxx, ...\` line directly after the existing OrdinaryDiffEq import block. Corresponding \`[deps]\` (with registry-resolved UUIDs) and \`[compat]\` entries (mostly = "2"; "3" for LowStorageRK/Differentiation/Core's variants) are added to each affected benchmark Project.toml. Folders touched: AstroChem, AutomaticDifferentiation, Bio, ComplicatedPDE, DAE, DynamicalODE, ModelingToolkit, MultiLanguage, NonStiffBVP, NonStiffDDE, NonStiffODE, SimpleHandwrittenPDE, StiffDDE, StiffODE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
…compat After registry sync (catching up the local General registry to upstream so the v7 stack registrations are visible), Manifest.toml was regenerated for the 24 benchmark folders that resolve cleanly under the new compat. The remaining 14 still need targeted fixes for transitively-pinned packages that don't yet support DiffEqBase v7 / SciMLBase v3 (Catalyst <16, ODEInterfaceDiffEq <4, Sundials <6, AdvancedHMC, ParU_jll, PDESystemLibrary, DifferentialEquations 8 transitively, ODEInterfaceDiffEq vs sublibraries). Compat tweaks in this commit: - SciMLLogging "2" -> "1, 2" (v7 stack still pins SciMLLogging 1.7-1.9) - DiffEqDevTools "4" -> "3" (v4.0.0 was yanked from the registry) - OrdinaryDiffEqCore "5" -> "4" (v5.0.0 was yanked from the registry) Folders with regenerated Manifest.toml: AdaptiveSDE, AutomaticDifferentiation, AutomaticDifferentiationSparse, DynamicalODE, GlobalOptimization, IntervalNonlinearProblem, ModelingToolkit, MultiLanguage, NBodySimulator, NeuralNetworks, NonStiffBVP, NonStiffDDE, NonStiffSDE, Optimization, OptimizationCUTEst, OptimizationFrameworks, PINNErrorsVsTime, PINNOptimizers, SimpleHandwrittenPDE, StiffBVP, StiffDDE, StiffSDE, Surrogates, Testing. Folders still failing (Manifests left as-is until their non-v7 deps upstream support DiffEqBase v7 / SciMLBase v3 — see commit body): AstroChem, BayesianInference, Bio, ComplicatedPDE, DAE, HybridJumps, Jumps, LinearSolve, MethodOfLinesPDE, NonStiffODE, NonlinearProblem, ParameterEstimation, StiffODE, Symbolics. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Targeted compat bumps to unblock several benchmark resolves:
Catalyst 14, 15 -> 16
ODEInterfaceDiffEq 3.x -> 4
Sundials 5 -> 6
ModelingToolkit 9 -> 11
Symbolics 6 -> 7 (AstroChem)
ReactionNetworkImporters 0.16 -> 1 (Jumps)
IRKGaussLegendre 0.2 -> 1 (NonStiffODE)
Manifest.toml regenerated where resolve succeeds:
AdaptiveSDE, AutomaticDifferentiation, AutomaticDifferentiationSparse,
ComplicatedPDE, DynamicalODE, GlobalOptimization, IntervalNonlinearProblem,
ModelingToolkit, MultiLanguage, NBodySimulator, NeuralNetworks,
NonStiffBVP, NonStiffDDE, NonStiffODE, NonStiffSDE, Optimization,
OptimizationCUTEst, OptimizationFrameworks, PINNErrorsVsTime,
PINNOptimizers, SimpleHandwrittenPDE, StiffBVP, StiffDDE, StiffODE,
StiffSDE, Surrogates, Testing.
Folders still failing — blocked on upstream packages that haven't yet
released a DiffEqBase v7 / SciMLBase v3 -compatible version:
AstroChem Catalyst 16 still caps DiffEqBase at 6.218
BayesianInference Turing 0.42 caps OrdinaryDiffEq at 6
Bio Catalyst 16 (same as AstroChem)
DAE DASSL 2 caps DiffEqBase at 6.218
HybridJumps Catalyst 16 (same as AstroChem)
Jumps PiecewiseDeterministicMarkovProcesses 0.0.10 caps Sundials at 4
LinearSolve ParU_jll 1 vs current Julia (needs jll bump)
MethodOfLinesPDE PDESystemLibrary 0.1 caps deps below v7 stack
NonlinearProblem Sundials 5 chain forcing SciMLBase v2 path; SpeedMapping
0.4 + NonlinearProblemLibrary 0.1 also pin SciMLBase v2
ParameterEstimation DiffEqParamEstim 2 caps SciMLBase at 2.155
Symbolics Catalyst 16 (same)
These will unblock as their upstreams cut releases compatible with
DiffEqBase v7 / SciMLBase v3 / OrdinaryDiffEq v7. The benchmark scripts
themselves are already migrated to the v7 API in earlier commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
DASSL v3.0.0 and DASKR v3.1.0 are registered with DiffEqBase = "6.217.0 - 7" compat, so they pick up the v7 stack cleanly. The DAE benchmark Project.toml was pinning the older v2 series of each, which capped DiffEqBase at v6.218 and was the only blocker for DAE/Manifest.toml regeneration. After this bump, 28 of 38 benchmark Manifests now resolve under the v7 stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Two API patterns slipped through earlier passes — neither would have been caught by my grep set: - benchmarks/HybridJumps/Synapse.jmd:1983 u_modified!(integrator, false) -> derivative_discontinuity!(integrator, false) Per SciMLBase v3, u_modified! was renamed to derivative_discontinuity! (the callback system cares about derivative discontinuity, not whether u changed). u_modified! had a deprecation shim in v2 that's removed in v3. - benchmarks/OptimizationFrameworks/optimal_powerflow.jmd:581,803 cost = sol.minimum -> cost = sol.objective Per SciMLBase v3, AbstractOptimizationSolution dropped the .minimizer/.minimum property aliases — use sol.u / sol.objective directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
…ed v7 deps NonlinearProblemLibrary v0.1.5 (just registered) widens SciMLBase compat to "2, 3", which unblocks the SciMLBenchmarks.jl NonlinearProblem benchmark under the v7 stack (DiffEqBase 7 / SciMLBase 3 / RAT 4). MethodOfLinesPDE also now resolves cleanly — picks up the registered PDESystemLibrary v0.1.1. Total resolving manifests: 30 of 38. Remaining failures are upstream-blocked: - AstroChem, Bio, HybridJumps, Symbolics — Catalyst (out of scope here) - Jumps — PiecewiseDeterministicMarkovProcesses (3rd-party, out of scope) - BayesianInference — DiffEqBayes/Turing/AdvancedHMC chain - ParameterEstimation — DiffEqParamEstim master ready, awaiting registration - LinearSolve — ParU_jll / Yggdrasil bump Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
# Conflicts: # benchmarks/ComplicatedPDE/Filament.jmd
|
Error while trying to register: Register Failed |
Member
|
The only real way to do this is to kick off the machine and see what comes back, so I'll kick off the big one and scan the results. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bring the entire
benchmarks/tree onto the OrdinaryDiffEq v7 / SciMLBase v3 / RecursiveArrayTools v4 / DiffEqBase v7 / StochasticDiffEq v7 / DelayDiffEq v6 / DiffEqDevTools v3 stack, per the migration story in OrdinaryDiffEq.jl NEWS.md.30 of 38 benchmark folders' Manifest.toml regenerate cleanly under the new stack. The remaining 8 are blocked on upstream issues out of scope for this PR (notes below).
Changes
Compat bumps across all 38
benchmarks/*/Project.tomlOrdinaryDiffEqXxxsublibrariesScript-level v7 API migrations across 60+
.jmdfilesautodiff = false / true→AutoFiniteDiff() / AutoForwardDiff()(re-exported by OrdinaryDiffEq v7)verbose = false / true→verbose = SciMLLogging.None() / SciMLLogging.Standard(), withusing SciMLLoggingadded to import blocks (multi-lineusingstatements handled correctly)sol.destats→sol.stats;has_destats(...)→has_stats(...)sol.minimum→sol.objective(Optimization v3 path)u_modified!(integrator, false)→derivative_discontinuity!(integrator, false)(callback API rename)sol[i]→sol.u[i],length(sol)→length(sol.t)where the intent was timestep semanticsprob_func(prob, i, repeat)→prob_func(prob, ctx)withctx.i/ctx.repeat; same foroutput_func(sol, i)→output_func(sol, ctx)precs = X(removed) → moved intolinsolve = KrylovJL_GMRES(; precs = X)(40 occurrences across Bio, StiffODE, SimpleHandwrittenPDE)using OrdinaryDiffEqXxxsublibrary imports added where non-default solvers are used (sinceusing OrdinaryDiffEqonly exports the default-set in v7)Manifest.toml regeneration
Resolved cleanly under the v7 stack on Julia 1.11 for 30 of 38 folders.
Folders still blocked (left out of this PR — separable concerns)
AstroChem,Bio,HybridJumps,SymbolicsJumpsPiecewiseDeterministicMarkovProcesses0.0.10 caps Sundials at 4 (3rd-partyrveltz/...) andJumpProblemLibrary v2.0(PR SciML/DiffEqProblemLibrary.jl#184) needs to register + Jumps benchmark scripts need migration to the newbuild_jump_problemAPI — also disregarded per directionBayesianInferenceDiffEqBayesPR #374 (compat-bumped + runic CI fix) merge + Turing 0.45 + AdvancedHMC chainParameterEstimationDiffEqParamEstimmaster is at v2.5.0 with v3 compat already merged (#290, #293, #294) — needs@JuliaRegistrator register()LinearSolveParU_jllv1 has unsatisfiable Julia compat with current 1.11 — Yggdrasil bumpMethodOfLinesPDENonlinearProblemRelated upstream PRs that fed into this work
Test plan
benchmarks/*/Manifest.tomlresolve cleanly withPkg.resolve()on Julia 1.11Pkg.test()(ExplicitImports + weave_file onbenchmarks/Testing) passes.jmdseparately)🤖 Generated with Claude Code