Skip to content

Restart multistep history at d_discontinuities - #4932

Open
singhharsh1708 wants to merge 7 commits into
SciML:masterfrom
singhharsh1708:d-discontinuities-restart
Open

singhharsh1708 wants to merge 7 commits into
SciML:masterfrom
singhharsh1708:d-discontinuities-restart

Conversation

@singhharsh1708

@singhharsh1708 singhharsh1708 commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

When the integrator lands on a d_discontinuities entry, update_fsal! now sets integrator.derivative_discontinuity, so the step after it restarts the multistep history from the post-discontinuity state, as an empty DiscreteCallback at the same time already does. The flag is set inside apply_step!, after loopheader! checks it, and loopfooter_reset! clears it before callbacks run, so only the first attempt of that step's perform_step! sees it; a rejected attempt is retried without the flag, which is why the VCAB and QNDF2 startup paths must handle rejection themselves.

Turning the flag on exposed two restart bugs, which this PR also fixes. VCAB3/4/5 and VCABM3/4/5 advanced their step counter and step-size history on a rejected startup step, so after a restart the retry took an Adams step from that history with a zero error estimate; this already broke VCAB and VCABM on the callback path on master (0.019 for VCAB3 and 5.8e-6 for VCABM3 with an empty callback). A rejected startup step now leaves the history alone, as a rejected full-order step already does. The same fix covers a rejected first step of a solve: on u' = -3u with dt = 0.4 and reltol 1e-8, VCAB3 on master has an error of 1.1e-3 and now 1.2e-9, and VCABM5 goes from 4.4e-7 to 5.0e-8.

QNDF2 accepted its first step after a restart with EEst set to 1 while using the step size from before the restart. That step now gets a backward Euler error estimate, (u - uprev - dt f(uprev)) / 2. With a mass matrix the residual M (u - uprev) - dt f(uprev) is mapped back to the units of u through the Newton matrix, as (M - γdt J) \ r, so scaling M and f by 1e-9 gives the same 9.6e-8 error as M = I. That solve reuses the nonlinear solver's own factorization of W and is taken only when can_smooth_est(nlsolver) is true, the check SDIRK uses for its smoothed estimate: NLNewton, or NonlinearSolveAlg in place with W reuse. Without a reusable W (out-of-place NonlinearSolveAlg, NLFunctional, NLAnderson), QNDF2 with a mass matrix keeps its history at a discontinuity, as master does with d_discontinuities, and now also after a callback, where master restarted and accepted the first step with EEst = 1. On u' = -3u + (t > 0.5) with a 3x3 identity Matrix as the mass matrix and out-of-place NonlinearSolveAlg, this gives 6.0e-8 with either d_discontinuities or an empty callback, where master gives 6.0e-8 and 0.059. The first step of a solve keeps its old behaviour.

On u' = -3u + (t > 0.5) at reltol 1e-9, setting the flag without the solver fixes gives errors of 5.9e-4, 3.1e-5, 1.6e-5 and 0.059 for VCAB3, VCAB4, VCAB5 and QNDF2; with them the errors are 4.8e-11, 3.4e-11, 2.3e-11 and 9.6e-8. Master, which does not restart there, gives 5.6e-11, 3.4e-11, 8.6e-11 and 8.7e-8. The solver fixes leave QNDF, FBDF, QNDF1 and ABDF2 unchanged. Other methods that read the flag now see it at a d_discontinuities entry too, as they already do after a callback. In-place RadauIIA5, RadauIIA9 and AdaptiveRadau lose accuracy there through a W-reuse bug that a callback triggers on master as well; #4941 fixes it and should land first.

One case does not improve. On an index-1 DAE whose algebraic variable jumps at the discontinuity, QNDF2 returns Unstable on master with d_discontinuities and now also with an empty callback, where master with the callback returned Success with a 0.059 error. With the callback the restart step is accepted, but the jump stays in QNDF2's difference history and the next step fails its error test. With d_discontinuities the algebraic variable is not reinitialized, so the restart step's error estimate stays near the jump size and never passes. QNDF and FBDF solve it to 1e-8 or better. Keeping the history after a callback costs one case that master solved: on u1' = -u1 + u2, 0 = u2 - (t > 0.5) with M = [1 0; 0 0] and an empty callback at 0.5, out-of-place NonlinearSolveAlg now returns Unstable at t = 0.5, where master returns Success with a 4.4e-6 error. NLNewton on the same problem goes from 4.4e-6 to 8.0e-7.

The new tests check that d_discontinuities alone gives the same t and u as the empty callback, plus a Success return code, the final time, and the error against the exact solution, for VCAB3/4/5, VCABM3/4/5, FBDF, QNDF and QNDF2, for VCAB3/4/5 and VCABM3/4/5 through the callback path, and for QNDF2 with a mass matrix of 1e-9, in place and out of place, with NLNewton and NonlinearSolveAlg; 14 of them fail without the solver fixes.

Fixes #4913

AI Disclosure: Claude assisted with this change.

@singhharsh1708

Copy link
Copy Markdown
Member Author

🤖 Automated comment from an AI agent running as @singhharsh1708, not written or reviewed by the account holder.

Independent review (Codex CLI 0.163.0, model gpt-6.1-sol, reasoning effort high): CHANGES, risk medium-high (MH1). Reviewed head: 62391721266f18d87366b4137f986b9a135f0e68.

Full review

VERDICT: CHANGES
PR: #4932
Round: 1
Head: 6239172
Base: 45ff46a
Slot: 1
Reviewer: Codex CLI 0.163.0 / gpt-6.1-sol
Session: 01a19773-4dbe-7062-9c4f-3a6ed49605a5
Reviewed: 2026-11-01T14:23:59Z
Summary: The PR makes declared derivative discontinuities restart multistep history through the shared integrator flag. Its new regression assertions fail before and pass after, but the change also causes a severe, independently reproduced Radau accuracy regression.

Risk

medium-high (MH1: Changes shared discontinuity handling used by the FBDF main lane and multiple other solver families; also L2 and L5.)

Risk assessment

  • Risk: medium-high (MH1: A shared integrator flag changes solver history, Jacobian refresh, and error-estimation behavior.)
  • Blast radius: FBDF, QNDF, Adams methods, FIRK, other consumers of derivative_discontinuity, and propagated DelayDiffEq discontinuities.
  • Evidence: Both changed test files pass on head; their 10 new assertions fail on the supplied base. A closed-form accuracy probe confirms substantial Radau regressions. Relevant sublibrary CI is green, but seven other jobs fail.
  • Verdict: CHANGES
  • Round: 1
  • Blocking priorities: P1, P2
  • Merge: needs changes (resolve the Radau regression and add independent numerical regression tests)

Blocking findings

  1. [P1] Setting the shared flag introduces silent Radau accuracy regressions - confirmed by running. At lib/OrdinaryDiffEqCore/src/integrators/integrator_utils.jl:223, the new assignment also activates FIRK discontinuity handling. For the PR’s switched equation, u′ = -3u + isodd(count(<(t), sw)), with sw = 0.5:0.5:2.5, u(0)=0, reltol=1e-8, and abstol=1e-11, maximum errors against the analytic solution are:

    Algorithm Base error Head error
    RadauIIA5 1.3175e-10 1.5659e-6
    RadauIIA9 6.9970e-12 3.3656e-5
    AdaptiveRadau 4.8682e-12 1.0455e-5

    Every run returns Success and reaches t=3. At reltol=1e-10, RadauIIA9’s head error is approximately 15,961 times reltol. The PR description acknowledges the W-reuse problem and identifies Reuse in-place FIRK W only at the dt it was factored for #4941 as a prerequisite, but that repair is absent from this head. Land and verify the prerequisite repair before enabling this flag globally, then add analytic regression coverage for these in-place Radau methods.

  2. [P2] The added numerical tests lack an independent accuracy oracle (G1) - suspected from reading. lib/OrdinaryDiffEqBDF/test/bdf_regression_tests.jl:347 and lib/OrdinaryDiffEqAdamsBashforthMoulton/test/abm_discontinuity_restart_tests.jl:77 compare two runs of the same solver, both using the modified discontinuity handler. Equality verifies callback equivalence, but cannot establish numerical correctness. The existing analytic callback tests do not exercise this new d_discontinuities path. Retain the equivalence checks and add accuracy or convergence assertions against the switched equation’s closed form, including successful completion.

Non-blocking findings

None.

What I ran

All Julia execution used the prescribed wrapper: Julia 1.13.1, four Julia threads, one OpenBLAS thread, isolated temporary directories, and the global lock. Commands below abbreviate that wrapper as jrun, the permitted scratch directory as $PROBES, and the supplied base worktree as $BASE.

  • Git inspection, all exit 0:

    git rev-parse HEAD
    git merge-base HEAD origin/master
    git diff origin/master...HEAD
    git log -1 --format=%B
    git log -1 --no-merges --format=%B
    git diff origin/master...HEAD --check

    HEAD and merge-base match the specified SHAs. The diff contains four files, 34 insertions, and one deletion. The newest commit contains neither agent trailer.

  • Read the linked issue, exit 0:

    gh issue view 4913 --repo SciML/OrdinaryDiffEq.jl
  • Initial head test command, exit 1:

    jrun --project=lib/OrdinaryDiffEqBDF -e 'using TestEnv; TestEnv.activate(); using Test, SafeTestsets; println("Julia ", VERSION); @safetestset "BDF changed file" include("lib/OrdinaryDiffEqBDF/test/bdf_regression_tests.jl"); push!(LOAD_PATH, joinpath(pwd(), "lib/OrdinaryDiffEqAdamsBashforthMoulton")); @safetestset "ABM changed file" include("lib/OrdinaryDiffEqAdamsBashforthMoulton/test/abm_discontinuity_restart_tests.jl")'

    Output:

    BDF changed file | 129  129
    Package OrdinaryDiffEqLowOrderRK ... is required but does not seem to be installed
    ABM changed file | Error 1
    

    This was my environment setup error. I corrected it by developing local ABM and FIRK into TestEnv’s temporary environment.

  • Corrected head command, exit 0:

    jrun --project=lib/OrdinaryDiffEqBDF "$PROBES/targeted_review.jl" head

    The scratch runner executes the complete changed ABM test file and the analytic Radau probe. Output tail:

    RadauIIA9 tol=1e-10 ret=Success end=3.0 max_abs_error=1.5960972e-06 error/tol=15960.972
    Targeted review | 217  217
    
  • Copied both changed test files into the supplied merge-base worktree and verified byte equality:

    cp lib/OrdinaryDiffEqBDF/test/bdf_regression_tests.jl "$BASE/lib/OrdinaryDiffEqBDF/test/bdf_regression_tests.jl"
    cp lib/OrdinaryDiffEqAdamsBashforthMoulton/test/abm_discontinuity_restart_tests.jl "$BASE/lib/OrdinaryDiffEqAdamsBashforthMoulton/test/abm_discontinuity_restart_tests.jl"
  • From that base worktree, exit 1, as expected:

    jrun --project=lib/OrdinaryDiffEqBDF "$PROBES/targeted_review.jl" base

    This executes both copied files and the identical accuracy probe, loading local Core 4.19.0. Output tail:

    Targeted review   | 336  10  346
      BDF changed file | 125   4  129
      ABM changed file | 211   6  217
    

    All 10 failures are the newly added equality assertions. Existing assertions pass. The accuracy probe uses the independent piecewise exponential solution, advancing each interval by
    u(t+h) = c/3 + (u(t)-c/3)exp(-3h).

  • Inspected the supplied check and failed-log snapshots for PR sublibrary CI, PR CI, and PR downstream tests. Core, BDF, ABM, and FIRK jobs pass. Compared failures against master sublibrary CI, master CI, and master downstream tests. Six failing jobs have matching signatures: GPUCompiler warnings, ReverseDiff tape errors, unexpected stiff-adjoint passes, Enzyme’s mi2 assertion, and the Newton line-search exception. The ModelingToolkit equation-orientation failure has no match in the supplied master logs.

No source files were changed. Generated manifests were removed; only the authorized copied tests remain modified on base.

What I did not verify

  • The proposed Reuse in-place FIRK W only at the dt it was factored for #4941 repair combined with this head.
  • GPU, AD, DDE, or other solver families specifically at discontinuities.
  • Package-wide or downstream suites locally.
  • The cause of the unmatched ModelingToolkit CI failure.
  • Base AdaptiveRadau already shows poor accuracy at reltol=1e-10; the three-method regression comparison above uses 1e-8.

AUTHOR_GRADE: {"task":"bugfix","score":2,"difficulty":3,"rounds":1,"author_harness":"unknown","author_model":"unknown","note":"The intended regression tests fail on base and pass on head, but the shared flag introduces severe silent Radau accuracy regressions and lacks independent accuracy assertions. The task requires coordinating restart and cache behavior across several solver families."}


🤖 Posted by an AI agent · harness: Claude Code · model: claude-opus-5-5 (coordinating gpt-6.1-sol reviewers)
Review: unreviewed
Session: 01a19773-4dbe-7062-9c4f-3a6ed49605a5

@singhharsh1708
singhharsh1708 force-pushed the d-discontinuities-restart branch from 6239172 to ef16b2e Compare November 3, 2026 07:39
@singhharsh1708
singhharsh1708 force-pushed the d-discontinuities-restart branch from ef16b2e to 04db522 Compare November 3, 2026 18:58
@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated comment from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris.

PR: #4932

Reviewed head: 04db522
Raw SHA-256: 9aa8020f4a7b959bc66fc5404eb899dcfcac3f357e65b379b118304a03d14de4
Source SHA-256: 046605b969597e9fef6e160f712a461ad332896b2f928277a219ae810257bb28
Source: standard-review-v1 (harness-log)
Reviewer session: 01a1d257-0337-7b32-a630-eaecddc8d1be
Slot: 1
Confidence: unknown
Own risk fixed before author rating/discussion: yes

Risk assessment

  • Risk: medium-high (MH1)
  • Verdict: CHANGES
  • Reviewed head: 04db522

Review report

VERDICT: CHANGES
Round: 1
Head: 04db522
Slot: 1
CI: CI is not green (regress:SciMLSensitivity.jl/Core4/1 / Downstream Tests - Core4)
Session: 01a1d257-0337-7b32-a630-eaecddc8d1be

Risk

medium-high (MH1: shared discontinuity handling changes the FBDF main lane and history/Jacobian reset behavior across solver families).

Blocking findings

  • [P1] The new flag introduces silent accuracy regressions in VCAB3/4/5. lib/OrdinaryDiffEqCore/src/integrators/integrator_utils.jl:223 activates startup resets in methods omitted from the new test loop. Confirmed by running the one-switch problem below: all three return Success, but their errors increase by several orders of magnitude compared with the same head with only this assignment removed in memory. VCAB3's maximum error remains 0.0005881778614161108 at relative tolerances 1e-6, 1e-9, and 1e-12. The five-switch probe also reproduces the regression with scalar out-of-place states. Suspected mechanism from reading adams_bashforth_moulton_perform_step.jl:637: startup counters and divided-difference history advance before the startup step's acceptance is known. Make the newly activated restart paths safe under rejected startup attempts and add analytic accuracy tests for VCAB3/4/5.

  • [P1] QNDF2 accepts an uncontrolled backward-Euler step after each discontinuity. lib/OrdinaryDiffEqCore/src/integrators/integrator_utils.jl:223 now invokes reset_qndf2_history!, which clears the history and resets the success counter. The first-step branch in lib/OrdinaryDiffEqBDF/src/bdf_perform_step.jl:785 sets the error estimate to 1 without measuring accuracy. Confirmed by running: the one-switch problem returns u(1) = 0.19999999999999996, Success, and zero rejected steps at all three tolerances above; the analytic answer is 0.2589566132838567. Reinitialize the step size or provide meaningful startup error control when restarting, and include QNDF2 in the analytic regression coverage. Testing QNDF does not exercise this implementation.

Both findings reproduce with:

using OrdinaryDiffEqAdamsBashforthMoulton, OrdinaryDiffEqBDF
f!(du, u, p, t) = (du[1] = -3 * u[1] + (t > 0.5); nothing)
prob = ODEProblem(f!, [0.0], (0.0, 1.0))
exact(t) = -expm1(-3 * max(t - 0.5, 0.0)) / 3
for alg in (VCAB3(), VCAB4(), VCAB5(), QNDF2())
    sol = solve(prob, alg; d_discontinuities = [0.5],
        reltol = 1e-9, abstol = 1e-11)
    @show typeof(alg) sol.retcode
    @show maximum(abs(u[1] - exact(t)) for (t, u) in zip(sol.t, sol.u))
end

Observed maximum saved-point absolute errors for this problem at reltol=1e-9, abstol=1e-11:

Algorithm Reviewed head Only the new assignment removed in memory
VCAB3 5.881778614161108e-4 5.582428963535335e-11
VCAB4 3.096686726480544e-5 3.429267980692430e-11
VCAB5 1.575324890466627e-5 8.561381752191302e-11
QNDF2 5.895661328385676e-2 8.723186423198470e-8

The control redefines only update_fsal! with the added assignment removed; it is not a complete default-branch test run. No checkout source was edited.

  • Verdict: CHANGES
  • Round: 1
  • Blocking priorities: P1, P1
  • Deferred P2 issues: None supplied; no findings deferred.

Non-blocking findings

  • lib/OrdinaryDiffEqAdamsBashforthMoulton/test/abm_discontinuity_restart_tests.jl:98 and lib/OrdinaryDiffEqBDF/test/bdf_regression_tests.jl:349: the equality assertions do not establish successful integration or independent numerical accuracy. Confirmed by adding only maxiters=1 to the review's in-memory copies of the new tests: all ten assertions pass although every solve returns MaxIters and ends around 1e-4, before the first discontinuity. Assert successful return codes and the final time, and compare with the analytic solution. Keep the useful callback-equivalence checks as additional coverage.

What I ran

  • git diff origin/master...HEAD, source/test inspection, newest-commit trailers, and final git status --porcelain / git rev-parse HEAD. The checkout is clean and matches the required head. The newest commit supplies neither attribution trailer, so both author attribution fields below are unknown. No PR discussion, author rating, or earlier review was read.
  • Exported the requested TMPDIR for commands and preserved the supplied JULIA_DEPOT_PATH. Used git archive HEAD to create a scratch source mirror so package resolution could not write into the checkout. julia --version: julia version 1.13.2. Resolved local Core 4.20.2, DiffEqBase 7.21.6, and SciMLBase 3.58.0.
  • For each affected solver package, GROUP=Core julia --project="$review_scratch/source/lib/<package>" -e 'using Pkg; Pkg.test(; allow_reresolve=true)': Adams passed 381 assertions; BDF passed 1,873 assertions with two existing broken tests in DAE initialization. Both commands exited 0.
  • analytic_probes.jl ABM and analytic_probes.jl BDF, then their base controls: forward/backward integration, in-place/scalar states, saved-point/dense errors, and two tolerances, against a 256-bit analytic switching solution. Head results were 198 pass / 26 fail for Adams and 144 pass / 16 fail for BDF. Some coarse reverse-direction failures also occurred in the control; those are not presented as PR regressions. The blocking findings isolate newly introduced forward errors.
  • minimal_probe.jl and minimal_bdf_probe.jl, each on head and base, reduced both findings to the one-switch example above and checked reltol=1e-6, 1e-9, 1e-12, with abstol=reltol/100. All scripts completed and printed the numerical evidence.
  • compare_tests.jl <ABM|BDF> <head|base>: the added tests pass all ten assertions on head and fail all ten with the assignment removed. The early-termination mutations pass 6/6 Adams and 4/4 BDF assertions despite MaxIters. An initial Adams mutation probe errored on an unsupported verbose=false keyword; removing that review-only keyword and rerunning produced the stated result.
  • adjacent_probe.jl: eight solves with/without a tstop at nextfloat(0.5) all returned Success; maximum error was about 7.98e-10. This probe did not establish another blocker.
  • uv run --locked python --version: Python 3.15.0, with the warning that --locked has no effect outside a project.

What I did not verify

  • The supplied SciMLSensitivity downstream CI failure was not rerun or attributed to this change. No GitHub discussion or CI logs were consulted.
  • Other Julia versions, QA, GPU, threaded groups, documentation, and the full monorepo suite were not run.
  • The checkout contains no committed Julia Manifest, interpreter version pin, Python project, or uv.lock. Consequently no repository-locked interpreter/dependency environment was available; the Julia checks used the declared package projects and scratch-generated manifests. The Python version command did not establish a locked Python environment.

AUTHOR_GRADE: {"task":"bugfix","score":2,"difficulty":3,"rounds":1,"author_harness":"unknown","author_model":"unknown","note":"The intended equivalence tests fail before and pass after, and both native Core suites pass, but independent analytic probes confirm silent accuracy regressions in VCAB3/4/5 and QNDF2."}

Head link: 04db522


🤖 Posted by an AI agent — harness: codex-cli 0.164.0 · model: gpt-6.1-sol
Conversation: local session 01a1d257-0337-7b32-a630-eaecddc8d1be (no session URL)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

d_discontinuities does not restart multistep methods: FBDF and VCABM4 keep their pre-discontinuity history

2 participants