Add lightsim2grid powerflow solver

[BDonnot]

With some discussion from example with @albanpuech I tried to "connect" lightsim2grid with the datakit.

To a relatively good surprise, this was done rather simply (I took the powsybl integration as an example).

I tested the data generation process on the cases 14, 118 and 2000 with no actions and some random actions.

I also made a branch in lightsim2grid: https://github.com/Grid2op/lightsim2grid/tree/dev_gfm_datakit to further speed up the computation. Wie the released lightsim2grid version on pypi you have to build the grid from scracth each time when the physical parameters (r,x, g, or b) are modified.

The development here does not depend on it, but if lightsim2grid is installed from source from this branch (instruction in the docs) this will make lightsim2grid even faster.

This developements have been assisted by Claude (Sonnet 5), all generated code has been reviewed and validated by myself.

I also noticed that there were discrepencies between DC pf between lightsim2grid and powermodels. For DC modelling, lightsim2grid strictly follow the matpower equations (heavily tested). I am not quite sure what is the convention used by powermodels.

Let me know if something is missing

Best

Benjamin

[romeokienzler]

@BDonnot Thanks for this — connecting lightsim2grid as a third PF engine, and doing it by mirroring the powsybl layout, makes for a clean and easy-to-follow addition. Against our CONTRIBUTING checklist this is in very good shape already: new dep in pyproject.toml, pf_solver propagated to scripts/config/default.yaml and all four tests/config/*.yaml, docs (new manual page + docs/components/ markers + mkdocs.yml + installation/getting_started), Google-style docstrings and type hints throughout, and unit tests covering the interesting edges (divergence, in-place-vs-rebuild equivalence, sub-tolerance updates). Nice.

What's needed

• Add Alban Puech as a reviewer (per CONTRIBUTING) — no reviewer is currently requested.
• The lightsim2grid>=1.1.0 lower bound in pyproject.toml is a bit misleading: as your own TODO and docs note, update_powerlines_parameters / update_trafos_parameters aren't in any release yet, so >=1.1.0 can install a version the fast path can't use. The api.py guard is on init_from_matpower, not those methods — worth a one-line confirmation that the in-place path degrades gracefully (rebuild) rather than erroring on such a version, since the update_lightsim2grid except Exception should catch a missing-method AttributeError and fall back. Just flagging so a reviewer knows it's intentional.
• CI: the only check reporting is DCO (green); mergeStateStatus is UNSTABLE but I don't see a failing test workflow attributable to your diff — looks fine, not a blocker. Please make sure pytest tests/ -n <cores> -v passes locally (the new tests skip cleanly when lightsim2grid isn't installed, which is good).

Nothing above is a merge blocker in my view — I'll leave the merge/approval call to a maintainer.

— 🤖 _automated pre-review; a maintainer will follow up_

[albanpuech]

@romeokienzler Could you please guide @BDonnot on how to make sure the pytest tests for the new lightsim2grid folder are being run by the CLI? Maybe we should do something similar to what we did for PowSyBl and Dynawo, with a separate CLI?

tests/lightsim2grid/test_lightsim2grid_solver.py::test_mapping_and_ac_pf_are_consistent SKIPPED
tests/lightsim2grid/test_lightsim2grid_solver.py::test_diverging_pf_raises SKIPPED
tests/lightsim2grid/test_lightsim2grid_solver.py::test_generate_pf_mode SKIPPED
tests/lightsim2grid/test_lightsim2grid_solver.py::test_in_place_update_matches_a_rebuild SKIPPED
tests/lightsim2grid/test_lightsim2grid_solver.py::test_structural_change_triggers_a_rebuild SKIPPED
tests/lightsim2grid/test_lightsim2grid_solver.py::test_in_place_update_applies_changes_bel

Thank you both :)

[romeokienzler]

@albanpuech @BDonnot Happy to. The tests SKIP because CI never installs the solver: the main pytests job only installs .[test], so is_lightsim2grid_available() is False and every tests/lightsim2grid test is skipped. This is exactly the PowSyBl/Dynawo situation — those don't run in pytests either; they run in the separate dynamic-pytests job (.github/workflows/ci-build.yaml), which installs the heavy extras and points pytest at their folders:

pip install -e ".[test,dynamic,powsybl]"
...
pytest tests/dynamic tests/powsybl -v

The lightsim2grid extra already exists in pyproject.toml (lightsim2grid = ["lightsim2grid>=1.1.0"]), so the smallest change is to fold it into that same job:

• add lightsim2grid to the install line → pip install -e ".[test,dynamic,powsybl,lightsim2grid]", and
• add the folder to the run line → pytest tests/dynamic tests/powsybl tests/lightsim2grid -v.

(A dedicated lightsim2grid-pytests job mirroring dynamic-pytests is also fine if you'd rather keep it isolated — it just costs an extra runner. Since lightsim2grid installs from a wheel with no Julia/Dynawo prerequisites, reusing dynamic-pytests is the cheaper option.)

One thing to confirm once it runs: pip will pull a released lightsim2grid (which lacks update_powerlines_parameters / update_trafos_parameters), so CI will exercise the rebuild fallback path, not the in-place one. test_in_place_update_matches_a_rebuild and friends should still pass on that path — worth eyeballing the first green run to be sure they don't silently assert the in-place branch was taken.

@BDonnot — this is a CI-only change (ci-build.yaml); no source changes needed. Note the branch is now BEHIND main, so a quick rebase before the next push will keep the run clean.

— 🤖 _automated pre-review; a maintainer will follow up_

[BDonnot]

Hello @romeokienzler

The lightsim2grid>=1.1.0 lower bound in pyproject.toml is a bit misleading: as your own TODO and docs note, update_powerlines_parameters / update_trafos_parameters aren't in any release yet, so >=1.1.0 can install a version the fast path can't use. The api.py guard is on init_from_matpower, not those methods — worth a one-line confirmation that the in-place path degrades gracefully (rebuild) rather than erroring on such a version, since the update_lightsim2grid except Exception should catch a missing-method AttributeError and fall back. Just flagging so a reviewer knows it's intentional.

The lightsim2grid >= 1.1.0 (and even 1.0.0 to be honest) is enough to make this powerflow working. What is missing there (I just merged the feature in the development branch of lightsim2grid). This is why I put a try / except, maybe a bit broad (Exception and not AttributeError) I will fix it and replace it with :

  try:
      _update_in_place(ls_net, net, mapping, old)
  except AttributeError:
      # half-updated: nothing about the LSGrid can be trusted anymore
      return to_lightsim2grid(net)

his is a CI-only change (ci-build.yaml);

I did not dare changing the CI but that is definitely something I can do. I'll match what is done for pypowsybl and dynawo.
However, if I may, long term it would be better to have separate jobs for each solver. This is not the case for lightsim2grid but there might be some version conflicts in installing some packages "all at once" (this happened to me a lot with pandapower, pandas, numpy etc.).
I will not modify the way it's one though, just do as Alban pointed:
pip install -e ".[test,dynamic,powsybl,lightsim2grid]"

Thanks

Benjamin

[romeokienzler]

@BDonnot Both plans sound right to me.

• except AttributeError: agreed, and your snippet matches the current fallback exactly (update_lightsim2grid already does return to_lightsim2grid(net) on the except path, leaving old untouched so the rebuild is clean). Narrowing to AttributeError is the correct tightening — it targets precisely the missing-method case on released versions while letting a genuine bug in _update_in_place surface instead of being silently swallowed into a rebuild.
• CI: folding into dynamic-pytests (pip install -e ".[test,dynamic,powsybl,lightsim2grid]" + adding tests/lightsim2grid to the run line) is the smallest change and matches powsybl/dynawo. Your point about per-solver dependency conflicts (pandapower/pandas/numpy) is fair — for now lightsim2grid is a self-contained wheel so it co-installs cleanly, but a dedicated lightsim2grid-pytests job is equally fine if you prefer the isolation; either works for this PR. Splitting all solvers into separate jobs is a reasonable follow-up but out of scope here.

One thing to eyeball on the first green run: since CI will pull a released lightsim2grid without the in-place methods, it exercises the rebuild path — so confirm test_in_place_update_matches_a_rebuild and friends pass without asserting the in-place branch was actually taken.

Also please rebase — the branch is BEHIND main. Merge/approval is a maintainer's call.

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

@BDonnot Both changes landed exactly as discussed — nice.

• except AttributeError (convert.py): confirmed. The narrowed catch falls back to a clean rebuild (return to_lightsim2grid(net), leaving old untouched) only on the missing-method case, so a genuine bug in _update_in_place now surfaces instead of being swallowed.
• CI (ci-build.yaml): confirmed — lightsim2grid is folded into dynamic-pytests (pip install -e ".[test,dynamic,powsybl,lightsim2grid]" + pytest tests/dynamic tests/powsybl tests/lightsim2grid -v), matching powsybl/dynawo, so tests/lightsim2grid no longer skips in CI.

That covers everything I'd raised. One thing to eyeball on the first green dynamic-pytests run: since pip pulls a released lightsim2grid (no update_powerlines_parameters/update_trafos_parameters), CI exercises the rebuild path — so just confirm test_in_place_update_matches_a_rebuild and friends pass there without asserting the in-place branch was taken.

DCO is green and the branch is mergeable; the BLOCKED state is just the pending review. Looks ready for a maintainer's look — approval/merge is their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[BDonnot]

What I did:

• Rebase and sync with master
• Add the attribute Error and small comment (more explicit)
• Add the lightsim2grid in CI along with pypowsybl and dynaflow
• ran pytest tests/ -n 2 -v as Alban asked and it's green: "348 passed, 239 skipped overall (391s)"

(plus an issue with DCO...)

[romeokienzler]

@BDonnot Thanks — your checklist matches what I confirmed on the last pass (rebase, AttributeError narrowing, CI fold-in), and the local 348 passed, 239 skipped run is reassuring.

On the DCO hiccup: it's green now — all five commits carry Signed-off-by: DONNOT Benjamin, and the DCO check passed at 14:43. So that's resolved; nothing further needed there.

The only thing still worth a glance is the one I flagged before: on the first CI dynamic-pytests run, pip pulls a released lightsim2grid (no update_powerlines_parameters/update_trafos_parameters), so tests/lightsim2grid exercises the rebuild path — just confirm test_in_place_update_matches_a_rebuild and friends pass there without asserting the in-place branch was taken.

Otherwise this looks ready for a maintainer's look — approval/merge is their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[BDonnot]

Oh sorry I missed your comment on that.

Indeed, it is exercising only the "rebuild" pass as of now, just like someone who would use the datakit right now. So this test is not exercising what it claims to do (yet) this is a matter of days, 2-3 weeks at most.

[romeokienzler]

@BDonnot No worries — and your read is exactly right: right now tests/lightsim2grid covers the rebuild path, which is precisely what any current datakit user hits until a lightsim2grid release ships update_powerlines_parameters/update_trafos_parameters. So the tests aren't wrong, they're just testing the only path that exists on PyPI today — that's the honest state and it's fine for this PR.

The one thing worth doing so the in-place coverage doesn't quietly fall through the cracks in a few weeks: leave a small marker for future-you — e.g. a # TODO(lightsim2grid>=X): CI still exercises the rebuild path; bump the pin and confirm the in-place branch is taken near the pin in pyproject.toml, or a short note on the test. Then when the release lands, bumping the lower bound flips CI onto the in-place path automatically. Not a blocker.

Everything I raised earlier is resolved — this looks ready for a maintainer's look; approval/merge is their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[albanpuech]

@BDonnot

(It is Alban talking to you, not claude code ;) )

The integration looks great so far. I did this small test. I think you will like it

Ran case500_goc on both PF engines: 1k load scenarios, 2 topology variants, k: 5, same seed. Only pf_solver changed. lightsim2grid used the fast path (in-place updates, no model rebuild). PowerModels used its fast PF solver (pf_fast: true). Both runs were validated.

What matched

	lightsim2grid	PowerModels
convergence	1998 / 2000	1998 / 2000
Failed cases	636, 769	636, 769
AC voltages	same (max diff ~1e-8)	same

The last row of the table is really good news!
Also gridfm_datakit validate (checks all the power system assumptions we make in graphkit so the data works for all models built with graphkit) passed for both

Speed

	lightsim2grid	PowerModels
AC power flow	2.5 ms	9.6 ms (3.8× slower)
DC power flow	0.5 ms	3.3 ms (6.5× slower)
Full generate	141 s	126 s

lightsim2grid is faster at power flow. The full run is not, because both paths still do OPF with PowerModels, and that step dominates (difference is noise)

DC PF residuals are not the same because the solvers use different DC models: lightsim2grid follows MATPOWER (1/(x·tap)), PowerModels uses x/(r²+x²).

[BDonnot]

I made claude happy (trying to at least) with the fix test, I will push ASAP :-)

[albanpuech]

cool! Lmk what you think of the convergence/ speedup/ validation results above :)

[romeokienzler]

@BDonnot That's exactly the fix — thanks for chasing it down.

• test_in_place_update_matches_a_rebuild: confirmed. Gating assert n_rebuilt == 1 behind the hasattr(..., "update_powerlines_parameters") check means CI (which pulls a released lightsim2grid) exercises and asserts only the rebuild path it actually takes, instead of silently asserting a branch that never ran. Good catch that it was asserting the wrong thing before.
• Cross-referenced TODOs (pyproject.toml ↔ the test): nice — bumping the lower bound to the release with update_powerlines_parameters/update_trafos_parameters will flip supports_in_place to True and re-arm the n_rebuilt == 1 assertion automatically, so the in-place coverage won't quietly slip.

CI just re-triggered on this push and is still pending; worth a glance once dynamic-pytests goes green, but nothing outstanding on my side. Everything I'd raised is now resolved — this looks ready for a maintainer's look; approval/merge is their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[BDonnot]

The speed-ups are (much) lower than what I observed outside of the datakit (but close to what I observed on my machine with the datakit). But I think it's because the datakit does not optimize the same way.

One grid at a time is much slower than computing a batch of grids. But still 4x faster is pretty nice already :-)

[BDonnot]

I'll fix the failing tests tomorrow.

Probably something I missed with the lightsim2grid version on pypi... Sorry about that

[romeokienzler]

@BDonnot No need to apologize — and this is actually the system working as intended: the red check is dynamic-pytests, which is new red only because CI now genuinely runs tests/lightsim2grid instead of skipping it. So the earlier "everything skips in CI" problem is fixed; we're now seeing real signal from the released-PyPI path.

One pointer to save you time tomorrow: your last commit (7f29aa0) already gates assert n_rebuilt == 1 behind hasattr(..., "update_powerlines_parameters"), so on a released lightsim2grid that assertion should no-op rather than fire. That means the failure is most likely elsewhere on the rebuild path — worth checking whether it's the extra installing cleanly on the runner, or a rebuild-path value/API-signature difference versus your local wheel, rather than the in-place assertion. (If it turns out the runner is pulling a different lightsim2grid version than your local box, that would explain a green-local / red-CI split.)

No urgency — tomorrow is totally fine. Everything else stands where it was, and merge/approval remains a maintainer's call.

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

@BDonnot Thanks — the new commit (7568d1, "Only fall back to a rebuild when branch parameters change") is exactly the right fix for the red dynamic-pytests.

• Rebuild-path regression resolved: moving the getattr(ls_net, "update_powerlines_parameters") lookup inside the if rows.size and branch-params-changed guard means a load/generator-only update no longer touches the missing methods on a released lightsim2grid, so it stays in place instead of raising AttributeError and forcing a rebuild. That's precisely what was breaking test_in_place_update_applies_changes_below_lightsim2grid_tolerance. Good catch.

One new thing this push introduced, though:

• DCO is now red (ACTION_REQUIRED) — and it's the only reporting check. The cause is this last commit: unlike your first six (all carrying Signed-off-by: DONNOT Benjamin), 7568d1 is authored as Claude <noreply@anthropic.com> with a Co-Authored-By trailer and no Signed-off-by line. A quick git commit --amend -s (or git rebase --signoff) to add your sign-off, then force-push, should turn DCO green again.

Once DCO is back to green, this looks ready for a maintainer's look — the CI fix itself is sound. Approval/merge remains a maintainer's call.

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

@BDonnot Thanks — DCO is green again. The latest commit (45f8775, "Only fall back to a rebuild when branch parameters change") now carries Signed-off-by: DONNOT Benjamin alongside the Assisted-by: Claude Code trailer, so the missing sign-off from the previous push is resolved and the DCO check passes.

That was the only thing I had outstanding. Everything raised earlier (rebase, AttributeError narrowing, CI fold-in, the rebuild-path fix and its test gating) is addressed, and the one remaining item — confirming the dynamic-pytests run stays green on the released-PyPI (rebuild) path — is a maintainer's eyeball on the next CI run rather than anything code-side.

Looks ready for a maintainer's look; approval/merge remains their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[albanpuech]

@BDonnot Just to let you know I will need some time to go thru all this... will do my best to review before our meeting with Nicolas and François

[BDonnot]

No problem, honestly it's almost a copy paste from the pypowsybl one : pypowsybl being more complex I mainly "simplified" it and change the calls to use lightsim2grid.

No hurry on my side however

Best

Benjamin

[romeokienzler]

@BDonnot Just one new thing since the last pass — no urgency, and nothing in the code review changed.

• The branch now has a merge conflict with main (mergeStateStatus flipped to DIRTY / CONFLICTING after the 8aa5243 merge-from-main, as main has since advanced). A quick rebase onto latest main (or re-merging and resolving) will clear it so CI can run clean again. Per CONTRIBUTING a rebase onto latest main before merge is the expected state anyway.
• DCO is still green and is the only reporting check; once the conflict is resolved, dynamic-pytests should re-trigger — the earlier rebuild-path items are all resolved on my side.

Everything substantive from before stands as addressed; this is purely the branch going stale against main. Approval/merge remains a maintainer's call.

— 🤖 _automated pre-review; a maintainer will follow up_

[romeokienzler]

@BDonnot Thanks — the merge-from-main (30487092) clears the one thing I'd flagged last time: the branch is now MERGEABLE again (was DIRTY/conflicting against main), so the stale-branch item is resolved.

• DCO: green on this push, so the new merge commit is properly signed off — no repeat of the earlier sign-off hiccup.
• CI: only DCO is reporting so far; dynamic-pytests should re-trigger on this push. Worth an eyeball once it reports — and the standing note still applies: CI pulls a released lightsim2grid, so it exercises the rebuild path, not the in-place one.

Everything substantive from the earlier passes stays addressed; this was purely the branch going stale against main. Looks ready for a maintainer's look — approval/merge remains their call.

— 🤖 _automated pre-review; a maintainer will follow up_

[yemine0x01]

LGTM! Nothing to flag from a code review perspective. @albanpuech, I see you started running some benchmarks. Did anything come up that would block approval?