Report actual requested memory on allocation failure; add memory_pool::free()

[mohitt31]

Report actual requested memory on allocation failure; add a way to release the memory pool

Addresses two of the three asks in #118 ("(Still) Excessive memory usage"):

1. "Add a hint displaying the actual amount of missing memory in case of
   COSMA being able to catch the OOM event" — memory_pool::resize() and
   memory_pool::reserve() already catch bad_alloc/length_error and print
   a hint, but the message was generic and didn't say how much memory COSMA
   actually tried to allocate. It now reports the exact requested size, in
   both human-readable units and raw element count, e.g.:

   COSMA (memory pool): failed to allocate 3.42 GB (459276288 elements)
   per rank. Try lowering the CPU memory limit (see environment variable
   COSMA_CPU_MAX_MEMORY) so that COSMA uses more sequential steps and less
   memory per rank.
   

   This doesn't query the OS for how much memory is actually free (that's
   platform-specific and unreliable across the batch/container environments
   COSMA runs in), but it gives users a concrete number to compare against
   what they know is available, rather than a generic message.

2. "Provide a function to ask COSMA to release its buffers" — the
   existing memory_pool::reset() only marks the pool as logically empty;
   it never shrinks the underlying std::vector, so COSMA's peak memory
   pool capacity stays reserved for the lifetime of the process (relevant
   since cosma_context is a long-lived Meyer's singleton). Added
   memory_pool::free(), which actually releases the capacity back to the
   OS via the swap-with-empty-vector idiom, plus a thin free_memory_pool()
   wrapper on cosma_context and a top-level cosma::free_memory_pool<Scalar>()
   convenience function mirroring the existing get_context_instance<Scalar>()
   pattern. This lets an application (e.g. one that interleaves COSMA calls
   with other memory-hungry libraries) give memory back between
   multiplications without tearing down and recreating the whole context.

   Important caveat, documented in both docstrings: this is only safe to
   call when no CosmaMatrix/Buffer objects from a previous multiply()
   are still alive, since those cache raw pointers into the pool's backing
   storage. It's safe between independent multiply() /
   multiply_using_layout() calls (which build fresh matrix objects
   internally each time — the common ScaLAPACK-replacement / pxgemm usage
   pattern), but calling it while manually holding onto CosmaMatrix
   objects is a use-after-free. I found this the hard way with a segfault
   in local testing before adding the guard comment — worth flagging in
   review in case a more defensive runtime check is wanted instead of a
   comment.

I did not attempt (1) from the issue, a general explanation of COSMA's
memory scaling behavior, beyond what's now in these two docstrings — that
felt better suited to a wiki/docs page than code, and I didn't want to
guess at wording without maintainer input.

What I found but did not change

While tracing this I noticed a real discrepancy between two memory
estimates: Strategy::memory_used (the "Required memory per rank" value
printed to the user, and the value checked against COSMA_CPU_MAX_MEMORY
during strategy selection) is computed with a buffer-reuse-aware heuristic
(memory_with_buffer_optimization, which only counts the two largest
per-matrix buffers). The memory actually reserved at runtime, in
multiply_using_layout() via CosmaMatrix::required_memory() →
memory_pool::reserve(), is the sum of every buffer allocated across
the whole recursion tree for A, B and C combined, further inflated by the
1.2x default COSMA_MEMORY_POOL_AMORTIZATION factor. That reserve call
doesn't account for the LIFO alloc/free pattern the buffers actually
follow during execution. I believe this mismatch is the real reason the
strategy selector's OOM-avoidance check (and the number printed to users)
can look safely under COSMA_CPU_MAX_MEMORY while the process still OOMs
— matching what's described in #97 and the COSMA_ADAPT_STRATEGY=OFF
workaround mentioned in this issue's comments.

I'm flagging this rather than attempting a fix, for two reasons: making
memory_pool::reserve() peak-aware (instead of sum-aware) is a real
algorithmic change to a hot path with correctness risk if I get the
buffer-lifetime accounting wrong, and I have no multi-node/MPI cluster to
validate that a fix actually reduces measured RSS at scale without
regressing performance or correctness. Happy to take a swing at it in a
follow-up if a maintainer confirms this reading of the code is right and
can help validate on a real cluster.

About me / testing scope

First-time contributor here. My background is single-node C++ performance
work (cache-blocking, AVX2 GEMM tiling), not distributed-memory algorithms,
so I want to be upfront about what I could and couldn't verify:

• Built locally on Apple Silicon (OpenBLAS + Open MPI, COSMA_SCALAPACK=OFF,
  no GPU backend) and ran the full existing test suite (ctest) — all pass,
  no regressions.
• Added tests/memory_pool.cpp (non-MPI gtest, same pattern as the
  existing test.mapper): free() actually zeroes capacity(), the pool
  is reusable afterward, reset() vs free() behave differently as
  documented, and an intentionally-oversized request throws with the new,
  more informative message.
• Added tests/memory_pool_mpi.cpp: runs a full multiply() via the
  existing test_cosma() correctness harness, calls free_memory_pool(),
  then runs another multiply() and checks the result against the
  reference — verified locally at both 4 and 8 (oversubscribed) ranks on
  a single machine.
• What I could not verify: whether this actually reduces peak RSS on a
  real multi-node job the way the reporters in (Still) Excessive memory usage #118 need, or how it
  behaves under a real distributed OOM. This PR is scoped to what I could
  responsibly test — it doesn't claim to close (Still) Excessive memory usage #118 on its own.

Fixes part of #118.

[simonpintarelli]

cscs-ci run GH200

[mohitt31]

Hi — just checking in. CI on GH200 looks green. Is there anything you'd like me to change, or is this good to merge as-is? Happy to address any feedback.

[simonpintarelli]

Hi — just checking in. CI on GH200 looks green. Is there anything you'd like me to change, or is this good to merge as-is? Happy to address any feedback.

Hi, thanks for the PR. I'm looking at it now.

[simonpintarelli]

cscs-ci run GH200