Fix count loss when pooling Visium HD bins

[psgundla]

Fixes #143.

As reported in #143, several source bins can share the same destination index, but the indexed += update does not accumulate repeated indices. This drops counts during 16 µm to 128 µm pooling and makes the result depend on the chunk size and input order.

This replaces that update with np.add.at, converting sparse chunks to dense arrays before accumulation. The existing arguments, spatial grid, and float32 output stay unchanged. The change keeps the current chunked implementation; it does not introduce a sparse-only pooling path or remove its dense-memory cost.

Verification

The regression test covers 72 combinations: dense, CSR, and CSC matrices; in-memory and backed storage; four chunk sizes, including the default 50,000; and three input orders. It checks exact pooled counts, zero-count bins, gene order, coordinates, and unchanged input counts.

From the repository root, with HEST installed:

PYTHONPATH=src:tests python -m unittest hest_tests.TestVisiumHDPooling -v

That test passes through the actual HEST package on Python 3.11.16/macOS, without mocking its imports.

Real-data validation uses the complete filtered 16 µm matrix from 10x Genomics' human colorectal-cancer Visium HD sample, processed with Space Ranger 3.0.0: 137,051 bins, 18,085 genes, and 266,691,885 UMIs.

Chunk size	Original output UMIs	Fixed output UMIs
1,024	213,383,291	266,691,885
4,096	121,371,935	266,691,885

The fixed outputs match every bin/gene entry in an independent integer sparse-matrix aggregation, with no lost counts. Changing the chunk size or reversing the barcode order produces the same output matrix. The total also agrees with the original 10x metrics CSV at whole-UMI precision.

The real-data rerun calls the patched function through the installed HEST package. The original comparison uses the unchanged function extracted from upstream commit 3ddb5eaf.

Coordinates are matched by barcode, microns_per_pixel comes from the scalefactors file, and pooling uses src_bin_size_um=16, dst_bin_size_um=128. Counts are not normalized, genes are not selected, and the tissue is not cropped.

Dataset files:

• Original 10x binned outputs archive — approximately 15.9 GB. Validation uses these three files inside the archive:
  • binned_outputs/square_016um/filtered_feature_bc_matrix.h5 — raw filtered counts.
  • binned_outputs/square_016um/spatial/tissue_positions.parquet — barcode coordinates.
  • binned_outputs/square_016um/spatial/scalefactors_json.json — physical pixel scale.
• Original 10x metrics summary CSV — independent vendor cross-check: 137,051 bins × 1,945.9316969595368 mean UMIs/bin agrees with 266,691,885 UMIs at whole-UMI precision.

The dataset is provided by 10x Genomics under CC BY 4.0. No dataset files are included in this PR.

Reproduction files

Supporting evidence is kept on a separate fork branch so the upstream diff stays limited to the fix and regression test. These links are pinned to the evidence commit:

• Validation report and reproduction commands
• Runnable real-data validation script
• Machine-readable results, input checksums, and dependency versions

Limits

The full WSI reader, image processing, and broader HEST data suite have not been run. This is one real sample, not a revalidation of all 25 slides reported in the issue. The default 50,000-row chunk is covered by the regression test, but was not used for the full real-data matrix because of its dense-memory requirements.