Skip to content

Read the stored geometry count instead of recounting the RTree on load - #497

Merged
Andy2003 merged 2 commits into
masterfrom
fix/rtree-count-from-metadata
Nov 18, 2026
Merged

Andy2003 merged 2 commits into
masterfrom
fix/rtree-count-from-metadata

Conversation

@Andy2003

@Andy2003 Andy2003 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Every time an RTree layer was loaded (each spatial.* procedure call), the whole tree was traversed to count its geometries. On large layers this made every query and write slow.

Loading now uses the totalGeometryCount stored on the metadata node. The tree is only counted when the value is missing (stores written by 5.x releases), or when it is 0 although the root has entries, or non-zero although the root has none. add and remove keep the stored count current, and removing the last node resets it to 0.

Behaviour change

The recount used to correct the count after indexed nodes were deleted directly with DETACH DELETE. It no longer does. The docs now say to call spatial.removeNode before deleting an indexed node. Automatic syncing is tracked in #496.

Resolves #313
Resolves #145

Every layer load traversed the whole tree to count its geometries, which
made queries and writes on large layers slow. The count stored on the
metadata node is now used; the tree is only counted when it is missing or
contradicts the tree (0 for a non-empty tree, or non-zero for an empty one).
Removing the last node resets the count to 0.

A direct DETACH DELETE of an indexed node no longer gets corrected by that
recount. The docs now say to call spatial.removeNode first (#496).
@Andy2003
Andy2003 requested a lite review from Copilot November 18, 2026 16:22
@Andy2003 Andy2003 self-assigned this Nov 18, 2026
@Andy2003 Andy2003 added the fix Bugfix label Nov 18, 2026
@Andy2003 Andy2003 added the fix Bugfix label Nov 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical active issue remains in RTreeIndex.isEmpty(tx), which can accept stale positive feature counts after deleting all geometries from a multi-level tree.

1 open issue
What changed in this PR

This pull request improves R-tree layer loading by reusing the persisted geometry count instead of traversing the full tree on every load, while retaining fallback recounting for legacy or clearly inconsistent metadata.

Changes:

  • Stores, validates, and maintains totalGeometryCount during R-tree operations.
  • Recounts legacy or inconsistent trees and resets the count when the final indexed node is removed.
  • Adds regression and upgrade-compatibility coverage for stored counts and documents the required procedure for modifying indexed nodes.
File Description
server-plugin/​src/​test/​java/​org/​neo4j/​gis/​spatial/​LayersTest.java Adds regression tests for stored geometry counts, legacy fallback recounting, stale counts, and final-node removal behavior.
server-plugin/​src/​main/​java/​org/​neo4j/​gis/​spatial/​rtree/​RTreeIndex.java Loads and maintains persisted geometry counts with fallback recounting for missing or inconsistent metadata.
README.md Documents the requirement to use spatial procedures when modifying indexed nodes.
docs/​docs/​modules/​ROOT/​pages/​overview/​introduction.adoc Adds user-facing guidance about keeping spatial indexes synchronized with graph changes.
compat-tests/​src/​test/​java/​org/​neo4j/​spatial/​compat/​UpgradeCompatibilityTest.java Verifies stored geometry counts across compatibility and upgrade fixtures.

Review effort: Lite

@github-actions

Copy link
Copy Markdown

Test Results

435 tests  +3   413 ✅ +3   3m 4s ⏱️ -2s
 33 suites ±0    22 💤 ±0 
 33 files   ±0     0 ❌ ±0 

Results for commit 2bca662. ± Comparison against base commit 3c1272f.

@Andy2003
Andy2003 merged commit e390381 into master Nov 18, 2026
7 checks passed
@Andy2003
Andy2003 deleted the fix/rtree-count-from-metadata branch November 18, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Bugfix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slow spatial queries with spatial 0.24 on Neo4J 3.1.1 and 3.1.4 Extremely slow queries on heavy write load

2 participants