Skip to content

store: create a copy's attribute indexes from the source's index list - #6758

Open
lx3133584 wants to merge 1 commit into
graphprotocol:masterfrom
lx3133584:fix/copy-source-indexes
Open

lx3133584 wants to merge 1 commit into
graphprotocol:masterfrom
lx3133584:fix/copy-source-indexes

Conversation

@lx3133584

Copy link
Copy Markdown

Fixes #6722

Problem

Since #6434 (v0.44.0) graphman copy creates the destination's schema from the
layout's default index set. Only the indexes to_postpone() covers are taken
from the source afterwards; the GiST (reference, block_range) indexes, the
BTree on id and any manually created index are written from the layout,
whatever the source carries. index_def was dropped from the copy path in that
change, and nothing in it refers to the behaviour #5140 asked for.

For an operator who drops indexes from a large deployment before copying it out
to rebalance shards, every copy comes back with all of them: +1.35 TB on a 1.6 TB
deployment in the report, +35 indexes on a 228 MB one. Maintaining them also
slows the copy's catch-up and pushed a destination shard over its disk threshold
mid-copy.

Fix

Pass the source's index list down to the DDL again, as #5425 had it:

  • SubgraphStore::copy_deployment loads the source's indexes and hands them to
    create_deployment.
  • Layout::create_relational_schema_with_source_indexes (new; the old entry
    point passes None) forwards them to Table::as_ddl, which writes the
    attribute indexes from that list instead of from the layout.

Two details that are not simply "restore the old code":

  • The time-travel indexes (brin_*, *_block_range_closed) and the aggregate
    indexes stay layout-driven. They describe how history is stored rather than
    something an operator tunes, and every deployment has them. A source list that
    filtered them would otherwise drop something the schema needs.
  • indexes_for_table skips the attr_N_0_<table>_id style indexes because the
    layout recreates them anyway, which is not true once the attribute indexes
    come from the source. indexes_for_copy_table keeps them, and still drops the
    ones the layout always creates.

Grafts are unaffected: a column that exists only on the destination has no entry
in the source list, so it keeps the layout's indexes for that column.

prune.rs creates the destination of a truncation, not a copy, and passes
None, which keeps its behaviour.

Testing

Added source_indexes_replace_the_layout_indexes_on_creation in
store/postgres/src/relational/ddl_tests.rs, which works on the generated DDL
without a database:

  • with an empty source list, create index attr_0_1_thing_big_thing ... using gist — which the layout describes — is not written
  • the time-travel indexes (brin_thing, thing_block_range_closed) are still
    written, since they are not the source's to drop
  • a source list carrying that gist index does produce it

Reverting the Table::as_ddl change makes the first of those fail with the
generated DDL printed, so the test is pinned to the behaviour and not to the
ordering of the output.

$ cargo test -p graph-store-postgres --lib
test result: ok. 64 passed; 0 failed; 0 ignored

cargo fmt --all -- --check and cargo clippy -p graph-store-postgres --all-targets are clean.

The graphman copy create path itself is covered by postgres::graft::copy in
store/test-store, which needs the two-shard config.sharded.toml. Those tests
fail on unmodified master on this machine (could not connect to server "shard_sgd" after a fresh database), so I could not use them to confirm the
end-to-end behaviour here; the unit test above and the DDL path it exercises are
what this change was verified against.

Deployment note

A copy made under v0.44.0+ of a slimmed-down source has the full default index
set. That is not corrected by this change — it needs a re-copy, or dropping the
indexes again on the destination.

Since graphprotocol#6434 (v0.44.0) a copied deployment is created from the layout's
default index set instead of the source's indexes. Only the indexes that
`to_postpone()` covers are taken from the source afterwards; the GiST
reference indexes, the BTree on `id` and any manually created index come
from the layout, blind to what the source carries. An operator who drops
indexes from a large deployment to save space gets all of them back on every
copy - 1.35 TB on a 1.6 TB deployment in the report - which also slows the
copy's catch-up and can push the destination shard over its disk threshold.

graphprotocol#5140 asked for the opposite and graphprotocol#5425 implemented it: create the indexes
based on the index creation DDL of the source. This restores that by passing
the source's index list down to the DDL again.

The time-travel and aggregate indexes stay layout-driven. They describe how
history is stored rather than something an operator tuned, and they exist on
every deployment, so taking them from the source would let a filter drop
something the schema needs.

Grafts keep working: a column that only exists on the destination still gets
the layout's indexes for it, because the source list has no entry to match.

Signed-off-by: Liang Xu <24490353+lx3133584@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

graphman copy: since v0.44.0 the destination gets the layout's GiST/id indexes regardless of the source's index set (#5140 / #5425)

1 participant