feat: add check replica command to identify broken snapshot chains - #561
Open
Hassanzadeh-sd wants to merge 1 commit into
Open
feat: add check replica command to identify broken snapshot chains#561Hassanzadeh-sd wants to merge 1 commit into
Hassanzadeh-sd wants to merge 1 commit into
Conversation
Add 'longhornctl check replica' which inspects the snapshot chains in the Longhorn replica data directories on each node and reports integrity issues: - snapshots referencing a missing parent (broken chain) - disk files without metadata files, and metadata files without disk files - missing or corrupted volume.meta and disk metadata files - a missing volume head, unexpected extra volume head files, and metadata declaring a mismatching disk name - loops in the snapshot chain The command follows the existing DaemonSet pattern: the remote command deploys privileged pods that run 'longhornctl-local check replica' against the host data directory, then aggregates the per-node JSON results into YAML. Results can be filtered with --volume-name and --name, and include the reachable snapshot chain from the volume head for context. A warning is emitted when the replica is in use, since findings may be transient while the engine is modifying the chain. Longhorn 9102 Signed-off-by: Sajjad Hassanzadeh <hassanzadeh.sd@gmail.com>
Hassanzadeh-sd
force-pushed
the
feat-check-replica-snapshot-chain
branch
from
July 26, 2026 06:55
d3312fc to
ebe7bbc
Compare
Author
|
Hi @derekbit, this PR is complete from my side and ready for review:
I would appreciate a review when you have time, and if everything looks good, consideration for merging toward the v1.13.0 milestone. I will address any feedback promptly. Also, as discussed in the issue, feel free to assign longhorn/longhorn#9102 to me. Thanks! |
Hassanzadeh-sd
added a commit
to Hassanzadeh-sd/website
that referenced
this pull request
Aug 9, 2026
Add the new check replica command (longhorn/cli#561, Issue longhorn/longhorn#9102) to the longhornctl troubleshooting command list for v1.13.0. Signed-off-by: Sajjad Hassanzadeh <hassanzadeh.sd@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Issue longhorn/longhorn#9102
What this PR does / why we need it:
Adds a new
longhornctl check replicacommand that identifies broken snapshot chains in the Longhorn replica data directories, for the v1 data engine.The remote command follows the existing DaemonSet pattern (same as
longhornctl get replica): it deploys privileged pods that runlonghornctl-local check replicaagainst the host data directory on each node, then aggregates the per-node JSON results into YAML.For every replica directory (filterable with
--volume-nameand--name), the checker:volume.metaand everyvolume-head-*.img.meta/volume-snap-*.img.metadisk metadata fileParentreference resolves to an existing disk, including snapshot tree branches that are not part of the volume head chainvolume.metaexists, and warns about unexpected extravolume-head-*.imgfilesA warning is emitted when the replica is in use, since findings may be transient while the engine is modifying the chain (for example during snapshot creation or purge).
Example output for a replica with a missing parent snapshot:
Special notes for your reviewer:
pkg/local/replica/checker.go(validateSnapshotChain) and is covered by unit tests using synthetic replica directories: healthy chain, snapshot tree branch after revert, missing parent, missing disk file, missing metadata file, missing volume head, chain loop, extra head file, corrupted metadata, and name mismatch.pkg/local/replica/getter.gohelper methodsgetReplicaNamesInDirectoryandisReplicaInUsewere refactored into package-level functions so the checker can reuse them; no behavior change.Additional documentation or context
docs/longhornctl_check_replica.mdis generated with thedoccommand, following the repo's doc generation flow.longhornctlsupports identifying the broken snapshot chain longhorn#9102.