sync: SVS v3 large-group revision (mhash, PARTIAL, announce+pull) by Taranum01 · Pull Request #190 · named-data/ndnd

Taranum01 · 2026-06-29T17:01:51Z

Summary

Implements the SVS v3 large-group revision (approved by Tianyuan Yu): inline FULL/PARTIAL sync, membership hash (mhash), and announce+pull recovery via published full vectors at 32=sv/<version>.

Event	`size ≤ threshold`	`size > threshold`
Publication	Inline FULL	Inline PARTIAL
Periodic sync	Inline FULL	Announce + pull
`mhash` mismatch	Pull or sender announce	Pull or sender announce

Backward compatible: SyncVectorThreshold defaults to 0 (legacy mode: StateVector-only wire, no mhash/VectorType, no announce+pull recovery — matches pre-revision DV prefix-table SVS). Large-group modes activate when threshold > 0.

Spec: docs/svs-v3-revision.md

Main changes

TLV: MemberSetHash, VectorType, SvsDataRef on SvsData (std/ndn/svs/v3)
Core: ComputeMhash, PARTIAL encode, publish at 32=sv, announce-only Sync, pull + merge (std/sync)
Example: std/examples/svs/large-sync (--threshold)
Tests: unit + in-process multi-node integration (std/sync/*_test.go, see svs_mesh_test.go)

Open item

Mixed-version interoperability (revised vs plain SVS v3 peers in one group) — spec §8; not addressed in this PR.

Test plan

go test ./std/sync/...
GitHub e2e (make e2e, sprint topology)
NFD smoke: 3× large-sync --threshold=80 on /ndn/svs/{alice,bob,carol} with multicast on /ndn/svs

Implements the approved large-group SVS revision with backward-compatible defaults (SyncVectorThreshold=0). Adds mhash and PARTIAL/announce+pull paths, 32=sv publish/pull recovery, tests, and a large-sync example. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

Stop blind pulls to unpublished 32=sv prefixes on inline FULL mhash mismatch (spec §5.6: merge inline first). Clone SvMap before encoding outside the mutex to avoid concurrent map mutation panics. Co-authored-by: Cursor <cursoragent@cursor.com>

DV prefix-table SVS uses default SyncVectorThreshold=0. Recovery was still forcing announce-only sync on mhash mismatch, breaking step-2 prefix propagation while router reachability looked fine. Gate large-group recovery behind threshold > 0 for true legacy behavior. Co-authored-by: Cursor <cursoragent@cursor.com>

DV prefix-table sync uses default threshold 0. Emit StateVector-only SvsData (no mhash/VectorType) so step-2 SVS matches pre-revision behavior on the wire while large-group features stay behind threshold > 0. Co-authored-by: Cursor <cursoragent@cursor.com>

Adds topo.min.conf (6 nodes) and make e2e-local for reliable local validation on Mac Docker/Colima. Full sprint topo remains make e2e for CI. Co-authored-by: Cursor <cursoragent@cursor.com>

Publish the revision spec alongside the std/sync implementation in PR named-data#190. Co-authored-by: Cursor <cursoragent@cursor.com>

tianyuan129 · 2026-07-03T23:37:59Z

@@ -0,0 +1,16 @@
+# Small topology for local e2e (6 nodes, minimal link delays).


Can we not introduce new topologies in e2e tests?

Removed in 79b79ee — topo.min.conf, make e2e-local, and the docker script were local Mac dev helpers only. CI continues to use topo.sprint.conf.

tianyuan129 · 2026-07-03T23:38:22Z

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash


Can you explain why we need this extra things?

Removed in 79b79ee for the same reason — not needed in the upstream repo.

tianyuan129 · 2026-07-03T23:39:37Z

@@ -0,0 +1,82 @@
+package face


I am completely lost, we are revising the SVS protocol. Why invent the multicast face here?

Fair point. Removed std/engine/face/multicast_face.go in 79b79ee. The in-process mesh tests now use a test-only multicast hub in svs_mesh_test.go (same coverage, no new production face type).

tianyuan129 · 2026-07-03T23:46:36Z

+		}
+		svsData = buildAnnounceSvsData(stateSnap, ref)
+	} else {
+		svsData = buildSvsDataForSend(


Should use arg struct to avoid long parameter list.

Done in 79b79ee — buildSvsDataForSend now takes a svsSendInput struct.

tianyuan129 · 2026-07-03T23:48:36Z

+			mtimeSnap,
+		)
+	}
+	logSyncSend(s, reason, svsData)


Is this a leftover from your debugging?

Yes — removed in 79b79ee. Deleted svs_debug.go and the logSyncSend / logSyncRecv calls on the send/recv path.

tianyuan129 · 2026-07-04T00:08:17Z

+	}
+	return exceedsSyncThreshold(threshold, inlineFullSize(state))
+}
+


I worry those helper functions are not helping with readability and mainainability.

Addressed in 79b79ee — merged svs_announce.go into svs_pull.go (announce+pull recovery) and svs_encode.go (inline FULL/PARTIAL build) to reduce file sprawl.

tianyuan129 · 2026-07-04T00:10:40Z

@@ -0,0 +1,89 @@
+package sync


I don't see it is necessary to have this file -- if this is only for debugging.

Removed svs_debug.go entirely in 79b79ee.

tianyuan129 · 2026-07-04T00:26:24Z

@@ -0,0 +1,157 @@
+package sync


Can you describe what this test is about?

Documented in 79b79ee — renamed to svs_mesh_test.go with a file-level comment. It is an in-process multi-node SVS harness (real engine + SvSync per node, test-only multicast hub) that exercises small-group sync, PARTIAL publication, and announce+pull recovery without mininet/NFD.

Remove local-only e2e topology and docker runner. Drop svs_debug and production multicast_face; keep mesh integration tests with a test-only hub in svs_mesh_test.go. Merge announce helpers into svs_pull, use svsSendInput for buildSvsDataForSend, and remove debug send logging. Co-authored-by: Cursor <cursoragent@cursor.com>

Taranum01 · 2026-07-04T16:54:51Z

Thanks for the review, @tianyuan129 — addressed in 79b79ee:

Removed local-only e2e topology / docker runner (topo.min.conf, e2e-local, e2e-local-docker.sh)
Removed production multicast_face; mesh integration tests use a test-only hub in svs_mesh_test.go
Removed svs_debug.go and debug send/recv logging
buildSvsDataForSend now takes a svsSendInput struct
Merged svs_announce.go into svs_pull.go + svs_encode.go

Also updated the PR description: SyncVectorThreshold=0 uses legacy StateVector-only wire (no mhash), which preserves DV prefix-table behavior and is what fixed e2e.

go test ./std/sync/... passes locally; CI should re-run on the new commit.

Remove dead large-sync debug flag and tuName helper, clarify threshold-0 comments, trim pull-path debug logging, rename announce tests to svs_pull_test, and test pullRefFromSyncDataWire with a signed Sync Data wire. Co-authored-by: Cursor <cursoragent@cursor.com>

Taranum01 · 2026-07-04T17:10:23Z

Pushed 60474b3 — minor style polish only (no logic changes): aligned `large-sync` with other SVS examples, clarified threshold-0 comments, trimmed pull-path debug logs, renamed `svs_announce_test.go` → `svs_pull_test.go`, and fixed `TestPullRefFromSyncDataWire` to use a real Sync Data wire.

Taranum01 changed the base branch from main to psvs June 29, 2026 17:07

Taranum Wasu and others added 6 commits June 29, 2026 22:43

fix lint: sort imports in svs_mhash_test.go

8540360

Co-authored-by: Cursor <cursoragent@cursor.com>

e2e: add minimal local topology and docker runner script

2b36d34

Adds topo.min.conf (6 nodes) and make e2e-local for reliable local validation on Mac Docker/Colima. Full sprint topo remains make e2e for CI. Co-authored-by: Cursor <cursoragent@cursor.com>

docs: add SVS v3 large-group revision specification

05623e2

Publish the revision spec alongside the std/sync implementation in PR named-data#190. Co-authored-by: Cursor <cursoragent@cursor.com>

tianyuan129 requested changes Jul 4, 2026

View reviewed changes

		@@ -0,0 +1,16 @@
		# Small topology for local e2e (6 nodes, minimal link delays).

Uh oh!

Conversation

Taranum01 commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Main changes

Open item

Test plan

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Taranum01 commented Jul 4, 2026

Uh oh!

Taranum01 commented Jul 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Taranum01 commented Jun 29, 2026 •

edited

Loading