Skip to content

fix(msg): keep read sequences monotonic across devices - #3820

Open
John-Funcity wants to merge 1 commit into
openimsdk:mainfrom
John-Funcity:codex/fix-monotonic-read-seq
Open

John-Funcity wants to merge 1 commit into
openimsdk:mainfrom
John-Funcity:codex/fix-monotonic-read-seq

Conversation

@John-Funcity

Copy link
Copy Markdown

Concurrent read reports from two devices can both pass the read-before-write check and then move hasReadSeq backwards. A delayed MongoDB cache fill or a bulk read update can also overwrite a newer Redis value, causing already-read messages to appear unread again.

This change makes read progress monotonic per user and conversation:

  • Merge Redis read sequences atomically, using decimal-string comparisons to preserve the full int64 range. Apply the same rule to single and bulk updates and cache fills, preserving MongoDB baselines on cold caches and fencing legacy RocksCache refill owners.
  • Persist read_seq with MongoDB $max, enforce a unique user/conversation index, and retry concurrent duplicate-key upserts against the existing row. Other sequence fields retain their $set semantics.
  • Return effective progress through the storage interfaces and use it in read-receipt notifications and the sender progress queued for persistence. Equal or stale reports can still emit receipts to allow downstream repair.

Validation:

  • Reproduced Redis regressions before the fix: 346 overwritten with 200, stale fills returning 100 after a concurrent advance, and bulk writes lowering cached or MongoDB baseline progress.
  • Passed go test -mod=readonly -race ./pkg/common/storage/cache/redis ./pkg/common/storage/database/mgo ./pkg/common/storage/controller ./internal/rpc/msg -run '^TestReadSeq' -count=1 -timeout=90s.
  • Passed go build -mod=readonly ./... and git diff --check.
  • Redis tests execute Lua with miniredis; MongoDB tests inspect driver mock commands, including unique-index creation and duplicate-key retry. These are not live MongoDB or Redis Cluster integration tests.

Rollout: resolve any existing duplicate seq_user rows before deploying. Startup creates user_conversation_unique; it does not delete or merge historical data. The index can coexist with the old non-unique index on MongoDB 5.0+; reconcile any manually installed equivalent unique index names beforehand. See rollout notes.

The existing in-memory notification queue and MQ architecture are preserved. Reliable delivery/compensation for lost persistence events and reconciliation of existing Redis/MongoDB differences are outside this change.

@pull-request-size pull-request-size Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🤖 All Contributors have signed the CLA.
The signed information is recorded here
Posted by the CLA Assistant Lite bot.

@John-Funcity

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant