You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem. AOF replay can run long after the original commands, and commands with an absolute
deadline then behave differently. Example, both commands originally running before T:
SET k 10 EXAT T
INCRBY k 5
Originally, k = 15 with deadline T. Replayed after T, SET ... EXAT T sees an elapsed
deadline and deletes the key, and INCRBY recreates it as 5 with no TTL. The replayed state is
wrong, and the key even outlives its deadline. Suspending lazy and active expiry during replay does
not help, because the command itself compares the deadline with the transaction clock.
Goal. Replay produces the state the original commands produced. Keys whose deadline has passed
by the end of replay are then expired normally.
Options under consideration (part of this issue is choosing one):
No expiry during replay. Replay never deletes because of a past deadline: SET and the
expire family store the past deadline, lazy and active expiry are off, and the base loader keeps
expired keys. Valkey does the same while loading its AOF.
Sound because the journal records the outcome of every time decision: a SET or EXPIRE
whose deadline had already passed is journaled as DEL, other deadlines as absolute PEXPIREAT, and lazy/active expiry as DEL.
No log format change.
Requires every journaled deadline to be absolute. Set-member FIELDEXPIRE is journaled with a
relative TTL today, and auto-journaled commands with relative TTLs need an audit.
The bootstrap load of --dbfilename must keep expired keys too.
Could also fix replicas, which delete a key when they apply SET ... PXAT T after T.
Logical time. Persist each record's original time (the transaction's time_now_ms_, not JournalItem::time_ms), and replay each record with its transaction clock pinned to it. The
base is loaded as of its cut time.
Handles relative TTLs without changing what is journaled.
Needs a time per record in the log (see AOF: segment format and AofSegmentWriter #8407), a clock hook in JournalExecutor, and an
audit of command paths that read the wall clock directly.
Done when: an option is chosen and documented, and the example above, plus the same pattern
with PX/PXAT, PEXPIREAT, GETEX and hash-field expiry, replays to the original state after
the deadline and then expires normally.
Open edge cases to settle: whether a snapshot serializes a key whose deadline passes between the
cut and its serialization.
Part of #8404 (AOF MVP). Design: Time-based commands.
Problem. AOF replay can run long after the original commands, and commands with an absolute
deadline then behave differently. Example, both commands originally running before
T:Originally,
k = 15with deadlineT. Replayed afterT,SET ... EXAT Tsees an elapseddeadline and deletes the key, and
INCRBYrecreates it as5with no TTL. The replayed state iswrong, and the key even outlives its deadline. Suspending lazy and active expiry during replay does
not help, because the command itself compares the deadline with the transaction clock.
Goal. Replay produces the state the original commands produced. Keys whose deadline has passed
by the end of replay are then expired normally.
Options under consideration (part of this issue is choosing one):
SETand theexpire family store the past deadline, lazy and active expiry are off, and the base loader keeps
expired keys. Valkey does the same while loading its AOF.
SETorEXPIREwhose deadline had already passed is journaled as
DEL, other deadlines as absolutePEXPIREAT, and lazy/active expiry asDEL.FIELDEXPIREis journaled with arelative TTL today, and auto-journaled commands with relative TTLs need an audit.
--dbfilenamemust keep expired keys too.SET ... PXAT TafterT.time_now_ms_, notJournalItem::time_ms), and replay each record with its transaction clock pinned to it. Thebase is loaded as of its cut time.
JournalExecutor, and anaudit of command paths that read the wall clock directly.
Done when: an option is chosen and documented, and the example above, plus the same pattern
with
PX/PXAT,PEXPIREAT,GETEXand hash-field expiry, replays to the original state afterthe deadline and then expires normally.
Open edge cases to settle: whether a snapshot serializes a key whose deadline passes between the
cut and its serialization.