Skip to content

Remove duplicate memories via session state managing - #14

Open
dannyjameswilliams wants to merge 8 commits into
groundwork-1from
remove_duplicate_session_memories
Open

dannyjameswilliams wants to merge 8 commits into
groundwork-1from
remove_duplicate_session_memories

Conversation

@dannyjameswilliams

Copy link
Copy Markdown

Manages session state through a minimal file system structure that keeps a text file of memory IDs, saved by session_id, which have been either stored or saved in that session (saved in different files for each).

When searching memories, all previously injected are excluded from being re-injected. All previously stored in this session are also excluded. To alleviate loss of memories in larger conversations, we fetch 30 memories and then take the top 5 non-duplicates.

On compacting, these files are reset. On session start, files older than 1 week are deleted as cleanup.

dannyjameswilliams and others added 8 commits October 7, 2026 14:32
Exclusion runs after the search, so with the server's default of 10 results a
session working in one area soon got the same ten back, all excluded, and
injected nothing while relevant memories below rank 10 never surfaced. Every
injected id was also recorded as shown even when the block was big enough for
the host to spill it to a file, hiding memories the model never saw.

Search now asks for 30 and injects the first 5 that survive exclusion, which
also keeps the block small. Setting a limit needs an explicit retrieval type;
hybrid is the one the Engram docs recommend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Memories this session wrote were excluded because the assistant already had
the conversation they came from. Compaction replaces that conversation with a
summary, just as it drops injected memories, so after it those facts were
excluded while no longer in context. PostCompact now resets own as well as
shown.

Session-scoped rows carrying this session's id stay excluded by the property
filter; the compaction summary stands in for those.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
clean() ran its whole sweep under one try, so the first file it failed on
ended the sweep. Two sessions starting together can race to remove the same
file, which only skipped the rest that once; but an entry that can never be
removed, such as a stray directory, would have stopped cleanup at the same
point on every session start. Each file now fails on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two blank lines before _tag, a final newline in search.py, and no trailing
blank line in store.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Search results are always the SDK's Memory objects. The dict fallback for
their content could never run, and since the exclusion checks read m.id first
it would only have been reached after a dict had already raised.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SessionStart is now the first hook to run, so on a fresh install it also
builds the plugin venv. Without a status message the assistant looked frozen
at startup for as long as that took.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A store that merges into an existing memory puts that memory's id in
memories_updated. Recording those as the session's own hid long-lived
memories for the rest of the session, most of whose text came from earlier
sessions — the whole of DeveloperPreferences, a single bounded row, after any
turn that added one preference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dannyjameswilliams
dannyjameswilliams added this pull request to stack #15 October 7, 2026 14:56

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant