Skip to content

[Feat] Add record inventory actions and background CLI - #172

Merged
iamalwaysuncomfortable merged 32 commits into
mainfrom
feat/record-inventory
Oct 23, 2026
Merged

iamalwaysuncomfortable merged 32 commits into
mainfrom
feat/record-inventory

Conversation

@iamalwaysuncomfortable

@iamalwaysuncomfortable iamalwaysuncomfortable commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Private token balances need consolidation for larger spends and separate records for concurrent trades. This PR adds record inventory planning, execution, reservations, and recovery through Veil's existing account, record-provider, and proving interfaces.

  • Core actions inspect inventory, join or split records, select records for consolidation, plan a target count/distribution, execute dependent steps, and reconcile saved transactions. RecordAsset requires only a program; record-name and amount-field overrides support compatible custom assets.
  • Record management stays off until the client sets recordManagement.enabled: true. Attaching recordActions() alone does not enable it. Participating writes atomically reserve execution inputs and checkpoint the proved transaction before broadcast.
  • Joins prefer available helpers for more than two records: up to 16 inputs for credits or dedicated stablecoin helpers, and 15 for dynamic ARC helpers. Helper selection follows the asset, network, and validated interface. Explicit tokenJoin routers remain supported; tokenJoin: false uses native pairwise joins. Credits splits account for their intrinsic 10,000-microcredit deduction.
  • Managed local signers support private fees. Selection prefers an eligible supplied fee record or chooses sufficient unreserved credits, excluding execution inputs. The prover must spend that exact fee record; confirmed rejected executions retain the charged fee record's exclusion. Wallet RPC writes require public fees or FeeMaster and cannot enforce a pre-broadcast fee ceiling. Nonempty wallet rebalance plans reject such ceilings before scanning; no-op plans remain valid.
  • Shield Swap wrappers resolve underlying token programs. Automatic local swap selection retries other scanned candidates only after reservation conflicts before proving. Quote terms and explicit record inputs stay fixed; submission and confirmation failures do not trigger replacement swaps.
  • @provablehq/shield-swap-cli provides shield-swap inventory inspect, plan, rebalance, run, and status. Maintenance includes count ranges, budgets, cooldowns, dry-run defaults, and a durable journal. run --execute stays in the foreground for use with a service manager. There is no standalone Veil CLI package.

Core exposes an injectable RecordInventoryStore and an in-memory implementation. SQLite is confined to the CLI package; its inventory commands and optional VEIL_INVENTORY_DB trading integration require Node 22.13+. The journal stores record identities and pending proved transactions, without keys or record plaintext.

Recovery rebroadcasts the same proof and transaction id. Failed rebroadcasts retain all reservations, record rebroadcastAttempts and lastRebroadcastError, and allow later entries to be reconciled. CLI status exposes those diagnostics; maintenance reports waiting while any spend remains unresolved. Storage failures and non-404 confirmation lookup errors still propagate. A successful rebroadcast or confirmed outcome clears the last error.

Documentation includes the record management guide, concurrent swaps example, and inventory CLI configuration and recovery reference.

Validation

Current follow-up checks:

  • Root suite: 2,649 passed, 312 gated tests skipped across 227 passing test files.
  • Type-level suite: 32 passed, no type errors.
  • All workspace package typechecks, the reference dApp typecheck, and the documentation build passed.
  • Regressions cover feasible minimum-bound preserve plans, credits deductions, early RPC fee-budget rejection, no-op plans, repeated failed rebroadcasts, later-entry reconciliation, retained private-fee reservations, transaction-id mismatches, storage failures, and SQLite diagnostic persistence across restarts.
  • Existing read-only network examples ran in the root suite. Documentation follows the repository voice guidance and Humanizer review.

Funded testnet runs completed earlier on this branch used managed clients with delegated proving and useFeeMaster: true:

  • concurrentInventory.e2e.test.ts: 3 tests passed, covering six concurrent USDCx-to-ETH swaps, six return swaps, all 12 claims, and restored balanced inventories. Each batch reached six overlapping executions. Representative confirmed swap: at1uv947jk4ndn7thlmxxqkhufr8r6mujf6xja39vq80l78t29l9s9qhuxnaq.
  • recordInventoryGuards.e2e.test.ts: 8 tests passed, with 10 confirmed inventory transactions, native joins and restoring splits, fee-ceiling rejection before broadcast for both write actions, cancellation after submission, and journal recovery. Representative confirmed join: at1mm34y2daufk95nlk9lc4pwt22y59qztjflahue3n3x9j3hgva5pq6a8t9n.

These funded runs exercise the managed client-side broadcast path; they were not repeated for this follow-up. The new failure cases use controlled transport and storage errors.

Coordination limits

Reservations protect clients sharing the same account/chain scope and store. Unmanaged wallets and raw submissions bypass them. SQLite coordinates reservations across processes; the bundled blinded-identity store does not coordinate identity counters across processes. Concurrent trading must separately coordinate those identities or stay in one process with a shared store object.

Concurrent swaps are independent transactions. Each needs a sufficiently large available input record and enforces its own output floor; automatic selection does not join records during a swap. Some trades may succeed while others fail, so each successful output must be claimed.

A missing transaction or failed broadcast does not establish rejection. Uncertain spends retain reservations and block maintenance until reconciled. Recovery does not force-unlock entries or undo completed chain operations.

@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
veil-loyalty-dapp Ready Ready Preview Oct 23, 2026 4:17pm UTC

Request Review

@Roee-87

Roee-87 commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

We can route all token records to use main_aj_arc20_2_15.aleo for join transactions.
The function signature for all 15 functions follows the general pattern:

fn join_4(
          public token_program: identifier,
          r0: dyn record,
          r1: dyn record,
          r2: dyn record,
          r3: dyn record,
      ) -> dyn record

Explorer link: https://explorer.provable.com/program/main_aj_arc20_2_15.aleo

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

RPC rebalances can submit before detecting a missing commitment grant, leaving partially completed inventory.

2 open findings
What changed in this PR

Adds record inventory planning, execution, coordination, and recovery across Veil, including a durable maintenance CLI and Shield Swap integration.

Changes:

  • Adds core ARC20, ARC22, and credits inventory actions with batch routing and reservations.
  • Adds scanner synchronization, proof checkpointing, SQLite recovery, and background maintenance.
  • Updates Shield Swap integrations, tests, examples, and documentation.
File Description
vitest.config.ts Aliases CLI storage in tests.
tsconfig.json Maps the CLI storage export.
site/​sidebars.ts Adds inventory and CLI documentation.
site/​docs/​packages/​cli.md Documents the CLI package.
site/​docs/​guides/​record-inventory.md Adds the inventory guide.
README.md Introduces inventory management.
pnpm-lock.yaml Registers new workspace dependencies.
packages/​shield-swap/​test/​actions/​inventory.test.ts Tests DEX inventory integration.
packages/​shield-swap/​src/​utils/​records.ts Excludes reserved trading records.
packages/​shield-swap/​src/​index.ts Exports inventory wrappers.
packages/​shield-swap/​src/​decorators/​shieldSwapActions.ts Adds inventory decorator actions.
packages/​shield-swap/​src/​actions/​inventory/​rebalanceInventory.ts Executes DEX inventory plans.
packages/​shield-swap/​src/​actions/​inventory/​planInventory.ts Resolves and plans underlying assets.
packages/​shield-swap/​README.md Documents DEX inventory support.
packages/​shield-swap-cli/​src/​session.ts Enables shared SQLite reservations.
packages/​shield-swap-cli/​README.md Documents journal sharing.
packages/​shield-swap-cli/​package.json Adds optional CLI integration.
packages/​provable-sdk/​test/​scannerSync.test.ts Tests scanner synchronization.
packages/​provable-sdk/​test/​delegatedBuild.test.ts Tests imports and checkpoints.
packages/​provable-sdk/​src/​index.ts Adds sync waits and proof checkpoints.
packages/​provable-sdk/​README.md Documents inventory proving behavior.
packages/​core/​test/​inventory/​inventory.test.ts Covers planning, routing, and recovery.
packages/​core/​test/​fixtures/​programs/​main_aj_arc20_2_15.aleo Provides batch-router test fixture.
packages/​core/​src/​types/​proving.ts Adds proving progress callbacks.
packages/​core/​src/​inventory/​types.ts Defines inventory APIs and storage.
packages/​core/​src/​inventory/​tokenJoin.ts Validates and resolves routed joins.
packages/​core/​src/​inventory/​store.ts Implements in-memory reservations.
packages/​core/​src/​inventory/​spending.ts Coordinates managed record spending.
packages/​core/​src/​inventory/​recordActions.ts Adds the inventory decorator.
packages/​core/​src/​inventory/​planner.ts Builds bounded inventory plans.
packages/​core/​src/​inventory/​internal.ts Implements scoping and identities.
packages/​core/​src/​index.ts Exports inventory APIs.
packages/​core/​src/​actions/​wallet/​writeContract.ts Coordinates managed writes.
packages/​core/​src/​actions/​wallet/​executeContract.ts Adds managed execution checkpoints.
packages/​core/​src/​actions/​records/​splitRecord.ts Implements record splitting.
packages/​core/​src/​actions/​records/​reconcileRecordInventory.ts Reconciles pending transactions.
packages/​core/​src/​actions/​records/​rebalanceRecordInventory.ts Executes dependent inventory plans.
packages/​core/​src/​actions/​records/​planRecordInventory.ts Plans scanner-backed inventory.
packages/​core/​src/​actions/​records/​joinRecords.ts Implements native and routed joins.
packages/​core/​src/​actions/​records/​internal.ts Validates assets and records.
packages/​core/​src/​actions/​records/​getRecordInventory.ts Scans and classifies inventory.
packages/​core/​src/​actions/​records/​autoJoin.ts Implements automatic consolidation.
packages/​core/​README.md Documents core inventory actions.
packages/​cli/​tsup.config.ts Configures CLI builds.
packages/​cli/​tsconfig.json Configures CLI typechecking.
packages/​cli/​test/​storage.test.ts Tests durable atomic storage.
packages/​cli/​test/​runner.test.ts Tests maintenance policies.
packages/​cli/​src/​storage.ts Implements the SQLite journal.
packages/​cli/​src/​runner.ts Runs maintenance cycles.
packages/​cli/​src/​index.ts Implements CLI commands.
packages/​cli/​src/​config.ts Validates CLI configuration.
packages/​cli/​README.md Documents operation and recovery.
packages/​cli/​package.json Defines the new CLI package.
package.json Adds the workspace CLI command.
examples/​e2e-demo.ts Adds an inventory consumer example.
apps/​loyalty-dapp/​src/​lib/​aleo.ts Adds reference-dApp inventory integration.
.gitignore Ignores local journals.
.changeset/​record-inventory.md Records release changes.
.changeset/​config.json Adds CLI package versioning.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/core/src/actions/records/rebalanceRecordInventory.ts
Comment thread README.md Outdated

@iamalwaysuncomfortable iamalwaysuncomfortable left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the core, SDK, Shield Swap, and CLI changes. The design holds up: management is opt-in and additive on every interface, SQLite stays in the CLI, and the two earlier Copilot findings are fixed.

One real bug blocks merge: the default preserve planner rejects feasible targets (reproduced on this branch, details inline). The other inline comments are a late-failing validation, a live-untested proving path, and a docs placement nit.

One item with no file anchor: the PR description says it adds @provablehq/veil-cli, but the CLI moved into @provablehq/shield-swap-cli in d032080. Worth updating the body so the merge commit and release notes don't carry the stale package name.

Comment thread packages/core/src/inventory/planner.ts Outdated
Comment thread packages/core/src/actions/records/rebalanceRecordInventory.ts
Comment thread packages/provable-sdk/src/index.ts
Comment thread site/docs/packages/cli.md Outdated

@iamalwaysuncomfortable iamalwaysuncomfortable left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at d18b88c.

Resolved

  • Planner preserve bug (896496b). Verified beyond the new unit cases with a randomized check on top of the branch: 3,855 ARC20 and 3,863 credits inputs whose totals cover the target after deductions, zero throws and zero final lots below the minimum. The replay-style test pins both counterexamples plus a credits case with deductions.
  • Docs page folded into shield-swap-cli.md (d18b88c); sidebar entry removed and the one inbound link updated. No dangling references to packages/cli remain.

New, non-blocking (inline on the runner): a prepared transaction the node deterministically rejects at broadcast wedges maintenance for every asset, and the documented way out is hand-editing the journal.

Carried over, non-blocking

  • rebalanceRecordInventory with an RPC account and a run-level maxFeeMicrocredits still fails inside the first step as interrupted rather than throwing up front.
  • Managed delegated proving (client-side broadcast when a progress handler is present) still deserves one funded testnet run with useFeeMaster on before the release that ships this.
  • PR body still says @provablehq/veil-cli; the package is @provablehq/shield-swap-cli.

Nothing here blocks merge.

Comment thread packages/shield-swap-cli/src/inventory/runner.ts
@iamalwaysuncomfortable
iamalwaysuncomfortable merged commit a978150 into main Oct 23, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
Preview — eb070689 Deployed Oct 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants