Skip to content

[M13] Convert highest-value full-table Scan to assigned-to-index GSI query - #53

Open
bsatrom wants to merge 1 commit into
mainfrom
fix/infra/m13-scan-to-gsi
Open

bsatrom wants to merge 1 commit into
mainfrom
fix/infra/m13-scan-to-gsi

Conversation

@bsatrom

@bsatrom bsatrom commented Jul 2, 2026

Copy link
Copy Markdown
Member

M13 — Convert full-table Scans to GSI queries

Problem

Several Lambda handlers perform DynamoDB Scan operations with FilterExpression, which read the entire table before filtering. As tables grow these become slow and costly (RCU-heavy), and they run on hot paths.

Fix (this PR)

Converts the highest-value scan — getDevicesAssignedToUser() in lambda/api-users/index.ts — from a full-table Scan to a Query against a new assigned-to-index GSI on the devices table.

This scan is the cheapest high-value conversion because it runs:

  • on every user listing (once per user), and
  • on each device (re)assignment / user deletion.

Changes:

  • lib/storage-construct.ts — add assigned-to-index GSI (assigned_to HASH, last_seen RANGE, ProjectionType.ALL).
  • lambda/api-users/index.ts — getDevicesAssignedToUser() now uses QueryCommand with KeyConditionExpression on the GSI instead of ScanCommand + FilterExpression.
  • Test updates asserting the new GSI (CDK synth-snapshot) and the query path.

Verification

  • npm run build (tsc) — clean.
  • npm test (vitest) — 451 passed. storage-construct.test.ts asserts all three GSIs (fleet-index, status-index, assigned-to-index).

Deferred scans (documented, not addressed here)

Each of these requires either a new access-key design or a table-level rearchitecture and is deferred to a follow-up to keep this PR minimal and low-risk:

  • lambda/api-activity (x6) — time-range/aggregation scans over activity data; best addressed by a dedicated time-series access pattern (partition-by-day + sort key) rather than a single GSI. Larger design change.
  • lambda/api-devices/index.ts:129 — list-all-devices scan; a table-wide listing has no single high-cardinality key to index on. Candidate for pagination + a sparse status/fleet GSI already present; needs product decision on default filter.
  • lambda/api-alerts/index.ts:148 — alert listing scan; needs an alert-status/time GSI design.
  • lambda/api-users/index.ts:147 — a second scan on a different path; folded conceptually with the assigned-to pattern but the remaining call site needs its own key model.
  • lambda/api-commands/index.ts:416 — command history scan; needs a device+time GSI.
  • lambda/analytics/chat-query.ts:488 — analytics scan on a path that already tolerates higher latency; lowest priority.

Follow-up tickets should design the appropriate GSIs / time-series keys for each before conversion.

Closes review finding M13

… GSI query

Adds an assigned-to-index GSI on the devices table and converts
getDevicesAssignedToUser() from a full-table Scan with FilterExpression
to a Query against the new index. This is the highest-value scan
(runs per user on listing and on every device (re)assignment).

Closes review finding M13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant