feat(config): add state-get/state-set, guard --set against API siblings

CLAUDE.md

@@ -389,6 +389,16 @@ kbagent config row-create --project NAME --component-id ID --config-id ID --name
 kbagent config row-update --project NAME --component-id ID --config-id ID --row-id ID [--name N] [--description D] [--configuration JSON|@file|-] [--change-description TEXT] [--is-disabled | --is-enabled] [--branch ID] [--allow-plaintext-on-encrypt-failure]
 kbagent config row-delete --project NAME --component-id ID --config-id ID --row-id ID [--branch ID] [--yes]
 kbagent config oauth-url --project NAME --component-id ID --config-id ID [--redirect-url URL]
+kbagent config state-get --project NAME --component-id ID --config-id ID [--row-id ID] [--branch ID]
+kbagent config state-set --project NAME --component-id ID --config-id ID [--row-id ID] --state JSON|@file|- [--branch ID] [--dry-run] [--yes]
+# state-get/state-set (0.84.2+, #593): read/write a config's runtime state via the dedicated
+#   PUT .../state endpoint -- closes the gap where `config update --set 'state...'` looked
+#   successful but silently wrote to configuration.state.* instead (runtime state untouched).
+#   Since 0.84.2, `config update --set` / `config row-update --set` REJECT (exit 2) any path
+#   whose first segment is a top-level API field (state, rows, name, description, id, version,
+#   currentVersion, changeDescription, created, creatorToken, isDeleted, isDisabled) -- use
+#   state-set / --name / --description / row-update --is-disabled instead, or --configuration
+#   for a genuine configuration.<prefix> key.
 
 kbagent search QUERY [--project NAME] [--type table|bucket|config|flow|data-app|transformation] [--search-type textual|config-based] [--regex] [--limit N]
 # --regex (0.67.0+): opt-in regex mode (mode=regex). Case-insensitive whole-term match on ENTITY NAMES

plugins/kbagent/skills/kbagent/SKILL.md

@@ -126,6 +126,8 @@ When working inside a git repository or project directory, run `kbagent init` (o
 | Update an existing configuration row | `kbagent config row-update --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --row-id ROW-ID` |
 | Delete a configuration row | `kbagent config row-delete --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --row-id ROW-ID` |
 | Requires master token. | `kbagent config oauth-url --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
+| Read the runtime ``state`` dict of a configuration or one of its rows | `kbagent config state-get --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
+| Overwrite the runtime ``state`` dict of a configuration or one of its rows | `kbagent config state-set --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --state STATE` |
 | List data apps across one or more registered projects | `kbagent data-app list` |
 | Show merged Data Science + Storage detail for one data app | `kbagent data-app detail --project PROJECT --app-id APP-ID` |
 | Create a Keboola data app end-to-end (POST + encrypt + PUT + deploy) | `kbagent data-app create --project PROJECT --name NAME --slug SLUG` |
@@ -409,6 +411,7 @@ For detailed response parsing rules and common pitfalls, see [gotchas](reference
 | **Semantic layer (metastore)** -- models, metrics, datasets, constraints, glossary; validate / export / diff / promote / build / token | [semantic-layer-workflow](references/semantic-layer-workflow.md) |
 | **Developer Portal** (identity CRUD, list/get apps, create/patch/upload-icon/publish/deprecate; TTY-confirm on writes) | [dev-portal-workflow](references/dev-portal-workflow.md) |
 | **Config metadata** (list/get/set/delete arbitrary key-value metadata on a configuration) | [config-metadata-workflow](references/config-metadata-workflow.md) |
+| **Config runtime state** (`state-get`/`state-set`; root vs row; seeding a dev branch before testing `changed_since: adaptive`) | [config-state-workflow](references/config-state-workflow.md) |
 | **Storage descriptions** (describe bucket / table / column, batch from YAML) | [storage-describe-workflow](references/storage-describe-workflow.md) |
 | **Deep column-level lineage** (`lineage build --ai`, column graph, ER + HTML output) | [lineage-deep-workflow](references/lineage-deep-workflow.md) |
 | **Session permissions firewall** (`--deny-writes` / `--deny-destructive`, persisted policies, `permissions check`) | [permissions-workflow](references/permissions-workflow.md) |

plugins/kbagent/skills/kbagent/references/commands-reference.md

@@ -142,6 +142,8 @@ Requires a **super-admin** Manage API token (same kind as `org setup`). Same def
 - `config row-update --project NAME --component-id ID --config-id ID --row-id ID [--name N] [--description D] [--configuration JSON|@file|-] [--change-description TEXT] [--is-disabled | --is-enabled] [--branch ID] [--allow-plaintext-on-encrypt-failure]` -- update an existing configuration row. Pass only the fields you want to change; omitted fields are preserved. `--is-disabled` / `--is-enabled` toggle the row's enabled state. `--change-description` sets the new row version's `changeDescription` audit line (default: auto-generated). `#`-prefixed secrets auto-encrypt before write (fail-closed; since 0.54.0, #378).
 - `config row-delete --project NAME --component-id ID --config-id ID --row-id ID [--branch ID] [--yes]` -- delete a configuration row. Destructive (gated behind `--allow-destructive`). Branch-aware. Without `--yes` and outside `--json` mode, prompts for interactive confirmation; `--json` mode auto-skips the prompt.
 - `config oauth-url --project NAME --component-id ID --config-id ID [--redirect-url URL]` -- return the OAuth authorization URL for a component that uses OAuth authentication. **Requires a master Storage API token** (canManageTokens privilege) -- non-master tokens fail with `MISSING_MASTER_TOKEN` exit 3 on a fail-fast pre-flight check before any HTTP write happens. Open the URL in a browser to complete the OAuth flow.
+- `config state-get --project NAME --component-id ID --config-id ID [--row-id ID] [--branch ID]` -- (since 0.84.2, #593) read a configuration's runtime `state`. Without `--row-id` returns the root config's state; with `--row-id` returns that row's state (a missing row id fails loudly, it does not silently return `{}`). For row-based components the root state node is unused -- read the row state instead. See [config-state-workflow](references/config-state-workflow.md).
+- `config state-set --project NAME --component-id ID --config-id ID [--row-id ID] --state JSON|@file|- [--branch ID] [--dry-run] [--yes]` -- (since 0.84.2, #593) write a configuration's runtime `state` via the dedicated branch-scoped `PUT .../state` endpoint. `--state` must be a JSON object (not an array/scalar) under 4 MB. `--row-id` targets a row's state instead of the root. `--dry-run` previews the current-vs-new diff (same shape as `config update --dry-run`) without writing; a no-op (new state equals current) short-circuits with `changed: false` and no write. Guarded write: prompts for confirmation unless `--yes` or `--json`. This is the fix for `config update --set 'state...'`, which never reached runtime state (see [gotchas](references/gotchas.md)). See [config-state-workflow](references/config-state-workflow.md) for the seed-before-migrate playbook.
 
 ## Cross-Project Search
 - `search QUERY [--project NAME] [--type table|bucket|config|flow|data-app|transformation] [--search-type textual|config-based] [--regex] [--limit N]` -- search for items across one or more projects. **Textual** mode (default, fast) matches item names via the Storage API `global-search` endpoint. **Config-based** mode scans full configuration JSON bodies (slow, complete). `--type` is repeatable; `--limit` applies per project in textual mode (1-100, default 50). `--project` is repeatable for multi-project scope. `--regex` (0.67.0+) is opt-in regex mode: a case-insensitive whole-term match on ENTITY NAMES only (`report` != `monthly_report`; use `.*report.*`), textual-only (error with `--search-type config-based`); regex does NOT match columns. In textual mode, a table matched via a column name carries `matched_columns` (JSON) / a "Matched columns" column. **Both modes match case-insensitively** -- reach for `config search --query` when you need a case-sensitive body scan.

plugins/kbagent/skills/kbagent/references/config-state-workflow.md

@@ -0,0 +1,215 @@
+# Config State Workflow -- Reading and Seeding Runtime State
+
+`config state-get` / `config state-set` (since v0.84.2, #593) read and write a
+configuration's runtime `state` -- the checkpoint dict incremental components
+persist between jobs (last sync cursors, `lastImportId`, OAuth intermediate
+data). This closes the gap where the only write path was the Keboola UI
+(`/raw` -> *Update State* tab), and where `config update --set 'state...'`
+looked successful but silently no-oped (see
+[gotchas](gotchas.md#config-update---set-state-is-now-a-hard-error-not-a-silent-no-op-since-v0842)).
+
+## When to use this
+
+- **Backfill / replay**: reprocess from a chosen checkpoint after a
+  downstream bug, without re-importing everything.
+- **Reset after a bad run**: a component wrote a corrupt checkpoint and keeps
+  skipping data until state is corrected.
+- **Seeding a dev branch**: `branch create` always starts with `state: {}`
+  (a fresh runtime state, not a copy of production's). Testing incremental
+  behaviour on a branch at all requires seeding.
+- **Migrating `processed_tags`/`query` -> `changed_since: adaptive`** in file
+  input mapping (Keboola is retiring the former; see the [changelog
+  announcement](https://changelog.keboola.com/deprecating-processed-tags-and-query-in-file-input-mapping/)).
+  This is the case that forced #593 -- see the dedicated section below.
+
+## Quick reference
+
+| Command | Purpose | Permission |
+|---------|---------|------------|
+| `config state-get` | Read root or row state | read |
+| `config state-set` | Write root or row state (guarded) | write |
+
+```bash
+kbagent --json config state-get --project ALIAS --component-id keboola.python-transformation-v2 --config-id 25344315
+kbagent --json config state-get --project ALIAS --component-id keboola.python-transformation-v2 --config-id 25344315 --row-id row1
+
+kbagent --json config state-set --project ALIAS --component-id keboola.python-transformation-v2 --config-id 25344315 \
+  --state '{"storage": {"input": {"files": [{"tags": ["import"], "lastImportId": "176200172"}]}}}' \
+  --branch 1234 --dry-run     # preview first, then re-run without --dry-run
+```
+
+## Root vs row state
+
+- **Root state** (`state-get`/`state-set` without `--row-id`) is the
+  configuration-level checkpoint. This is what `config detail --with-state`
+  shows as the top-level `state` key.
+- **Row state** (`--row-id ROW_ID`) is that row's own checkpoint, embedded in
+  the row object under `config detail`'s `rows[]` array.
+- **For row-based components the root state node is unused** -- if your
+  configuration has rows (per-table extractors, per-endpoint writers, ...),
+  read/write the *row* state, not the root. Writing to the root on a
+  row-based config is a no-op for the component's actual behaviour, even
+  though the write itself succeeds.
+- A missing/typo'd `--row-id` fails loudly (named in the error), it does not
+  silently return or write an empty `{}` -- that would repeat the exact class
+  of bug #593 is about.
+
+## Migration playbook: `processed_tags`/`query` -> `changed_since: adaptive`
+
+The API-verified trap: `changed_since: adaptive` with an **empty** state does
+not mean "start watching from now" -- it triggers a full reload of the
+component's entire file history (see
+[gotchas](gotchas.md#changed_since-adaptive-with-empty-state-reloads-the-entire-file-history-since-v0842)).
+Since a dev branch always starts with `state: {}`, validating this migration
+on a branch -- the safe place to test it -- reproduces the full reload every
+time unless you seed state first. This is the one manual step (issue #593)
+that used to require the Keboola UI; it is now fully scriptable.
+
+1. **Create an isolated branch** (see
+   [branch-workflow](branch-workflow.md)):
+
+   ```bash
+   kbagent --json branch create --project ALIAS --name "migrate-adaptive-25344315"
+   ```
+
+   This auto-activates the branch; subsequent commands default to it. Note
+   its `branch_id` for the explicit `--branch` flags below (state and job
+   commands accept it explicitly too, which is clearer in scripts).
+
+2. **Edit the file input mapping** to drop `processed_tags`/`query` and add
+   `"changedSince": "adaptive"`. Fetch fresh, change only the mapping, push
+   back (`config update`) or edit locally and `sync push` if the project is
+   under GitOps sync -- see [sync-workflow](sync-workflow.md) and the
+   [safe-write-workflow](safe-write-workflow.md) (fetch -> dry-run -> confirm
+   -> push).
+
+3. **Find the checkpoint to seed** (see below), then seed the branch's state
+   *before* the first run:
+
+   ```bash
+   kbagent --json config state-set --project ALIAS --component-id keboola.python-transformation-v2 \
+     --config-id 25344315 --branch <branch_id> \
+     --state '{"storage": {"input": {"files": [{"tags": ["import"], "lastImportId": "<checkpoint>"}]}}}' \
+     --dry-run   # verify the diff, then drop --dry-run to apply
+   ```
+
+   Seed root or row state depending on whether the config uses rows (see
+   above). This is the safer direction: an empty state is what triggers the
+   full reload, a seeded checkpoint is what avoids it.
+
+4. **Run the job on the branch** and let it complete:
+
+   ```bash
+   kbagent --json job run --project ALIAS --component-id keboola.python-transformation-v2 \
+     --config-id 25344315 --branch <branch_id> --wait
+   ```
+
+   Watch the run duration and file count -- a seeded checkpoint should
+   produce a run comparable to a normal incremental run (seconds, only new
+   files), not a full-history reload (minutes, thousands of files).
+
+5. **Verify the state advanced**:
+
+   ```bash
+   kbagent --json config state-get --project ALIAS --component-id keboola.python-transformation-v2 \
+     --config-id 25344315 --branch <branch_id>
+   ```
+
+   `lastImportId` (or the equivalent cursor) should have moved past your
+   seeded checkpoint to the newest file processed.
+
+6. **Merge when satisfied.** `branch merge` returns a Keboola UI URL for
+   manual review -- it does not merge automatically:
+
+   ```bash
+   kbagent --json branch merge --project ALIAS
+   ```
+
+   Branch merge propagates **configs only, not state** -- production's state
+   is untouched by the merge, so repeat the seed step (3) against production
+   with production's own checkpoint before the first production run under
+   the new mapping, if production also starts from a cleared/uncertain
+   state. If production already has a healthy `state` under the old mapping
+   shape, verify with `state-get` whether it needs reshaping for `adaptive`
+   before relying on it as-is.
+
+## Finding the right checkpoint
+
+`lastImportId` (or your component's equivalent cursor field) should point at
+a file the component has already durably processed, so the next run picks up
+only what comes after it. Two common sources:
+
+- **An already-healthy production state**: `config state-get` on the
+  production config (if one exists under the old mapping) may already carry
+  a comparable checkpoint you can carry over.
+- **The Storage Files listing**: `kbagent --json storage files --project ALIAS --tag TAG --limit N`
+  lists files with their ids and creation times sorted for inspection --
+  pick the id of the most recent file you know is already fully imported
+  downstream, and seed with that.
+
+Never guess a future or non-existent id "to be safe" -- an id past the true
+checkpoint skips real files; use `--dry-run` and `state-get` to confirm
+before committing to a value on a config that matters.
+
+## State document shape for file input mapping
+
+The shape mirrors the file input mapping's own `tags` selection criteria:
+the state you write should describe the same tag set the mapping filters on,
+and `tags` is an array, not a single string.
+
+**Treat the example below as structure, not as authoritative types.** Issue
+#593 verified the `lastImportId` checkpoint field itself, but a component's
+state document is component-defined -- whether an id serializes as a string
+or a number is not something to infer from a doc example. The reliable move
+is always the same: `state-get` an already-migrated config in the same
+project, and mirror exactly what comes back. Seeding a wrong shape does not
+error; it silently behaves like an empty state, which for `adaptive` means
+the full-history reload this whole workflow exists to avoid.
+
+```json
+{
+  "storage": {
+    "input": {
+      "files": [
+        {
+          "tags": ["import"],
+          "lastImportId": "176200172"
+        }
+      ]
+    }
+  }
+}
+```
+
+If your input mapping selects files by more than one tag set (multiple
+entries under `storage.input.files` in the configuration itself), the state
+document needs one matching entry per selection. When unsure, `state-get` a
+comparable already-migrated config in the same project as ground truth
+before writing.
+
+## When to use `--dry-run`
+
+Always, the first time, on any config that matters. `--dry-run` computes the
+current-vs-new diff without writing (same shape as `config update --dry-run`)
+and never prompts. A no-op write (new state identical to current) is
+detected and skipped even without `--dry-run` (`changed: false`, no API
+call) -- so re-running the seed step is safe and idempotent.
+
+`state-set` without `--dry-run` is a guarded write: it prompts for
+interactive confirmation unless `--yes` is passed or the command runs under
+`--json` (consistent with the rest of the `config` group -- `--json` skips
+the prompt without requiring `--yes`, since this is a `write`, not
+`destructive`, operation).
+
+## Anti-patterns
+
+- **`config update --set 'state.x=y'`** -- rejected since v0.84.2 (exit 2);
+  use `state-set` instead. See [gotchas](gotchas.md) for the full history.
+- **Seeding with an empty `{}`** to "reset and be safe" on an `adaptive`
+  file input mapping -- this is the *expensive* direction for this specific
+  mapping type (full reload), the inverse of the usual "clear state = safe
+  reset" assumption for other incremental strategies.
+- **Merging a branch and assuming production's state came along** -- branch
+  merge propagates configs, not runtime state; seed production separately.
+- **Writing root state on a row-based component** -- succeeds but has no
+  effect on the component's actual incremental behaviour; use `--row-id`.