Skip to content

Remove the usage file - #810

Draft
janvanicek wants to merge 1 commit into
mainfrom
jv-PAT-2006-remove-usage-file
Draft

Remove the usage file#810
janvanicek wants to merge 1 commit into
mainfrom
jv-PAT-2006-remove-usage-file

Conversation

@janvanicek

@janvanicek janvanicek commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Context

Linear: PAT-2006

The usage file has never worked. A component could report its own consumption by writing out/usage.json; job-runner's UsageFile read it, validated it, and handed it to Client::addJobUsage() — a method with an empty body (// todo implement this) since it was introduced on 2019-07-04 (af3033b). Verified at the version pinned into job-runner's production build (keboola/job-queue-internal-api-php-client 25.5.0). job-runner was the only caller, and nothing subclasses Client.

So usageData is [] on every queue v2 job (100% of jobs in the last 90 days — 23,010,147). Nothing consumes it either: zero hits for usageData/usage_data in telemetry-billing, billing-api, telemetry-billing-gcp or connection, and the telemetry kbc_job extract has no usage column.

UsageFileTest mocked addJobUsage and asserted it was called, so it stayed green against a method that does nothing — that is why this survived seven years.

Only one component in the org writes a usage file: keboola.ex-google-analytics-v4. Its reports have been discarded all along, so nothing changes for it.

Deployment order for this batch

  1. keboola/docker-bundleRunner::run() loses its fifth parameter
  2. keboola/job-runner — then composer update keboola/dockerbundle
  3. keboola/job-queue — removes addJobUsage() from the client. The standalone keboola/job-queue-internal-api-php-client is synced from the monorepo, so it picks this up automatically and then needs a new major tag.
  4. keboola/developers-docs + keboola/connection-docs — last, so the docs do not get ahead of the deployed behaviour

What this PR does

Removes the whole read path: UsageFileInterface, NullUsageFile, Configuration\Usage and its adapter, the (unreferenced) Resources/schemas/usage.json, the TestUsageFile double, the two usage tests in RunnerTest, and the $usageFile parameter threaded through Runner::run(), runRow(), runComponent() and runImages(). Five now-stale phpstan-baseline.neon entries go with them.

Verified: composer phpstan (analyses src and Tests, so all ~70 Runner::run() call sites are argument-checked) and composer phpcs both clean. The full test suite needs live Storage API/ECR credentials and was not run.

Release Notes

Justification, description
out/usage.json was read and validated on every main-image run and then thrown away, because addJobUsage() is an empty method. This removes the dead path rather than leaving a parameter that can only ever be a no-op. It also removes a dind/no-dind divergence: the k8sContainers runtime never had a usage-file counterpart.

Plans for Customer Communication
None needed. No customer-visible feature is being withdrawn — the feature never did anything. The documentation that advertised it is removed in the paired developers-docs and connection-docs PRs.

Impact Analysis
BC break for library consumers: Runner::run() loses its fifth parameter. job-runner is the only consumer of keboola/dockerbundle (runner-sync-api has been DinD-free since sync actions moved to pods) and is updated in the paired PR.

One behaviour change: a malformed out/usage.json no longer fails an otherwise successful job. storeUsage() ran in a finally block, so a bad usage file surfaced as an application error — the feature's only observable effect. Jobs that used to fail this way will now succeed, which matches how the same job already behaves on k8sContainers. Only keboola.ex-google-analytics-v4 writes a usage file at all, and it writes a valid one.

Deployment Plan
Merge first in the batch (see order above), then bump the lock in job-runner.

Rollback Plan
Revert the commit and revert the paired job-runner PR. No data migration, no state to unwind.

Post-Release Support Plan
Nothing to watch specifically. If a component starts failing right after the job-runner release, this change can only make a previously-failing usage-file job pass, not the reverse.

`out/usage.json` was read, validated and then handed to
`Client::addJobUsage()`, which has had an empty body since 2019 — the payload was
always discarded, on every runtime.

Removes the whole read path: `UsageFileInterface`, `NullUsageFile`,
`Configuration\Usage` and its adapter, the (unreferenced) usage JSON schema, the
`TestUsageFile` double, and the `$usageFile` parameter threaded through
`Runner::run()`, `runRow()`, `runComponent()` and `runImages()`.

BC break: `Runner::run()` loses its fifth parameter.

Behaviour change: a malformed `out/usage.json` no longer fails an otherwise
successful job. That validation was the feature's only observable effect, and it
had no counterpart on the k8sContainers runtime, so this also removes a
dind/no-dind divergence.
@linear-code

linear-code Bot commented Aug 10, 2026

Copy link
Copy Markdown

PAT-2006

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant