Skip to content

DEVEX: detect silent common freeze + fail loudly + alert Slack - #161

Merged
pdodgen-revparts merged 3 commits into
mainfrom
DEVEX-php-common-bump-freeze-detect
Aug 10, 2026
Merged

pdodgen-revparts merged 3 commits into
mainfrom
DEVEX-php-common-bump-freeze-detect

Conversation

@pdodgen-revparts

Copy link
Copy Markdown
Contributor

Background

Fleet-wide 10-day silent regression 2026-07-31 → 2026-08-10:

Common v12.247 pinned `revolutionparts/ebay-fulfillment-api-php-client` to exact `1.0.3`. Apps whose composer.lock had `1.0.1` for that peer had `composer update revolutionparts/common` return "Nothing to modify in lock file" and exit 0, because composer's resolver correctly determined that upgrading common alone was unsatisfiable within the update whitelist. Nightly Common Update reported green every night while composer.lock stayed at v12.246.2 for 10 days on 7 repos.

The user-facing failure mode was invisible: no red X, no email, no Slack post. Only surfaced when a human happened to notice their app was on an old common.

Fix

Add a `Detect silent freeze` step between `Update Common` and `Commit Common Bump`. When composer update produces no lock change, compare the lock's common version against Packagist's latest via `composer show revolutionparts/common --latest --format=json`:

State Action
lock changed normal update path — commit + push
lock unchanged AND at latest legitimate no-op → exit 0
lock unchanged AND lock < latest SILENT FREEZE — dump `composer why-not` to job summary, exit 1, fire Slack (if configured)

Add optional `slack_webhook_url` secret. When set, failures also post to Slack. When not set, only the GH workflow failure signal fires (email, mobile, PR check).

Design vs the closed alternative

Considered making `composer update` more aggressive with `--with-all-dependencies` in #160 (now closed). Rejected because it papers over the real issue: upstream constraint choices that break the fleet should surface as failures we can respond to, not be silently worked around. The right forcing function is the workflow failure + Slack alert.

Coverage

Applies to every caller of `php-common-bump.yaml@main` — every RT participant's Nightly Common Update + Upgrade Common workflows pick it up automatically the next time they run.

Callers that pass `slack_webhook_url`

None currently. For those to get the Slack alert, each caller needs to add `slack_webhook_url: ${{ secrets.SLACK_WEBHOOK_URL }}` under `secrets:` in their Upgrade Common / Nightly Common Update workflow. That's a small opt-in sweep — happy to draft it as a follow-up PR set, but not required for this change to be useful (workflow failures still surface without Slack).

Companion PRs

  • encodium/common — unpin `ebay-fulfillment-api-php-client` from exact `1.0.3` back to `^1.0.3` (root-cause fix for the specific freeze that motivated this PR)
  • 7 per-repo unstick PRs (radmin / catalog_api / listings-url-service / payments / returns-api / rp_api / webstore) — manually push each stuck lock forward to peer 1.0.3 + common v12.255.3

Test plan

  • Merge
  • Confirm next Nightly Common Update on a repo already at latest common: freeze-check step logs "at latest", commit step logs "No changes to commit", workflow green (same as before)
  • Confirm next Nightly Common Update on a repo with a lock update available: normal commit + push path (same as before)
  • Once the seven unstick PRs merge, verify no stuck repos remain. Then, if a future common constraint change re-freezes any repo, expect the alert (red X + Slack)

## Background

Fleet-wide 10-day silent regression 2026-07-31 → 2026-08-10:

Common v12.247 pinned `revolutionparts/ebay-fulfillment-api-php-client`
to exact "1.0.3". Apps with 1.0.1 in their lock had `composer update
revolutionparts/common` return "Nothing to modify in lock file" and
exit 0 because composer's resolver correctly determined that upgrading
common alone was unsatisfiable within the update whitelist. Nightly
Common Update reported green every night while composer.lock stayed
at v12.246.2 for 10 days on 7 repos.

The user-facing failure mode was invisible: no red X, no email, no
Slack post. Only surfaced when a human happened to notice their app
was on an old common.

## Fix

Add a `Detect silent freeze` step between `Update Common` and
`Commit Common Bump`. When composer update produces no lock change,
compare the lock's common version against Packagist's latest via
`composer show revolutionparts/common --latest --format=json`:

- Same version → legitimate no-op, exit 0 (workflow succeeds)
- Different → SILENT FREEZE. Print `composer why-not` diagnostic to
  the job summary, exit 1 (workflow fails), fire Slack alert if a
  webhook was passed.

Add optional `slack_webhook_url` secret to the workflow_call inputs.
When set, the failure ALSO posts to Slack. When not set, only the
GH workflow failure signal fires (email/mobile/PR check).

## Coverage

Applies to every caller of php-common-bump.yaml — every RT participant's
Nightly Common Update + Upgrade Common workflows.

## Design decisions vs the alternative

Considered `composer update revolutionparts/common --with-all-dependencies`
in #160 (now closed). Rejected because it papers over
the real issue: upstream constraint choices that break the fleet
should surface as failures we can respond to, not be silently worked
around. The right forcing function is the workflow failure + Slack
alert.

## Companion PRs

- encodium/common: unpin `ebay-fulfillment-api-php-client: "1.0.3"`
  back to `"^1.0.3"` (the actual root-cause fix for THIS freeze)
- 7 per-repo unstick PRs (radmin, catalog_api, listings-url-service,
  payments, returns-api, rp_api, webstore): manually push each stuck
  lock forward to peer 1.0.3 + common v12.255.3
@cursor

cursor Bot commented Aug 10, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes fleet-wide reusable CI behavior: legitimate “already at latest” paths stay green, but repos stuck behind Packagist will start failing nightly bumps until locks are fixed—intentional but operationally noisy until callers opt into Slack.

Overview
Extends the reusable php-common-bump workflow so nightly/common-bump runs can’t report success when composer update revolutionparts/common leaves composer.lock unchanged but Packagist has a newer revolutionparts/common.

After Update Common, a new Detect silent freeze step skips when the lock changed; otherwise it compares the locked common version to composer show … --latest (JSON parsed via printf + jq). If lock < latest, it writes composer why-not and recovery steps to the job summary, sets frozen=true, and fails the job. An optional slack_webhook_url secret triggers a Post freeze alert to Slack step on that failure (incoming webhook mode, mirroring existing deploy notifications).

Callers without the webhook still get a red workflow; Slack is opt-in per caller.

Reviewed by Cursor Bugbot for commit 86d58e1. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fbf3b9d. Configure here.

Comment thread .github/workflows/php-common-bump.yaml
@pdodgen-revparts
pdodgen-revparts marked this pull request as ready for review August 10, 2026 21:00
@pdodgen-revparts
pdodgen-revparts requested a review from a team as a code owner August 10, 2026 21:00
…ze alert

Cursor Bugbot caught this on the initial PR: slackapi/slack-github-action@v1.24.0
defaults to Workflow Builder mode when SLACK_WEBHOOK_TYPE is unset,
silently no-op-ing incoming-webhook POSTs. Mirrors what sibling
slack-deploy-notification.yaml already does.

Without this the freeze alert would look fine (green ✓) but silently
drop the message — exactly the kind of invisible failure this whole
PR exists to prevent.
Local end-to-end test of the freeze-check step on radmin's stuck lock
(v12.189.0 vs Packagist v12.255.7) revealed that `echo "$SHOW_JSON" | jq`
mangles composer's JSON output. Common's composer.json ships PSR-4
autoload keys like `"RevolutionParts\\\\": "src/"` — four literal
backslashes → two backslash chars in JSON string escape → one literal
backslash in the parsed string. bash's `echo` collapses the escapes
in transit, corrupting the JSON before jq parses it:

  jq: parse error: Invalid numeric literal at line 34, column 38

Result before this fix: LATEST would come back empty every time, the
step would decide "at latest → legit no-op", and NO freeze would ever
be surfaced. The entire safeguard would be dead code.

Fix: use `printf '%s' "$SHOW_JSON" | jq` which preserves the bytes
verbatim. Verified locally on radmin — freeze correctly detected:
CURRENT=v12.189.0 LATEST=v12.255.7 → SILENT FREEZE, exit 1.

Confidence-in-safeguard: manual end-to-end run against a stuck repo
now trips the exit-1 branch as designed.
@pdodgen-revparts
pdodgen-revparts merged commit 6d6e73e into main Aug 10, 2026
1 of 2 checks passed
@pdodgen-revparts
pdodgen-revparts deleted the DEVEX-php-common-bump-freeze-detect branch August 10, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant