Skip to content

fix(ci): stabilize BrowserStack Test workflow - #1321

Merged
crisryantan merged 13 commits into
mainfrom
fix/browserstack-cbt-stability
Aug 14, 2026
Merged

fix(ci): stabilize BrowserStack Test workflow#1321
crisryantan merged 13 commits into
mainfrom
fix/browserstack-cbt-stability

Conversation

@crisryantan

@crisryantan crisryantan commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes the BrowserStack Test workflow reliably green on main. This is not caused by a recent product PR: master run 26452312354 (May 26) and PR run 31624825318 fail the same way.

Failure modes in the logs:

  1. Edge 15.15063 (EdgeHTML) deterministically failed ~47 of ~1050 tests. Dropped from the matrix (review: Edge 15–18 is ~0.02% share). Chromium Edge betas in the beta workflow are unchanged. No src/ changes — EdgeHTML workarounds were reverted after dropping the browser.
  2. Occasional 6-hour hangs until GitHub's default job timeout cancels the run.
  3. BrowserStack tunnel flake — concurrent runs hijacking tunnels (transport error, has not captured in 60000 ms, Rate Limit Exceeded).
  4. Firefox timing flakes on remaining browsers — Firefox 51 (legacy matrix) and Firefox 153 (beta) still needed a few test waits after Edge was gone.

Changes

  • test/cross-browser-testing/ — remove bs_edge_windows_15; share karma settings; pin sessions to BROWSERSTACK_LOCAL_IDENTIFIER; raise capture / no-activity / mocha timeouts.
  • .github/workflows/cross-browser-testing.ymltimeout-minutes: 30 and a shared concurrency group for push + pull_request.
  • test/src/config/utils.js — raise waitForCondition default timeout for slow BrowserStack VMs.
  • test/src/tests-core-sdk.js — wait for in-flight identify before retrying in the failed-config spec (Firefox 51).
  • test/src/tests-mparticle-instance-manager.ts — wait for all instance cookies before asserting (Firefox 153 beta).

Coverage tradeoff

Legacy EdgeHTML is no longer in the BrowserStack matrix (~0.02% share). Consent / cookie-sync coverage is not skipped on remaining browsers.

Test plan

  • npm run lint clean
  • npm run test:jest — 589/589
  • BrowserStack on this branch (run 31711843399) — 5344 SUCCESS in ~3.5 min, including Edge 15 (before it was dropped)
  • BrowserStack after dropping Edge 15 — remaining matrix green within 30 min
  • BrowserStack Beta — green after Firefox 153 cookie wait

@crisryantan
crisryantan requested a review from a team as a code owner August 13, 2026 14:29
github.ref differs for those events, so the previous group still allowed
two matrices for the same branch to contend for BrowserStack sessions.
@cursor

cursor Bot commented Aug 13, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Changes are limited to CI workflows, Karma test infrastructure, and test timing/assertions; no production SDK behavior is modified in the provided diff.

Overview
Stabilizes BrowserStack Test by capping jobs at 30 minutes, adding a concurrency group so push and PR on the same branch do not run two matrices at once, and applying the same timeout on the beta workflow.

Karma / BrowserStack: Duplicated config moves into browserstack.karma.shared.js (getSharedKarmaSettings). When BROWSERSTACK_LOCAL_IDENTIFIER is set, Karma uses the CI tunnel (startTunnel: false). Shared settings raise Mocha timeout to 20s; the main matrix also raises capture and browser no-activity timeouts. Edge 15 (EdgeHTML) is removed from the legacy matrix (~0.02% share).

Tests: waitForCondition default timeout increases from 200ms to 3s. Core SDK and multi-instance specs wait for the right async state (in-flight identify, per-instance cookies) instead of assuming fast VMs.

Reviewed by Cursor Bugbot for commit 54cef9e. Bugbot is set up for automated code reviews on this repo. Configure here.

@sonarqubecloud

Copy link
Copy Markdown

@crisryantan
crisryantan marked this pull request as draft August 13, 2026 14:32

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit baf751e. Configure here.

Comment thread test/cross-browser-testing/browserstack.karma.config.js Outdated
The dotted browserstack.localIdentifier key is ignored by the launcher,
so CI sessions were not pinned to the workflow tunnel.
@crisryantan
crisryantan marked this pull request as ready for review August 13, 2026 14:53

@rmi22186 rmi22186 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edge 15–18 marketshare is only .02%. let's just drop edge from our browserstack

Edge 15–18 is ~0.02% share. Keep Chromium Edge in the beta workflow.
@crisryantan

Copy link
Copy Markdown
Contributor Author

Dropped bs_edge_windows_15 from the BrowserStack matrix in 2e5f396 (Edge 15–18 ~0.02% share). Chromium Edge in the beta workflow is unchanged. The performCookieSync try/catch stays as production hardening for invalid pixel URLs.

@sonarqubecloud

Copy link
Copy Markdown

@crisryantan crisryantan changed the title fix: stabilize BrowserStack Test workflow on EdgeHTML 15 fix(ci): stabilize BrowserStack Test workflow Aug 14, 2026
@crisryantan
crisryantan merged commit 5451301 into main Aug 14, 2026
65 of 66 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants