✅ [RUM-17233] Add Shopify checkout views and actions e2e test by lierniel · Pull Request #4941 · DataDog/browser-sdk

lierniel · 2026-08-12T12:49:03Z

Motivation

RUM-17233: cover the Shopify integration with e2e tests. The dedicated browser-rum-shopify bundle (#4878) explicitly scoped e2e tests out; this closes that gap by exercising the full checkout flow against a real Shopify dev store.

Changes

Added e2e test framework support for Shopify: buildShopifyUrl/unlockShopifyStorePassword (dev store password gate), withShopifyApp()/shopifySetup (intercepts the CDN bundle request and injects RUM configuration into both the storefront's Theme Liquid snippet and the checkout's Custom Pixel sandbox), and a getShopifyStorePassword() secret accessor.
Added shopify.scenario.ts, covering the full flow: storefront → product page → add to cart → checkout → thank-you page → back to storefront. Asserts:
- A single session id shared across the Theme Liquid and Custom Pixel SDK instances
- 5 ordered view events with the correct URLs (checkout/thank-you matched via the same path regex used in browser-rum-shopify)
- Ordered click actions with the correct target and page for each
- No unexpected browser console errors, filtering known store-side noise
Runs serially with a 15s cooldown between invocations, and test.slow(), since this hits a live dev store that's sensitive to request volume (Cloudflare bot protection, checkout rate limiting) and the full flow runs close to the default test timeout.

Test instructions

To run locally:

SHOPIFY_STORE_PASSWORD=<dev store password> yarn test:e2e -g "shopify"

Or trigger the according pipeline on the gitlab: https://gitlab.ddbuild.io/DataDog/browser-sdk/-/pipelines

Chromium only (--disable-web-security is required to bypass the dev store's CSP/CORS restrictions). Runs against the live custom-pixel-e2e.myshopify.com dev store, so repeated back-to-back local runs may need a few minutes' gap to avoid tripping Cloudflare's verification wall.

Checklist

Tested locally
Tested on staging
Added unit tests for this change.
Added e2e/integration tests for this change.
Updated documentation and/or relevant AGENTS.md file

cit-pr-commenter-54b7da · 2026-08-12T12:49:35Z

Bundles Sizes Evolution

📦 Bundle Name	Base Size	Local Size	𝚫	𝚫%	Status
Rum	180.44 KiB	180.44 KiB	0 B	0.00%	✅
Rum Profiler	8.43 KiB	8.43 KiB	0 B	0.00%	✅
Rum Recorder	21.12 KiB	21.12 KiB	0 B	0.00%	✅
Logs	57.04 KiB	57.04 KiB	0 B	0.00%	✅
Rum Salesforce	N/A	138.47 KiB	N/A	N/A	N/A
Rum Slim	138.47 KiB	138.47 KiB	0 B	0.00%	✅
Worker	22.96 KiB	22.96 KiB	0 B	0.00%	✅
Rum Shopify	N/A	200.02 KiB	N/A	N/A	N/A
Rum-shopify Profiler	N/A	8.43 KiB	N/A	N/A	N/A
Rum-shopify Recorder	N/A	3.72 KiB	N/A	N/A	N/A

datadog-prod-us1-4 · 2026-08-12T12:50:33Z

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

DataDog/browser-sdk | unit
🧪 1 Test failed
console debug observable should notify debug from Chrome Headless 151.0.0.0 (Linux 0.0.0)
TypeError: Cannot read properties of undefined (reading &#39;args&#39;)
    at UserContext.&lt;anonymous&gt; (/go/src/github.com/DataDog/browser-sdk/packages/browser-core/src/domain/console/consoleObservable.spec.ts:36:54 &lt;- /tmp/_karma_webpack_24420/commons.js:388745:60)
    at &lt;Jasmine&gt;

❄️ 1 New flaky test detected

shopify checkout views and actions from shopify.scenario.ts

expect(received).toBe(expected) // Object.is equality

[chromium] › shopify.scenario.ts:61:13 › shopify checkout views and actions ──────────────────────

    Error: expect(received).toBe(expected) // Object.is equality

    Expected: 1
    Received: 2

      161 |         ...intakeRegistry.rumActionEvents.map((e) =&gt; e.session.id),
...

New test introduced in this PR is flaky.

View in Flaky Test Management

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: eb67446 | Docs | Datadog PR Page | Give us feedback!}

…fy bundle

BeltranBulbarellaDD

LGTM! Just some minor comments.

BeltranBulbarellaDD · 2026-08-13T09:37:36Z

+
+export function buildShopifyUrl(): string {
+  return SHOPIFY_STORE_URL
+}


Maybe since it's a constant we don't need this?

BeltranBulbarellaDD · 2026-08-13T09:39:58Z

+// Matches the CDN URL used by the store's Theme Liquid snippet and Custom Pixel for the main
+// bundle and its dynamically-imported chunks (e.g. the session replay recorder), served from
+// `https://www.datadoghq-browser-agent.com/<site>/v<major>/[chunks/]<name->]datadog-rum-shopify.js`
+const SHOPIFY_ASSET_URL_PATTERN =


Okay not related specifically to your PR but how about making a function to reuse with Salesforce? Since it's the same pattern except for datadog-rum-shopify vs datadog_rum_salesforce?

BeltranBulbarellaDD · 2026-08-14T12:04:11Z

+  // enough to trip Cloudflare's wall or Shopify's checkout rate limit
+  test.beforeAll(async () => {
+    await new Promise((resolve) => setTimeout(resolve, 15_000))
+  })


I think we can remove this. Since for now we only have 1 test and it's scheduled.

Actually we still could have a few retries happening sequentially even with scheduled setup. I would keep it just in case, I believe it doesn't hurt to have anyway

BeltranBulbarellaDD · 2026-08-14T12:06:25Z

+          const cart = (await response.json()) as { item_count: number }
+          return cart.item_count
+        })
+      await expect.poll(getCartItemCount, { timeout: 10000 }).toBeGreaterThan(0)


I think we can remove the timeout. At least I did locally and we are good.

This timeout is only for polling /cart.js with the interval (playwright docs). We're waiting for the amount of the items in cart in the response to be greater than 0 in order to avoid clicking on the "Checkout" button before items is actually landed to the cart. It might bypass without this polling a few times, but in a higher volumes of runs it could hit.
I was running it in the loop of 40 sequential runs when I hit it and after this fix it doesn't happen anymore, therefore I'd rather keep it - but let me know if you disagree and want to discuss!

Oh okay, makes sense, it's just that with the first promise and then the timeout seemed redundant at least in 1/2 runs I tried.

Oh, indeed, the timeout option may be redundant since it defaults to 30s - I'll remove it 👍

BeltranBulbarellaDD · 2026-08-14T12:07:49Z

+
+      // Card fields sync back to the parent form via an async postMessage; give it a moment before
+      // clicking "Pay now" so the click doesn't land before the form is submittable.
+      await page.waitForTimeout(500)


By default the page.getByRole wait's 30s. So I don't think this is necessary.

Yeah, but the thing is: the Pay now button is available immediately so page.getByRole will be executed right away without waiting for the element to appear (because it's already on the page from the start). But before clicking on Pay now I want to make sure that card fields form (which is on separate iframe) is synced with the parent form on the top frame.
Without that it might happen that regardless of the card fields was already filled, clicking on the Pay now button fails because card details are missing (the parent form haven't received the update from card fields form).
That's why I added this waitForTimeout(500) - it's enough time for the card details form to finish syncing with parent form and now clicking on the Pay now button never fails

BeltranBulbarellaDD · 2026-08-14T12:08:12Z

+
+      await page.getByRole('button', { name: 'Pay now' }).click()
+
+      await expect(page.getByRole('heading', { name: /thank you/i })).toBeVisible({ timeout: 30000 })


(I tested locally and it passed)

BeltranBulbarellaDD · 2026-08-14T12:14:51Z

+
+      const orderedViews = [...intakeRegistry.rumViewEvents]
+        .sort((a, b) => a.date - b.date)
+        .filter((event, index, events) => events.findIndex((e) => e.view.id === event.view.id) === index)


All of this is because we navigate back. Do we care about this assert? To go back to the page? Since we know the SDK worked correctly until now?

I'm not sure I got it properly, but we want to verify that the session stays the same after transition storefront page -> checkout page and vice verca. Here I'm sorting view events by time and filter them by uniq id to confirm that they came in correct order with right url

Oh yeah vice verca I understand. Well it makes sense.

BenoitZugmeyer · 2026-08-14T12:32:25Z

+// Shopify apps don't serve a locally-generated page body; this factory only intercepts the
+// bootstrap script request and injects the RUM configuration read by the store's Theme Liquid
+// snippet and Custom Pixel.
+export async function shopifySetup(options: SetupOptions, servers: Servers, page: Page): Promise<string> {


suggestion: those setup functions are templates to render an HTML page the e2e test will load. It seems like you are doing very different things here. What about moving this logic in shopify.scenario.ts?

I was following the example of the Salesforce, which has similar setup pattern

It seems like you are doing very different things here

Actually not that different - the part of rendering an HTML page for e2e test is injecting the local sdk version and passing the init options for it - and this is exactly what I'm doing here. This method doesn't render the html - it's rendered by predefined Shopify dev-store - but I still need to override the sdk bundle path and pass the init config for it. Therefore I believe it's the right place to do this, it would look overly cumbersome in shopify.scenario.ts - but let me know if you disagree, I'm open to the discussion :)

BenoitZugmeyer · 2026-08-14T12:36:53Z

+  })
+
+  // Flow: add to cart -> checkout -> thank you -> back to storefront
+  createTest('shopify checkout views and actions')


nitpick: this test is huge. Is there any way to split it, so it's easier to maintain and understand what's failing?

Yeah, but the reason behind putting everything in one test is the Claudflare bot wall we were discussing on Wednesday. I wanted to avoid parallel runs as much as possible - so I've created one big test to cover everything. It's true that with this approach it's harder to maintain and understand what is failing, but it's a trade of for avoiding multiple runs from happening at the same time to bypass Claudflare.
However I believe we could soften the assertions with expect.soft() so test would keep running even if some assertions fail to better understand that's really broken, wdyt?

BenoitZugmeyer · 2026-08-14T12:39:56Z

suggestion: this file is not just about shopify urls. Maybe rename it to shopify.ts or shopifyUtils.ts?

lierniel added 2 commits August 12, 2026 00:39

✅ Add Shopify e2e test framework support

d160743

✅ Add Shopify checkout views and actions e2e test

fd9496c

lierniel added 2 commits August 12, 2026 19:06

🐛 fix(shopify-e2e): intercepted recorder chunk requests for the shopi…

7c81df8

…fy bundle

👷 chore(shopify-e2e): moved shopify e2e test to a daily schedule

ef6cdb5

lierniel marked this pull request as ready for review August 12, 2026 21:20

lierniel requested a review from a team as a code owner August 12, 2026 21:20

sbarrio requested a review from thomas-lebeau August 13, 2026 07:03

BeltranBulbarellaDD reviewed Aug 14, 2026

View reviewed changes

BenoitZugmeyer reviewed Aug 14, 2026

View reviewed changes

🐛 fix(shopify-e2e): addressed PR review feedback

eb67446


		await page.getByRole('button', { name: 'Pay now' }).click()

		await expect(page.getByRole('heading', { name: /thank you/i })).toBeVisible({ timeout: 30000 })

Uh oh!

Conversation

lierniel commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Test instructions

Checklist

Uh oh!

cit-pr-commenter-54b7da Bot commented Aug 12, 2026

Bundles Sizes Evolution

Uh oh!

datadog-prod-us1-4 Bot commented Aug 12, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

BeltranBulbarellaDD left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BeltranBulbarellaDD Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lierniel Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lierniel commented Aug 12, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Aug 12, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

BeltranBulbarellaDD Aug 14, 2026 •

edited

Loading

lierniel Aug 14, 2026 •

edited

Loading