Skip to content

feat: add wp-env, PHPUnit and Playwright testing setup with Copilot support - #532

Merged
Soare-Robert-Daniel merged 12 commits into
developmentfrom
feat/agentic-testing-setup
Jul 24, 2026
Merged

feat: add wp-env, PHPUnit and Playwright testing setup with Copilot support#532
Soare-Robert-Daniel merged 12 commits into
developmentfrom
feat/agentic-testing-setup

Conversation

@Alexia-Soare

@Alexia-Soare Alexia-Soare commented Jul 23, 2026

Copy link
Copy Markdown

The plugin had no runnable local test environment — PHPUnit needed a hand-built MySQL/SVN setup and there was no E2E coverage at all, which made agent-assisted development blind (issue #530). This ports otter-blocks' setup: one wp-env instance serves development, PHPUnit, and Playwright, and a Copilot setup workflow boots coding agents into that environment ready to run both suites.

What changed

  • wp-env environmentyarn run env:start reuses a running instance or picks a free port and pins it in the gitignored .wp-env.override.json (the Playwright config reads the same file). An afterStart script provisions permalinks and the plugin's default settings so the classic settings screens are directly testable. PHPUnit runs inside the container pinned to a separate wptests_ table prefix — without it the suite install would wipe the development site, which shares the database.

  • PHPUnit suites — the admin-ajax API (subscribe, contact, reset settings, select page, wizard steps, newsletter, subscribers export/empty, notices) is covered at the handler seams; access control (role bypass, exclude list by slug/IP, bot detection), the public helpers (GA-code sanitizer, capability filters, form fields, shortcodes), the page-template integration, the Retry-After backtime, and the v1.8 → v2 settings migration each get their own suite.

  • Playwright E2E suite — user journeys across the whole surface: the maintenance lifecycle (visitor 503, admin/wp-login unaffected, the wp-admin warning notice), access rules (excluded URLs, bot bypass both ways, feeds and REST staying reachable, Retry-After and robots meta), Design-tab customization and reset, the countdown/social/contact/GDPR/Google Analytics modules, a subscriber's full journey (client-side validation, subscribing, admin count, CSV export as a real download, emptying the list), and a subscriber-role login that bypasses maintenance only while the frontend-role setting allows it. Specs run with workers: 1 since maintenance mode is site-wide state; tests/e2e/utils.js encodes the headless-browser quirks documented in AGENTS.md.

  • CI and Copilot — a new Test E2E workflow runs the Playwright suite on pull requests, and copilot-setup-steps.yml prepares the Copilot coding agent's environment with dependencies installed and wp-env already running. The PHPUnit job moves to PHP 7.4 (the current WordPress test suite refuses 7.2). AGENTS.md documents the commands, environment rules, and testing practices.

  • Verify toolchain and PHPStan — the composer dev tools move to current majors (WPCS 3.4, installer ^1, PHPUnit ^9.6, sniffer platform at PHP 7.4; codeinwp/phpcs-ruleset dropped since it pinned WPCS below 3), the handful of violations the newer sniffs surfaced are fixed, and PHPStan runs at level 6 with WordPress stubs — matching otter-blocks — with phpstan-baseline.neon capturing the 110 pre-existing findings so CI only fails on new ones. Stubs for the optional integrations (Otter, Elementor, nginx-helper) and dynamic constants live in tests/static-analysis-stubs/. A PHPStan job joins the Test PHP workflow, and the lint job moves off PHP 7.2/composer 2.1, which cannot install the new stack.

  • Node and lockfile hygiene — every yarn install now runs with --ignore-engines (the legacy eslint toolchain caps Node below 20 while the new test tooling needs 18+), the zip/deploy workflows move from Node 14 to 20, package-lock.json is ignored by the repository (an npm install rewrites the tracked yarn.lock into a format yarn 1 cannot parse, which would break the deploy workflow), and the release zip now excludes AGENTS.md and the new dev-only files via .distignore.

Note

The E2E environment pins the classic maintenance screen (wpmm_new_look=0): the modules settings UI only renders in that mode, and the block-based wizard flow needs Otter Blocks — it overlaps PR #524's scope and is intentionally out of this suite. AJAX guard paths written as plain die() are also uncovered by design (they would terminate the PHPUnit process), and the PHP 7.4 bump in test-php.yml also exists in PR #524 — an identical one-line conflict to resolve at merge.

Testing workflow

flowchart LR
    A[Contributor] --> B[New:<br/>yarn run env:start]:::added
    B --> C[New:<br/>single wp-env site,<br/>free port pinned]:::added
    C --> D[New:<br/>PHPUnit in container,<br/>wptests_ tables]:::added
    C --> E[New:<br/>Playwright E2E,<br/>serial specs]:::added
    F[Pull request] --> G[New:<br/>Test E2E workflow]:::added
    G --> E
    H[Copilot coding agent] --> I[New:<br/>copilot-setup-steps]:::added
    I --> B

    classDef added fill:#1a7f37,color:#fff,stroke:#116329,stroke-width:3px
Loading

QA

  1. With Docker running, install and start the environment:

    composer install && yarn install --frozen-lockfile --ignore-engines
    npx playwright install chromium
    npm run test:unit:php

    Expect: the wp-env site starts (port printed, default 8888) and the PHPUnit suite finishes green without any local MySQL/SVN setup.

  2. Run npm run test:e2e, then open the site in a browser at the printed port and log in (admin/password).

    Expect: the suite finishes green, and the site afterwards is intact — maintenance mode off, homepage public, plugin settings reachable under the LightStart menu.

  3. Run the PHPUnit suite twice in a row (npm run test:unit:php:base), then reload the site.

    Expect: the development site is untouched — the suite installs into its own wptests_ tables instead of reinstalling over the site.

  4. In the Actions tab, run the "Copilot Setup Steps" workflow manually (workflow_dispatch).

    Expect: the job finishes green, ending with the wp-env environment started.

  5. Build a release artifact with bash bin/dist.sh and inspect the zip.

    Expect: no tests/, .wp-env.json, AGENTS.md, or workflow files inside.

🤖 Generated with Claude Code

Alexia-Soare and others added 5 commits July 23, 2026 11:35
…test scripts

One wp-env instance (Docker) serves development, PHPUnit, and E2E.
bin/wp-env-up.js reuses a running instance or pins a free port in the
gitignored .wp-env.override.json; bin/wp-env-setup.sh provisions
permalinks and the plugin defaults after start. PHPUnit runs inside the
container with WORDPRESS_TABLE_PREFIX=wptests_ so the suite install
cannot wipe the development site, which shares the database.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exercises wpmm_add_subscriber, wpmm_send_contact, wpmm_reset_settings,
wpmm_select_page, wpmm_skip_wizard, wpmm_skip_insert_template,
wpmm_change_template_category, wpmm_toggle_gutenberg, wpmm_subscribe,
wpmm_subscribers_empty_list, wpmm_subscribers_export and
wpmm_dismiss_notices through _handleAjax, asserting on options, post
meta, the subscribers table and the mock mailer.

The suite's die handler is replaced with one that throws an Error-based
signal because several handlers wrap wp_send_json_*() in
catch ( Exception ), which would swallow the suite's exception and
answer a second time. Guard paths using a plain die() stay uncovered:
they would terminate the PHPUnit process.

The asset-suffix assertion in the generic test now accounts for
SCRIPT_DEBUG, which wp-env enables.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bscribe flow

Specs run serially (workers: 1) because maintenance mode is site-wide
state. Covered: visitors get the 503 maintenance page while admins and
wp-login stay reachable, disabling restores the site, and a visitor can
subscribe from the maintenance page through the real admin-ajax call.

The helpers in tests/e2e/utils.js encode the non-obvious constraints:
headless Chromium stops producing animation frames after any settings
save (form POST + redirect), so interactions after a save use
force: true with explicit state assertions; the hash-driven settings
tabs survive same-document navigations, so tabs are activated
explicitly; and the save round-trip is detected by its ?updated=1
redirect response because WordPress strips the query from the URL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
copilot-setup-steps.yml boots the Copilot coding agent into a checkout
with dependencies installed and the wp-env instance already running, so
both test suites work out of the box. AGENTS.md documents the new
commands, the wp-env instance rules, the E2E constraints, and the TDD
practices for this repository.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
yarn.lock is the tracked lockfile (deploy runs yarn install
--frozen-lockfile); an npm install creates package-lock.json and also
rewrites yarn.lock into a format yarn 1 cannot parse, so the npm
lockfile must never reach the repository.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Alexia-Soare Alexia-Soare added the pr-checklist-skip Allow this Pull Request to skip checklist. label Jul 23, 2026
@pirate-bot pirate-bot added the pr-checklist-complete The Pull Request checklist is complete. (automatic label) label Jul 23, 2026
The legacy eslint toolchain caps its node engines below current node
versions while the new test tooling requires node 18+, so every yarn
install now runs with --ignore-engines and the zip/deploy workflows move
from node 14 to 20. The PHPUnit job moves to PHP 7.4, required by the
latest WordPress test suite (same fix as #524; trivial conflict).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pirate-bot

pirate-bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Plugin build for 24d3ad4 is ready 🛎️!

@Alexia-Soare Alexia-Soare linked an issue Jul 23, 2026 that may be closed by this pull request
Alexia-Soare and others added 3 commits July 23, 2026 12:36
…les end to end

New unit suites: check_user_role (role bypass and the frontend_role
setting), check_exclude (default and custom slugs, visitor IPs, trailing
comments), check_search_bots (bot bypass on/off, regular browsers), the
GA-code sanitizer, capability filters, settings-form hidden fields, and
wpmm_get_option unslashing. New E2E specs: excluded URLs stay reachable
while maintenance mode is active, and search bots see the site when bot
bypass is enabled.

The ajax suite's hooks moved from the class-level fixture into set_up():
the WordPress suite snapshots hook state once per process and restores
it after every test, so class-fixture hooks evaporate after the first
test whenever another test class runs first. The newsletter HTTP mocks
are scoped to the subscribe route for the same reason — the restored
_maybe_update_* hooks make admin_init hit wordpress.org otherwise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…emantics

New journeys: custom title/heading reach the maintenance page and the
design reset button restores defaults; the countdown, social (with
configured profile links), contact, and GDPR consent modules render for
visitors; a real subscriber logs in through wp-login and bypasses
maintenance mode only while the frontend-role setting allows it; search
bots are blocked with bypass off and admitted with it on; feeds and the
REST API stay reachable during maintenance; and the 503 response
carries Retry-After plus the configured robots meta tag.

The roles spec uses a per-run username and sweeps leftovers first, so a
previously failed run cannot make user creation collide.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… admin journeys

Unit: the plugin page template shows in the templates dropdown and swaps
in for pages that use it, Retry-After follows the countdown settings,
the Google Analytics snippet renders only when enabled and strips
markup from the code, activating over v1.8 options migrates them into
the v2 settings, and wpmm_do_shortcode expands shortcodes.

E2E: an admin sees the subscriber count, exports the list as a real CSV
download, and empties it in place; client-side validation blocks an
invalid subscribe email before any request; a configured GA code
reaches the visitor-facing markup; and the wp-admin warning notice
appears while maintenance mode is active and disappears with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Soare-Robert-Daniel

Copy link
Copy Markdown
Contributor

@Alexia-Soare let's update the dev composer tools that verify.

Also, let's add PHPStan along with the baseline for the current issues (check how it is done in Otter & PPOM).

Alexia-Soare and others added 3 commits July 23, 2026 15:34
…baseline

Composer dev tools move to current majors: WPCS 3.4 (with the sniffer
platform at PHP 7.4), dealerdirect installer ^1, PHPUnit ^9.6, and the
codeinwp/phpcs-ruleset package is dropped — it pinned WPCS below 3 and
only supplied the unused phpcbf-fix-exit-0 wrapper. The loose-comparison
exclusion moves to its WPCS 3 sniff name so the ruleset keeps its
intent, the remaining new violations are fixed (formatting via phpcbf,
reserved-keyword parameter renames, one missing method visibility, one
wrong docblock type), and lint no longer crashes on PHP 8.4 hosts.

PHPStan runs at level 6 with WordPress stubs, matching otter-blocks:
phpstan-baseline.neon captures the 110 pre-existing findings so CI only
fails on new ones, tests/static-analysis-stubs/ declares the optional
integrations (Otter, Elementor, nginx-helper) and dynamic constants,
and a PHPStan job joins the Test PHP workflow. composer run phpstan /
phpstan:generate:baseline are documented in AGENTS.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dependency lock is resolved for the PHP 7.4 platform and the new
sniffer installer needs composer-plugin-api 2.2, so the pinned composer
2.1 and PHP 7.2 cannot install it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The renames were part of the toolchain commit but were lost when a
static-analysis probe in the same file was reverted wholesale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Soare-Robert-Daniel
Soare-Robert-Daniel merged commit 617cb0d into development Jul 24, 2026
9 checks passed
@Soare-Robert-Daniel
Soare-Robert-Daniel deleted the feat/agentic-testing-setup branch July 24, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-checklist-complete The Pull Request checklist is complete. (automatic label) pr-checklist-skip Allow this Pull Request to skip checklist.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Dev] Add testing setup and Copilot

3 participants