Say what configfailurepolicy actually does

[juherr]

The -configfailurepolicy row in the command-line table promises:

Whether TestNG should continue to execute the remaining tests in the suite or skip them if an @before* method fails.

TestNG does that for exactly one of the four configuration levels. Measured against master:

Failing method	skip	continue
@BeforeSuite	test skipped	test skipped
@BeforeTest	test skipped	test runs
@BeforeClass	test skipped	test skipped
@BeforeMethod	test skipped	test skipped

The shipped DTD has always had the accurate framing — "whether to continue attempting Before/After Class/Methods after they've failed once or just skip remaining" — so this rewrites the row around that, and spells out what each policy actually invalidates: continue narrows a configuration failure to the failing instance (@BeforeClass) or the failing invocation (@BeforeMethod) rather than poisoning the whole class, a failed @BeforeTest invalidates no instance at all so those test methods run, and a @BeforeSuite failure stops the suite either way. The failure is reported in every case, so continue never turns a broken setup into a green run.

The gap between this row and the runner is the sole basis of testng-team/testng#2731, which is still open. Companion PR on the code side, adding a characterization test for the whole matrix and the same wording in the javadoc: testng-team/testng#3426.

Verified with ./mvnw clean package; the rendered cell was checked in target/html/running_testng.html and target/html/documentation.html.

Summary by CodeRabbit

• Documentation
  • Expanded -configfailurepolicy guidance to clarify how configuration failures affect test execution.
  • Documented behavior across suite, test, class, and method setup stages, including skipped tests, invalidated scopes, suite termination, and failure reporting.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 05004922-ae51-4e3d-87e9-b1f4f9fb7224

📥 Commits

Reviewing files that changed from the base of the PR and between c59d463 and 3d52203.

📒 Files selected for processing (1)

• src/main/asciidoc/docs/running_testng.adoc

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The -configfailurepolicy documentation now explains continue behavior for configuration failures across multiple configuration annotations, including scope invalidation, skipped methods, suite termination, and failure reporting.

Changes

Configuration Failure Policy

Layer / File(s)	Summary
Document continue semantics src/main/asciidoc/docs/running_testng.adoc	The documentation describes how continue handles failures in @BeforeClass, @BeforeMethod, @BeforeTest, and @BeforeSuite. It also documents skipped methods, affected scopes, suite termination, and failure reporting.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 3d522

This PR only clarifies the documented behavior of the existing configuration-failure policy without changing runtime behavior. No actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: documenting the actual behavior of configfailurepolicy.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch juherr/clarify-configfailurepolicy

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}