Skip to content

GHA-213 Add sandbox integration tests for Jira actions - #116

Closed
nils-werner-sonarsource wants to merge 3 commits into
masterfrom
nw/add-test-fixtures-for-jira
Closed

nils-werner-sonarsource wants to merge 3 commits into
masterfrom
nw/add-test-fixtures-for-jira

Conversation

@nils-werner-sonarsource

Copy link
Copy Markdown
Contributor

Summary

  • Replaces fake integration test jobs (which never ran real assertions) with real sandbox integration tests using scripts/jira-sandbox/ fixtures
  • Tests use Vault credentials to create real Jira state, call the action under test, assert non-empty outputs, and always clean up

Test plan

  • All unit-tests jobs pass
  • sandbox-integration-test jobs: setup creates sandbox state, action produces expected outputs, cleanup always runs

🤖 Generated with Claude Code

nils-werner-sonarsource and others added 2 commits March 10, 2026 21:36
Add scripts/jira-sandbox/ with Python helpers to run a full end-to-end
integration test cycle against the Jira sandbox:
- setup.py: creates a version, Feature/Bug/Maintenance issues, and a REL
  release ticket; writes state to test_state.json
- verify.py: runs 7 lifecycle verifications (release version, create next
  version, release notes, integration ticket, status transition)
- cleanup.py: deletes all created objects and removes test_state.json
- run_all.py: orchestrator that always runs cleanup even on failure
- config.py / jira_client.py: shared constants and authenticated client

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… create-jira-version

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Add sandbox integration tests for Jira actions GHA-213 Add sandbox integration tests for Jira actions Mar 10, 2026
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Mar 10, 2026 •

Copy link
Copy Markdown

GHA-213

Generate the version name inline instead of using setup.py, which
requires project admin rights the CI Jira user doesn't have. The action
fetches its own Vault credentials internally for the create operation;
cleanup reuses the same Vault path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sonarqube-agent

sonarqube-agent AI commented Mar 10, 2026 •

Copy link
Copy Markdown
Contributor

Remediation Agent Summary 📊

🤖 To review: The 3 issues found require manual fixes.
Issues requiring manual fix (3)

QualityIssue
Maintainability
🔴 High
Define a constant instead of duplicating this literal "Version exists" 3 times.

Why is this an issue?

Why is this an issue?

Duplicated string literals make the process of refactoring complex and error-prone, as any change would need to be propagated on all occurrences.

Exceptions

No issue will be raised on:

  • duplicated string in decorators
  • strings with less than 5 characters
  • strings with only letters, numbers and underscores


Maintainability
🟠 Medium
Add replacement fields or use a normal string instead of an f-string.

Why is this an issue?

Why is this an issue?

A format string is a string that contains placeholders, usually represented by special characters such as "%s" or "{}", depending on the technology in use. These placeholders are replaced by values when the string is printed or logged. Thus, it is required that a string is valid and arguments match replacement fields in this string.

This applies to the % operator, the str.format method, and loggers from the logging module. Internally, the latter use the %-formatting. The only difference is that they will log an error instead of raising an exception when the provided arguments are invalid.

Formatted string literals (also called "f-strings"; available since Python 3.6) are generally simpler to use, and any syntax mistake will cause a failure at compile time. However, it is easy to forget curly braces, which will not lead to any detectable errors.

This rule raises an issue when:

  • A string formatted with % will not return the expected text because some arguments are unused.
  • A string formatted with str.format will not return the expected string because some arguments are unused.
  • An "f-string" doesn’t contain any replacement field, which probably means some curly braces are missing.
  • Loggers will log an error because their message is not formatted properly.

Rule S2275 covers cases where formatting a string will raise an exception.

Resources


Maintainability
🟠 Medium
Add replacement fields or use a normal string instead of an f-string.

Why is this an issue?

Why is this an issue?

A format string is a string that contains placeholders, usually represented by special characters such as "%s" or "{}", depending on the technology in use. These placeholders are replaced by values when the string is printed or logged. Thus, it is required that a string is valid and arguments match replacement fields in this string.

This applies to the % operator, the str.format method, and loggers from the logging module. Internally, the latter use the %-formatting. The only difference is that they will log an error instead of raising an exception when the provided arguments are invalid.

Formatted string literals (also called "f-strings"; available since Python 3.6) are generally simpler to use, and any syntax mistake will cause a failure at compile time. However, it is easy to forget curly braces, which will not lead to any detectable errors.

This rule raises an issue when:

  • A string formatted with % will not return the expected text because some arguments are unused.
  • A string formatted with str.format will not return the expected string because some arguments are unused.
  • An "f-string" doesn’t contain any replacement field, which probably means some curly braces are missing.
  • Loggers will log an error because their message is not formatted properly.

Rule S2275 covers cases where formatting a string will raise an exception.

Resources


Note

Help us improve the Agent!
Have a suggestion or found an issue? Share your feedback here.

@sonarqubecloud

Copy link
Copy Markdown

SonarQube reviewer guide

Review in SonarQube

Summary: Refactored integration testing approach for Jira actions—replacing mock tests with real sandbox E2E tests that execute actual action logic against a live Jira instance, plus support infrastructure (setup/verify/cleanup orchestration).

Review Focus:

  • The GitHub Actions workflow change: migration from multiple isolated mock tests to a single, unified sandbox integration test that leverages Vault authentication and performs real cleanup
  • New Python test helper scripts: ensure they correctly import and call action modules, handle state persistence robustly, and provide clear error reporting
  • Risk: Vault integration and real Jira API calls introduce external dependencies; verify credentials and cleanup logic are bulletproof to avoid leaving test fixtures behind

Start review at: .github/workflows/test-create-jira-version.yml (lines 41–104). This is where the behavioral shift is most critical—replacing multiple mock scenarios with a single real integration test that uses Vault to retrieve credentials, creates actual Jira objects, verifies outputs, and cleans up. Understanding this workflow change is essential before evaluating the supporting Python scripts.

💬 Please send your feedback

Quality Gate Failed Quality Gate failed

Failed conditions
4 New issues

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants