diff --git a/.Pipelines/CI-AND-RELEASE-PIPELINES.md b/.Pipelines/CI-AND-RELEASE-PIPELINES.md index d8d18d00..11a9baeb 100644 --- a/.Pipelines/CI-AND-RELEASE-PIPELINES.md +++ b/.Pipelines/CI-AND-RELEASE-PIPELINES.md @@ -7,37 +7,44 @@ including what each pipeline does, when it runs, and how to trigger a release. ## Pipeline Files -| File | Purpose | -|------|---------| -| [`azure-pipelines.yml`](../azure-pipelines.yml) | PR gate and post-merge CI — calls the shared template with `runPublish: false` | -| [`pipeline-publish.yml`](pipeline-publish.yml) | Release pipeline — manually queued, builds and publishes to PyPI | -| [`template-pipeline-stages.yml`](template-pipeline-stages.yml) | Shared stages template — PreBuildCheck, Validate, and CI stages reused by both pipelines | -| [`credscan-exclusion.json`](credscan-exclusion.json) | CredScan suppression file for known test fixtures | +| File | ADO Pipeline | Purpose | +|------|-------------|---------| +| [`azure-pipelines.yml`](../azure-pipelines.yml) | [MSAL.Python-PR-OneBranch-Official (3064)](https://dev.azure.com/IdentityDivision/IDDP/_build?definitionId=3064) | PR gate, post-merge CI, and performance benchmarks — calls the shared template with `runPublish: false`; runs benchmarks on post-merge pushes to `dev` | +| [`pipeline-publish.yml`](pipeline-publish.yml) | [MSAL.Python-Publish (3067)](https://dev.azure.com/IdentityDivision/IDDP/_build?definitionId=3067) | Release pipeline — manually queued, builds and publishes to PyPI | +| [`template-pipeline-stages.yml`](template-pipeline-stages.yml) | — | Shared stages template — PreBuildCheck, Validate, UnitTests, and E2ETests stages reused by both pipelines | +| [`credscan-exclusion.json`](credscan-exclusion.json) | — | CredScan suppression file for known test fixtures | --- -## PR / CI Pipeline (`azure-pipelines.yml`) +## PR / CI Pipeline — [MSAL.Python-PR-OneBranch-Official (3064)](https://dev.azure.com/IdentityDivision/IDDP/_build?definitionId=3064) ### Triggers | Event | Branches | |-------|----------| -| Pull request opened / updated | all branches | -| Push / merge | `dev`, `azure-pipelines` | +| Pull request opened / updated | `dev` (PRs targeting `dev` only) | +| Push / merge | `dev` | | Scheduled | Daily at 11:45 PM Pacific, `dev` branch (only when there are new changes) | +Fast unit-test feedback for PRs targeting **other** branches (e.g. `release-x.y.z`) +is provided separately by the GitHub Actions workflow +[`.github/workflows/python-package.yml`](../.github/workflows/python-package.yml), +which runs the package build and unit tests on every PR. + ### Stages ``` -PreBuildCheck ─► CI +PreBuildCheck ─► UnitTests ─► E2ETests ─► Benchmark (post-merge to dev only) ``` -| Stage | What it does | -|-------|-------------| -| **PreBuildCheck** | Runs SDL security scans: PoliCheck (policy/offensive content), CredScan (leaked credentials), and PostAnalysis (breaks the build on findings) | -| **CI** | Runs the full test suite on Python 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14 | +| Stage | What it does | When it runs | +|-------|-------------|-------------| +| **PreBuildCheck** | Runs SDL security scans: PoliCheck (policy/offensive content), CredScan (leaked credentials), and PostAnalysis (breaks the build on findings) | Always | +| **UnitTests** | Runs the unit test suite on Python 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14 (no Key Vault required) | After PreBuildCheck | +| **E2ETests** | Fetches the MSID Lab certificate from Key Vault and runs `tests/test_e2e.py` + `tests/test_fmi_e2e.py` on the same Python matrix. On forked PRs the stage still runs, but the Key Vault tasks are skipped and the E2E tests self-skip (because `LAB_APP_CLIENT_CERT_PFX_PATH` is unset), so the stage reports green with all E2E tests marked Skipped in the Tests tab. | After UnitTests | +| **Benchmark** | Runs performance benchmarks on Python 3.9 and publishes `benchmark-results` artifact | Post-merge pushes to `dev` and manual runs only | -The Validate stage is **skipped** on PR/CI runs (it only applies to release builds). +The `Validate` stage is **skipped** on PR/CI runs (it only applies to release builds). > **SDL coverage:** The PreBuildCheck stage satisfies the OneBranch SDL requirement. > It runs on every PR, every merge to `dev`, and on the daily schedule — ensuring @@ -45,7 +52,7 @@ The Validate stage is **skipped** on PR/CI runs (it only applies to release buil --- -## Release Pipeline (`pipeline-publish.yml`) +## Release Pipeline — [MSAL.Python-Publish (3067)](https://dev.azure.com/IdentityDivision/IDDP/_build?definitionId=3067) ### Triggers @@ -62,18 +69,25 @@ with both parameters filled in. ### Stage Flow ``` -PreBuildCheck ─► Validate ─► CI ─► Build ─┬─► PublishMSALPython (publishTarget == 'test.pypi.org (Preview / RC)') - └─► PublishPyPI (publishTarget == 'pypi.org (ESRP Production)') +PreBuildCheck ─► Validate ─► UnitTests ─► E2ETests ─► Build ─┬─► PublishMSALPython (publishTarget == 'test.pypi.org (Preview / RC)') + └─► PublishPyPI (publishTarget == 'pypi.org (ESRP Production)') ``` | Stage | What it does | Condition | |-------|-------------|-----------| | **PreBuildCheck** | PoliCheck + CredScan scans | Always | | **Validate** | Asserts the `packageVersion` parameter matches `msal/sku.py __version__` | Always (release runs only) | -| **CI** | Full test matrix (Python 3.9–3.14) | After Validate passes | -| **Build** | Builds `sdist` and `wheel` via `python -m build`; publishes `python-dist` artifact | After CI passes | +| **UnitTests** | Unit test matrix (Python 3.9–3.14) | After Validate passes | +| **E2ETests** | E2E test matrix (Python 3.9–3.14) with MSID Lab cert from Key Vault | After UnitTests passes | +| **Build** | Builds `sdist` and `wheel` via `python -m build`; publishes `python-dist` artifact | After E2ETests passes | | **PublishMSALPython** | Uploads to test.pypi.org | `publishTarget == test.pypi.org (Preview / RC)` | -| **PublishPyPI** | Uploads to PyPI via ESRP; requires manual approval | `publishTarget == pypi.org (ESRP Production)` | +| **PublishPyPI** | Uploads to PyPI via ESRP (`EsrpRelease@12`); requires manual approval | `publishTarget == pypi.org (ESRP Production)` | + +> ⚠️ **TestPyPI publishing is currently a no-op.** The `MSAL-Test-Python-Upload` +> service connection has not yet been created (pending a test.pypi.org API +> token), so the `PublishMSALPython` stage prints a skip message rather than +> uploading. Until the SC exists, use the `pypi.org (ESRP Production)` path +> with an RC version (e.g. `1.36.0rc1`) for end-to-end validation. --- diff --git a/.Pipelines/pipeline-publish.yml b/.Pipelines/pipeline-publish.yml index 818dfddb..12133d72 100644 --- a/.Pipelines/pipeline-publish.yml +++ b/.Pipelines/pipeline-publish.yml @@ -6,7 +6,7 @@ # Publish targets: # test.pypi.org (Preview / RC) — preview releases via MSAL-Test-Python-Upload SC # (SC creation pending test.pypi.org API token) -# pypi.org (ESRP Production) — production releases via ESRP (EsrpRelease@9) using MSAL-ESRP-AME SC +# pypi.org (ESRP Production) — production releases via ESRP (EsrpRelease@12) using MSAL-ESRP-AME SC # # For pipeline documentation, see .Pipelines/CI-AND-RELEASE-PIPELINES.md. @@ -27,12 +27,12 @@ pr: none # Stage flow: # -# PreBuildCheck ─► Validate ─► CI ─► Build ─► PublishMSALPython (publishTarget == Preview) -# └─► PublishPyPI (publishTarget == ESRP Production) +# PreBuildCheck ─► Validate ─► UnitTests ─► E2ETests ─► Build ─► PublishMSALPython (publishTarget == Preview) +# └─► PublishPyPI (publishTarget == ESRP Production) stages: -# PreBuildCheck, Validate, and CI stages are defined in the shared template. +# PreBuildCheck, Validate, UnitTests, and E2ETests stages are defined in the shared template. - template: template-pipeline-stages.yml parameters: packageVersion: ${{ parameters.packageVersion }} @@ -43,8 +43,8 @@ stages: # ══════════════════════════════════════════════════════════════════════════════ - stage: Build displayName: 'Build package' - dependsOn: CI - condition: eq(dependencies.CI.result, 'Succeeded') + dependsOn: E2ETests + condition: eq(dependencies.E2ETests.result, 'Succeeded') jobs: - job: BuildDist displayName: 'Build sdist + wheel (Python 3.12)' @@ -130,10 +130,10 @@ stages: # ══════════════════════════════════════════════════════════════════════════════ # Stage 4b · Publish to PyPI (ESRP Production) -# Uses EsrpRelease@9 via the MSAL-ESRP-AME service connection. +# Uses EsrpRelease@12 via the MSAL-ESRP-AME service connection. # IMPORTANT: configure a required manual approval on this environment in # ADO → Pipelines → Environments → MSAL-Python-Release → Approvals and checks. -# IMPORTANT: EsrpRelease@9 requires a Windows agent. +# IMPORTANT: EsrpRelease@12 requires a Windows agent. # ══════════════════════════════════════════════════════════════════════════════ - stage: PublishPyPI displayName: 'Publish to PyPI (ESRP Production)' @@ -159,7 +159,7 @@ stages: artifactName: python-dist targetPath: $(Pipeline.Workspace)/python-dist - - task: EsrpRelease@9 + - task: EsrpRelease@12 displayName: 'Publish to PyPI via ESRP' inputs: connectedservicename: 'MSAL-ESRP-AME' diff --git a/.Pipelines/template-pipeline-stages.yml b/.Pipelines/template-pipeline-stages.yml index 972121c3..e6b68438 100644 --- a/.Pipelines/template-pipeline-stages.yml +++ b/.Pipelines/template-pipeline-stages.yml @@ -9,13 +9,13 @@ # Parameters: # packageVersion - Version to validate against msal/sku.py # Required when runPublish is true; unused otherwise. -# runPublish - When true: also runs the Validate stage before CI. -# When false (PR / merge builds): only PreBuildCheck + CI run. +# runPublish - When true: also runs the Validate stage before UnitTests. +# When false (PR / merge builds): only PreBuildCheck + tests run. # # Stage flow: # -# runPublish: true → PreBuildCheck ─► Validate ─► CI -# runPublish: false → PreBuildCheck ─► CI (Validate is skipped) +# runPublish: true → PreBuildCheck ─► Validate ─► UnitTests ─► E2ETests +# runPublish: false → PreBuildCheck ─► UnitTests ─► E2ETests (Validate is skipped) # # Build and Publish stages are defined in pipeline-publish.yml (not here), # so that the PR build never references PyPI service connections. @@ -68,6 +68,16 @@ stages: GdnBreakGdnToolCredScan: true GdnBreakGdnToolPoliCheck: true + - task: securedevelopmentteam.vss-secure-development-tools.build-task-publishsecurityanalysislogs.PublishSecurityAnalysisLogs@3 + displayName: 'Publish Security Analysis Logs (TSA)' + condition: succeededOrFailed() + inputs: + tsaConfigFile: '$(Build.SourcesDirectory)/.Pipelines/tsaConfig.json' + + - task: mspremier.PostBuildCleanup.PostBuildCleanup-task.PostBuildCleanup@3 + displayName: 'Clean agent directories' + condition: always() + # ══════════════════════════════════════════════════════════════════════════════ # Stage 1 · Validate — verify packageVersion matches msal/sku.py __version__ # Skipped when runPublish is false (PR / merge builds). @@ -106,12 +116,13 @@ stages: displayName: 'Verify version parameter matches msal/sku.py' # ══════════════════════════════════════════════════════════════════════════════ -# Stage 2 · CI — run the full test matrix across all supported Python versions. -# Always runs. Waits for Validate when runPublish is true; +# Stage 2 · UnitTests — run unit tests across all supported Python versions. +# No Key Vault or service connection needed. +# Waits for Validate when runPublish is true; # runs immediately when Validate is skipped (PR / merge builds). # ══════════════════════════════════════════════════════════════════════════════ -- stage: CI - displayName: 'Run tests' +- stage: UnitTests + displayName: 'Unit test' dependsOn: - PreBuildCheck - Validate @@ -121,31 +132,128 @@ stages: in(dependencies.Validate.result, 'Succeeded', 'Skipped') ) jobs: - - job: Test - displayName: 'Run unit tests' + - job: Pytest + displayName: 'pytest' pool: - vmImage: ubuntu-latest + vmImage: ubuntu-22.04 + timeoutInMinutes: 30 strategy: matrix: - Python39: - python.version: '3.9' - Python310: - python.version: '3.10' - Python311: - python.version: '3.11' - Python312: - python.version: '3.12' - Python313: - python.version: '3.13' - Python314: - python.version: '3.14' + Python39: { python.version: '3.9' } + Python310: { python.version: '3.10' } + Python311: { python.version: '3.11' } + Python312: { python.version: '3.12' } + Python313: { python.version: '3.13' } + Python314: { python.version: '3.14' } + maxParallel: 6 steps: - # Retrieve the MSID Lab certificate from Key Vault (via AuthSdkResourceManager SC). - # Matches the pattern used by MSAL.js (install-keyvault-secrets.yml) and MSAL Java. - # Skipped on forked PRs — service connections are not available to forks. - # E2E tests self-skip when LAB_APP_CLIENT_CERT_PFX_PATH is unset. + - task: UsePythonVersion@0 + displayName: 'Use Python $(python.version)' + inputs: + versionSpec: '$(python.version)' + + - bash: | + set -euo pipefail + python -m pip install --upgrade pip + pip install -r requirements.txt + pip install pytest pytest-azurepipelines pytest-timeout + displayName: 'Install Python dependencies' + + - bash: | + set -o pipefail + mkdir -p test-results + pytest -vv \ + --benchmark-skip \ + --timeout=120 \ + --junitxml=test-results/junit-unit.xml \ + --ignore=tests/test_e2e.py \ + --ignore=tests/test_e2e_manual.py \ + --ignore=tests/test_fmi_e2e.py \ + --deselect tests/test_cryptography.py::CryptographyTestCase::test_ceiling_should_be_latest_cryptography_version_plus_three \ + --deselect tests/test_cryptography.py::CryptographyTestCase::test_should_be_run_with_latest_version_of_cryptography \ + 2>&1 | tee test-results/pytest-unit.log + displayName: 'Run pytest (unit)' + env: + # Force unbuffered stdout so ADO logs stream in real time through the tee pipe. + PYTHONUNBUFFERED: '1' + + # Run cryptography version-gating tests separately as a warning-only check. + # These tests fail whenever a new `cryptography` release ships (signalling a + # required ceiling bump in setup.cfg). We deliberately swallow the non-zero + # exit code with `|| true` so the step always succeeds at the shell level — + # this keeps the stage result as 'Succeeded' (not 'SucceededWithIssues') so + # downstream publish gates can stay strict. Failures are still surfaced as + # JUnit test failures in the Tests tab via PublishTestResults below + # (failTaskOnFailedTests: false), giving maintainers visibility without + # blocking unrelated PRs or releases. + - bash: | + pytest -vv \ + --timeout=60 \ + tests/test_cryptography.py::CryptographyTestCase::test_ceiling_should_be_latest_cryptography_version_plus_three \ + tests/test_cryptography.py::CryptographyTestCase::test_should_be_run_with_latest_version_of_cryptography \ + --junitxml=test-results/junit-crypto-ceiling.xml \ + || true + displayName: 'Check cryptography ceiling (warning only)' + env: + PYTHONUNBUFFERED: '1' + + - task: PublishTestResults@2 + displayName: 'Publish unit test results' + condition: succeededOrFailed() + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'test-results/junit-unit.xml' + failTaskOnFailedTests: true + testRunTitle: 'Unit · Python $(python.version)' + + - task: PublishTestResults@2 + displayName: 'Publish cryptography ceiling results' + condition: succeededOrFailed() + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'test-results/junit-crypto-ceiling.xml' + failTaskOnFailedTests: false + testRunTitle: 'Cryptography ceiling check $(python.version)' + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 3 · E2ETests — runs only if unit tests pass. Fetches the MSID Lab +# certificate from Key Vault (mirrors MSAL.NET's +# build/template-install-keyvault-secrets.yaml). +# Fork behaviour: the stage still runs on forked PRs, but the +# Key Vault retrieval and certificate decoding steps are skipped +# via `ne(System.PullRequest.IsFork, 'True')`. The pytest step then +# self-skips each test because LAB_APP_CLIENT_CERT_PFX_PATH is unset +# (see tests/test_e2e.py). Result: green stage on forks with all +# E2E tests reported as Skipped in the Tests tab. +# ═══════════════════════════════════════════════════════════════════════════ +- stage: E2ETests + displayName: 'E2E tests' + dependsOn: UnitTests + + condition: eq(dependencies.UnitTests.result, 'Succeeded') + jobs: + - job: Pytest + displayName: 'pytest' + pool: + vmImage: ubuntu-22.04 + timeoutInMinutes: 60 + strategy: + matrix: + Python39: { python.version: '3.9' } + Python310: { python.version: '3.10' } + Python311: { python.version: '3.11' } + Python312: { python.version: '3.12' } + Python313: { python.version: '3.13' } + Python314: { python.version: '3.14' } + maxParallel: 6 + steps: + - task: UsePythonVersion@0 + displayName: 'Use Python $(python.version)' + inputs: + versionSpec: '$(python.version)' + - task: AzureKeyVault@2 - displayName: 'Retrieve lab certificate from Key Vault' + displayName: 'Fetch MSID Lab certificate from Key Vault' condition: ne(variables['System.PullRequest.IsFork'], 'True') inputs: azureSubscription: 'AuthSdkResourceManager' @@ -156,49 +264,48 @@ stages: - bash: | set -euo pipefail if [ -z "${LAB_AUTH_B64:-}" ]; then - echo "##vso[task.logissue type=error]LabAuth secret is empty or was not injected — Key Vault retrieval may have failed." + echo "##vso[task.logissue type=error]LabAuth secret is empty — Key Vault retrieval failed." exit 1 fi CERT_PATH="$(Agent.TempDirectory)/lab-auth.pfx" printf '%s' "$LAB_AUTH_B64" | base64 -d > "$CERT_PATH" echo "##vso[task.setvariable variable=LAB_APP_CLIENT_CERT_PFX_PATH]$CERT_PATH" echo "Lab cert written to: $CERT_PATH ($(wc -c < "$CERT_PATH") bytes)" - displayName: 'Write lab certificate to disk' + displayName: 'Decode lab certificate to PFX' condition: ne(variables['System.PullRequest.IsFork'], 'True') env: LAB_AUTH_B64: $(LabAuth) - - task: UsePythonVersion@0 - inputs: - versionSpec: '$(python.version)' - displayName: 'Set up Python' - - - script: | + - bash: | + set -euo pipefail python -m pip install --upgrade pip pip install -r requirements.txt - displayName: 'Install dependencies' + pip install pytest pytest-azurepipelines pytest-timeout + displayName: 'Install Python dependencies' - # Use bash: explicitly; set -o pipefail so that pytest failures aren't hidden by the pipe to tee. - # Without pipefail, tee exits 0 and the step can succeed even when tests fail. - # (set -o pipefail also works in script: steps, but bash: makes the shell choice explicit.) - bash: | - pip install pytest pytest-azurepipelines - mkdir -p test-results set -o pipefail - pytest -vv --junitxml=test-results/junit.xml 2>&1 | tee test-results/pytest.log - displayName: 'Run tests' + mkdir -p test-results + pytest -vv \ + --timeout=300 \ + --junitxml=test-results/junit-e2e.xml \ + tests/test_e2e.py tests/test_fmi_e2e.py \ + 2>&1 | tee test-results/pytest-e2e.log + displayName: 'Run pytest (E2E)' env: + # Force unbuffered stdout so ADO logs stream in real time through the tee pipe. + PYTHONUNBUFFERED: '1' LAB_APP_CLIENT_CERT_PFX_PATH: $(LAB_APP_CLIENT_CERT_PFX_PATH) - task: PublishTestResults@2 - displayName: 'Publish test results' + displayName: 'Publish E2E test results' condition: succeededOrFailed() inputs: testResultsFormat: 'JUnit' - testResultsFiles: 'test-results/junit.xml' + testResultsFiles: 'test-results/junit-e2e.xml' failTaskOnFailedTests: true - testRunTitle: 'Python $(python.version)' + testRunTitle: 'E2E · Python $(python.version)' - bash: rm -f "$(Agent.TempDirectory)/lab-auth.pfx" - displayName: 'Clean up lab certificate' + displayName: 'Remove lab certificate from agent' condition: always() diff --git a/.Pipelines/tsaConfig.json b/.Pipelines/tsaConfig.json new file mode 100644 index 00000000..f20b3111 --- /dev/null +++ b/.Pipelines/tsaConfig.json @@ -0,0 +1,17 @@ +{ + "codebaseName": "MSAL Python", + "notificationAliases": [ + "IdentityDevExDotnet@microsoft.com" + ], + "codebaseAdmins": [ + "EUROPE\\aadidagt" + ], + "instanceUrl": "https://identitydivision.visualstudio.com", + "projectName": "IDDP", + "areaPath": "IDDP\\DevEx-Client-SDK\\Python", + "iterationPath": "IDDP\\Unscheduled", + "tools": [ + "credscan", + "policheck" + ] +} diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml index a956b091..f34f7753 100644 --- a/.github/workflows/python-package.yml +++ b/.github/workflows/python-package.yml @@ -1,30 +1,57 @@ -# This workflow will install Python dependencies, run tests and lint with a variety of Python versions -# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions +# Build verification + unit tests for the msal Python package. +# +# This workflow runs on every PR (against any target branch) to give contributors +# fast feedback that the package still builds and that the unit tests pass across +# all supported Python versions. +# +# Post-merge validation on dev, E2E tests, benchmarks, SDL scans, and PyPI +# publishing are NOT run here. Those run in the ADO pipelines: +# - azure-pipelines.yml (PRs + pushes to dev: unit + E2E + SDL) +# - .Pipelines/pipeline-publish.yml (manual release to TestPyPI / PyPI) -name: CI/CD +name: Build and Unit Tests on: - push: pull_request: - branches: [ dev ] - - # This guards against unknown PR until a community member vet it and label it. - types: [ labeled ] + # No `branches` filter — run on PRs against any target branch. jobs: + build: + name: Build package (sdist + wheel) + permissions: + contents: read + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.12' + cache: 'pip' + - name: Install build tooling + run: | + python -m pip install --upgrade pip + python -m pip install build twine + - name: Build sdist and wheel + run: python -m build --sdist --wheel --outdir dist/ . + - name: Verify built artifacts + # `twine check` catches broken long_description / metadata that would fail PyPI upload. + run: twine check dist/* + - name: Upload built artifacts + uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + retention-days: 7 + ci: + name: Unit tests Python ${{ matrix.python-version }} permissions: contents: read - env: - # Fake a TRAVIS env so that the pre-existing test cases would behave like before - TRAVIS: true - LAB_APP_CLIENT_ID: ${{ secrets.LAB_APP_CLIENT_ID }} - LAB_APP_CLIENT_CERT_BASE64: ${{ secrets.LAB_APP_CLIENT_CERT_BASE64 }} - LAB_APP_CLIENT_CERT_PFX_PATH: lab_cert.pfx - # Derived from https://docs.github.com/en/actions/guides/building-and-testing-python#starting-with-the-python-workflow-template runs-on: ubuntu-22.04 strategy: + fail-fast: false matrix: python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', '3.14'] @@ -41,91 +68,14 @@ jobs: - name: Install dependencies run: | python -m pip install --upgrade pip - python -m pip install flake8 pytest - if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - - name: Populate lab cert.pfx - # https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#storing-base64-binary-blobs-as-secrets - run: echo $LAB_APP_CLIENT_CERT_BASE64 | base64 -d > $LAB_APP_CLIENT_CERT_PFX_PATH - - name: Test with pytest - run: pytest --benchmark-skip - - name: Lint with flake8 - run: | - # stop the build if there are Python syntax errors or undefined names - #flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics - # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide - #flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics - - cb: - # Benchmark only after the correctness has been tested by CI, - # and then run benchmark only once (sampling with only one Python version). - needs: ci - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - uses: actions/checkout@v4 - - name: Set up Python 3.9 - uses: actions/setup-python@v5 - with: - python-version: 3.9 - cache: 'pip' - - name: Install dependencies - run: | - python -m pip install --upgrade pip + python -m pip install pytest if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - - name: Setup an updatable cache for Performance Baselines - uses: actions/cache@v4 - with: - path: .perf.baseline - key: ${{ runner.os }}-performance-${{ hashFiles('tests/test_benchmark.py') }} - restore-keys: ${{ runner.os }}-performance- - - name: Run benchmark - run: pytest --benchmark-only --benchmark-json benchmark.json --log-cli-level INFO tests/test_benchmark.py - - name: Render benchmark result - uses: benchmark-action/github-action-benchmark@v1 - with: - tool: 'pytest' - output-file-path: benchmark.json - fail-on-alert: true - - name: Publish Gibhub Pages - run: git push origin gh-pages - cd: - needs: ci - # Note: github.event.pull_request.draft == false WON'T WORK in "if" statement, - # because the triggered event is a push, not a pull_request. - # This means each commit will trigger a release on TestPyPI. - # Those releases will only succeed when each push has a new version number: a1, a2, a3, etc. - if: | - github.event_name == 'push' && - ( - startsWith(github.ref, 'refs/tags') || - startsWith(github.ref, 'refs/heads/release-') - ) - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - name: Set up Python 3.9 - uses: actions/setup-python@v5 - with: - python-version: 3.9 - cache: 'pip' - - name: Build a package for release + - name: Run unit tests + # Skip benchmarks and E2E tests — those require lab credentials and run in ADO. run: | - python -m pip install build --user - python -m build --sdist --wheel --outdir dist/ . - - name: | - Publish to TestPyPI when pushing to release-* branch. - You better test with a1, a2, b1, b2 releases first. - uses: pypa/gh-action-pypi-publish@v1.13.0 - if: startsWith(github.ref, 'refs/heads/release-') - with: - user: __token__ - password: ${{ secrets.TEST_PYPI_API_TOKEN }} - repository_url: https://test.pypi.org/legacy/ - - name: Publish to PyPI when tagged - if: startsWith(github.ref, 'refs/tags') - uses: pypa/gh-action-pypi-publish@v1.13.0 - with: - user: __token__ - password: ${{ secrets.PYPI_API_TOKEN }} + pytest tests/ \ + --benchmark-skip \ + --ignore=tests/test_e2e.py \ + --ignore=tests/test_e2e_manual.py \ + --ignore=tests/test_fmi_e2e.py diff --git a/RELEASE_GUIDE.md b/RELEASE_GUIDE.md index 17abd14d..c9ec405c 100644 --- a/RELEASE_GUIDE.md +++ b/RELEASE_GUIDE.md @@ -1,179 +1,137 @@ # MSAL Python — Release Guide -This document provides step-by-step instructions for releasing a new version of `msal` to PyPI. +How to ship a new version of `msal` to PyPI. Everything happens in Azure DevOps +(no GitHub Releases, no Git-tag-triggered automation). --- -## Prerequisites +## Before you start — one-time prerequisites -- You have push access to the [AzureAD/microsoft-authentication-library-for-python](https://github.com/AzureAD/microsoft-authentication-library-for-python) repository. -- The following GitHub repository secrets are configured: - - `TEST_PYPI_API_TOKEN` — API token for [TestPyPI](https://test.pypi.org/) - - `PYPI_API_TOKEN` — API token for [PyPI](https://pypi.org/) +Confirm these are set up in ADO (`IdentityDivision` → `IDDP` project) — the +release will fail at runtime if any are missing: + +- [ ] Pipeline **MSAL.Python-Publish** (definition `3067`) exists +- [ ] Service connection **`MSAL-ESRP-AME`** exists and is authorized +- [ ] Environment **`MSAL-Python-Release`** has a **required manual approval** + configured under *Approvals and checks* +- [ ] Key Vault **`MSALVault`** contains cert **`MSAL-ESRP-Release-Signing`** + +> **Note on TestPyPI:** The TestPyPI publish path +> (`publishTarget = test.pypi.org (Preview / RC)`) is currently a **no-op** — +> the `MSAL-Test-Python-Upload` service connection has not been created yet, +> so that stage prints a skip message and uploads nothing. Until it's wired up, +> use an RC version (e.g. `1.36.0rc1`) on the production path for dry runs. --- -## Version Location +## Release in 4 steps -The package version is defined in a single file: +### 1. Bump the version on `dev` -``` -msal/sku.py → __version__ = "x.y.z" -``` +The package version lives in **one file**: [msal/sku.py](msal/sku.py). -`setup.cfg` reads it dynamically via `version = attr: msal.__version__`, so **no other file needs updating**. +```python +__version__ = "1.36.0" # final release +# or +__version__ = "1.36.0rc1" # RC / dry run +``` ---- +Open a PR, get it merged into `dev`. -## Branch Strategy +### 2. Cut the release branch -``` -dev (all development happens here) - │ - │── feature/fix PR → merged into dev - │ - ├──► release-1.35.0 (version branch, cut from dev when ready) - │ │ - │ ├── TestPyPI publish (automatic on push) - │ │ - │ ├── bug found? fix on dev, merge dev → release-1.35.0 - │ │ │ - │ │ └── TestPyPI re-publish (automatic) - │ │ - │ ├── tag 1.35.0 (via GitHub Release) → PyPI publish - │ │ - │ │ ── post-release hotfix needed? ── - │ │ - │ ├── fix on dev, merge dev → release-1.35.0 - │ │ │ - │ │ ├── bump sku.py to 1.35.1 - │ │ │ - │ │ ├── TestPyPI re-publish (automatic) - │ │ │ - │ │ └── tag 1.35.1 (via GitHub Release) → PyPI publish - │ │ - │ └── (repeat for further patches: 1.35.2, 1.35.3, ...) - │ - ├──► release-1.36.0 (next minor version, cut from dev) - │ │ - │ ├── TestPyPI publish - │ │ - │ ├── tag 1.36.0 → PyPI publish - │ │ - │ └── patches: merge dev → bump → tag 1.36.1, 1.36.2, ... - ... +```bash +git checkout dev && git pull +git checkout -b release-1.36.0 +git push origin release-1.36.0 ``` -- **`dev`** — All feature work, bug fixes, and PRs land here. -- **`release-x.y.z`** — Version branch cut from `dev` when ready to release. Used for final validation and TestPyPI testing. -- **Tags** — Created from the version branch via GitHub Releases to trigger production PyPI publish. - ---- +Pushing the branch does **not** publish anything — the pipeline is manual. -## Step-by-Step Release Process +### 3. Queue the publish pipeline -### 1. Complete All Work on `dev` +ADO → **IDDP** → **MSAL.Python-Publish (3067)** → **Run pipeline**: -- All features, fixes, and version bumps should be merged into `dev` via PRs. -- Ensure CI passes on `dev`. -- Update the version in `msal/sku.py` before cutting the release branch: - ```python - __version__ = "1.35.0" - ``` +| Field | Value | +|-------|-------| +| Branch | `release-1.36.0` | +| **Package version to publish** | `1.36.0` (must match `msal/sku.py` exactly) | +| **Publish target** | `pypi.org (ESRP Production)` | -### 2. Create a Version Branch from `dev` +The pipeline runs: -```bash -git checkout dev -git pull origin dev -git checkout -b release-1.35.0 -git push origin release-1.35.0 +``` +PreBuildCheck → Validate → UnitTests → E2ETests → Build → PublishPyPI ``` -This push triggers the CD pipeline: -- CI runs tests (must pass). -- CD publishes to **TestPyPI** automatically. -- Verify at: https://test.pypi.org/project/msal/ +### 4. Approve, then verify -### 3. Apply Patches (If Needed) +When `PublishPyPI` starts, it pauses for approval on the +`MSAL-Python-Release` environment. An approver clicks **Approve** in +ADO → Pipelines → Environments → MSAL-Python-Release. -If bugs are found during validation: +ESRP signs the artifact and uploads to PyPI. Verify: -1. Fix the bug on `dev` first (via a PR to `dev`). -2. Merge `dev` into the version branch: - ```bash - git checkout release-1.35.0 - git merge dev - git push origin release-1.35.0 - ``` -3. This triggers another TestPyPI publish. Bump the version to `1.35.1` if the previous version was already published. +- https://pypi.org/project/msal/ +- `pip install msal==1.36.0` -### 4. Create a GitHub Release (Production Publish) +--- -Once the version branch is validated: +## Dry run (before the real release) -1. Go to **GitHub → Releases → Create a new release**. -2. Click **"Choose a tag"** and type the version (e.g., `1.35.0`) — select **"Create new tag on publish"**. -3. Set **Target** to the `release-1.35.0` branch. -4. Set **Release title** to `1.35.0`. -5. Add release notes (changelog, breaking changes, etc.). -6. Click **"Publish release"**. +Recommended flow to exercise the pipeline end-to-end without burning a real +version number: -This creates a tag, which triggers the CD pipeline to publish to **PyPI**. +1. Set `msal/sku.py` to an RC version: `__version__ = "1.36.0rc1"` +2. Open a PR, merge to `dev` +3. Cut `release-1.36.0` from `dev` +4. Queue **MSAL.Python-Publish**: + - Package version: `1.36.0rc1` + - Publish target: `pypi.org (ESRP Production)` +5. Approve when prompted +6. Confirm `pip install msal==1.36.0rc1 --pre` works +7. Then bump to `1.36.0` and run the real release -Verify at: https://pypi.org/project/msal/ +RC versions are not installed by default by `pip install msal` (they require +`--pre`), so they're safe to push to production PyPI for verification. -### 5. Post-Release +--- -- Verify installation: `pip install msal==1.35.0` -- If the version on `dev` hasn't been bumped yet, open a PR to bump `msal/sku.py` to the next dev version (e.g., `1.36.0`). +## Hotfix (patch a released version) ---- +```bash +# Fix on dev first +git checkout dev +# ...PR, merge fix into dev... -## Hotfix Releases +# Merge into the existing release branch +git checkout release-1.36.0 +git merge dev -For urgent fixes on an already-released version: +# Bump patch version in msal/sku.py: 1.36.0 → 1.36.1 +git commit -am "bump to 1.36.1" +git push origin release-1.36.0 +``` -1. Fix the issue on `dev` (via PR). -2. Merge `dev` into the existing `release-x.y.z` branch. -3. Update `msal/sku.py` to the patch version (e.g., `1.35.0` → `1.35.1`). -4. Push the version branch (triggers TestPyPI). -5. Create a GitHub Release with tag `1.35.1` targeting `release-1.35.0`. +Then re-queue **MSAL.Python-Publish** with `packageVersion = 1.36.1` and +`pypi.org (ESRP Production)`. Approve. --- -## How the CI/CD Pipeline Works +## What changed from the old flow -| Job | Trigger | Purpose | -|-----|---------|---------| -| **ci** | Every push and labeled PR to `dev` | Runs tests on Python 3.8–3.14 | -| **cb** | After CI passes | Runs benchmarks | -| **cd** | Push to `release-*` branch or tag | Builds and publishes the package | +| Old (GitHub-Actions) | New (ADO) | +|---|---| +| Push to `release-*` auto-published to TestPyPI | Manual queue with `test.pypi.org` target (currently no-op) | +| Push a Git tag auto-published to PyPI | Manual queue with `pypi.org (ESRP Production)` target | +| `PYPI_API_TOKEN` GitHub secret | ESRP service connection `MSAL-ESRP-AME` | +| Anyone with tag-push could ship | Required approver on `MSAL-Python-Release` environment | -| Trigger | Target | -|---------|--------| -| Push to `release-*` branch | **TestPyPI** | -| Tag (created via GitHub Release) | **PyPI** (production) | +Git tags are still allowed for source-tracking, but they trigger nothing. --- -## Quick Reference +## Pipeline architecture -```bash -# 1. Ensure dev is ready, version bumped in msal/sku.py -# 2. Cut version branch -git checkout dev && git pull -git checkout -b release-1.35.0 -git push origin release-1.35.0 -# → TestPyPI publish happens automatically - -# 3. If patches needed: fix on dev, then merge into release branch -git checkout release-1.35.0 -git merge dev -git push origin release-1.35.0 - -# 4. Production release: create a GitHub Release -# → GitHub.com → Releases → New release -# → Tag: 1.35.0, Target: release-1.35.0 -# → PyPI publish happens automatically -``` +See [.Pipelines/CI-AND-RELEASE-PIPELINES.md](.Pipelines/CI-AND-RELEASE-PIPELINES.md) +for stage-by-stage detail, triggers, and the shared template structure. diff --git a/azure-pipelines.yml b/azure-pipelines.yml index b800165e..f155db7f 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -1,20 +1,22 @@ # PR gate and branch CI for the msal Python package. -# Runs on pushes to dev/azure-pipelines, on all pull requests, and on a daily schedule. -# Delegates all stages to .Pipelines/template-pipeline-stages.yml with -# runPublish: false — PreBuildCheck (SDL scans) + CI (test matrix) only. +# Runs on pushes to dev, on all PRs to dev, and on a daily schedule. +# Delegates test stages to .Pipelines/template-pipeline-stages.yml with +# runPublish: false — PreBuildCheck (SDL scans) + UnitTests + E2ETests only. trigger: -- dev -- azure-pipelines + branches: + include: + - dev pr: branches: include: - - '*' + - dev + drafts: false schedules: -- cron: '45 7 * * *' # 07:45 UTC daily (11:45 PM PST / 12:45 AM PDT) — matches legacy MSAL-Python-SDL-CI schedule - displayName: 'Daily SDL + CI (dev)' +- cron: '45 7 * * *' # 07:45 UTC daily (11:45 PM PST / 12:45 AM PDT) + displayName: 'Daily CI (dev)' branches: include: - dev @@ -24,3 +26,55 @@ stages: - template: .Pipelines/template-pipeline-stages.yml parameters: runPublish: false + +- stage: Benchmark + displayName: 'Run benchmarks' + dependsOn: E2ETests + # Only run on post-merge pushes to dev — not on PRs or scheduled runs. + condition: | + and( + succeeded('E2ETests'), + eq(variables['Build.SourceBranch'], 'refs/heads/dev'), + or( + eq(variables['Build.Reason'], 'IndividualCI'), + eq(variables['Build.Reason'], 'BatchedCI'), + eq(variables['Build.Reason'], 'Manual') + ) + ) + jobs: + - job: Benchmark + displayName: 'Performance benchmarks (Python 3.9)' + pool: + vmImage: ubuntu-latest + steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: '3.9' + displayName: 'Set up Python 3.9' + + - script: | + python -m pip install --upgrade pip + pip install pytest -r requirements.txt + displayName: 'Install dependencies' + + - task: Cache@2 + displayName: 'Restore performance baseline cache' + inputs: + key: 'perf-baseline | "$(Agent.OS)" | tests/test_benchmark.py' + path: .perf.baseline + + - bash: | + pytest --benchmark-only --benchmark-json benchmark.json --log-cli-level INFO tests/test_benchmark.py + displayName: 'Run benchmarks' + + - bash: | + [ -f benchmark.json ] && echo "##vso[task.setvariable variable=benchmarkFileExists]true" + displayName: 'Check benchmark output exists' + condition: succeededOrFailed() + + - task: PublishPipelineArtifact@1 + displayName: 'Publish benchmark results' + condition: and(succeededOrFailed(), eq(variables['benchmarkFileExists'], 'true')) + inputs: + targetPath: 'benchmark.json' + artifact: 'benchmark-results' diff --git a/msal/sku.py b/msal/sku.py index 19ff0138..8b1f1daa 100644 --- a/msal/sku.py +++ b/msal/sku.py @@ -2,5 +2,5 @@ """ # The __init__.py will import this. Not the other way around. -__version__ = "1.35.2rc1" +__version__ = "1.37.0rc1" SKU = "MSAL.Python"