[Project Automation] Create High level tracker workflow + readme + helper script #288
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,268 @@ | ||
| name: RAPIDS High Level Tracking | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| PROJECT_ID: | ||
| description: "Project ID" | ||
| default: "PVT_kwDOAp2shc4ArK9w" | ||
| type: string | ||
|
|
||
| # ------------- Field IDs ------------- | ||
| WEEK_FIELD_ID: | ||
| description: "Week Field ID" | ||
| default: "PVTIF_lADOAp2shc4ArK9wzgiTHg8" | ||
| type: string | ||
| RELEASE_FIELD_ID: | ||
| description: "Release Field ID" | ||
| default: "PVTF_lADOAp2shc4ArK9wzgpEYKw" | ||
| type: string | ||
| STAGE_FIELD_ID: | ||
| description: "Stage Field ID" | ||
| default: "PVTSSF_lADOAp2shc4ArK9wzgiTH8g" | ||
| type: string | ||
| START_DATE_FIELD_ID: | ||
| description: "Start Date Field ID" | ||
| default: "PVTF_lADOAp2shc4ArK9wzgiTHmQ" | ||
| type: string | ||
| END_DATE_FIELD_ID: | ||
| description: "End Date Field ID" | ||
| default: "PVTF_lADOAp2shc4ArK9wzgiTHmU" | ||
| type: string | ||
|
|
||
| # ------------- Option IDs ------------- | ||
| # Stage options | ||
| STAGE_PLANNED_NAME: | ||
| description: "📓 Planned option ID" | ||
| default: "📓 Planned" | ||
| type: string | ||
| STAGE_DEVELOPMENT_NAME: | ||
| description: "💻 Development option ID" | ||
| default: "💻 Development" | ||
| type: string | ||
| STAGE_BURNDOWN_NAME: | ||
| description: "🔥 Burndown option ID" | ||
| default: "🔥 Burndown" | ||
| type: string | ||
| STAGE_CODE_FREEZE_NAME: | ||
| description: "❄ Code Freeze option ID" | ||
| default: "❄ Code Freeze" | ||
| type: string | ||
| STAGE_HOTFIX_NAME: | ||
| description: "🩹 Hotfix option ID" | ||
| default: "🩹 Hotfix" | ||
| type: string | ||
|
|
||
| secrets: | ||
| ADD_TO_PROJECT_GITHUB_TOKEN: | ||
| description: "Project Access Token" | ||
| required: true | ||
|
|
||
| jobs: | ||
| process-branch-name: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| release: ${{ steps.process-branch-name.outputs.branch-name }} | ||
| steps: | ||
| - name: Extract branch name | ||
| id: process-branch-name | ||
| run: | | ||
| branch=${{ github.event.pull_request.base.ref }} | ||
| release=${branch#branch-} | ||
| echo "release=$release" >> "$GITHUB_OUTPUT" | ||
|
|
||
| get-stage-field: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| stage-id: ${{ steps.get-stage-field.outputs.stage-id }} | ||
| needs: | ||
| - process-branch-name | ||
| steps: | ||
| # Add cache step for releases.json | ||
| # Pinned to the 4.2 commit SHA | ||
| - name: Cache releases.json | ||
| id: cache-releases | ||
| uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c30a57 | ||
| with: | ||
| path: /tmp/releases.json | ||
| # Cache key based on date - refresh daily to ensure we get updates | ||
| key: ${{ runner.os }}-releases-json-${{ github.run_id }}-${{ github.run_number }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-releases-json- | ||
|
|
||
| - name: Get stage field ID | ||
| id: get-stage-field | ||
| run: | | ||
| # Use the "nightly" info from this JSON: | ||
| # https://raw.githubusercontent.com/rapidsai/docs/refs/heads/main/_data/releases.json | ||
| # to determine the stage for the new release | ||
|
|
||
| # Check if cache exists, download only if needed | ||
| if [ ! -f "/tmp/releases.json" ] || [ "$(find /tmp/releases.json -mtime +1 -print)" ]; then | ||
| echo "Cache miss or expired. Downloading releases.json..." | ||
| curl -s https://raw.githubusercontent.com/rapidsai/docs/main/_data/releases.json -o /tmp/releases.json | ||
|
|
||
| # Validate the downloaded JSON | ||
| if ! jq empty /tmp/releases.json 2>/dev/null; then | ||
| echo "Error: Downloaded file is not valid JSON" | ||
| rm -f /tmp/releases.json | ||
| exit 1 | ||
| fi | ||
| else | ||
| echo "Using cached releases.json" | ||
| fi | ||
|
|
||
| # Use the cached file | ||
| RELEASES_JSON=$(cat /tmp/releases.json) | ||
|
|
||
| # Extract current repo name from GitHub context | ||
| REPO_NAME=$(echo "$GITHUB_REPOSITORY" | cut -d '/' -f 2) | ||
| echo "Repository: \"$REPO_NAME\"" | ||
|
|
||
| # Check if repo is in cudf_dev group | ||
| CUDF_DEV_REPOS=("cuDF" "RMM" "rapids-cmake" "raft" "dask-cuda" "ucx-py") | ||
| IS_CUDF_DEV="false" | ||
| for repo in "${CUDF_DEV_REPOS[@]}"; do | ||
| if [[ "$REPO_NAME" == "$repo" ]]; then | ||
| IS_CUDF_DEV="true" | ||
| break | ||
| fi | ||
| done | ||
| echo "Is cudf_dev group: \"$IS_CUDF_DEV\"" | ||
|
|
||
| # Get the release from branch name | ||
| RELEASE="${{ needs.process-branch-name.outputs.release }}" | ||
| echo "Release: $RELEASE" | ||
|
|
||
| # Get current date in YYYY-MM-DD format | ||
| CURRENT_DATE=$(date +%Y-%m-%d) | ||
| echo "Current date: $CURRENT_DATE" | ||
|
|
||
| # Extract dates from the releases.json for the current release | ||
| if [[ "$IS_CUDF_DEV" == "true" ]]; then | ||
| PLANNING_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].cudf_dev.planning.start') | ||
| DEV_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].cudf_dev.development.start') | ||
| BURNDOWN_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].cudf_dev.burndown.start') | ||
| FREEZE_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].cudf_dev.freeze.start') | ||
| RELEASE_DATE=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].date') | ||
| else | ||
| PLANNING_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].planning.start') | ||
| DEV_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].development.start') | ||
| BURNDOWN_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].burndown.start') | ||
| FREEZE_START=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].freeze.start') | ||
| RELEASE_DATE=$(echo "$RELEASES_JSON" | jq -r --arg rel "$RELEASE" '.[$rel].date') | ||
| fi | ||
|
|
||
| echo "Planning start: $PLANNING_START" | ||
| echo "Development start: $DEV_START" | ||
| echo "Burndown start: $BURNDOWN_START" | ||
| echo "Freeze start: $FREEZE_START" | ||
| echo "Release date: $RELEASE_DATE" | ||
|
|
||
| # Determine stage based on current date | ||
| STAGE_ID="${{ inputs.STAGE_PLANNED_NAME }}" | ||
|
|
||
| if [[ "$CURRENT_DATE" < "$DEV_START" ]]; then | ||
| STAGE_ID="${{ inputs.STAGE_PLANNED_NAME }}" | ||
| echo "Stage: Planned" | ||
| elif [[ "$CURRENT_DATE" < "$BURNDOWN_START" ]]; then | ||
| STAGE_ID="${{ inputs.STAGE_DEVELOPMENT_NAME }}" | ||
| echo "Stage: Development" | ||
| elif [[ "$CURRENT_DATE" < "$FREEZE_START" ]]; then | ||
| STAGE_ID="${{ inputs.STAGE_BURNDOWN_NAME }}" | ||
| echo "Stage: Burndown" | ||
| elif [[ "$CURRENT_DATE" < "$RELEASE_DATE" ]]; then | ||
| STAGE_ID="${{ inputs.STAGE_CODE_FREEZE_NAME }}" | ||
| echo "Stage: Code Freeze" | ||
| else | ||
| STAGE_ID="${{ inputs.STAGE_HOTFIX_NAME }}" | ||
| echo "Stage: Hotfix" | ||
|
Comment on lines
+165
to
+179
Member
There was a problem hiding this comment. We should move as many of the That goes for this block and any other lines in this workflow.
Contributor
Author
There was a problem hiding this comment. Yeah fair, if these stay as variables vs going hard-coded per James' comment above I'll update. I don't think this is a vector for script injection though because the only way these variables get set is inside the action (they aren't affected by a user defined field in a workflow-triggering PR). If a bad actor attempted to set the values in their own branch, it wouldn't have an effect unless we merged it. One of the frustrating parts of these workflows is that testing is challenging because they need to be merged into the default branch to be functional, but it does provide protection against script injection as a benefit in many cases! |
||
| fi | ||
|
|
||
| # Set output | ||
| echo "stage-id=$STAGE_ID" >> "$GITHUB_OUTPUT" | ||
|
|
||
| get-project-id: | ||
| uses: ./.github/workflows/project-get-item-id.yaml | ||
| secrets: inherit | ||
| permissions: | ||
| contents: read | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
|
|
||
| update-release-field: | ||
| uses: ./.github/workflows/project-set-text-date-numeric-field.yaml | ||
| needs: | ||
| - get-project-id | ||
| - process-branch-name | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| FIELD_ID: ${{ inputs.RELEASE_FIELD_ID }} | ||
| FIELD_TYPE: text | ||
| SET_VALUE: ${{ needs.process-branch-name.outputs.release }} | ||
| ITEM_PROJECT_ID: "${{ needs.get-project-id.outputs.ITEM_PROJECT_ID }}" | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
| UPDATE_LINKED_ISSUES: true | ||
| secrets: inherit | ||
|
|
||
| update-stage-field: | ||
| uses: ./.github/workflows/project-get-set-single-select-field.yaml | ||
| needs: | ||
| - get-project-id | ||
| - process-branch-name | ||
| - get-stage-field | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| SINGLE_SELECT_FIELD_ID: ${{ inputs.STAGE_FIELD_ID }} | ||
| SINGLE_SELECT_FIELD_NAME: "Stage" | ||
| SINGLE_SELECT_OPTION_VALUE: ${{ needs.get-stage-field.outputs.stage-id }} | ||
| ITEM_PROJECT_ID: "${{ needs.get-project-id.outputs.ITEM_PROJECT_ID }}" | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
| UPDATE_ITEM: true | ||
| UPDATE_LINKED_ISSUES: true | ||
| secrets: inherit | ||
|
|
||
| set-opened-date-field: | ||
| uses: ./.github/workflows/project-set-text-date-numeric-field.yaml | ||
| if: github.event.action == 'opened' || github.event.action == 'workflow_dispatch' | ||
| needs: | ||
| - get-project-id | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| FIELD_ID: ${{ inputs.START_DATE_FIELD_ID }} | ||
| FIELD_TYPE: date | ||
| SET_VALUE: ${{ github.event.pull_request.created_at }} | ||
| ITEM_PROJECT_ID: "${{ needs.get-project-id.outputs.ITEM_PROJECT_ID }}" | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
| UPDATE_LINKED_ISSUES: true | ||
| secrets: inherit | ||
|
|
||
| set-closed-date-field: | ||
| uses: ./.github/workflows/project-set-text-date-numeric-field.yaml | ||
| if: github.event.action == 'closed' || github.event.action == 'workflow_dispatch' | ||
| needs: | ||
| - get-project-id | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| FIELD_ID: ${{ inputs.END_DATE_FIELD_ID }} | ||
| FIELD_TYPE: date | ||
| SET_VALUE: ${{ github.event.pull_request.closed_at }} | ||
| ITEM_PROJECT_ID: "${{ needs.get-project-id.outputs.ITEM_PROJECT_ID }}" | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
| UPDATE_LINKED_ISSUES: true | ||
| secrets: inherit | ||
|
|
||
| update-week-field: | ||
| uses: ./.github/workflows/project-get-set-iteration-field.yaml | ||
| needs: | ||
| - get-project-id | ||
| with: | ||
| PROJECT_ID: ${{ inputs.PROJECT_ID }} | ||
| ITERATION_FIELD_ID: ${{ inputs.WEEK_FIELD_ID }} | ||
| ITERATION_FIELD_NAME: "Week" | ||
| ITEM_PROJECT_ID: "${{ needs.get-project-id.outputs.ITEM_PROJECT_ID }}" | ||
| ITEM_NODE_ID: "${{ github.event.pull_request.node_id }}" | ||
| UPDATE_ITEM: true | ||
| UPDATE_LINKED_ISSUES: true | ||
| secrets: inherit | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General comment for all of these
default:values with what look like identifiers from a service.... what are they? Where did the values come from? Are they intended to be overridden?If every user of this is expected to provide these values at the site where this is called, then I think that:
required: trueIf these values are intended to be used literally and never overridden, then I think that:
env.context (https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#env-context)If these values could be used literally but might be overridden, then I think that:
required: trueThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the identifiers
They are graphQL
nodeIDs; they are static values associated with GitHub projects. Unfortunately, there's no clean way of dealing with these; either we hard-code them to store them, or we re-query their values using logic shown in the helper python script in docs every time we run them. While they look like private keys etc, they're totally public and anyone can query those values.On defaults:
If this setup isn't something we're happy with, then I'd go with the hard-code in env. Goal 1 is more important than goal 2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok but how? Let's imagine you're not available and this breaks complaining about a project with this ID not existing... how would I go find the appropriate values?
The other thing I'm confused about... if it's ok to hard-code a single project's ID, then why is this workflow getting checked into
shared-workflows(which contains workflows that are intended to be re-used in many places), instead of in whatever repo the project board is associated with?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I see now that you have docs on how to get these values. Great!
https://github.com/jarmak-nv/shared-workflows/blob/high-level-tracker/docs/project-automation-readme.md#getting-project-and-field-ids
It'd still be helpful to have comments in these files indicated where these specific defaults came from. That's information I'd need to know to update these, or to choose whether or not to override those defaults in calls to these workflows from other repos.