rapids-s3-path is currently the source of truth for computing proper paths for CI workflow artifacts.
The problem with rapids-s3-path is that it only returns S3 URIs (e.g. s3://rapids-downloads/ci/cudf/...).
There are a lot of different areas where we call rapids-s3-path and then replace s3://rapids-downloads/ with https://downloads.rapids.ai/:
|
s3_dl_path=$(echo "${s3_dl_path}" | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|") |
|
ARTIFACTS_URL=${browsable_url/s3:\/\/${RAPIDS_DOWNLOADS_BUCKET}\//https:\/\/downloads.rapids.ai\/} |
|
ARTIFACTS_URL=$(rapids-s3-path | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|") |
To prevent this manual replacement from being necessary, we should add a rapids-http-path script that computes the https://downloads.rapids.ai path.
The scripts should ultimately be refactored to work like this:
_rapids-path - a new script that's called by rapids-s3-path and rapids-http-path to generate the URI path. This script is prefixed with an underscore (_) to denote that it's not intended by be used publicly (only by other gha-tools scripts).
rapids-s3-path - should be refactored to call _rapids-path and prepend the s3://rapids-downloads/ string
rapids-http-path - a new script that calls _rapids-path and prepends the https://downloads.rapids.ai/ string
rapids-s3-path is currently the source of truth for computing proper paths for CI workflow artifacts.
The problem with
rapids-s3-pathis that it only returns S3 URIs (e.g.s3://rapids-downloads/ci/cudf/...).There are a lot of different areas where we call
rapids-s3-pathand then replaces3://rapids-downloads/withhttps://downloads.rapids.ai/:gha-tools/tools/_rapids-download-from-s3
Line 18 in 71f9e9a
gha-tools/tools/rapids-upload-to-s3
Line 36 in 71f9e9a
gha-tools/tools/rapids-upload-artifacts-dir
Line 32 in 71f9e9a
To prevent this manual replacement from being necessary, we should add a
rapids-http-pathscript that computes thehttps://downloads.rapids.aipath.The scripts should ultimately be refactored to work like this:
_rapids-path- a new script that's called byrapids-s3-pathandrapids-http-pathto generate the URI path. This script is prefixed with an underscore (_) to denote that it's not intended by be used publicly (only by othergha-toolsscripts).rapids-s3-path- should be refactored to call_rapids-pathand prepend thes3://rapids-downloads/stringrapids-http-path- a new script that calls_rapids-pathand prepends thehttps://downloads.rapids.ai/string