Skip to content

Add multi-team lookup to secrets backends#65689

Closed
PrithviBadiga wants to merge 1 commit intoapache:mainfrom
PrithviBadiga:prithvi/multi-team-secrets-backends
Closed

Add multi-team lookup to secrets backends#65689
PrithviBadiga wants to merge 1 commit intoapache:mainfrom
PrithviBadiga:prithvi/multi-team-secrets-backends

Conversation

@PrithviBadiga
Copy link
Copy Markdown
Contributor

Adds multi-team lookup support to three secrets backends:

  • AzureKeyVaultBackend
  • KubernetesSecretsBackend
  • LockboxSecretBackend

This update makes the backends:

  • look up team-scoped secrets first when team_name is provided
  • fall back to the global secret when no team-scoped secret is found
  • avoid resolving team-scoped identifiers as global secrets when team_name is not provided

Updates:

  • Azure Key Vault:
    • looks up {prefix}-{team_name}-{secret_id} before {prefix}-{secret_id}
  • Kubernetes:
    • adds team_label support
    • looks up {id_label}={secret_id},{team_label}={team_name} before falling back to unlabeled global secrets
  • Yandex Lockbox:
    • looks up {prefix}/{team_name}/{secret_id} before {prefix}/{secret_id}

Also updates unit tests and backend docs for all three providers.

Verification:

  • AIRFLOW_HOME=$(mktemp -d) PYTHONPATH=/Users/prith/Desktop/Codex/airflow-65682/airflow-core/src:/Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/src /Users/prith/Desktop/Codex/airflow/.venv/bin/python -m pytest /Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/tests/unit/cncf/kubernetes/secrets/test_kubernetes_secrets_backend.py
  • AIRFLOW_HOME=$(mktemp -d) PYTHONPATH=/Users/prith/Desktop/Codex/airflow-65682/airflow-core/src:/Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/src /Users/prith/Desktop/Codex/airflow/.venv/bin/python -m pytest /Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/tests/unit/microsoft/azure/secrets/test_key_vault.py
  • AIRFLOW_HOME=$(mktemp -d) PYTHONPATH=/Users/prith/Desktop/Codex/airflow-65682/airflow-core/src:/Users/prith/Desktop/Codex/airflow-65682/providers/yandex/src /Users/prith/Desktop/Codex/airflow/.venv/bin/python -m pytest /Users/prith/Desktop/Codex/airflow-65682/providers/yandex/tests/unit/yandex/secrets/test_lockbox.py
  • /Users/prith/Desktop/Codex/airflow/.venv/bin/python -m ruff check /Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/src/airflow/providers/microsoft/azure/secrets/key_vault.py /Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/secrets/kubernetes_secrets_backend.py /Users/prith/Desktop/Codex/airflow-65682/providers/yandex/src/airflow/providers/yandex/secrets/lockbox.py /Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/tests/unit/microsoft/azure/secrets/test_key_vault.py /Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/tests/unit/cncf/kubernetes/secrets/test_kubernetes_secrets_backend.py /Users/prith/Desktop/Codex/airflow-65682/providers/yandex/tests/unit/yandex/secrets/test_lockbox.py
  • /Users/prith/Desktop/Codex/airflow/.venv/bin/python -m ruff format --check /Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/src/airflow/providers/microsoft/azure/secrets/key_vault.py /Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/secrets/kubernetes_secrets_backend.py /Users/prith/Desktop/Codex/airflow-65682/providers/yandex/src/airflow/providers/yandex/secrets/lockbox.py /Users/prith/Desktop/Codex/airflow-65682/providers/microsoft/azure/tests/unit/microsoft/azure/secrets/test_key_vault.py /Users/prith/Desktop/Codex/airflow-65682/providers/cncf/kubernetes/tests/unit/cncf/kubernetes/secrets/test_kubernetes_secrets_backend.py /Users/prith/Desktop/Codex/airflow-65682/providers/yandex/tests/unit/yandex/secrets/test_lockbox.py

This closes: #65682

@vincbeck
Copy link
Copy Markdown
Contributor

Can you split it in 3 different PRs please?

@PrithviBadiga
Copy link
Copy Markdown
Contributor Author

@vincbeck
Split into 3 separate PRs by backend:

Closing this combined PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add multi-team support to AzureKeyVaultBackend, KubernetesSecretsBackend, and LockboxSecretBackend

2 participants