Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
92a6876
add redshift support
brendan-cook-87 Nov 24, 2023
8ba69db
Merge branch 'main' into REDSHIFT_SUPPORT
hanslemm Oct 21, 2024
6ad19c4
chore(profiles): clean up
hanslemm Oct 21, 2024
2be379f
chore(type_helpers): redshift SUPER type for JSON and ARRAY
hanslemm Oct 21, 2024
dfe7572
Merge branch 'brooklyn-data:main' into REDSHIFT_SUPPORT
hanslemm Dec 19, 2024
72f5e4a
Update dbt_project.yml
hanslemm Dec 19, 2024
42b11cd
Merge branch 'main' into REDSHIFT_SUPPORT
hanslemm Jun 12, 2025
d6bd6b3
Update README.md
hanslemm Jun 12, 2025
70d0081
Update type_helpers.sql
shiv-io Aug 5, 2025
ab81712
Update macros/database_specific_helpers/type_helpers.sql
hanslemm Sep 23, 2025
c5f3ed1
Update CONTRIBUTING.md
hanslemm Sep 23, 2025
4069ed7
Add Trino array type macro to type_helpers.sql
hanslemm Sep 23, 2025
637586e
Merge branch 'main' into REDSHIFT_SUPPORT
hanslemm Sep 23, 2025
c58f50d
Merge branch 'REDSHIFT_SUPPORT' into REDSHIFT_SUPPORT
hanslemm Aug 12, 2026
2021662
Merge pull request #1 from shiv-io/REDSHIFT_SUPPORT
hanslemm Aug 12, 2026
91ad526
chore: merge upstream 2.11.0 into REDSHIFT_SUPPORT
hanslemm Sep 8, 2026
495aecf
chore: nest generic test arguments under arguments (dbt 1.10+)
hanslemm Sep 8, 2026
b93ed16
feat: dim_dbt__current_relations for deferral state
hanslemm Sep 11, 2026
54bb9cb
feat: export_state run-operation for deferral state
hanslemm Sep 11, 2026
f1b2849
fix: validate and escape export_state's interpolated arguments
hanslemm Sep 11, 2026
a455f14
fix: filter target_name in Jinja and widen the identifier pattern
hanslemm Sep 11, 2026
d152d7a
fix(dim_dbt__current_relations): rank per (node_id, target_name); ord…
hanslemm Sep 11, 2026
2179d12
fix(export_state): fall back to a node's own target; emit ISO 8601 ti…
hanslemm Sep 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,18 @@ Tox will take care of installing the dependencies for each environment, so you d
tox -e integration_snowflake # For the Snowflake tests
tox -e integration_databricks # For the Databricks tests
tox -e integration_bigquery # For the BigQuery tests
tox -e integration_redshift # For the Redshift testss
Comment thread
hanslemm marked this conversation as resolved.
Outdated
```

The Spark tests require installing the [ODBC driver](https://www.databricks.com/spark/odbc-drivers-download). On a Mac,
DBT_ENV_SPARK_DRIVER_PATH should be set to `/Library/simba/spark/lib/libsparkodbc_sbu.dylib`. Spark tests have not yet
been added to the integration tests.

The Redshift tests require your AWS credentials configured in the current environment (either as environment variables or in your credentials
file - see [Configure the AWS cli](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)). They are currently configured with IAM
authorisation, so your principal will require the redshift:getClusterCredentials permission to retrieve the password for the specified redshift
database user.

If you don't have access to a particular database type, this isn't a problem. Test on the one you do have, and let us know in the PR.

#### SQLFluff
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The package currently supports
- Postgres :white_check_mark:
- SQL Server :white_check_mark:
- Trino :white_check_mark:
- Redshift ✅

Models included:

Expand Down
4 changes: 4 additions & 0 deletions integration_test_project/example-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ export DBT_ENV_SECRET_DATABRICKS_TOKEN=
export DBT_ENV_SECRET_GCP_PROJECT=
export DBT_ENV_SPARK_DRIVER_PATH= # /Library/simba/spark/lib/libsparkodbc_sbu.dylib on a Mac
export DBT_ENV_SPARK_ENDPOINT= # The endpoint ID from the Databricks HTTP path
export DBT_ENV_SECRET_REDSHIFT_HOST=
export DBT_ENV_SECRET_REDSHIFT_CLUSTER_ID=
export DBT_ENV_SECRET_REDSHIFT_DB=
export DBT_ENV_SECRET_REDSHIFT_USER=

# dbt environment variables, change these
export DBT_VERSION="1_5_0"
Expand Down
11 changes: 11 additions & 0 deletions integration_test_project/profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,17 @@ dbt_artifacts:
trust_cert: True
Encrypt: False
user: dbt
password: "123"
redshift:
Comment thread
hanslemm marked this conversation as resolved.
type: redshift
method: iam
threads: 8
host: "{{ env_var('DBT_ENV_SECRET_REDSHIFT_HOST') }}"
port: 5439
dbname: "{{ env_var('DBT_ENV_SECRET_REDSHIFT_DB') }}"
user: "{{ env_var('DBT_ENV_SECRET_REDSHIFT_USER') }}"
schema: dbt_artifacts_test_commit_{{ env_var('DBT_VERSION', '') }}_{{ env_var('GITHUB_SHA_OVERRIDE', '') if env_var('GITHUB_SHA_OVERRIDE', '') else env_var('GITHUB_SHA') }}
cluster_id: "{{ env_var('DBT_ENV_SECRET_REDSHIFT_CLUSTER_ID') }}"
password: "123Administrator"
trino:
type: trino
Expand Down
4 changes: 4 additions & 0 deletions macros/_macros.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ macros:
description: |
Dependent on the adapter type, returns the native type for storing JSON.

- name: type_string
description: |
Dependent on the adapter type, returns the native type for storing a string.

## MIGRATION ##
- name: migrate_from_v0_to_v1
description: |
Expand Down
21 changes: 20 additions & 1 deletion macros/database_specific_helpers/type_helpers.sql
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,14 @@
json
{% endmacro %}

{% macro redshift__type_json() %}
super
{% endmacro %}

{#- ARRAY -#}

{% macro type_array() %}
{{ return(adapter.dispatch('type_array', 'dbt_artifacts')()) }}
{{ return(adapter.dispatch('type_array', 'dbt_artifacts')()) }}
{% endmacro %}

{% macro default__type_array() %}
Expand All @@ -44,6 +48,21 @@
array<string>
{% endmacro %}

{% macro redshift__type_array() %}
super
{% endmacro %}

{% macro type_string() %}
{{ return(adapter.dispatch('type_string', 'dbt_artifacts')()) }}
{% endmacro %}

{% macro default__type_string() %}
{{ return(api.Column.translate_type("string")) }}
{% endmacro %}

{% macro redshift__type_string() %}
varchar(max)

Comment thread
hanslemm marked this conversation as resolved.
Outdated
{% macro trino__type_array() %}
array(varchar)
{% endmacro %}
Expand Down
16 changes: 16 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ profiles_dir = integration_test_project

[testenv]
passenv =
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
DBT_PROFILES_DIR
GITHUB_SHA_OVERRIDE
GITHUB_SHA
Expand All @@ -84,6 +87,10 @@ passenv =
DBT_ENV_SECRET_GCP_PROJECT
DBT_ENV_SPARK_DRIVER_PATH
DBT_ENV_SPARK_ENDPOINT
DBT_ENV_SECRET_REDSHIFT_HOST
DBT_ENV_SECRET_REDSHIFT_CLUSTER_ID
DBT_ENV_SECRET_REDSHIFT_DB
DBT_ENV_SECRET_REDSHIFT_USER
GOOGLE_APPLICATION_CREDENTIALS
DBT_CLOUD_PROJECT_ID
DBT_CLOUD_JOB_ID
Expand Down Expand Up @@ -304,6 +311,15 @@ commands =
dbt deps
dbt build --target bigquery --vars '"my_var": "my value"'

# Redshift integration test
[testenv:integration_test_redshift]
changedir = integration_test_project
deps = dbt-redshift~=1.7.0
commands =
dbt clean
dbt deps
dbt build --target redshift --vars '"my_var": "my value"'

[testenv:integration_bigquery_1_9_0]
changedir = integration_test_project
deps = dbt-bigquery~=1.9.0
Expand Down