Skip to content

Commit 6592d15

Browse files
pedronauckpedronauck
authored
feat(repo): Add postgres standalone (#307)
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * SQLite support (file & in-memory), selectable DB driver, provider-based multi-DB plumbing, migrations, and driver-aware startup/health guidance. * Config: database driver, SQLite path/busy timeout, and vector DB entries with validation and normalization. * **Documentation** * Added "Database Overview" and "Database Troubleshooting" links. * **Tests** * Extensive multi-driver unit & integration coverage for migrations, repos, transactions, concurrency, and edge cases. * **Refactor** * API key lookup renamed to use "fingerprint" across interfaces and tests. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 51ba575 commit 6592d15

116 files changed

Lines changed: 17009 additions & 234 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎cli/cmd/auth/bootstrap/shared.go‎

Lines changed: 4 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,8 @@ import (
88

99
"github.com/compozy/compozy/engine/auth/bootstrap"
1010
authuc "github.com/compozy/compozy/engine/auth/uc"
11-
"github.com/compozy/compozy/engine/infra/postgres"
1211
"github.com/compozy/compozy/engine/infra/repo"
1312
"github.com/compozy/compozy/pkg/config"
14-
"github.com/compozy/compozy/pkg/logger"
1513
)
1614

1715
// defaultAuthBootstrapDBTimeout is the bounded timeout for bootstrap DB operations.
@@ -35,31 +33,15 @@ func (f *DefaultServiceFactory) CreateService(ctx context.Context) (*bootstrap.S
3533
dbCtx, cancel := context.WithTimeout(ctx, defaultAuthBootstrapDBTimeout)
3634
defer cancel()
3735
// NOTE: Auth bootstrap always uses Postgres to enforce consistent credential handling.
38-
dbCfg := &postgres.Config{
39-
ConnString: cfg.Database.ConnString,
40-
Host: cfg.Database.Host,
41-
Port: cfg.Database.Port,
42-
User: cfg.Database.User,
43-
Password: cfg.Database.Password,
44-
DBName: cfg.Database.DBName,
45-
SSLMode: cfg.Database.SSLMode,
46-
PingTimeout: cfg.Database.PingTimeout,
47-
HealthCheckTimeout: cfg.Database.HealthCheckTimeout,
48-
HealthCheckPeriod: cfg.Database.HealthCheckPeriod,
49-
ConnectTimeout: cfg.Database.ConnectTimeout,
50-
}
51-
drv, err := postgres.NewStore(dbCtx, dbCfg)
36+
dbCfg := cfg.Database
37+
dbCfg.Driver = "postgres"
38+
provider, providerCleanup, err := repo.NewProvider(dbCtx, &dbCfg)
5239
if err != nil {
5340
return nil, nil, fmt.Errorf("failed to connect to database: %w", err)
5441
}
55-
provider := repo.NewProvider(drv.Pool())
5642
authRepo := provider.NewAuthRepo()
5743
cleanup := func() {
58-
closeCtx, cancelClose := context.WithTimeout(context.WithoutCancel(ctx), defaultAuthBootstrapDBTimeout)
59-
defer cancelClose()
60-
if err := drv.Close(closeCtx); err != nil {
61-
logger.FromContext(ctx).Error("Failed to close database", "error", err)
62-
}
44+
providerCleanup()
6345
}
6446
factory := authuc.NewFactory(authRepo)
6547
service := bootstrap.NewService(factory)

‎docs/content/docs/cli/compozy-start.mdx‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,47 @@ compozy start [flags]
6767

6868
Additional standalone fields (`bind_ip`, `namespace`, `cluster_name`, `enable_ui`, `log_level`, `start_timeout`) are configured via `compozy.yaml` or environment variables.
6969

70+
### Database Flags
71+
72+
<Table>
73+
<TableHeader>
74+
<TableRow>
75+
<TableHead>Flag</TableHead>
76+
<TableHead>Description</TableHead>
77+
<TableHead>Default</TableHead>
78+
</TableRow>
79+
</TableHeader>
80+
<TableBody>
81+
<TableRow>
82+
<TableCell>`--db-driver`</TableCell>
83+
<TableCell>Selects the database backend (`postgres` or `sqlite`).</TableCell>
84+
<TableCell>`postgres`</TableCell>
85+
</TableRow>
86+
<TableRow>
87+
<TableCell>`--db-path`</TableCell>
88+
<TableCell>SQLite database path or `:memory:`. Required when `--db-driver=sqlite`.</TableCell>
89+
<TableCell>None</TableCell>
90+
</TableRow>
91+
<TableRow>
92+
<TableCell>`--db-conn-string`</TableCell>
93+
<TableCell>PostgreSQL connection string (`postgres://user:pass@host:port/db`). Overrides individual host/port flags.</TableCell>
94+
<TableCell>None</TableCell>
95+
</TableRow>
96+
</TableBody>
97+
</Table>
98+
99+
```bash title="SQLite file-backed"
100+
compozy start --db-driver=sqlite --db-path=./data/compozy.db
101+
```
102+
103+
```bash title=":memory: mode"
104+
compozy start --db-driver=sqlite --db-path=:memory:
105+
```
106+
107+
<Callout type="warning">
108+
When `--db-driver=sqlite`, configure an external vector database (Qdrant, Redis, or filesystem) because SQLite cannot host embeddings.
109+
</Callout>
110+
70111
## Examples
71112

72113
<Tabs items={["Remote", "Standalone"]}>

‎docs/content/docs/cli/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
"project-commands",
88
"dev-commands",
99
"compozy-start",
10+
"migrate",
1011
"auth-commands",
1112
"config-commands",
1213
"knowledge-commands",

‎docs/content/docs/cli/migrate.mdx‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
---
2+
title: "compozy migrate"
3+
description: "Database migration commands with automatic driver detection for PostgreSQL and SQLite."
4+
icon: Database
5+
---
6+
7+
Run schema migrations for the active database driver. Compozy detects PostgreSQL or SQLite based on your configuration and applies the correct migration set.
8+
9+
## Usage
10+
11+
```bash
12+
compozy migrate [command]
13+
```
14+
15+
### Commands
16+
17+
| Command | Description |
18+
|---------|-------------|
19+
| `up` | Applies all pending migrations. |
20+
| `down` | Rolls back the most recent migration batch. |
21+
| `status` | Displays applied vs. pending migrations. |
22+
23+
## Multi-Database Support
24+
25+
```bash
26+
# Apply migrations (auto-detects driver from compozy.yaml or CLI flags)
27+
compozy migrate up
28+
29+
# Check migration status
30+
compozy migrate status
31+
32+
# Roll back last migration batch
33+
compozy migrate down
34+
```
35+
36+
- PostgreSQL and SQLite use **separate migration files** tuned for each SQL dialect.
37+
- Switching drivers? Run `compozy migrate up` after updating `database.driver` to ensure the new backend schema exists.
38+
- For SQLite, keep the `.db` file available before running migrations; the CLI creates it if missing.
39+
40+
<Callout type="warning">
41+
Always back up your database before running destructive commands such as `compozy migrate down`, especially in production PostgreSQL environments.
42+
</Callout>
43+
44+
## Driver Overrides
45+
46+
```bash
47+
# Force PostgreSQL migrations even if config defaults apply
48+
compozy migrate up --db-driver=postgres --db-conn-string=postgres://...
49+
50+
# Migrate an SQLite file
51+
compozy migrate up --db-driver=sqlite --db-path=./data/compozy.db
52+
```
53+
54+
Flags mirror those available on `compozy start`, ensuring parity between runtime and migration workflows (`--db-driver`, `--db-path`, `--db-conn-string`).
55+
56+
## Troubleshooting
57+
58+
- `no such table`: migrations were not applied for the selected driver—rerun `compozy migrate up`.
59+
- `database is locked` (SQLite): close other processes and consider enabling WAL mode before retrying.
60+
- `permission denied` (PostgreSQL): ensure the database user can create tables, indexes, and sequences.
61+
62+
## Related Pages
63+
64+
<ReferenceCardList>
65+
<ReferenceCard
66+
title="Database Overview"
67+
description="Understand how driver selection impacts migrations."
68+
href="/docs/database/overview"
69+
icon="Database"
70+
/>
71+
<ReferenceCard
72+
title="PostgreSQL Driver"
73+
description="Production-ready configuration and pgvector guidance."
74+
href="/docs/database/postgresql"
75+
icon="ServerCog"
76+
/>
77+
<ReferenceCard
78+
title="SQLite Driver"
79+
description="Lightweight deployments and concurrency limits."
80+
href="/docs/database/sqlite"
81+
icon="HardDrive"
82+
/>
83+
</ReferenceCardList>

‎docs/content/docs/configuration/database.mdx‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
---
2+
title: "Database Configuration"
3+
description: "Field-by-field reference for configuring PostgreSQL and SQLite in Compozy."
4+
icon: Database
5+
---
6+
7+
## `database.driver`
8+
9+
Select the backing store for Compozy metadata.
10+
11+
| Property | Value |
12+
|----------|-------|
13+
| **Type** | `string` |
14+
| **Options** | `postgres` \| `sqlite` |
15+
| **Default** | `postgres` |
16+
| **Environment Variable** | `DB_DRIVER` or `COMPOZY_DB_DRIVER` |
17+
18+
Omitting `driver` keeps PostgreSQL as the default.
19+
20+
## PostgreSQL Fields
21+
22+
Use these fields when `driver: postgres` (or omitted):
23+
24+
<Table>
25+
<TableHeader>
26+
<TableRow>
27+
<TableHead>Field</TableHead>
28+
<TableHead>Type</TableHead>
29+
<TableHead>Required</TableHead>
30+
<TableHead>Description</TableHead>
31+
</TableRow>
32+
</TableHeader>
33+
<TableBody>
34+
<TableRow>
35+
<TableCell>`host`</TableCell>
36+
<TableCell>string</TableCell>
37+
<TableCell>Yes*</TableCell>
38+
<TableCell>Hostname or IP address of the PostgreSQL server (*omit when using `conn_string`).</TableCell>
39+
</TableRow>
40+
<TableRow>
41+
<TableCell>`port`</TableCell>
42+
<TableCell>string</TableCell>
43+
<TableCell>No</TableCell>
44+
<TableCell>Port number (default `5432`).</TableCell>
45+
</TableRow>
46+
<TableRow>
47+
<TableCell>`user`</TableCell>
48+
<TableCell>string</TableCell>
49+
<TableCell>Yes*</TableCell>
50+
<TableCell>Database user used for migrations and runtime operations.</TableCell>
51+
</TableRow>
52+
<TableRow>
53+
<TableCell>`password`</TableCell>
54+
<TableCell>string</TableCell>
55+
<TableCell>Yes*</TableCell>
56+
<TableCell>Password for the specified user.</TableCell>
57+
</TableRow>
58+
<TableRow>
59+
<TableCell>`dbname`</TableCell>
60+
<TableCell>string</TableCell>
61+
<TableCell>Yes*</TableCell>
62+
<TableCell>Name of the PostgreSQL database where Compozy stores metadata.</TableCell>
63+
</TableRow>
64+
<TableRow>
65+
<TableCell>`conn_string`</TableCell>
66+
<TableCell>string</TableCell>
67+
<TableCell>No</TableCell>
68+
<TableCell>PostgreSQL DSN (`postgres://user:pass@host:port/db?sslmode=require`). When provided, overrides individual connection fields.</TableCell>
69+
</TableRow>
70+
<TableRow>
71+
<TableCell>`sslmode`</TableCell>
72+
<TableCell>string</TableCell>
73+
<TableCell>No</TableCell>
74+
<TableCell>SSL/TLS mode (`disable`, `require`, `verify-ca`, `verify-full`). Defaults to `require` in production builds.</TableCell>
75+
</TableRow>
76+
<TableRow>
77+
<TableCell>`max_open_conns`</TableCell>
78+
<TableCell>int</TableCell>
79+
<TableCell>No</TableCell>
80+
<TableCell>Upper bound for concurrent connections in the pool.</TableCell>
81+
</TableRow>
82+
<TableRow>
83+
<TableCell>`max_idle_conns`</TableCell>
84+
<TableCell>int</TableCell>
85+
<TableCell>No</TableCell>
86+
<TableCell>Number of warm idle connections the pool maintains.</TableCell>
87+
</TableRow>
88+
</TableBody>
89+
</Table>
90+
91+
```yaml title="PostgreSQL Example"
92+
database:
93+
driver: postgres
94+
host: postgres.internal
95+
port: 5432
96+
user: compozy
97+
password: ${DB_PASSWORD}
98+
dbname: compozy
99+
sslmode: verify-full
100+
max_open_conns: 50
101+
max_idle_conns: 10
102+
```
103+
104+
## SQLite Fields
105+
106+
Use these fields when `driver: sqlite`:
107+
108+
<Table>
109+
<TableHeader>
110+
<TableRow>
111+
<TableHead>Field</TableHead>
112+
<TableHead>Type</TableHead>
113+
<TableHead>Required</TableHead>
114+
<TableHead>Description</TableHead>
115+
</TableRow>
116+
</TableHeader>
117+
<TableBody>
118+
<TableRow>
119+
<TableCell>`path`</TableCell>
120+
<TableCell>string</TableCell>
121+
<TableCell>Yes</TableCell>
122+
<TableCell>Path to the SQLite file. Use `:memory:` for ephemeral, in-memory deployments.</TableCell>
123+
</TableRow>
124+
<TableRow>
125+
<TableCell>`busy_timeout`</TableCell>
126+
<TableCell>int</TableCell>
127+
<TableCell>No</TableCell>
128+
<TableCell>Milliseconds SQLite waits before returning `database is locked`.</TableCell>
129+
</TableRow>
130+
<TableRow>
131+
<TableCell>`journal_mode`</TableCell>
132+
<TableCell>string</TableCell>
133+
<TableCell>No</TableCell>
134+
<TableCell>Transaction journal mode. `wal` is recommended for moderate concurrency.</TableCell>
135+
</TableRow>
136+
<TableRow>
137+
<TableCell>`synchronous`</TableCell>
138+
<TableCell>string</TableCell>
139+
<TableCell>No</TableCell>
140+
<TableCell>Durability level: `full`, `normal`, or `off`.</TableCell>
141+
</TableRow>
142+
</TableBody>
143+
</Table>
144+
145+
```yaml title="SQLite Example"
146+
database:
147+
driver: sqlite
148+
path: ./data/compozy.db
149+
busy_timeout: 5000
150+
journal_mode: wal
151+
synchronous: normal
152+
```
153+
154+
<Callout type="error">
155+
SQLite deployments **must** configure an external vector database provider. See the [vector database guide](/docs/knowledge-bases/vector-databases) for supported options.
156+
</Callout>
157+
158+
## Environment Variables
159+
160+
CLI flags and environment variables override configuration values:
161+
162+
| Variable | Description |
163+
|----------|-------------|
164+
| `COMPOZY_DB_DRIVER` | Overrides `database.driver`. |
165+
| `COMPOZY_DB_HOST` | PostgreSQL host. |
166+
| `COMPOZY_DB_PORT` | PostgreSQL port. |
167+
| `COMPOZY_DB_NAME` | PostgreSQL database name. |
168+
| `COMPOZY_DB_USER` | PostgreSQL user. |
169+
| `COMPOZY_DB_PASSWORD` | PostgreSQL password. |
170+
| `COMPOZY_DB_CONN_STRING` | PostgreSQL connection string. |
171+
| `COMPOZY_DB_PATH` | SQLite file or `:memory:` path. |
172+
| `COMPOZY_DB_BUSY_TIMEOUT` | Millisecond timeout for SQLite locks. |
173+
174+
CLI precedence follows the order: **flags → environment → configuration file → defaults**.
175+
176+
## Related Topics
177+
178+
<ReferenceCardList>
179+
<ReferenceCard
180+
title="Database Overview"
181+
description="Decide when to run PostgreSQL or SQLite."
182+
href="/docs/database/overview"
183+
icon="Database"
184+
/>
185+
<ReferenceCard
186+
title="PostgreSQL Driver"
187+
description="Production setup and tuning guidance."
188+
href="/docs/database/postgresql"
189+
icon="ServerCog"
190+
/>
191+
<ReferenceCard
192+
title="SQLite Driver"
193+
description="Edge deployments and concurrency limits."
194+
href="/docs/database/sqlite"
195+
icon="HardDrive"
196+
/>
197+
</ReferenceCardList>

‎docs/content/docs/configuration/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
"overview",
88
"mode-configuration",
99
"redis",
10+
"database",
1011
"temporal"
1112
]
1213
}

0 commit comments

Comments
 (0)