Repository navigation
Conversation
…188) - Make DROP and moonlink calls robust; handle quoted identifiers; remove load_files - Add AGENTS.md, CLAUDE.md and docs/ARCHITECTURE.md - Drop mirrors in moonlink only at top-level commit; follow Postgres identifier rules; reconnect after a moonlink restart Co-authored-by: Isaac <no-reply@databricks.com>
- Add an end-to-end release gate for pg_mooncake Co-authored-by: Isaac <no-reply@databricks.com>
- e2e gate: re-create after a warned DROP, restart with a dropped source, build provenance - e2e gate: require the moonlink-down path for the re-create checks - e2e gate: provenance comes only from a build-info matching the loaded binary - e2e gate: name the cause of bgworker errors; stricter orphan DROP and build-info - e2e gate: known bug for a SIGTERM panic at shutdown; NOT-SEEN for intermittent bugs - e2e gate: check logs after the final shutdown; accept an errored mirror after DDL - e2e gate: provenance fails closed - e2e gate docs: say what a build-info attests Co-authored-by: Isaac <no-reply@databricks.com>
- e2e gate: blocking and limitation known bugs; GATE BLOCKED - e2e gate: worker exits at stops, error causes, one-line DDL match - e2e gate: limitation XFAILs pass; strict known bugs; final-stop exits - e2e gate: close false-PASS paths in leftovers, known bugs, stops Co-authored-by: Isaac <no-reply@databricks.com>
- e2e gate: review nits on known-bugs loading and bgworker errors - e2e gate: config reload, writes to mirrors, and role checks - e2e gate: stricter known-bugs file, typed leftovers, hung-stop result - e2e known bugs: shutdown panic in new scenarios; dead-table re-create - e2e gate: name panics in a hung stop; shutdown panic can hang a stop - e2e gate: session checks that prove their claim; fail-closed tooling Co-authored-by: Isaac <no-reply@databricks.com>
- e2e gate: round-2 review nits on the session and stop checks Co-authored-by: Isaac <no-reply@databricks.com>
…#196) - e2e gate: last nits on log scans, crash forms and the loader - e2e gate: PostgreSQL 18 crash lines; abnormal shutdown counts Co-authored-by: Isaac <no-reply@databricks.com>
…er signals, pg_duckdb fork (#194) - Clean up orphaned moonlink tables when re-creating a mirror - Prepare 0.3.0: upgrade script, CHANGELOG, README install and self-hosting guide - Resolve dependencies that build with the pinned Rust toolchain - Add a CI workflow: build and make test on PostgreSQL 14-18, e2e gate on pushes - Reject writes to mirrors before pg_duckdb plans them - Document the pg_duckdb reload crash and who can read mirrors - Handle shutdown signals only on the moonlink worker's main thread - make test: fail when pg_regress tests fail - Dockerfile: include the extension upgrade scripts - Reject TRUNCATE, COPY FROM, renames and attaching mirrors - create_table: after a failure, drop only the table this call created - Upgrade script: tolerate 0.2.0 installs without load_files - Drive moonlink from a thread created with shutdown signals blocked - Docs: fix the upgrade, slot, reload and pg_duckdb notes; mark post-bump statements - CI: stricter e2e job, separate fmt job, clippy, DuckDB version checks - Fail release tags that still carry release markers - Restart the moonlink worker after a SIGTERM outside a server shutdown - Bundle pg_duckdb with the upstream fix for its TimeZone hook - Re-raise an early SIGTERM for moonlink; give its thread the main thread's stack size - Docs: database renames and column changes; name the pg_duckdb tag Co-authored-by: Isaac <no-reply@databricks.com>
…tests/mirror_diff) (#192) - tests/mirror_diff: differential mirror tests driven by the PostgreSQL regression suite - tests/mirror_diff: known differences on the current submodules - TESTING.md: how to run tests/mirror_diff - tests/mirror_diff: typed, ordered comparison and checks that can't pass vacuously - tests/mirror_diff: known differences pinned to their signatures - tests/mirror_diff: document the comparison rules and the gate - tests/mirror_diff: close the remaining false-PASS paths - tests/mirror_diff: unsupported-types covers only the types mirrors lack - tests/mirror_diff: document SKIPs, check names and known gaps - tests/mirror_diff: close five more false-PASS paths, with a test each - tests/mirror_diff: composites are not an unsupported type; pin probe rules to their events Co-authored-by: Isaac <no-reply@databricks.com>
…nown limitation (#198) - tests/mirror_diff: composites as DuckDB STRUCT text are a limitation Co-authored-by: Isaac <no-reply@databricks.com>
- Bump moonlink to its 0.3 stack (de5de55) - Docs: describe moonlink 0.3 without its write-ahead log - Docs: composite columns read back as DuckDB STRUCT values - CHANGELOG: moonlink 0.3 fixes; resolve the post-bump release markers - Bump moonlink to 977dc54: faster initial copy - CHANGELOG: correct character(n), full-disk and refusal notes; early-restart limitation Co-authored-by: Isaac <no-reply@databricks.com>
- e2e gate: crash kinds with hyphens; minor-independent pins; loader - e2e gate: probes for moonlink 0.3.0 RC1 - e2e known bugs: moonlink 0.3.0 RC1 - e2e known bugs: char(n) padding is a known limitation - e2e gate: keep moonlink down for the while-down checks - e2e known bugs: pg_duckdb's TimeZone hook is fixed - e2e known bugs: moonlink 0.3.0 RC2 fixes the decimal panic - Docs: final known limitations from the release gate - e2e known bugs: name the moonlink commit the list was checked on Co-authored-by: Isaac <no-reply@databricks.com>
…n tests (#201) - Pin rust-postgres to the revision moonlink uses - Makefile: honour a caller's PG_CONFIG and pass it to cargo pgrx - Tests: freshness after unrelated commits, prepared statements, pre-existing partitions - Review fixes: check PG_CONFIG for make test, absolute paths, rust-postgres check Co-authored-by: Isaac <no-reply@databricks.com>
Moves the moonlink submodule to its 0.3.0 release commit 0e38b43 and documents moonlink PRs 2371-2374: create_table returns only once its initial copy is durable, fails with an error on a full disk, and a drop during create leaves nothing behind. Scopes the restart note and adds the server-crash-during-create_table limitation. Updates the mirror_diff pins. Co-authored-by: Isaac <no-reply@databricks.com>
…ld, code and test fixes (#204) - mooncake.data_directory (postmaster setting, default pg_mooncake under PGDATA): moonlink's files and socket can live outside PGDATA, so pg_basebackup and standbys work; the directory is locked per server and checked like a data directory. - moonlink starts only once recovery finishes (not on standbys; on promotion). - create_table reports the cause of libpq failures without echoing connection strings; the unreleased 0.2 legacy key code is gone. - e2e gate requires PASS with no skips; restart and crash exactness are gated; unused harness machinery removed. - Docker image builds from the submodules and is started and checked in CI; pg_regress gets a PostgreSQL 14 output file. Co-authored-by: Isaac <no-reply@databricks.com>
Rewrites the release docs for 0.3.0 as the first release of the moonlink-based design: the CHANGELOG (overview, usage, behaviour and guarantees, performance, known limitations), the README (install, Coming from 0.1.x, backups and standbys, security), AGENTS.md as a contributor guide, ARCHITECTURE.md and TESTING.md. Removes the unreleased 0.2 upgrade script. Co-authored-by: Isaac <no-reply@databricks.com>
…pins (#205) Rewrites mirror_diff's rules for the 0.3.0 pins: one rule per refused-type message, limitation entries for the documented limitations, comparator events for numeric scale padding and char(n) padding that cannot hide real differences, and tests against the real expected.txt. A full PostgreSQL 17 pass: 2104 checks, 0 FAIL. mirror_diff is advisory, not a release gate. Co-authored-by: Isaac <no-reply@databricks.com>
Moves the moonlink submodule to its 0.3.0 release commit 37b040f9: dropping and re-creating a mirror keeps the new one (moonlink PR 2376), dependency upgrades for the open RustSec advisories (moonlink PR 2378), and test-only fixes. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pg_mooncake 0.3.0
This brings the 0.3.0 release candidate (
v0.3-devat1719f31) tomain. 0.3.0 is the first release of pg_mooncake built on moonlink: a columnstore mirror of a regular Postgres table, kept in Iceberg through logical replication and queried in DuckDB through pg_duckdb. It replaces the unreleased 0.2 line onmain; 0.1.x was a different design and is not upgraded in place (see the README's Coming from 0.1.x).Start here: CHANGELOG.md (overview, usage, behaviour and guarantees, performance, known limitations) and README.md (install, Docker, backups and standbys, security).
What changed against
main37b040f9, Mooncake-Labs/moonlinkv0.3-dev): service features pg_mooncake doesn't use are removed (REST, Kafka, OTEL, its own WAL, Delta, REST/Glue catalogs, GCS), and the CDC, recovery, storage and read paths are hardened. Each change is a reviewed moonlink PR, linked from the CHANGELOG.v1.0.0-mooncake.1(DuckDB v1.4.1 plus the upstream TimeZone-hook crash fix).DROPof mirrors, refusal of writes to mirrors,mooncake.data_directory(letspg_basebackupand standbys work), moonlink starting only after recovery, safer error messages, worker restart and signal handling.make test(pg_regress) on PostgreSQL 14-18; an end-to-end gate (scripts/e2e-smoke.sh: crash/restart exactness, DDL, roles, config reload, base backups, standbys) on 17 and 18; a differential suite driven by the PostgreSQL regression tests (tests/mirror_diff, advisory); a Docker build-and-run job.How it was validated
v0.3-devas its own reviewed PR (Mooncake-Labs/pg_mooncake PRs 188-206) with green CI.Still open before tagging
src_uri(scripts/check-release-markers.shlists them).v0.3-devinto itsmainpreserving SHAs and tagv0.3.0at the pinned commit, so the submodule pin stays reachable.Feedback especially welcome on: install from source and Docker, the Known limitations list (is anything there a blocker for you?), and whether the docs are clear for someone new to it.
This pull request and its description were written by Isaac.