Repository navigation
Release cleanup: mooncake.data_directory, strict e2e gate, Docker build, code and test fixes - #204
Merged
Merged
Conversation
- DROP and the orphan cleanup in create_table send only the current moonlink key. pg_mooncake 0.2 was never released, so no mirror is registered under the quoted name. - create_table's libpq failures are ordinary errors that carry the server's message (missing source, taken mirror name, connection failures) instead of panics that dropped it; pg_regress covers two of them. - One is_mirror helper for drop tracking and the write guard; mirror_name is private. - Comments: uri_encode cites RFC 3986; the worker comment no longer says pg_terminate_backend can reach it (it isn't in the ProcArray); history phrases removed. - Cargo.toml: remove the unused bincode; say why the bgworker feature exists. Co-authored-by: Isaac <no-reply@databricks.com>
writes.sql ran MERGE and SET ACCESS METHOD through DO blocks that printed the PostgreSQL 15+ message on 14. They now run directly; writes_1.out is the PostgreSQL 14 output (both are syntax errors there). Also ORDER BY for multi-row SELECT * results, and descriptive table names in identifiers.sql. Co-authored-by: Isaac <no-reply@databricks.com>
The image no longer copies sql/ (no upgrade scripts ship), copies rust-toolchain.toml and .cargo/config.toml, installs clang, libclang and cmake as the README lists, and builds pg_mooncake with the toolchain rust-toolchain.toml pins. It builds pg_duckdb from the pinned submodule on postgres:18 instead of taking pgduckdb/pgduckdb:18-main, whose DuckDB need not be the v1.4.1 that duckdb_mooncake is built for. .dockerignore keeps build outputs and duckdb_mooncake out of the context. make help lists every target and says that test runs the pg_regress tests. Co-authored-by: Isaac <no-reply@databricks.com>
- CI's e2e job passes only on exit 0 and "GATE e2e-smoke PASS"; E2E_MAX_SKIP and the count parsing that tolerated skips are gone. CI also builds the Docker image (pushes and manual runs, not pushed), runs tests/mirror_diff's unit tests in the fmt job, and runs clippy with the default features. - e2e-smoke.sh: the known-bugs file lists limitations only. Removed the blocking class and GATE BLOCKED (exit 4), "requires" rules, Intermittent / NOT-SEEN, the KNOWN_BUG_IDS hook, the canary regexes, INFO diagnostics and dead helpers. - exact_after_restart and exact_after_crash are gated: a restart or crash must neither lose nor duplicate rows. - mirror_writes' plain write controls must fail with "cannot modify mirror", not any error. - Cluster sockets live in E2E_DIR instead of /tmp. - WIRE_MOONLINK_RPC_REV is explained: duckdb_mooncake's locked moonlink_rpc, with an unchanged wire format. - e2e-known-bugs.txt: header without stale pins; the server-file-roles limitation names both roles, which reads need (checked: either alone fails). - e2e-build-info.sh no longer writes keys nothing reads. - tests/mirror_diff scripts with a shebang are executable. Co-authored-by: Isaac <no-reply@databricks.com>
moonlink's file names are longer than the 99 characters of Postgres' tar format, so pg_basebackup failed on any server with mirrors. The new postmaster setting mooncake.data_directory (default pg_mooncake, relative to the data directory, as before) can point outside the data directory. It is moonlink's base path and the location of the RPC socket that backends and DuckDB connect to. At startup the directory is created (0700) if missing and must be owned by the server user and writable. A socket path over 107 bytes is rejected. A directory whose socket already accepts connections (a backup copy on the same host inherits the setting) is refused. A non-default directory inside the data directory gets a warning. The worker now starts when recovery finishes instead of at consistent state. That is the same moment on a primary, crash recovery included. On a standby, moonlink ran but could not replicate from a read-only server, and its attempts errored mirrors it had restored. Now it starts on promotion. The e2e scenario basebackup takes plain, tar and standby backups, and checks a copy on its own directory: mirrors error until re-created, and re-creating resnapshots. It checks the refused shared directory, and a standby that runs no moonlink until promoted. Co-authored-by: Isaac <no-reply@databricks.com>
- The postgres image sets PG_VERSION to its Debian package version, which overrode the Makefile's PG_VERSION; the make calls pass PG_VERSION=pg18 and PG_CONFIG explicitly. - The runtime stage installs libcurl4t64 (pg_duckdb's built-in httpfs links it) and ca-certificates (CREATE EXTENSION downloads the mooncake DuckDB extension over HTTPS). - mooncake.data_directory defaults to /var/lib/postgresql/18/mooncake in the image: in the volume, next to PGDATA (/var/lib/postgresql/18/docker) and outside it, so pg_basebackup works. The image creates it owned by postgres, so a new volume gets it. - The CI docker job runs the image: it waits for the server, creates the extension, a source table and its mirror, reads the mirror back, checks the data directory setting and takes a base backup. Co-authored-by: Isaac <no-reply@databricks.com>
- The postmaster locks the directory with flock on moonlink.lock (holding its PID and data directory) and keeps it open for its lifetime, so a second server is refused even while the first one's moonlink isn't listening, for example on a standby or during recovery. The socket probe stays as a second check and no longer blocks. - An existing directory with group write or any access for others, a symbolic link, and the data directory or one of its parents are refused. ./pg_mooncake counts as the default and gets no warning. - Errors come with an errcode and a real hint (ERRCODE_CONFIG_FILE_ERROR, ERRCODE_OBJECT_IN_USE), also the warning of a DROP that moonlink did not see. - Loading pg_mooncake outside shared_preload_libraries is a clear ERROR. - create_table names the server it couldn't connect to by host, port, database and user; the connection string, which may hold a password, is never shown. The e2e basebackup scenario checks the lock against a running standby, a listening socket in an unlocked directory, and the refused directories. Co-authored-by: Isaac <no-reply@databricks.com>
pg_duckdb includes unicode/ucol.h, and pg_config's CPPFLAGS point at the libxml2 headers; the GitHub runners have both preinstalled, the postgres:18 image does not. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-release cleanup of code, tests, CI and build, and
mooncake.data_directoryCleanup before pg_mooncake 0.3.0, plus a fix for a release blocker:
pg_basebackupfailed on any server with mirrors.Why
$PGDATA/pg_mooncakereach about 124, so backups and new standbys failed while any mirror existed.RAISE, and an e2e gate that tolerated skips and carried rules nothing used.Changes
mooncake.data_directory(new, postmaster context)pg_mooncake, relative to the data directory, so the default behavior is unchanged. An absolute path is accepted../pg_mooncakecounts as the default).flockon<dir>/moonlink.lockand keeps it for its lifetime (its children inherit it, and the kernel releases it however the processes end);ERRCODE_OBJECT_IN_USE, with a hint);ATTACHfollow the setting.The moonlink worker starts when recovery finishes (was: at consistent state)
ALTER PUBLICATIONand left the mirrors it had restored in the errored state. Now it starts when the standby is promoted, and restored mirrors keep working.Code
shared_preload_librariesis a clear error (pg_mooncake must be loaded via shared_preload_libraries).HINT:inside the message.mooncake.create_tablenever shows a connection string, which may hold a password. A connection failure names the server by host, port, database and user.mooncake.create_tablereports libpq failures as errors with the server's message. Examples:could not read the columns of public.x: relation "public.x" does not exist,could not create public.t: relation "t" already exists. Before, these were panics that dropped the cause.is_mirrorhelper for drop tracking and the write guard.uri_encode, andpg_terminate_backendcan't reach the worker.bincodedependency is removed.Tests
writes.sqlruns MERGE and SET ACCESS METHOD directly.writes_1.outis the PostgreSQL 14 output.ORDER BYfor multi-row results, descriptive table names inidentifiers.sql, and adata_directorytest (the setting exists with postmaster context).requiresrules, intermittent bugs, the bug-id hook, canary regexes and INFO diagnostics are removed.exact_after_restartandexact_after_crashare gated checks.cannot modify mirror.E2E_DIR.basebackup: plain, tar and standby backups, a copy on its own directory, a refused shared directory, and a standby promoted.CI and build
GATE e2e-smoke PASS.tests/mirror_diff's unit tests.rust-toolchain.tomlpins, onpostgres:18. Before, the runtime base waspgduckdb/pgduckdb:18-main, whose DuckDB need not match the v1.4.1 that duckdb_mooncake is built for.sql/. It passesPG_VERSION=pg18andPG_CONFIGto make explicitly, because the postgres image setsPG_VERSIONto its Debian package version.libcurl4t64(pg_duckdb's built-in httpfs links it) andca-certificates.mooncake.data_directoryis/var/lib/postgresql/18/mooncake: in the volume, next to PGDATA (/var/lib/postgresql/18/docker) and outside it, so pg_basebackup works. The image creates the directory owned by postgres, so a new volume gets it.make helplists every target.Testing
cargo fmt --check,cargo clippy -- -D warningsmake test, PostgreSQL 14.18 / 17.10 / 18.4python3 tests/mirror_diff/test_mirror_diff.pyscripts/e2e-smoke.sh 17, all scenarios, cassert buildGATE e2e-smoke PASS: pass=447 fail=0 skip=0 xfail_limitation=15scripts/e2e-smoke.sh 18, all scenarios, cassert buildGATE e2e-smoke PASS: pass=447 fail=0 skip=0 xfail_limitation=15The
basebackupscenario (PostgreSQL 17 and 18):-Fp,-Ftand-Rsucceed withmooncake.data_directoryoutside the data directory.mooncake.data_directory "..." is in use by another server.mooncake.list_tables()is empty;Table postgres.public.b_mc not found;mooncake.create_tableresnapshots it (1001 rows) and CDC continues.mooncake.list_tables()fails withfailed to connect to moonlink;pg_ctl promotethe worker starts and re-creating the mirror resnapshots it.This pull request and its description were written by Isaac.