Repository navigation
Release docs for 0.3.0: first release of the moonlink design, no 0.2 upgrade - #203
Merged
Merged
Conversation
pg_mooncake 0.2.0 was never released; the only releases are 0.1.x, the earlier columnstore extension. So 0.3.0 is the first release of the moonlink-based design, and there is no upgrade from 0.2. Remove the 0.2.0 to 0.3.0 upgrade script, and rewrite the docs for someone new to 0.3. - CHANGELOG 0.3.0: an overview, usage, and "Behaviour and guarantees" in place of the Changed, Fixed and Fixed in moonlink lists, keeping the moonlink PR links, then the known limitations. Nothing is framed against 0.2, and each item is stated once: renames, unsupported types, numeric, infinite dates and timestamps and the create_table restart cases are known limitations. The Performance marker asks for absolute numbers. - README: "Coming from 0.1.x" replaces the upgrade section. The Known Issues section is folded into Security, which now says the privileges story once, and the ClickBench claim about the 0.1 engine is gone. - AGENTS.md is the contributor guide. It points to docs/ARCHITECTURE.md for the submodule pins and the wire format and to the CHANGELOG for known limitations, and drops the 0.2 quoted-name note and the upgrade-script rule. - docs/ARCHITECTURE.md lists the pins once, with the pg_duckdb tag, and loses the 0.2 _wal/ notes. - TESTING.md matches the harness and CI: run times, the mirror_writes contract, the known limitations and what GATE PASS means, and mirror_diff as an advisory test with a link to its README. - LICENSE: 2024-2026. Co-authored-by: Isaac <no-reply@databricks.com>
- CHANGELOG: fill in the Performance section with the release benchmark's numbers (write amplification,
freshness, scan latency, initial copy), which removes its marker. Supported types are "stored faithfully",
with the exceptions in Known limitations. Document table_config's keys. Name S3 among what this release
doesn't include. Multi-dimensional arrays are described the same way in both places. Remove the unmeasured
write-amplification claim and the build-toolchain note.
- Known limitations: cross-session freshness, pg_basebackup failing on a server with mirrors (moonlink's file
names exceed the 99 characters Postgres' tar format allows), and ALTER DATABASE RENAME failing while a
database has mirrors.
- README:
- freshness: about half a second from another session, and immediately in the writing session. The Quick
Start reads in the same session.
- The pg_duckdb build step is skipped only for this bundled fork, and the server dev package comes from PGDG.
- Roles lists why create_table needs a superuser in practice. pg_hba needs no replication lines for logical
replication.
- New "Table settings" paragraph; database renames and backups are described as tested.
- "Coming from 0.1.x" is a step list: keep data and keys, drop the extension, delete 0.1's data files, and
configure Postgres before CREATE EXTENSION.
- TESTING.md and docs/ARCHITECTURE.md: wording fixes.
Co-authored-by: Isaac <no-reply@databricks.com>
With mirrors under $PGDATA/pg_mooncake, pg_basebackup fails while any mirror exists: moonlink's file names there are longer than the 99 characters Postgres' tar format allows, which also prevents creating a standby with it. The upcoming mooncake.data_directory setting can move them outside the data directory. A new README section, "Backups and standbys", says so, and that mirror data is then not in base backups, so restored mirrors are dropped and re-created. "Where data lives", the CHANGELOG's Operations and its Known limitations entry point to the setting. Both statements of the setting carry a temporary release marker until its code lands. Co-authored-by: Isaac <no-reply@databricks.com>
Known limitations now list the column types mirrors store, as moonlink's type check accepts them, instead of naming only interval and inet. They also say that unconstrained numeric values and numeric array elements read back with 10 fractional digits, that numeric NaN, Infinity and values that don't fit fail create_table or error the mirror, and that enum columns compare and sort as text on a mirror, with comparisons against a literal failing in DuckDB. The README gains a short "Column types" note that points there. Co-authored-by: Isaac <no-reply@databricks.com>
- mooncake.data_directory as implemented: read at server start, pg_mooncake under the data directory by default or an absolute path, holding all of moonlink's files and its socket. The README covers its startup checks, the refusal to share it with another server (a copy made with pg_basebackup on the same host inherits it), and why pg_basebackup needs it outside the data directory. It also says what a restored copy shows, and that moonlink doesn't run on standbys but starts on promotion. The temporary release markers go. - CHANGELOG: the setting, standby behaviour, create_table errors that name their cause, and the Docker image's contents. The README's Docker marker now asks for the image to be published. - docs/ARCHITECTURE.md and AGENTS.md: the worker starts when recovery finishes, and the on-disk state, metadata store and ATTACH socket path follow mooncake.data_directory. - TESTING.md: the e2e job needs exit 0 and GATE e2e-smoke PASS, fmt also runs the mirror_diff unit tests, clippy runs with the default features, and there is a new docker build job. It also covers the basebackup scenario, the output without INFO lines, the known-bugs line format, and cluster sockets in E2E_DIR. Co-authored-by: Isaac <no-reply@databricks.com>
Known limitations and the create_table behaviour now cover what tests/mirror_diff expects and the CHANGELOG didn't say, each checked on PostgreSQL 17 at moonlink 0e38b43: - create_table refuses unlogged tables, not only unlogged partitions. - An array of a domain over a composite or an array type is refused, as are range types of any subtype, including ranges over a domain. Other arrays, composites and domains of supported types are stored. - Arrays with explicit bounds can't be stored, like multi-dimensional ones: they fail create_table, or error the mirror when written later. - A source gaining inheritance children puts its mirror into the errored state. - Queries on mirrors follow DuckDB's semantics in places: integer division, the regex operator matching the whole string, and results that depend on row order without ORDER BY. TESTING.md gives mirror_diff's measured run time, about 45 minutes on 8 cores. Co-authored-by: Isaac <no-reply@databricks.com>
…ding Match the code cleanup's latest changes: - The Docker image keeps mirrors in /var/lib/postgresql/18/mooncake, inside its volume but outside the data directory, so pg_basebackup works in it. With a bind mount or --user, that directory has to be created first, owned by the server's user with mode 0700. - mooncake.data_directory is locked through moonlink.lock: one server per directory. A refused start names the holder's data directory and PID, and the lock file stays after shutdown. The directory is checked like a data directory (mode 0700 or 0750, no symbolic link, not the data directory or a parent of it), and ./pg_mooncake counts as the default. - moonlink stores src_uri in its metadata store and repeats it in an error, so the docs advise a src_uri without a password. pg_mooncake's own connection errors show only host, port, database and user, in the format the CHANGELOG now gives. - pg_mooncake must be preloaded; otherwise CREATE EXTENSION, LOAD and pg_upgrade's library check get a clear ERROR. - TESTING.md: the e2e preflight needs python3, the basebackup scenario checks the lock and refused directories, and the docker job runs the image. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
pg_mooncake 0.2.0 was never released. The only releases are 0.1.0 to 0.1.2, the earlier columnstore extension, which is a different design. So 0.3.0 is the first release of the moonlink-based pg_mooncake, and there is no upgrade from 0.2 to support. This PR removes the upgrade script and rewrites the release docs for someone new to 0.3.0. It touches only the docs,
LICENSEandsql/, and is based on the final moonlink pin PR.Changes
sql/:pg_mooncake--0.2.0--0.3.0.sqlis deleted, which leavessql/empty.COPY sql sqlnow has nothing to copy, sodocker buildfails until that line goes. It belongs to the code and CI cleanup (Dockerfile owners), so it is not changed here. CI doesn't build the image.pg_mooncake.controlneeds no change.CHANGELOG, 0.3.0, rewritten as a first release:
mooncake.*procedures,DROP TABLE, and identifier rules;mooncake.load_filesisn't mentioned, and the moonlink features this release doesn't include are one line in the overview.interval/inetand unconstrainednumericare described only in Known limitations;create_tablefails, or the mirror errors), and multi-dimensional array values;DROP TABLEwhile moonlink is down" joins theDROP EXTENSION ... CASCADEleftover entry.README
DROP EXTENSION pg_mooncake CASCADE, then install 0.3.0.AGENTS.md is now titled as the contributor guide; the README's Contributing section already points to it.
[pins]statements point to the one pin list in docs/ARCHITECTURE.md.e2e-known-bugs.txt"the complete list". Section 7 is now one paragraph that points to the CHANGELOG.make check-rust-postgres).docs/ARCHITECTURE.md
v1.0.0-mooncake.1and no product version._wal/startup warning and data-directory row are removed.TESTING.md
v0.3-dev, and thefmtjob also runsmake check-rust-postgres.mirror_writesrequiresERROR: cannot modify mirrorfor every write form; the text said no code emitted it yet.XFAIL(limitation:...);GATE e2e-smoke PASS(exit 0) means no unexpected failure and no SKIP;LICENSE: 2024-2026.
Coordination with the other cleanup streams
COPY sql sqlmust be removed along with this PR (see above).scripts/e2e-smoke.shstill reportsBLOCKEDandXFAIL(<bugs>)for blocking entries. Every entry in today'sscripts/e2e-known-bugs.txtis alreadyclass=limitation, so a run on these pins shows none of them. The CIe2ejob's description is the current workflow: unexpected failures, skips aboveE2E_MAX_SKIP, or no passing check.tests/mirror_diff/README.md(line 169) linksTESTING.md#release-policy-blocking-and-limitation, which no longer exists. Its owner should point it at#known-limitations-and-the-gate, or drop it.TableName::moonlink_keysandlegacy_moonlink_name(src/functions.rs) are the code behind the removed quoted-name note. Their removal is in the code cleanup stream.Review fixes (second commit)
table_config, checked against moonlink0e38b435table_config.rsandconfig_utils.rs:mooncake.skip_index_merge,mooncake.skip_data_compactionandmooncake.mem_slice_bytes(default 16 MiB);create_tablealways overridesappend_onlyandrow_identity, unknown keys are ignored, and"iceberg"storage is documented as not supported in this release;NULLsrc_uri, an unknown key is ignored, andmem_slice_bytes: 0is rejected.v0.1.2source:$PGDATA/mooncake_local_tables/and its cache in$PGDATA/mooncake_local_cache/, or undermooncake_<db>_<table>_<oid>/inmooncake.default_bucket;CREATE TABLE AS, and configure Postgres beforeCREATE EXTENSION.ALTER DATABASE ... RENAME TOfails withdatabase "d1" is being accessed by other userswhile the database has mirrors, because moonlink's walsenders use it. The README now says so.pg_hba.confwith everyreplicationline removed:create_tableand replication work, because logical replication connections match the per-database entries. The README says those lines aren't needed, the opposite of the review's suggestion.CREATEon the schema,USAGEon schemamooncakeandREPLICATIONcreated a mirror; the publication already existed. Roles now lists whatcreate_tabledoes and says "in practice a superuser".pg_basebackupfails on a server with mirrors:ERROR: file name too long for tar format: "pg_mooncake/postgres/public.s_mc/data/ic-<uuid>-<uuid>.parquet". Postgres limits tar member names to 99 characters (src/port/tar.c), and moonlink's paths reach 124. After dropping every mirror,pg_basebackupworks. This is now a Known limitation and replaces the old Backups text, which assumedpg_basebackupworks. It also meanspg_basebackupcan't create a standby while mirrors exist.create_tablefor columns declared with more than one dimension; a multi-dimensional value in a one-dimensional column errors the mirror.0e38b435.LoadFilesrow and step 6 are reworded and wrapped.Backups and standbys (third commit)
pg_basebackupfails while mirrors exist, and the fix is a new server setting,mooncake.data_directory, coming from the code cleanup. This commit documents it:pg_mooncakeunder$PGDATA,pg_basebackupfails while any mirror exists, because of the tar name limit, so it can't create a standby either;mooncake.data_directoryto an absolute path outside$PGDATAmakespg_basebackupand standbys work;<!-- RC: confirm mooncake.data_directory semantics from the code PR -->until that PR lands. They are removed before the tag.$PGDATA/pg_mooncake)" is left for the code PR to update, along with the setting.Column types, numeric and enums (fourth commit)
From the mirror_diff run:
intervalandinetnow lists the column types mirrors store. The list is taken from moonlink's type check at0e38b435(TextFormatConverter::is_supported_type, plus the numeric typmod check):numericwith precision up to 38 and scale from 0 to the precision, arrays, composites and domains built on supported types, and enums. Every other type is refused atcreate_table, with examples.numeric:numericandnumericarray elements are stored asdecimal(38, 10), so they read back with 10 fractional digits (1as1.0000000000, andnumeric(10,2)[]{1.50}as{1.5000000000}, checked on PG17);NaN,Infinity, too many digits) failcreate_table, or error the mirror when written later.ORDER BYgivesblue, green, orange, purple, red, yellow, andmin/maxgiveblue/yellow;WHERE c = 'red'fails withType with name rainbow does not exist!, whilec::text = 'red'works.Matching the code cleanup (fifth commit)
The docs now describe the code cleanup PR as implemented, taken from its code and test results:
mooncake.data_directory:pg_mooncakeunder the data directory by default, or an absolute path;moonlink.sock;pg_basebackupon the same host inherits the setting;WARNINGfor a non-default directory inside the data directory, andpg_basebackupneeds a directory outside it;list_tables(), reading a mirror errors withTable ... not found, and drop plus re-create takes a new snapshot.create_tableerrors that name their cause, and the Docker image (postgres:18with the bundled pg_duckdb and pg_mooncake built from source). The README's Docker marker stays, because the image isn't published yet, but now asks for it to be published from this Dockerfile.<mooncake.data_directory>/moonlink.sock. The metadata store and the on-disk state section follow the setting too.GATE e2e-smoke PASS, so theE2E_MAX_SKIPtext is gone;fmtalso runstest_mirror_diff.py, clippy runs with the default features, and there is a newdockerbuild job;basebackup;INFOlines,KNOWNstatuses are FIXED, OPEN, UNKNOWN and NOT-RUN, and the known-bugs format isbug <name> <description>pluschecklines;E2E_DIR.git merge-tree).Limitations from the mirror_diff rules (sixth commit)
These are limitations the mirror_diff rules treat as expected but the CHANGELOG didn't mention. Each was checked on PostgreSQL 17 at moonlink
0e38b43:create_table. Postgres itself rejects them:cannot add relation "u" to publication.dpair[],dints[]) are refused, and so are range types, including a range over a domain. A domain over a composite, an array of composites, and a domain overint[]are all stored. Known limitations now states the exception.'[0:2]={1,2,3}'):create_tablefails on a source that holds one (invalid array: multi-dimensional arrays and arrays with explicit bounds are not supported), and one written later errors the mirror. These are documented together with multi-dimensional arrays.the source table changed: it gained inheritance children). This is now in the errored-state bullet.sum(1 / n)is 0 on the source and 0.75 on the mirror for n = 2 and 4;~matches the whole string:'abc1def' ~ '[0-9]'counts 1 row on the source and 0 on the mirror;ORDER BY, such asROWSwindow frames and ties, can differ.Code cleanup follow-ups (seventh commit)
This commit documents what the code cleanup PR added in its review round:
mooncake.data_directoryis/var/lib/postgresql/18/mooncake, inside the volume but outside the data directory, sopg_basebackupworks in it. With a bind mount or--user,18/mooncakemust exist first, owned by the server's user with mode 0700. The README's Docker marker stays, because the image isn't published yet.moonlink.lock: one server per directory. A second server's FATAL names the holder's data directory and postmaster PID. The file stays after shutdown, which is harmless. It is also a row in ARCHITECTURE.md's on-disk table../pg_mooncakecounts as the default.src_uri: moonlink stores it inmoonlink_metadata_store.sqliteundermooncake.data_directory, and itsTable <src_uri> not founderror repeats it. The docs advise asrc_uriwithout a password: in the README, and as a Known limitation. pg_mooncake's own connection errors show only host, port, database and user, and the CHANGELOG gives their format.CREATE EXTENSION,LOAD,pg_upgrade's library check), pg_mooncake raisesERROR: pg_mooncake must be loaded via shared_preload_libraries(55000). This is in the README and the CHANGELOG.Release markers
scripts/check-release-markers.shfinds exactly four: the release date (CHANGELOG.md:1), and in the README the Docker image (:28, reworded to "publish the image"), the PG version matrix (:44) andsrc_uri(:182). The temporarymooncake.data_directorymarkers are gone.Tests run
#anchorsin CHANGELOG.md, README.md, AGENTS.md, docs/ARCHITECTURE.md and TESTING.md (script that resolves each file and GitHub heading slug)tests/mirror_diff/README.mdone abovescripts/check-release-markers.sh-D warningsandmake test, PG17make package, PG17 (run on an earlier build of this change, which had the samesql/deletion)pg_mooncake.so,pg_mooncake--0.3.0.sqlandpg_mooncake.control.CREATE EXTENSION pg_mooncake CASCADEinstalls 0.3.0,pg_extension_update_pathsis empty, and a mirror matches its sourceThis pull request and its description were written by Isaac.