Skip to content

Release docs for 0.3.0: first release of the moonlink design, no 0.2 upgrade - #203

Merged
zhousun merged 7 commits into
v0.3-devfrom
v0.3/14-release-docs
Nov 17, 2026
Merged

zhousun merged 7 commits into
v0.3-devfrom
v0.3/14-release-docs

Conversation

@zhousun

@zhousun zhousun commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

What and why

pg_mooncake 0.2.0 was never released. The only releases are 0.1.0 to 0.1.2, the earlier columnstore extension, which is a different design. So 0.3.0 is the first release of the moonlink-based pg_mooncake, and there is no upgrade from 0.2 to support. This PR removes the upgrade script and rewrites the release docs for someone new to 0.3.0. It touches only the docs, LICENSE and sql/, and is based on the final moonlink pin PR.

Changes

sql/: pg_mooncake--0.2.0--0.3.0.sql is deleted, which leaves sql/ empty.

  • The Dockerfile's COPY sql sql now has nothing to copy, so docker build fails until that line goes. It belongs to the code and CI cleanup (Dockerfile owners), so it is not changed here. CI doesn't build the image.
  • pg_mooncake.control needs no change.

CHANGELOG, 0.3.0, rewritten as a first release:

  • an introduction: what pg_mooncake 0.3.0 does, and that nothing is migrated from 0.1.x;
  • Overview: what a mirror is, where moonlink runs and stores data, what this release doesn't include, and the bundled pg_duckdb tag;
  • Usage: the four mooncake.* procedures, DROP TABLE, and identifier rules;
  • Behaviour and guarantees, in place of "Changed", "Fixed" and "Fixed in moonlink":
    • covers creating a mirror, values, replication and durability, reads, the errored state, mirror tables, and operations;
    • each item is a statement of behaviour, keeping the moonlink PR links.
  • Nothing is framed against 0.2, and "Before, ..." is gone.
  • "Removed" is gone: mooncake.load_files isn't mentioned, and the moonlink features this release doesn't include are one line in the overview.
  • moonlink PR 2370 (initial-copy speed) is no longer cited, because its only line was the comparison with 0.2.
  • Each item is stated once:
    • renames, interval/inet and unconstrained numeric are described only in Known limitations;
    • the PR 2373/2374 bullet is three lines, and the restart cases moved to Known limitations;
    • new Known limitations entries: infinite dates and timestamps (create_table fails, or the mirror errors), and multi-dimensional array values;
    • "a DROP TABLE while moonlink is down" joins the DROP EXTENSION ... CASCADE leftover entry.
  • The Performance marker asks for absolute numbers: write amplification, apply lag (p50 and p95), initial copy throughput (rows/s), and scan latency.

README

  • "Upgrading from 0.2" is replaced by "Coming from 0.1.x":
    • 0.3.0 is a new architecture: mirrors of regular tables through logical replication, instead of columnstore tables;
    • nothing is migrated: copy any data you want to keep, DROP EXTENSION pg_mooncake CASCADE, then install 0.3.0.
  • "Known Issues" is folded into "Security", so the privileges story is told once. Roles keeps the requirements, and Security keeps the implications and links to the CHANGELOG's Known limitations.
  • The ClickBench "top 10" claim (the 0.1 engine) and its link are removed.

AGENTS.md is now titled as the contributor guide; the README's Contributing section already points to it.

  • [pins] statements point to the one pin list in docs/ARCHITECTURE.md.
  • Section 1 says "do not change the wire format" and links the description in ARCHITECTURE.md.
  • Section 2, item 5 links the drop flow in ARCHITECTURE.md.
  • Removed:
    • the 0.2 quoted-name invariant;
    • the upgrade-scripts table row and recipe step;
    • section 7's copy of the limitations. It also wrongly called e2e-known-bugs.txt "the complete list". Section 7 is now one paragraph that points to the CHANGELOG.
  • Pitfall 5 had gone stale: rust-postgres is pinned to moonlink's revision (make check-rust-postgres).

docs/ARCHITECTURE.md

  • The pins are listed once at the top, with the pg_duckdb tag v1.0.0-mooncake.1 and no product version.
  • The wire format section links the invariants in AGENTS.md section 1, and step 6 links section 2, item 5.
  • The 0.2 _wal/ startup warning and data-directory row are removed.

TESTING.md

  • CI runs on "the development branches", not v0.3-dev, and the fmt job also runs make check-rust-postgres.
  • The e2e run time is about 25 minutes, both locally and in CI (it said 20 and 35).
  • mirror_writes requires ERROR: cannot modify mirror for every write form; the text said no code emitted it yet.
  • The unfinished note about an empty extension directory is replaced: the community build has no build record, so the gate can't pass on it.
  • The marker sentence about "the moonlink submodule bump" is removed.
  • The blocking/limitation policy and the long setup-error list are replaced by a short section:
    • known-bugs entries are documented limitations, which report XFAIL(limitation:...);
    • GATE e2e-smoke PASS (exit 0) means no unexpected failure and no SKIP;
    • adding a limitation is a release decision.
  • mirror_diff is down to three lines, marked advisory and not a release gate, with a link to its README.

LICENSE: 2024-2026.

Coordination with the other cleanup streams

  • Dockerfile: COPY sql sql must be removed along with this PR (see above).
  • TESTING.md describes the e2e gate as it will be once the unused blocking/requires machinery is removed: only limitation entries, and gate states PASS, FAIL and INCOMPLETE. Until then, scripts/e2e-smoke.sh still reports BLOCKED and XFAIL(<bugs>) for blocking entries. Every entry in today's scripts/e2e-known-bugs.txt is already class=limitation, so a run on these pins shows none of them. The CI e2e job's description is the current workflow: unexpected failures, skips above E2E_MAX_SKIP, or no passing check.
  • tests/mirror_diff/README.md (line 169) links TESTING.md#release-policy-blocking-and-limitation, which no longer exists. Its owner should point it at #known-limitations-and-the-gate, or drop it.
  • Code: TableName::moonlink_keys and legacy_moonlink_name (src/functions.rs) are the code behind the removed quoted-name note. Their removal is in the code cleanup stream.

Review fixes (second commit)

  • Performance: filled in with the release benchmark's numbers. The Performance marker is gone, and 4 markers remain. The numbers come from the project's benchmark, which isn't in this repository, so there is no link.
  • Freshness: the README overview says changes are visible about half a second later from another session, and immediately in the writing session. The Quick Start reads in the same session, a paragraph after it explains why, and Known limitations has the cross-session delay.
  • table_config, checked against moonlink 0e38b435 table_config.rs and config_utils.rs:
    • the supported keys are mooncake.skip_index_merge, mooncake.skip_data_compaction and mooncake.mem_slice_bytes (default 16 MiB);
    • create_table always overrides append_only and row_identity, unknown keys are ignored, and "iceberg" storage is documented as not supported in this release;
    • checked on a live cluster: the README example works with a NULL src_uri, an unknown key is ignored, and mem_slice_bytes: 0 is rejected.
  • "Coming from 0.1.x", checked against the v0.1.2 source:
    • 0.1 stored local data in $PGDATA/mooncake_local_tables/ and its cache in $PGDATA/mooncake_local_cache/, or under mooncake_<db>_<table>_<oid>/ in mooncake.default_bucket;
    • it has no code that deletes them when a table or the extension is dropped, so the README says to delete them;
    • also added: re-add the keys after CREATE TABLE AS, and configure Postgres before CREATE EXTENSION.
  • Claims tested on PostgreSQL 17:
    • ALTER DATABASE ... RENAME TO fails with database "d1" is being accessed by other users while the database has mirrors, because moonlink's walsenders use it. The README now says so.
    • pg_hba.conf with every replication line removed: create_table and replication work, because logical replication connections match the per-database entries. The README says those lines aren't needed, the opposite of the review's suggestion.
    • A non-superuser that owns the source and has CREATE on the schema, USAGE on schema mooncake and REPLICATION created a mirror; the publication already existed. Roles now lists what create_table does and says "in practice a superuser".
    • pg_basebackup fails on a server with mirrors: ERROR: file name too long for tar format: "pg_mooncake/postgres/public.s_mc/data/ic-<uuid>-<uuid>.parquet". Postgres limits tar member names to 99 characters (src/port/tar.c), and moonlink's paths reach 124. After dropping every mirror, pg_basebackup works. This is now a Known limitation and replaces the old Backups text, which assumed pg_basebackup works. It also means pg_basebackup can't create a standby while mirrors exist.
  • Smaller fixes:
    • pg_duckdb build step: "skip only if this bundled pg_duckdb (v1.0.0-mooncake.1) is installed".
    • The server dev package comes from PGDG.
    • Supported types are "stored faithfully", with the exceptions in Known limitations.
    • Multi-dimensional arrays: refused at create_table for columns declared with more than one dimension; a multi-dimensional value in a one-dimensional column errors the mirror.
    • S3 is named among what this release doesn't include. GCS isn't, since it doesn't exist at 0e38b435.
    • Removed: the unmeasured write-amplification line, and the Rust-toolchain note.
    • TESTING.md says "among the"; the ARCHITECTURE.md LoadFiles row and step 6 are reworded and wrapped.

Backups and standbys (third commit)

pg_basebackup fails while mirrors exist, and the fix is a new server setting, mooncake.data_directory, coming from the code cleanup. This commit documents it:

  • README "Backups and standbys" (new section):
    • with the default pg_mooncake under $PGDATA, pg_basebackup fails while any mirror exists, because of the tar name limit, so it can't create a standby either;
    • setting mooncake.data_directory to an absolute path outside $PGDATA makes pg_basebackup and standbys work;
    • mirror data is then not in base backups, so after a restore the mirrors are dropped and re-created.
  • README "Where data lives" names the setting.
  • CHANGELOG: a line under Operations for the setting, and the Known limitations entry is rewritten around the default configuration.
  • Temporary markers: both descriptions of the setting carry <!-- RC: confirm mooncake.data_directory semantics from the code PR --> until that PR lands. They are removed before the tag.
  • docs/ARCHITECTURE.md's "On-disk state ($PGDATA/pg_mooncake)" is left for the code PR to update, along with the setting.

Column types, numeric and enums (fourth commit)

From the mirror_diff run:

  • Supported types: the Known limitations entry that named only interval and inet now lists the column types mirrors store. The list is taken from moonlink's type check at 0e38b435 (TextFormatConverter::is_supported_type, plus the numeric typmod check): numeric with precision up to 38 and scale from 0 to the precision, arrays, composites and domains built on supported types, and enums. Every other type is refused at create_table, with examples.
  • numeric:
    • unconstrained numeric and numeric array elements are stored as decimal(38, 10), so they read back with 10 fractional digits (1 as 1.0000000000, and numeric(10,2)[] {1.50} as {1.5000000000}, checked on PG17);
    • values that don't fit (NaN, Infinity, too many digits) fail create_table, or error the mirror when written later.
  • Enums: they compare and sort as text on a mirror. Checked on PG17:
    • ORDER BY gives blue, green, orange, purple, red, yellow, and min/max give blue/yellow;
    • WHERE c = 'red' fails with Type with name rainbow does not exist!, while c::text = 'red' works.
    • No earlier enum entry existed to merge with, so this is one new entry covering both.
  • README: a short "Column types" note that points to Known limitations.

Matching the code cleanup (fifth commit)

The docs now describe the code cleanup PR as implemented, taken from its code and test results:

  • mooncake.data_directory:
    • read at server start; pg_mooncake under the data directory by default, or an absolute path;
    • holds all of moonlink's files and moonlink.sock;
    • startup checks: created 0700, owned by the server user and writable, socket path at most 107 bytes;
    • refuses to start when another server's moonlink uses the directory, and a copy made with pg_basebackup on the same host inherits the setting;
    • WARNING for a non-default directory inside the data directory, and pg_basebackup needs a directory outside it;
    • a restored copy has empty list_tables(), reading a mirror errors with Table ... not found, and drop plus re-create takes a new snapshot.
    • The two temporary markers are removed.
  • Standbys: moonlink starts when recovery finishes. So it doesn't run on standbys, and mirrors can't be read there. It starts on promotion, and a promoted standby keeps the mirrors whose files are in its directory. This is in the README, the CHANGELOG (Operations and Known limitations) and ARCHITECTURE.md step 2.
  • CHANGELOG: create_table errors that name their cause, and the Docker image (postgres:18 with the bundled pg_duckdb and pg_mooncake built from source). The README's Docker marker stays, because the image isn't published yet, but now asks for it to be published from this Dockerfile.
  • AGENTS.md section 1 and the ARCHITECTURE diagram: the socket is <mooncake.data_directory>/moonlink.sock. The metadata store and the on-disk state section follow the setting too.
  • TESTING.md:
    • the e2e job needs exit 0 and GATE e2e-smoke PASS, so the E2E_MAX_SKIP text is gone;
    • fmt also runs test_mirror_diff.py, clippy runs with the default features, and there is a new docker build job;
    • the scenario list covers basebackup;
    • the output has no INFO lines, KNOWN statuses are FIXED, OPEN, UNKNOWN and NOT-RUN, and the known-bugs format is bug <name> <description> plus check lines;
    • cluster sockets are in E2E_DIR.
  • This commit merges cleanly with the code cleanup PR (checked with git merge-tree).

Limitations from the mirror_diff rules (sixth commit)

These are limitations the mirror_diff rules treat as expected but the CHANGELOG didn't mention. Each was checked on PostgreSQL 17 at moonlink 0e38b43:

  • Unlogged tables are refused by create_table. Postgres itself rejects them: cannot add relation "u" to publication.
  • Arrays of a domain over a composite or an array (dpair[], dints[]) are refused, and so are range types, including a range over a domain. A domain over a composite, an array of composites, and a domain over int[] are all stored. Known limitations now states the exception.
  • Arrays with explicit bounds ('[0:2]={1,2,3}'): create_table fails on a source that holds one (invalid array: multi-dimensional arrays and arrays with explicit bounds are not supported), and one written later errors the mirror. These are documented together with multi-dimensional arrays.
  • A source that gains inheritance children errors the mirror (the source table changed: it gained inheritance children). This is now in the errored-state bullet.
  • DuckDB semantics, one new entry:
    • integer division doesn't truncate: sum(1 / n) is 0 on the source and 0.75 on the mirror for n = 2 and 4;
    • ~ matches the whole string: 'abc1def' ~ '[0-9]' counts 1 row on the source and 0 on the mirror;
    • results that depend on row order without ORDER BY, such as ROWS window frames and ties, can differ.
    • Integer overflow errors on both engines, so it isn't listed.
  • README "Column types" mentions the exceptions and DuckDB semantics.
  • TESTING.md: a full mirror_diff run takes about 45 minutes on 8 cores.

Code cleanup follow-ups (seventh commit)

This commit documents what the code cleanup PR added in its review round:

  • Docker: the image's mooncake.data_directory is /var/lib/postgresql/18/mooncake, inside the volume but outside the data directory, so pg_basebackup works in it. With a bind mount or --user, 18/mooncake must exist first, owned by the server's user with mode 0700. The README's Docker marker stays, because the image isn't published yet.
  • moonlink.lock: one server per directory. A second server's FATAL names the holder's data directory and postmaster PID. The file stays after shutdown, which is harmless. It is also a row in ARCHITECTURE.md's on-disk table.
  • Directory checks: mode 0700 or 0750, not a symbolic link, not the data directory or a parent of it; ./pg_mooncake counts as the default.
  • src_uri: moonlink stores it in moonlink_metadata_store.sqlite under mooncake.data_directory, and its Table <src_uri> not found error repeats it. The docs advise a src_uri without a password: in the README, and as a Known limitation. pg_mooncake's own connection errors show only host, port, database and user, and the CHANGELOG gives their format.
  • Preloading: loaded any other way (CREATE EXTENSION, LOAD, pg_upgrade's library check), pg_mooncake raises ERROR: pg_mooncake must be loaded via shared_preload_libraries (55000). This is in the README and the CHANGELOG.
  • TESTING.md:
    • the e2e preflight needs python3;
    • the basebackup scenario checks that a standby's locked directory and refused directories (a symbolic link, mode 0777, the data directory or its parent) can't be used;
    • the docker job builds and runs the image.
  • This commit merges cleanly with the code cleanup PR's head.

Release markers

scripts/check-release-markers.sh finds exactly four: the release date (CHANGELOG.md:1), and in the README the Docker image (:28, reworded to "publish the image"), the PG version matrix (:44) and src_uri (:182). The temporary mooncake.data_directory markers are gone.

Tests run

Check Result
Relative links and #anchors in CHANGELOG.md, README.md, AGENTS.md, docs/ARCHITECTURE.md and TESTING.md (script that resolves each file and GitHub heading slug) All resolve. Over every Markdown file in the repo, the only broken link is the tests/mirror_diff/README.md one above
scripts/check-release-markers.sh Exactly the 4 markers above
clippy -D warnings and make test, PG17 Clean, 9/9 (on both commits)
Fresh install from make package, PG17 (run on an earlier build of this change, which had the same sql/ deletion) The package holds only pg_mooncake.so, pg_mooncake--0.3.0.sql and pg_mooncake.control. CREATE EXTENSION pg_mooncake CASCADE installs 0.3.0, pg_extension_update_paths is empty, and a mirror matches its source

This pull request and its description were written by Isaac.

zhousun and others added 7 commits November 17, 2026 00:26
pg_mooncake 0.2.0 was never released; the only releases are 0.1.x, the earlier columnstore extension. So 0.3.0 is
the first release of the moonlink-based design, and there is no upgrade from 0.2. Remove the 0.2.0 to 0.3.0
upgrade script, and rewrite the docs for someone new to 0.3.

- CHANGELOG 0.3.0: an overview, usage, and "Behaviour and guarantees" in place of the Changed, Fixed and Fixed in
  moonlink lists, keeping the moonlink PR links, then the known limitations. Nothing is framed against 0.2, and each
  item is stated once: renames, unsupported types, numeric, infinite dates and timestamps and the create_table
  restart cases are known limitations. The Performance marker asks for absolute numbers.
- README: "Coming from 0.1.x" replaces the upgrade section. The Known Issues section is folded into Security, which
  now says the privileges story once, and the ClickBench claim about the 0.1 engine is gone.
- AGENTS.md is the contributor guide. It points to docs/ARCHITECTURE.md for the submodule pins and the wire format
  and to the CHANGELOG for known limitations, and drops the 0.2 quoted-name note and the upgrade-script rule.
- docs/ARCHITECTURE.md lists the pins once, with the pg_duckdb tag, and loses the 0.2 _wal/ notes.
- TESTING.md matches the harness and CI: run times, the mirror_writes contract, the known limitations and what
  GATE PASS means, and mirror_diff as an advisory test with a link to its README.
- LICENSE: 2024-2026.

Co-authored-by: Isaac <no-reply@databricks.com>
- CHANGELOG: fill in the Performance section with the release benchmark's numbers (write amplification,
  freshness, scan latency, initial copy), which removes its marker. Supported types are "stored faithfully",
  with the exceptions in Known limitations. Document table_config's keys. Name S3 among what this release
  doesn't include. Multi-dimensional arrays are described the same way in both places. Remove the unmeasured
  write-amplification claim and the build-toolchain note.
- Known limitations: cross-session freshness, pg_basebackup failing on a server with mirrors (moonlink's file
  names exceed the 99 characters Postgres' tar format allows), and ALTER DATABASE RENAME failing while a
  database has mirrors.
- README:
  - freshness: about half a second from another session, and immediately in the writing session. The Quick
    Start reads in the same session.
  - The pg_duckdb build step is skipped only for this bundled fork, and the server dev package comes from PGDG.
  - Roles lists why create_table needs a superuser in practice. pg_hba needs no replication lines for logical
    replication.
  - New "Table settings" paragraph; database renames and backups are described as tested.
  - "Coming from 0.1.x" is a step list: keep data and keys, drop the extension, delete 0.1's data files, and
    configure Postgres before CREATE EXTENSION.
- TESTING.md and docs/ARCHITECTURE.md: wording fixes.

Co-authored-by: Isaac <no-reply@databricks.com>
With mirrors under $PGDATA/pg_mooncake, pg_basebackup fails while any mirror exists: moonlink's file names
there are longer than the 99 characters Postgres' tar format allows, which also prevents creating a standby with
it. The upcoming mooncake.data_directory setting can move them outside the data directory. A new README section,
"Backups and standbys", says so, and that mirror data is then not in base backups, so restored mirrors are
dropped and re-created. "Where data lives", the CHANGELOG's Operations and its Known limitations entry point to
the setting. Both statements of the setting carry a temporary release marker until its code lands.

Co-authored-by: Isaac <no-reply@databricks.com>
Known limitations now list the column types mirrors store, as moonlink's type check accepts them, instead of naming
only interval and inet. They also say that unconstrained numeric values and numeric array elements read back with
10 fractional digits, that numeric NaN, Infinity and values that don't fit fail create_table or error the mirror,
and that enum columns compare and sort as text on a mirror, with comparisons against a literal failing in DuckDB.
The README gains a short "Column types" note that points there.

Co-authored-by: Isaac <no-reply@databricks.com>
- mooncake.data_directory as implemented: read at server start, pg_mooncake under the data directory by default or
  an absolute path, holding all of moonlink's files and its socket. The README covers its startup checks, the
  refusal to share it with another server (a copy made with pg_basebackup on the same host inherits it), and why
  pg_basebackup needs it outside the data directory. It also says what a restored copy shows, and that moonlink
  doesn't run on standbys but starts on promotion. The temporary release markers go.
- CHANGELOG: the setting, standby behaviour, create_table errors that name their cause, and the Docker image's
  contents. The README's Docker marker now asks for the image to be published.
- docs/ARCHITECTURE.md and AGENTS.md: the worker starts when recovery finishes, and the on-disk state, metadata store
  and ATTACH socket path follow mooncake.data_directory.
- TESTING.md: the e2e job needs exit 0 and GATE e2e-smoke PASS, fmt also runs the mirror_diff unit tests, clippy
  runs with the default features, and there is a new docker build job. It also covers the basebackup scenario, the
  output without INFO lines, the known-bugs line format, and cluster sockets in E2E_DIR.

Co-authored-by: Isaac <no-reply@databricks.com>
Known limitations and the create_table behaviour now cover what tests/mirror_diff expects and the CHANGELOG didn't
say, each checked on PostgreSQL 17 at moonlink 0e38b43:
- create_table refuses unlogged tables, not only unlogged partitions.
- An array of a domain over a composite or an array type is refused, as are range types of any subtype, including
  ranges over a domain. Other arrays, composites and domains of supported types are stored.
- Arrays with explicit bounds can't be stored, like multi-dimensional ones: they fail create_table, or error the
  mirror when written later.
- A source gaining inheritance children puts its mirror into the errored state.
- Queries on mirrors follow DuckDB's semantics in places: integer division, the regex operator matching the whole
  string, and results that depend on row order without ORDER BY.

TESTING.md gives mirror_diff's measured run time, about 45 minutes on 8 cores.

Co-authored-by: Isaac <no-reply@databricks.com>
…ding

Match the code cleanup's latest changes:
- The Docker image keeps mirrors in /var/lib/postgresql/18/mooncake, inside its volume but outside the data
  directory, so pg_basebackup works in it. With a bind mount or --user, that directory has to be created first,
  owned by the server's user with mode 0700.
- mooncake.data_directory is locked through moonlink.lock: one server per directory. A refused start names the
  holder's data directory and PID, and the lock file stays after shutdown. The directory is checked like a data
  directory (mode 0700 or 0750, no symbolic link, not the data directory or a parent of it), and ./pg_mooncake counts
  as the default.
- moonlink stores src_uri in its metadata store and repeats it in an error, so the docs advise a src_uri without a
  password. pg_mooncake's own connection errors show only host, port, database and user, in the format the
  CHANGELOG now gives.
- pg_mooncake must be preloaded; otherwise CREATE EXTENSION, LOAD and pg_upgrade's library check get a clear ERROR.
- TESTING.md: the e2e preflight needs python3, the basebackup scenario checks the lock and refused directories, and
  the docker job runs the image.

Co-authored-by: Isaac <no-reply@databricks.com>
@zhousun
zhousun merged commit d8dd14a into v0.3-dev Nov 17, 2026
17 of 18 checks passed
@zhousun
zhousun deleted the v0.3/14-release-docs branch November 17, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant