Skip to content

tests/mirror_diff: rules for 0.3.0; advisory, passing on the release pins - #205

Merged
zhousun merged 2 commits into
v0.3-devfrom
v0.3/16-mirror-diff-rules
Nov 17, 2026
Merged

zhousun merged 2 commits into
v0.3-devfrom
v0.3/16-mirror-diff-rules

Conversation

@zhousun

@zhousun zhousun commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

tests/mirror_diff runs the PostgreSQL regression suite against source tables and their mirrors. Its known-differences list still described moonlink 0.2: the rules matched messages moonlink no longer prints, and 0.3's create_table refusals and errored state had no rules. A run on these pins therefore failed 211 checks for behavior the CHANGELOG documents.

This rewrites expected.txt against a full run (all layers, PostgreSQL 17.10 with assertions, moonlink 0e38b435). Every remaining difference is a documented limitation:

  • Refused types and table shapes: column types mirrors cannot store, unlogged tables, tables with inheritance children, tables with generated columns, and partitioned tables without a primary key or with unlogged partitions. One rule per message, anchored to each layer's signature and keyed on the reason moonlink gives, with the list of refused types spelled out, so a refusal of any other type still fails.
  • Values that cannot be stored: multi-dimensional arrays, numeric values beyond decimal(38, 10), NaN and Infinity, and infinite dates and timestamps. Both the refusal at create_table and the errored state after a later write match.
  • Source changes that error a mirror: TRUNCATE, a partition attached with rows, and a source gaining inheritance children.
  • Reading back: char(n) padding, composites as STRUCT text, unconstrained numeric scale, enum order (enum columns on mirrors compare and order as text), window queries over ties, and DuckDB semantics.

Bugs that moonlink 0.3 fixed are removed, including the panic rules pinned to code locations that no longer exist.

Comparator events: two differences now get their own events, the way composites printed as STRUCT text already do, so their rules match only verified cases:

  • numeric-scale: unconstrained numeric values read back with 10 fractional digits (1 as 1.0000000000). The event is reported only for numeric and numeric[] columns, and only when every differing mirror value is printed the way decimal(38, 10) prints it and equals the heap's value exactly. The comparison has no rounding precision, and 0 and -0.0000000000 differ.
  • char-padding: char(n) values read back without their trailing blanks. The event is reported only when every differing mirror value is the heap's value without its trailing blanks.

Any other difference in those columns is a plain value difference and fails.

Cleanup: the unused summarize.py and the bug-id map option are removed. The README now says mirror_diff is advisory, because its rules are measured on PostgreSQL 17 and don't cover other majors' regression files. It also gives the measured run time and links the renamed TESTING.md section.

Testing

  • A full run on these pins: 2104 checks, 1337 pass, 406 expected limitations, 361 skipped, 0 fail, 0 xpass, 0 unverified, GATE mirror_diff PASS. It took about 45 minutes on 8 cores.
  • python3 tests/mirror_diff/test_mirror_diff.py passes (49 tests). It includes tests that load the committed expected.txt and check that its rules match each documented limitation and leave other differences failing:
    • a refusal of another type, such as hstore or numeric(2,5);
    • a numeric-scale event on numeric(10,2);
    • an unlisted out-of-sync reason;
    • a char(n) difference that is not padding;
    • two 30-digit numerics that differ in the last digit.

This pull request and its description were written by Isaac.

zhousun and others added 2 commits November 17, 2026 04:28
The known-differences list still described moonlink 0.2: its rules
matched messages moonlink no longer prints, and 0.3's refusals and
errored state had none, so a run failed about 200 checks for documented
behavior. The list is rewritten against a full run on PostgreSQL 17:

- Limitations, one rule each where possible: types mirrors cannot store,
  refused table shapes (unlogged, inheritance parents, generated
  columns, partitioned without a primary key), multi-dimensional arrays,
  numeric values beyond decimal(38, 10), NaN or Infinity, infinite dates
  and timestamps, source changes that error a mirror (TRUNCATE, a
  partition attached with rows, new inheritance children), char(n)
  padding, the scale of unconstrained numeric, and enum order.
- Bugs that moonlink 0.3 fixed are gone, with the panic rules pinned to
  code locations that no longer exist.

Unconstrained numeric values read back with 10 fractional digits. The
comparator now checks that such values equal the heap's as numbers and
reports them as a numeric-scale event, instead of a value difference a
rule could only match by format.

The unused summarize.py and the bug-id map option are removed. The
README states that mirror_diff is advisory (its rules are measured on
PostgreSQL 17), gives the measured run time, and links the renamed
TESTING.md section.

Co-authored-by: Isaac <no-reply@databricks.com>
…nces

The numeric-scale comparison rounded to Decimal's 28-digit context, so
two 30-digit values that differ in their last digit compared equal. It
now compares exact values, only for unconstrained numeric and numeric[],
and only when every differing mirror value is printed the way
decimal(38, 10) prints it; 0 and -0.0000000000 differ.

The char(n) padding rule accepted any value difference in a char(n)
column. The comparator now reports a char-padding event when every
differing mirror value is the heap's value without its trailing blanks,
and the rule matches only that event.

Every rule is anchored to the signature of the layer it reads (refusal,
refused value, errored read, comparator event), the NaN and Infinity
reasons apply to numeric columns only, and the two refused-type rules
are one.

Tests load the committed expected.txt and check that its rules match
each documented limitation and leave other differences failing: a
refusal of another type, a numeric-scale event on numeric(10,2), an
unlisted out-of-sync reason, a char(n) difference that is not padding,
and the 30-digit numeric.

Co-authored-by: Isaac <no-reply@databricks.com>
@zhousun
zhousun force-pushed the v0.3/16-mirror-diff-rules branch from 1f4583c to 0156707 Compare November 17, 2026 04:28
@zhousun
zhousun merged commit f54bbce into v0.3-dev Nov 17, 2026
10 checks passed
@zhousun
zhousun deleted the v0.3/16-mirror-diff-rules branch November 17, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant