Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions tests/mirror_diff/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ table and that queries on the mirror (executed by DuckDB) return what Postgres r

The PostgreSQL files are read from `--pg-src` at run time. None are copied into this repository.

mirror_diff is advisory, not a release gate: its known differences (`expected.txt`) are measured
on PostgreSQL 17, and other majors have regression files and outputs that its rules don't cover.
Its `GATE` line says whether a run matches that list.

## Requirements

- An installed Postgres with pg_duckdb and pg_mooncake, plus the DuckDB `mooncake` extension.
Expand Down Expand Up @@ -36,18 +40,18 @@ Useful options:
| `--server-log FILE` | with `--host`: the server's log, so moonlink panics and backend crashes are seen |
| `--keep` | keep the databases and leave the server running, for debugging |
| `--expected FILE` | known-differences list (default `expected.txt` next to the script) |
| `--bug-ids FILE` | optional bug-id map: `<bug> <ids>` lines, shown as `ids=` on `KNOWN` lines |
| `--converge-secs N` | how long to keep reading a mirror that differs (default 10) |
| `--max-rejected-values N` | values a type probe may drop after `create_table` refuses them by name (default 40) |
| `--trace` | log every statement to `<workdir>/trace.sql` |
| `--resume` | continue an interrupted run in the same `--workdir` (finished units are listed in `done.txt`) |
| `--exit-regression` | exit with the regression rule instead of the gate (see Output) |

A full run (all layers, about 60 regress files, each in its own throwaway cluster) takes 1–3
hours, most of it spent waiting out moonlink hangs after panics. Artifacts in `<workdir>`:
A full run (all layers, about 60 regress files and the two extra files, each in its own
throwaway cluster) takes about 45 minutes on 8 cores with a cassert build of PostgreSQL 17.
Artifacts in `<workdir>`:

- `results.jsonl`: one JSON record per check with the full evidence (statement, events, sample
rows that differ, errors). `python3 summarize.py <workdir>/results.jsonl` renders it as markdown.
rows that differ, errors).
- `logs/<layer>-<file>.log`: the server log of each file's cluster (moonlink panics, backtraces).
- `trace.sql`: with `--trace`, every statement each session sent, with outcome and duration.

Expand Down Expand Up @@ -102,6 +106,12 @@ marker, so pointing `--workdir` at the wrong place cannot remove an unrelated cl
converted values equal the heap's, the difference is a `struct-text` event (the
`composite-as-struct` limitation); any other difference in such a column is an ordinary value
difference.
- An unconstrained `numeric` (or `numeric[]`) column whose differing mirror values all equal the
heap's exactly as numbers, printed the way `decimal(38, 10)` prints them (`1` vs
`1.0000000000`), is a `numeric-scale` event (the `numeric-scale-padded` limitation).
- A `char(n)` (or `char(n)[]`) column whose differing mirror values are all the heap's values
without their trailing blanks is a `char-padding` event (the `char-padding-lost` limitation).
- Any other difference in such columns is an ordinary value difference.
- `json` and `jsonb` cells are compared after JSON normalisation in the cdc layer: whitespace is
removed and object keys are sorted, while numbers keep their text and duplicate keys are kept.
The mirror's reformatting of JSON text is reported by the data layer (verdict `equivalent`).
Expand Down Expand Up @@ -159,14 +169,14 @@ RESULT <check> PASS|FAIL|XFAIL(<bugs>)|XFAIL(limitation:<bugs>)|XPASS(<bugs>)|SK
BUILD <k>=<v> what was tested (commits, library hashes, server version, cassert)
TYPE <type> <verdict> probes=<n> worst=<check>
STAT <counter> <n>
KNOWN <bug> OPEN|FIXED|NOT-SEEN <check>=<status>,... class=blocking|limitation [ids=<ids>]
KNOWN <bug> OPEN|FIXED|NOT-SEEN <check>=<status>,... class=blocking|limitation
SUMMARY mirror_diff pg<major> layers=<..> pass=N xpass=N fail=N xfail=N xfail_limitation=N skip=N unverified=N
BLOCKING mirror_diff <n> [<bug>,...]
GATE mirror_diff PASS|FAIL|BLOCKED|INCOMPLETE
```

- `GATE` and the exit status follow the
[release policy](../../TESTING.md#release-policy-blocking-and-limitation): FAIL (exit 1) >
[release policy](../../TESTING.md#known-limitations-and-the-gate): FAIL (exit 1) >
BLOCKED (exit 4) > INCOMPLETE (exit 3, the run was limited by `--files` or `--layers`) >
PASS (exit 0). `XFAIL(limitation:...)` does not count against the gate.
- Unlike the end-to-end gate, plain SKIPs don't count against this gate. They are queries that
Expand Down
87 changes: 87 additions & 0 deletions tests/mirror_diff/compare.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import json
import re
from decimal import Decimal, InvalidOperation

from psqlsession import NULL_TOKEN

Expand Down Expand Up @@ -474,6 +475,62 @@ def struct_to_row(cell):
return None


_NUMERIC_SPECIALS = ("NaN", "Infinity", "-Infinity")
# How a mirror prints an unconstrained numeric, which it stores as decimal(38, 10).
_DECIMAL_38_10 = re.compile(r"-?\d+\.\d{10}")


def _numeric(text):
"""The exact value of a numeric literal as (sign, digits, exponent) without trailing zeros. No
rounding: Decimal's arithmetic context has 28 digits, more than enough to hide a difference."""
if text in _NUMERIC_SPECIALS:
return text
sign, digits, exponent = Decimal(text).as_tuple()
digits = list(digits)
while len(digits) > 1 and digits[-1] == 0:
digits.pop()
exponent += 1
if digits == [0]:
exponent = 0 # the sign is kept: 0 and -0.0000000000 differ
return sign, tuple(digits), exponent


def numeric_key(v, array, mirror):
"""A cell of an unconstrained numeric column (or a one-dimensional array of them) as exact
numbers, so that 1 and 1.0000000000 compare equal. A mirror's value must be printed the way
decimal(38, 10) prints it. None if the cell isn't such a value."""
if v is None or v == NULL_TOKEN:
return v
s = str(v)
try:
elems = _parse_pg_array(s) if array else [s]
if elems is None or any(e is not None and e.startswith("{") for e in elems):
return None
if mirror and any(e is not None and not _DECIMAL_38_10.fullmatch(e) for e in elems):
return None
key = tuple(None if e is None else _numeric(e) for e in elems)
return key if array else key[0]
except (InvalidOperation, IndexError):
return None


def _unpadded(v, array):
"""A char(n) cell (or array of them) without the trailing blanks of each value."""
if v is None or v == NULL_TOKEN:
return v
if not array:
return str(v).rstrip(" ")
elems = _parse_pg_array(str(v))
return None if elems is None else tuple(None if e is None else e.rstrip(" ") for e in elems)


def _as_cells(v, array):
if not array or v is None or v == NULL_TOKEN:
return v
elems = _parse_pg_array(str(v))
return None if elems is None else tuple(elems)


def divergence_events(cols, heap_rows, mirror_rows, sample=3):
"""Events describing how a mirror's rows differ from the heap's. cols: [(name, type), ...].
Each event: {"kind": row-count|value|rows, "detail": text, ["col", "type"]}."""
Expand Down Expand Up @@ -503,6 +560,36 @@ def divergence_events(cols, heap_rows, mirror_rows, sample=3):
f"value(s) equal the heap's after STRUCT-to-row conversion, e.g. heap "
f"{str(h_only[0])[:60]} mirror {str(m_only[0])[:60]}"})
continue
# The values that differ, with their surplus counts on each side.
dh = {v: hv[v] - mv.get(v, 0) for v in h_only}
dm = {v: mv[v] - hv.get(v, 0) for v in m_only}
if typ in ("numeric", "numeric[]"):
hn, mn = {}, {}
for src, dst, mirror in ((dh, hn, False), (dm, mn, True)):
for v, k in src.items():
key = numeric_key(v, typ == "numeric[]", mirror)
dst[key] = dst.get(key, 0) + k
if None not in hn and None not in mn and hn == mn:
# Only the text form differs: the mirror prints the values with another scale.
out.append({"kind": "numeric-scale", "col": name, "type": typ,
"detail": f"numeric-scale col={name} type={typ}: {sum(dm.values())} "
f"value(s) equal the heap's as numbers, e.g. heap {str(h_only[0])[:60]} "
f"mirror {str(m_only[0])[:60]}"})
continue
if re.fullmatch(r"character\(\d+\)(\[\])?", typ):
array = typ.endswith("[]")
hp, mp = {}, {}
for src, dst, conv in ((dh, hp, _unpadded), (dm, mp, _as_cells)):
for v, k in src.items():
key = conv(v, array)
dst[key] = dst.get(key, 0) + k
if None not in hp and None not in mp and hp == mp:
# Only the trailing blanks of the heap's values are missing.
out.append({"kind": "char-padding", "col": name, "type": typ,
"detail": f"char-padding col={name} type={typ}: {sum(dm.values())} "
f"value(s) equal the heap's without trailing blanks, e.g. heap "
f"{str(h_only[0])[:60]!r} mirror {str(m_only[0])[:60]!r}"})
continue
out.append({"kind": "value", "col": name, "type": typ,
"detail": f"value col={name} type={typ} heap-only: {'; '.join(str(v)[:80] for v in h_only[:sample])} "
f"|| mirror-only: {'; '.join(str(v)[:80] for v in m_only[:sample])}"})
Expand Down
Loading