diff --git a/diam/peer/testdata/sctprestart/README.md b/diam/peer/testdata/sctprestart/README.md index 0449a7e..1de89c4 100644 --- a/diam/peer/testdata/sctprestart/README.md +++ b/diam/peer/testdata/sctprestart/README.md @@ -1,9 +1,19 @@ This local SCTP restart harness runs `peer.Manager` against two privileged Linux containers. The alternate peer runs in the primary client's namespace; captures on both `eth0` interfaces cover both associations. No SCTP socket runs on the host. The Go driver is excluded from ordinary package discovery by `testdata` and requires the `sctprestart` build tag. -Run from the repository root, with Docker and the existing `i81-wire-tools:go1.27.2` image available: +The runner requires Docker with SCTP support, Bash and Python 3. It uses +`scripts/sctp-wire/harness.sh`, the shared Dockerfile and tshark exporter. +Every runner independently builds `sctp-wire-tools:go1.27.2` when absent or +verifies it when present, then records and uses its immutable image ID. No sm +run is required first. Set `SCTP_WIRE_IMAGE` to select a different or fresh tag. +Python 3 runs the package-local analyzer; tshark runs in the recorded image. + +Invoke the runner from any directory using its path. Arguments remain +`[source checkout] [output root] [variant]`, with this checkout, a unique +`sctp-wire-peer-evidence-*` directory under `${TMPDIR:-/tmp}`, and `basic` +as defaults. For example, from the repository root: ```sh -diam/peer/testdata/sctprestart/run.sh "$PWD" /tmp/i93-wire basic +diam/peer/testdata/sctprestart/run.sh "$PWD" /path/to/evidence/peer basic ``` The third argument selects `basic`, `changed-identity`, `changed-apps`, `alternate-stream`, `fragment`, `watchdog`, or `deadline`. Each run writes evidence under a subdirectory named for that variant. Pass a frozen source checkout as the first argument to compare baseline and changed implementations; the builder overlays only this driver onto that checkout. The driver compiles against the public Manager APIs on both versions. @@ -18,17 +28,22 @@ Watchdog analysis uses the replacement CER's host and the restart hook's new gen The harness wrapper forwards each message synchronously to Manager. After `ServeDIAM` returns from protocol commit, it reads `Manager.Peers()` and records a `RECOVERY_CHECKPOINT` containing the processed CER/DWA command, both identifiers, connection generation and peer snapshot. A TCP control connection on server port 3869 delivers that checkpoint to the client. The client waits for the CER checkpoint before answering recovery probes, and for each DWA checkpoint before sending the next DWA. The analyzer requires REOPEN/ineligible at admission and after DWA 1 and 2, then OKAY/eligible after DWA 3 (RFC 3539 section 3.4.1 and Appendix A). Every checkpoint must match the corresponding captured message and precede the next answer's delivery. Asynchronous `OnPeerEvent` callbacks do not establish recovery boundaries; delayed, missing or queued recovery callbacks cannot affect this check. Initial admission and the alternate peer cannot satisfy recovery. No additional Manager accessor is required. -Run the analyzer's synthetic packet/log regression suite in Docker: +Run the analyzer's synthetic packet/log regression suite from the repository root: ```sh -docker run --rm --user "$(id -u):$(id -g)" \ - --mount "type=bind,src=$PWD,dst=/source,readonly" \ - -w /source/diam/peer/testdata/sctprestart \ - i81-wire-tools:go1.27.2 python3 -B -m unittest -v analyze_test +python3 -B -m unittest discover -s diam/peer/testdata/sctprestart -p '*_test.py' -v ``` These tests execute the analyzer CLI with synthetic tshark output. They check its acceptance and rejection boundaries; the live two-container watchdog run supplies packet capture evidence. -Missing Docker, the image, SCTP support, tshark, captures, restart evidence, or assertions fails the command. Evidence includes packet tables, raw application request bytes, captures and SHA-256 hashes, logs, build and dependency versions, image identity, kernel version, and receiver drop counters. Run-specific containers and networks are labeled, removed on every exit path, and checked by label and name. No volumes are created. +Missing Docker, SCTP support, tshark, captures, restart evidence, or assertions fails the command. Evidence includes packet tables, raw application request bytes, captures and SHA-256 hashes, logs, build and dependency versions, image identity, kernel version, and receiver drop counters. Run-specific containers and networks are labeled, removed on every exit path, and checked by label and name. No volumes are created. + +Compilation copies source into the builder's writable +container layer and mounts no host Go caches. Artifacts are copied out as the +calling user; privileged peers bind only the binary. Container/network names +use unique `sctp-wire-*` prefixes and temporary directories use +`sctp-wire-build.*`. Docker CLI configuration stays in that temporary +directory. Set `TMPDIR` to choose its location; no files are written outside +that directory and the output directory. The harness does not test TLS, dialed association renewal, packet loss/reordering, or peer state election races. Unit and scripted transport tests cover those policy and concurrency cases. Its packet checks and observed generations establish a real accepted SCTP association restart, not a fresh listener connection. diff --git a/diam/peer/testdata/sctprestart/analyze.py b/diam/peer/testdata/sctprestart/analyze.py index 2c08643..6ac378b 100755 --- a/diam/peer/testdata/sctprestart/analyze.py +++ b/diam/peer/testdata/sctprestart/analyze.py @@ -3,7 +3,7 @@ import json import pathlib import re -import subprocess +import runpy import sys @@ -92,18 +92,16 @@ def assert_watchdog_recovery(rows, log, cer, cea): out = pathlib.Path(sys.argv[1]).resolve() -image = 'i81-wire-tools:go1.27.2' -run_id = (out / 'run-id').read_text().strip() +tshark = runpy.run_path(str(pathlib.Path(__file__).resolve().parents[4] / 'scripts/sctp-wire/tshark.py'))['tshark'] fields = ['frame.number', 'frame.time_epoch', 'ip.src', 'sctp.chunk_type', 'sctp.srcport', 'sctp.dstport', 'diameter.flags.T', 'diameter.Session-Id', 'diameter.User-Name', 'sctp.data_sid', 'diameter.cmd.code', 'diameter.flags.request', 'diameter.hopbyhopid', 'diameter.endtoendid', 'diameter.Result-Code', 'diameter.Origin-Host', 'diameter.Auth-Application-Id', 'diameter.Origin-State-Id'] -cmd = ['docker', 'run', '--rm', '--label', f'harness={run_id}', '--mount', f'type=bind,src={out},dst=/evidence,readonly', image, 'tshark', '-r', '/evidence/client.pcapng', '-Y', 'sctp', '-T', 'fields'] +arguments = ['-Y', 'sctp', '-T', 'fields'] for field in fields: - cmd.extend(['-e', field]) -result = subprocess.run(cmd, text=True, capture_output=True, check=True) + arguments.extend(['-e', field]) +result = tshark(out, 'client.pcapng', *arguments) (out / 'packets.tsv').write_text(result.stdout) (out / 'tshark-errors.txt').write_text(result.stderr) rows = [dict(zip(fields, row)) for row in csv.reader(result.stdout.splitlines(), delimiter='\t')] -server_cmd = [arg.replace('/evidence/client.pcapng', '/evidence/server.pcapng') for arg in cmd] -server_result = subprocess.run(server_cmd, text=True, capture_output=True, check=True) +server_result = tshark(out, 'server.pcapng', *arguments) (out / 'server-packets.tsv').write_text(server_result.stdout) server_rows = [dict(zip(fields, row)) for row in csv.reader(server_result.stdout.splitlines(), delimiter='\t')] assert any(row['sctp.chunk_type'] == '6' for row in server_rows), 'ABORT did not reach receiver capture point' @@ -167,8 +165,7 @@ def assert_watchdog_recovery(rows, log, cer, cea): assert original['diameter.hopbyhopid'] != retry['diameter.hopbyhopid'], 'hop-by-hop not freshly allocated' for key in ('diameter.Session-Id', 'diameter.User-Name'): assert original[key] == retry[key] and original[key], f'{key} payload changed or absent' -raw_cmd = cmd[:cmd.index('-Y')] + ['-Y', 'diameter.cmd.code == 272 && diameter.flags.request == 1', '-T', 'jsonraw', '-j', 'diameter'] -raw = subprocess.run(raw_cmd, text=True, capture_output=True, check=True) +raw = tshark(out, 'client.pcapng', '-Y', 'diameter.cmd.code == 272 && diameter.flags.request == 1', '-T', 'jsonraw', '-j', 'diameter') (out / 'request-raw.json').write_text(raw.stdout) wire = [bytes.fromhex(p['_source']['layers']['diameter_raw'][0]) for p in json.loads(raw.stdout)] assert len(wire) == 2 and wire[0][20:] == wire[1][20:], 'complete serialized AVP payload changed on retry' diff --git a/diam/peer/testdata/sctprestart/run.sh b/diam/peer/testdata/sctprestart/run.sh index 1584c77..2427a88 100755 --- a/diam/peer/testdata/sctprestart/run.sh +++ b/diam/peer/testdata/sctprestart/run.sh @@ -1,80 +1,28 @@ #!/usr/bin/env bash -# Every SCTP socket and packet capture runs in isolated privileged containers. +# Scenario commands and timings; shared lifecycle lives in scripts/sctp-wire. set -euo pipefail harness=$(cd "$(dirname "$0")" && pwd) -source=$(cd "${1:-$harness/../../../..}" && pwd) +root=$(cd "$harness/../../../.." && pwd) +# shellcheck source=scripts/sctp-wire/harness.sh +source "$root/scripts/sctp-wire/harness.sh" +scenario=peer variant=${3:-basic} case "$variant" in basic|changed-identity|changed-apps|alternate-stream|watchdog|fragment|deadline) ;; *) echo "unknown variant: $variant" >&2; exit 2 ;; esac -rig=$(mktemp -d "${TMPDIR:-/tmp}/i93-wire-build.XXXXXX") -trap 'rm -rf "$rig"' EXIT -name=retained +source=$(cd "${1:-$harness/../../../..}" && pwd) +output=${2:-${TMPDIR:-/tmp}/sctp-wire-peer-evidence-$(date +%s)-$$} +output=$output/$variant +fixture=diam/peer/testdata/sctprestart/main.go +build=(build -tags=sctprestart -o /out/peer ./diam/peer/testdata/sctprestart) binary=peer -out=$(realpath -m "${2:-/tmp/i93-wire-evidence-$(date +%s)-$$}/$variant") -mkdir -p "$out" -run=i93-wire-${name}-${variant}-${rig##*.} -network=$run-net -server=$run-server -client=$run-client -builder=$run-build -analyzer=$run-analyze - -cleanup(){ - for peer in "$client" "$server"; do - for child in capture initial server alternate; do - signal=TERM - if [[ $child == capture ]]; then signal=INT; fi - if [[ $child == initial ]]; then signal=KILL; fi - if docker exec "$peer" test -s "/tmp/$child.pid" >/dev/null 2>&1; then - if ! docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then - docker exec "$peer" sh -c 'kill -"$1" "$(cat "/tmp/$2.pid")"' sh "$signal" "$child" >/dev/null 2>&1 || true - for ((i=0;i<10;i++)); do - if docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then break; fi - sleep .1 - done - fi - fi - done - role=${peer##*-} - for file in wire.pcapng capture.log server.log initial.log alternate.log; do - destination=$out/$role-$file - if [[ ! -e $destination ]]; then docker cp "$peer:/tmp/$file" "$destination" >/dev/null 2>&1 || true; fi - done - done - docker rm -f -v "$client" "$server" "$builder" "$analyzer" >/dev/null 2>&1 || true - docker network rm "$network" >/dev/null 2>&1 || true - docker ps -a --filter "label=harness=$run" --format '{{.Names}}' > "$out/cleanup-containers.txt" - docker volume ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-volumes.txt" - docker network ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-networks.txt" - docker ps -a --filter "name=$run" --format '{{.Names}}' > "$out/cleanup-name.txt" - docker volume ls --filter "name=$run" --format '{{.Name}}' > "$out/cleanup-name-volumes.txt" - docker network ls --filter "name=$run" --format '{{.Name}}' > "$out/cleanup-name-networks.txt" - rm -rf "$rig" -} -trap cleanup EXIT -trap 'exit 130' INT TERM -printf '%s\n' "$run" > "$out/run-id" -cp "$harness/Dockerfile" "$rig/Dockerfile" -docker image inspect i81-wire-tools:go1.27.2 > "$out/image-build.log" -mkdir -p "$rig/out" -docker run --rm --name "$builder" --label "harness=$run" \ - --mount "type=bind,src=$source,dst=/source,readonly" \ - --mount "type=bind,src=$harness,dst=/harness,readonly" \ - --mount "type=bind,src=$rig/out,dst=/out" \ - i81-wire-tools:go1.27.2 sh -c 'cp -a /source /tmp/build; mkdir -p /tmp/build/diam/peer/testdata/sctprestart; cp /harness/main.go /tmp/build/diam/peer/testdata/sctprestart/main.go; cd /tmp/build; for f in diam/server.go diam/accepted_generation.go diam/network_sctp.go diam/peer/manager.go diam/peer/actor.go diam/peer/session.go diam/peer/renewal.go diam/peer/watchdog.go diam/peer/send.go diam/peer/testdata/sctprestart/main.go go.mod go.sum; do if test -f "$f"; then sha256sum "$f"; fi; done > /out/source-hashes.txt; go build -tags=sctprestart -o /out/peer ./diam/peer/testdata/sctprestart' > "$out/build.log" 2>&1 - -cp "$rig/out/source-hashes.txt" "$out/source-hashes.txt" +wire_path=/wire +server_port=3868 +client_port=43868 +capture_suffix= +cleanup_attempts=10 +analyze_on_failure=true +hashes=(diam/server.go diam/accepted_generation.go diam/network_sctp.go diam/peer/manager.go diam/peer/actor.go diam/peer/session.go diam/peer/renewal.go diam/peer/watchdog.go diam/peer/send.go diam/peer/testdata/sctprestart/main.go go.mod go.sum) -docker network create --label "harness=$run" "$network" >/dev/null -for peer in "$server" "$client"; do - docker run -d --privileged --label "harness=$run" --name "$peer" --network "$network" --mount "type=bind,src=$rig/out/$binary,dst=/wire,readonly" i81-wire-tools:go1.27.2 sleep infinity >/dev/null -done -sip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$server") -cip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$client") -docker exec "$server" iptables -I INPUT -p sctp -s "$cip" --sport 43868 --dport 3868 -m sctp --chunk-types any ABORT -j DROP -for peer in "$server" "$client"; do - docker exec -d "$peer" sh -c 'tshark -i eth0 -f sctp -w /tmp/wire.pcapng >/tmp/capture.log 2>&1 & capture_pid=$!; printf "%s\n" "$capture_pid" >/tmp/capture.pid; wait "$capture_pid"; printf "%s\n" "$?" >/tmp/capture.exit' -done -sleep 1 +wire_scenario() { watchdog=false if [[ $variant == watchdog ]]; then watchdog=true; fi request_timeout=12s @@ -117,42 +65,6 @@ result=$? set -e printf '%s\n' "$result" > "$out/restart.exit" if [[ $variant == deadline ]]; then sleep 6; else sleep .5; fi -for peer in "$server" "$client"; do - docker exec "$peer" sh -c 'kill -INT "$(cat /tmp/capture.pid)"' - for ((i=0;i<30;i++)); do if docker exec "$peer" test -s /tmp/capture.exit; then break; fi; sleep .1; done - docker exec "$peer" test -s /tmp/capture.exit - docker cp "$peer:/tmp/wire.pcapng" "$out/${peer##*-}.pcapng" >/dev/null - docker cp "$peer:/tmp/capture.log" "$out/${peer##*-}-capture.log" >/dev/null - docker cp "$peer:/tmp/capture.exit" "$out/${peer##*-}-capture.exit" >/dev/null -done -docker cp "$server:/tmp/server.log" "$out/server.log" >/dev/null -docker cp "$client:/tmp/initial.log" "$out/initial.log" >/dev/null -docker cp "$client:/tmp/alternate.log" "$out/alternate.log" >/dev/null -docker cp "$client:/tmp/initial.exit" "$out/initial.exit" >/dev/null -docker exec "$server" iptables -nvxL INPUT > "$out/iptables.txt" -docker exec "$server" uname -a > "$out/kernel.txt" -docker exec "$server" /usr/local/go/bin/go version > "$out/go-version.txt" -docker exec "$server" /usr/local/go/bin/go version -m /wire > "$out/binary-build.txt" -docker inspect "$server" --format '{{.Image}}' > "$out/image-id.txt" -docker exec "$server" tshark --version > "$out/tshark-version.txt" -docker exec "$server" sh -c 'kill -TERM "$(cat /tmp/server.pid)"' -for ((i=0;i<30;i++)); do if docker exec "$server" test -s /tmp/server.exit; then break; fi; sleep .1; done -docker exec "$server" test -s /tmp/server.exit -printf '%s\n' "run=$run restart_exit=$result output=$out" -cat "$out/restarted.log" -cleanup -socket_gid=$(docker run --rm --label "harness=$run" --user "$(id -u):$(id -g)" \ - --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ - i81-wire-tools:go1.27.2 stat -c '%g' /var/run/docker.sock) -docker run --rm --name "$analyzer" --label "harness=$run" \ - --user "$(id -u):$(id -g)" --group-add "$socket_gid" \ - --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ - --mount type=bind,src=/usr/bin/docker,dst=/usr/bin/docker,readonly \ - --mount "type=bind,src=$harness/analyze.py,dst=/analyze.py,readonly" \ - --mount "type=bind,src=$out,dst=$out" \ - i81-wire-tools:go1.27.2 python3 -B /analyze.py "$out" > "$out/assertions.txt" 2>&1 || result=$? -cat "$out/assertions.txt" -sha256sum "$out"/*.pcapng > "$out/capture-sha256.txt" -cleanup -trap - EXIT -exit "$result" +} + +wire_run diff --git a/diam/sm/testdata/sctprestart/Dockerfile b/diam/sm/testdata/sctprestart/Dockerfile deleted file mode 100644 index f0e4138..0000000 --- a/diam/sm/testdata/sctprestart/Dockerfile +++ /dev/null @@ -1,3 +0,0 @@ -FROM golang:1.27.2-bookworm@sha256:5cf287a799e6b94384bad13d16b14904c531f51ba65792237e122ce42b392f61 -RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tshark iptables iproute2 && rm -rf /var/lib/apt/lists/* -WORKDIR /work diff --git a/diam/sm/testdata/sctprestart/README.md b/diam/sm/testdata/sctprestart/README.md index 8113ee3..c4990e9 100644 --- a/diam/sm/testdata/sctprestart/README.md +++ b/diam/sm/testdata/sctprestart/README.md @@ -7,35 +7,37 @@ The image pins Go 1.27.2 and installs tshark, iptables and iproute2. Each run ow a dedicated labeled Docker bridge, joins its children, copies captures out, and removes its containers and network. It mounts no host Go cache. -Run from the repository root, writing evidence outside the checkout: +All three restart scenarios use `scripts/sctp-wire/harness.sh`, its shared +Dockerfile, and its tshark exporter. Each runner builds +`sctp-wire-tools:go1.27.2` when absent and verifies it when present; no other +scenario needs to run first. `SCTP_WIRE_IMAGE` selects another tag, including a +fresh tag for an independent first run. A run records the immutable image ID +and uses it for compilation, peers and later analysis. Python 3 runs the local +analyzer; only tshark runs in an analysis container. -```sh -bash diam/sm/testdata/sctprestart/run.sh "$PWD" /tmp/i81-wire-fixed basic -``` - -The optional Go test entry point also runs inside Docker. Mount the repository -and a throwaway scratch directory at their original absolute host paths: the -Docker daemon must see those paths when the runner starts its peers. For example, -using a scratch directory chosen outside the checkout: +Invoke the runner from any directory using its path. The arguments remain +`[source checkout] [output root] [variant]`; source defaults to this checkout, +variant to `basic`, and output to a unique `sctp-wire-sm-evidence-*` directory +under `${TMPDIR:-/tmp}`. Evidence is placed in the variant subdirectory. +For example, from the repository root: ```sh -export I81_WIRE_SCRATCH=/absolute/path/to/scratch/i81-wire-test -mkdir -p "$I81_WIRE_SCRATCH" -docker run --rm --label harness=i81-wire-orchestrator \ - --mount "type=bind,src=$PWD,dst=$PWD,readonly" \ - --mount "type=bind,src=$I81_WIRE_SCRATCH,dst=$I81_WIRE_SCRATCH" \ - --mount type=bind,src=/usr/bin/docker,dst=/usr/bin/docker,readonly \ - --mount type=bind,src=/usr/libexec/docker/cli-plugins/docker-buildx,dst=/usr/libexec/docker/cli-plugins/docker-buildx,readonly \ - --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ - -e TMPDIR="$I81_WIRE_SCRATCH" -w "$PWD" i81-wire-tools:go1.27.2 \ - go test -tags=sctprestart ./diam/sm \ - -run '^TestAcceptedSCTPRestartWire$' -count=1 -v -timeout 3m +bash diam/sm/testdata/sctprestart/run.sh "$PWD" /path/to/evidence/sm basic ``` -Build `i81-wire-tools:go1.27.2` with the Dockerfile in this directory first. The -Docker CLI and buildx paths above are the installed paths used for validation; -adjust those mounts if the local installation differs. The test's temporary -directory is removed by Go. Use the standalone runner to retain packet captures. +The optional `sctprestart` tagged `TestAcceptedSCTPRestartWire` entry point +invokes the same runner. Run it inside a Linux Docker container with the Docker +CLI and socket available. Mount the checkout and `TMPDIR` at their original +absolute host paths so sibling containers can bind them. The test removes its +own temporary evidence; use the standalone runner to retain captures. + +Compilation copies source into the builder's writable container layer. No +host Go cache is mounted, and build artifacts are copied out as the calling +user. Privileged peers bind only the fixture binary. Every run uses unique +`sctp-wire-*` container/network names and a `sctp-wire-build.*` temporary +directory; cleanup touches only that run's labeled resources. Docker CLI +configuration stays in the temporary directory. Set `TMPDIR` to choose its +location; the runner writes only there and in its output directory. The client binds source port 43868, completes CER/CEA, sets SO_LINGER enabled with zero timeout, and is killed and reaped. The server drops client ABORT on @@ -57,16 +59,16 @@ operations and `ResetForRestart` to the product adapter; no dependency is patche To compare with the original defect using the exact same harness: ```sh -mkdir -p /tmp/i81-wire-baseline-source -git archive 49b6c8c | tar -x -C /tmp/i81-wire-baseline-source +mkdir -p /path/to/scratch/baseline-source +git archive 49b6c8c | tar -x -C /path/to/scratch/baseline-source bash diam/sm/testdata/sctprestart/run.sh \ - /tmp/i81-wire-baseline-source /tmp/i81-wire-baseline basic + /path/to/scratch/baseline-source /path/to/evidence/baseline basic ``` The runner copies the tagged peer into a throwaway source tree inside its build container, so a baseline lacking the helper is supported. The expected failure is `FAIL phase=2 CEA ... i/o timeout`, with the second CER present on the wire -and no matching CEA. `analyze.py /tmp/i81-wire-baseline/basic` independently +and no matching CEA. `analyze.py /path/to/evidence/baseline/basic` independently checks the capture and exits nonzero for that missing answer. Repeat the fixed run with `changed-apps`, `alternate-stream`, `watchdog`, and diff --git a/diam/sm/testdata/sctprestart/analyze.py b/diam/sm/testdata/sctprestart/analyze.py index 744784b..5594e4b 100755 --- a/diam/sm/testdata/sctprestart/analyze.py +++ b/diam/sm/testdata/sctprestart/analyze.py @@ -2,22 +2,20 @@ import csv import pathlib import re -import subprocess +import runpy import sys out = pathlib.Path(sys.argv[1]).resolve() -image = 'i81-wire-tools:go1.27.2' -run_id = (out / 'run-id').read_text().strip() +tshark = runpy.run_path(str(pathlib.Path(__file__).resolve().parents[4] / 'scripts/sctp-wire/tshark.py'))['tshark'] fields = ['frame.number', 'frame.time_epoch', 'ip.src', 'sctp.chunk_type', 'sctp.data_sid', 'diameter.cmd.code', 'diameter.flags.request', 'diameter.hopbyhopid', 'diameter.endtoendid', 'diameter.Result-Code', 'diameter.Origin-Host', 'diameter.Auth-Application-Id', 'diameter.Origin-State-Id'] -cmd = ['docker', 'run', '--rm', '--label', f'harness={run_id}', '--mount', f'type=bind,src={out},dst=/evidence,readonly', image, 'tshark', '-r', '/evidence/client.pcapng', '-Y', 'sctp', '-T', 'fields'] +arguments = ['-Y', 'sctp', '-T', 'fields'] for field in fields: - cmd.extend(['-e', field]) -result = subprocess.run(cmd, text=True, capture_output=True, check=True) + arguments.extend(['-e', field]) +result = tshark(out, 'client.pcapng', *arguments) (out / 'packets.tsv').write_text(result.stdout) (out / 'tshark-errors.txt').write_text(result.stderr) rows = [dict(zip(fields, row)) for row in csv.reader(result.stdout.splitlines(), delimiter='\t')] -server_cmd = [arg.replace('/evidence/client.pcapng', '/evidence/server.pcapng') for arg in cmd] -server_result = subprocess.run(server_cmd, text=True, capture_output=True, check=True) +server_result = tshark(out, 'server.pcapng', *arguments) (out / 'server-packets.tsv').write_text(server_result.stdout) server_rows = [dict(zip(fields, row)) for row in csv.reader(server_result.stdout.splitlines(), delimiter='\t')] assert any(row['sctp.chunk_type'] == '6' for row in server_rows), 'ABORT did not reach receiver capture point' diff --git a/diam/sm/testdata/sctprestart/run.sh b/diam/sm/testdata/sctprestart/run.sh index 08265f0..1c17ed6 100755 --- a/diam/sm/testdata/sctprestart/run.sh +++ b/diam/sm/testdata/sctprestart/run.sh @@ -1,77 +1,28 @@ #!/usr/bin/env bash -# Every SCTP socket and packet capture runs in isolated privileged containers. +# Scenario commands and timings; shared lifecycle lives in scripts/sctp-wire. set -euo pipefail harness=$(cd "$(dirname "$0")" && pwd) -source=$(cd "${1:-$harness/../../../..}" && pwd) +root=$(cd "$harness/../../../.." && pwd) +# shellcheck source=scripts/sctp-wire/harness.sh +source "$root/scripts/sctp-wire/harness.sh" +scenario=sm variant=${3:-basic} case "$variant" in basic|changed-apps|alternate-stream|watchdog|fragment) ;; *) echo "unknown variant: $variant" >&2; exit 2 ;; esac -rig=$(mktemp -d "${TMPDIR:-/tmp}/i81-wire-build.XXXXXX") -trap 'rm -rf "$rig"' EXIT -name=retained +source=$(cd "${1:-$harness/../../../..}" && pwd) +output=${2:-${TMPDIR:-/tmp}/sctp-wire-sm-evidence-$(date +%s)-$$} +output=$output/$variant +fixture=diam/sm/testdata/sctprestart/main.go +build=(build -tags=sctprestart -o /out/peer ./diam/sm/testdata/sctprestart) binary=peer -out=$(realpath -m "${2:-/tmp/i81-wire-evidence-$(date +%s)-$$}/$variant") -mkdir -p "$out" -run=i81-wire-${name}-${variant}-${rig##*.} -network=$run-net -server=$run-server -client=$run-client -builder=$run-build - -cleanup(){ - for peer in "$client" "$server"; do - for child in capture initial server; do - signal=TERM - if [[ $child == capture ]]; then signal=INT; fi - if [[ $child == initial ]]; then signal=KILL; fi - if docker exec "$peer" test -s "/tmp/$child.pid" >/dev/null 2>&1; then - if ! docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then - docker exec "$peer" sh -c 'kill -"$1" "$(cat "/tmp/$2.pid")"' sh "$signal" "$child" >/dev/null 2>&1 || true - for ((i=0;i<10;i++)); do - if docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then break; fi - sleep .1 - done - fi - fi - done - role=${peer##*-} - for file in wire.pcapng capture.log server.log initial.log; do - destination=$out/$role-$file - if [[ ! -e $destination ]]; then docker cp "$peer:/tmp/$file" "$destination" >/dev/null 2>&1 || true; fi - done - done - docker rm -f -v "$client" "$server" "$builder" >/dev/null 2>&1 || true - docker network rm "$network" >/dev/null 2>&1 || true - docker ps -a --filter "label=harness=$run" --format '{{.Names}}' > "$out/cleanup-containers.txt" - docker volume ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-volumes.txt" - docker network ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-networks.txt" - docker ps -a --filter "name=$run" --format '{{.Names}}' > "$out/cleanup-name.txt" - rm -rf "$rig" -} -trap cleanup EXIT -trap 'exit 130' INT TERM -printf '%s\n' "$run" > "$out/run-id" -cp "$harness/Dockerfile" "$rig/Dockerfile" -DOCKER_CONFIG="$rig/docker-config" docker build -t i81-wire-tools:go1.27.2 "$rig" > "$out/image-build.log" 2>&1 -mkdir -p "$rig/out" -docker run --rm --name "$builder" --label "harness=$run" \ - --mount "type=bind,src=$source,dst=/source,readonly" \ - --mount "type=bind,src=$harness,dst=/harness,readonly" \ - --mount "type=bind,src=$rig/out,dst=/out" \ - i81-wire-tools:go1.27.2 sh -c 'cp -a /source /tmp/build; mkdir -p /tmp/build/diam/sm/testdata/sctprestart; cp /harness/main.go /tmp/build/diam/sm/testdata/sctprestart/main.go; cd /tmp/build; for f in diam/server.go diam/accepted_generation.go diam/network_sctp.go diam/sm/sm.go diam/sm/accepted_watchdog.go; do if test -f "$f"; then sha256sum "$f"; fi; done > /out/source-hashes.txt; go build -tags=sctprestart -o /out/peer ./diam/sm/testdata/sctprestart' > "$out/build.log" 2>&1 - -cp "$rig/out/source-hashes.txt" "$out/source-hashes.txt" +wire_path=/wire +server_port=3868 +client_port=43868 +capture_suffix= +cleanup_attempts=10 +analyze_on_failure=false +hashes=(diam/server.go diam/accepted_generation.go diam/network_sctp.go diam/sm/sm.go diam/sm/accepted_watchdog.go) -docker network create --label "harness=$run" "$network" >/dev/null -for peer in "$server" "$client"; do - docker run -d --privileged --label "harness=$run" --name "$peer" --network "$network" --mount "type=bind,src=$rig/out/$binary,dst=/wire,readonly" i81-wire-tools:go1.27.2 sleep infinity >/dev/null -done -sip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$server") -cip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$client") -docker exec "$server" iptables -I INPUT -p sctp -s "$cip" --sport 43868 --dport 3868 -m sctp --chunk-types any ABORT -j DROP -for peer in "$server" "$client"; do - docker exec -d "$peer" sh -c 'tshark -i eth0 -f sctp -w /tmp/wire.pcapng >/tmp/capture.log 2>&1 & capture_pid=$!; printf "%s\n" "$capture_pid" >/tmp/capture.pid; wait "$capture_pid"; printf "%s\n" "$?" >/tmp/capture.exit' -done -sleep 1 +wire_scenario() { watchdog=false if [[ $variant == watchdog ]]; then watchdog=true; fi docker exec -d "$server" sh -c '/wire -mode server -addr "$1" -watchdog="$2" >/tmp/server.log 2>&1 & server_pid=$!; printf "%s\n" "$server_pid" >/tmp/server.pid; wait "$server_pid"; printf "%s\n" "$?" >/tmp/server.exit' sh "$sip:3868" "$watchdog" @@ -99,32 +50,6 @@ result=$? set -e printf '%s\n' "$result" > "$out/restart.exit" sleep .5 -for peer in "$server" "$client"; do - docker exec "$peer" sh -c 'kill -INT "$(cat /tmp/capture.pid)"' - for ((i=0;i<30;i++)); do if docker exec "$peer" test -s /tmp/capture.exit; then break; fi; sleep .1; done - docker exec "$peer" test -s /tmp/capture.exit - docker cp "$peer:/tmp/wire.pcapng" "$out/${peer##*-}.pcapng" >/dev/null - docker cp "$peer:/tmp/capture.log" "$out/${peer##*-}-capture.log" >/dev/null - docker cp "$peer:/tmp/capture.exit" "$out/${peer##*-}-capture.exit" >/dev/null -done -docker cp "$server:/tmp/server.log" "$out/server.log" >/dev/null -docker cp "$client:/tmp/initial.log" "$out/initial.log" >/dev/null -docker cp "$client:/tmp/initial.exit" "$out/initial.exit" >/dev/null -docker exec "$server" iptables -nvxL INPUT > "$out/iptables.txt" -docker exec "$server" uname -a > "$out/kernel.txt" -docker exec "$server" /usr/local/go/bin/go version > "$out/go-version.txt" -docker exec "$server" /usr/local/go/bin/go version -m /wire > "$out/binary-build.txt" -docker inspect "$server" --format '{{.Image}}' > "$out/image-id.txt" -docker exec "$server" tshark --version > "$out/tshark-version.txt" -docker exec "$server" sh -c 'kill -TERM "$(cat /tmp/server.pid)"' -for ((i=0;i<30;i++)); do if docker exec "$server" test -s /tmp/server.exit; then break; fi; sleep .1; done -docker exec "$server" test -s /tmp/server.exit -printf '%s\n' "run=$run restart_exit=$result output=$out" -cat "$out/restarted.log" -cleanup -trap - EXIT -if [[ "$result" == 0 ]]; then - "$harness/analyze.py" "$out" > "$out/assertions.txt" 2>&1 || result=$? - cat "$out/assertions.txt" -fi -exit "$result" +} + +wire_run diff --git a/diam/testdata/sctptlsrestart/README.md b/diam/testdata/sctptlsrestart/README.md index 563b32f..6987ab1 100644 --- a/diam/testdata/sctptlsrestart/README.md +++ b/diam/testdata/sctptlsrestart/README.md @@ -8,8 +8,17 @@ A separate client establishes SCTP from the same source port without sending DATA. The old TLS connection must close, fence writes and finish dispatch without creating another generation. Missing SCTP fails the explicit harness. -Build `i81-wire-tools:go1.27.2` with `diam/sm/testdata/sctprestart/Dockerfile`, -then run from the repository root with Docker, Bash and Python 3: +The runner requires Docker, Bash and Python 3. It uses +`scripts/sctp-wire/harness.sh`, the shared Dockerfile and tshark exporter. +It independently builds `sctp-wire-tools:go1.27.2` if absent and verifies it +otherwise; no sm or peer run is required first. `SCTP_WIRE_IMAGE` selects +another tag, including a fresh tag for an independent first run. Compilation, +peers and tshark use the immutable image ID recorded in the evidence. + +Invoke the runner from any directory using its path. Both arguments remain +required: `source checkout` and `output directory`. There is one TLS scenario +and evidence goes directly into the output directory. For example, from the +repository root: ```sh mkdir -p /path/to/scratch/temp @@ -19,7 +28,14 @@ TMPDIR=/path/to/scratch/temp \ The runner copies source into an internal throwaway tree, mounts no host Go cache, saves both namespace captures/logs/source hashes, joins explicit child -PIDs and removes its containers/network. Its tshark analyzer requires a second INIT/INIT ACK/COOKIE ECHO/COOKIE ACK, +PIDs and removes its containers/network. Artifacts are copied out as the calling +user; privileged peers bind only the test binary. Every run uses unique +`sctp-wire-*` container/network names and a `sctp-wire-build.*` temporary +directory. Cleanup touches only resources labeled for that run. Docker CLI +configuration stays in the temporary directory. Set `TMPDIR` to choose its +location; no files are written outside that directory and the output directory. + +Its tshark analyzer requires a second INIT/INIT ACK/COOKIE ECHO/COOKIE ACK, positive receiver INPUT drop, and no DATA from the restarted client. A successful server also must emit no stale TLS DATA after the restart; the runner checks client success and the killed initial client's exit 137. diff --git a/diam/testdata/sctptlsrestart/analyze.py b/diam/testdata/sctptlsrestart/analyze.py index c5fc32f..352c235 100644 --- a/diam/testdata/sctptlsrestart/analyze.py +++ b/diam/testdata/sctptlsrestart/analyze.py @@ -1,12 +1,12 @@ -import csv,pathlib,re,subprocess,sys +import csv,pathlib,re,runpy,sys out=pathlib.Path(sys.argv[1]).resolve() -run=(out/'run-id').read_text().strip() +tshark=runpy.run_path(str(pathlib.Path(__file__).resolve().parents[3]/'scripts/sctp-wire/tshark.py'))['tshark'] fields=['frame.number','frame.time_epoch','ip.src','sctp.chunk_type','sctp.srcport','sctp.dstport'] rows=[] for role in ['client','server']: - cmd=['docker','run','--rm','--label',f'harness={run}','--mount',f'type=bind,src={out},dst=/evidence,readonly','i81-wire-tools:go1.27.2','tshark','-r',f'/evidence/{role}-wire.pcapng','-Y','sctp','-T','fields'] - for f in fields:cmd+=['-e',f] - proc=subprocess.run(cmd,text=True,capture_output=True,check=True) + arguments=['-Y','sctp','-T','fields'] + for f in fields:arguments+=['-e',f] + proc=tshark(out,f'{role}-wire.pcapng',*arguments) (out/f'{role}-packets.tsv').write_text(proc.stdout) parsed=[dict(zip(fields,row)) for row in csv.reader(proc.stdout.splitlines(),delimiter='\t')] assert any('6' in r['sctp.chunk_type'].split(',') for r in parsed),'ABORT absent at capture point' diff --git a/diam/testdata/sctptlsrestart/run.sh b/diam/testdata/sctptlsrestart/run.sh index 7ce7273..fa9ceac 100755 --- a/diam/testdata/sctptlsrestart/run.sh +++ b/diam/testdata/sctptlsrestart/run.sh @@ -1,46 +1,26 @@ #!/usr/bin/env bash +# Scenario commands and timings; shared lifecycle lives in scripts/sctp-wire. set -euo pipefail harness=$(cd "$(dirname "$0")" && pwd) +root=$(cd "$harness/../../.." && pwd) +# shellcheck source=scripts/sctp-wire/harness.sh +source "$root/scripts/sctp-wire/harness.sh" +scenario=tls +variant=basic source=$(realpath "$1") -out=$(realpath -m "$2") -mkdir -p "$out" -rig=$(mktemp -d "${TMPDIR:-/tmp}/i81-f4-build.XXXXXX") -run=i81-f4-${rig##*.} -network=$run-net -server=$run-server -client=$run-client -cleanup(){ - for peer in "$client" "$server"; do - for child in initial server capture; do - signal=TERM; [[ $child == initial ]] && signal=KILL; [[ $child == capture ]] && signal=INT - if docker exec "$peer" test -s "/tmp/$child.pid" >/dev/null 2>&1; then - docker exec "$peer" sh -c 'test -s "/tmp/$2.exit" || kill -"$1" "$(cat "/tmp/$2.pid")"' sh "$signal" "$child" >/dev/null 2>&1 || true - for ((i=0;i<30;i++)); do if docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then break; fi; sleep .1; done - fi - done - for file in server.log initial.log capture.log capture.exit initial.exit server.exit wire.pcapng; do - docker cp "$peer:/tmp/$file" "$out/${peer##*-}-$file" >/dev/null 2>&1 || true - done - done - docker rm -f -v "$client" "$server" >/dev/null 2>&1 || true - docker network rm "$network" >/dev/null 2>&1 || true - docker ps -a --filter "label=harness=$run" --format '{{.Names}}' > "$out/cleanup-containers.txt" - docker network ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-networks.txt" - python3 -c 'import shutil,sys;shutil.rmtree(sys.argv[1])' "$rig" -} -trap cleanup EXIT -trap 'exit 130' INT TERM -printf '%s\n' "$run" > "$out/run-id" -mkdir "$rig/out" -docker run --rm --label "harness=$run" --mount "type=bind,src=$source,dst=/source,readonly" --mount "type=bind,src=$rig/out,dst=/out" i81-wire-tools:go1.27.2 sh -c 'cp -a /source /tmp/build; cd /tmp/build; sha256sum diam/server.go diam/network_sctp.go diam/accepted_generation.go diam/server_sctp_tls_restart_wire_test.go >/out/source-hashes.txt; go test -c -tags=sctptlsrestart -o /out/f4.test ./diam' > "$out/build.log" 2>&1 -cp "$rig/out/source-hashes.txt" "$out/source-hashes.txt" -docker network create --label "harness=$run" "$network" >/dev/null -for peer in "$server" "$client"; do docker run -d --privileged --label "harness=$run" --name "$peer" --network "$network" --mount "type=bind,src=$rig/out/f4.test,dst=/f4.test,readonly" i81-wire-tools:go1.27.2 sleep infinity >/dev/null; done -sip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$server") -cip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$client") -docker exec "$server" iptables -I INPUT -p sctp -s "$cip" --sport 43869 --dport 5868 -m sctp --chunk-types any ABORT -j DROP -for peer in "$server" "$client"; do docker exec -d "$peer" sh -c 'tshark -i eth0 -f sctp -w /tmp/wire.pcapng >/tmp/capture.log 2>&1 & pid=$!; echo "$pid" >/tmp/capture.pid; wait "$pid"; echo "$?" >/tmp/capture.exit'; done -sleep 1 +output=$(realpath -m "$2") +fixture= +build=(test -c -tags=sctptlsrestart -o /out/f4.test ./diam) +binary=f4.test +wire_path=/f4.test +server_port=5868 +client_port=43869 +capture_suffix=-wire +cleanup_attempts=30 +analyze_on_failure=true +hashes=(diam/server.go diam/network_sctp.go diam/accepted_generation.go diam/server_sctp_tls_restart_wire_test.go) + +wire_scenario() { docker exec -d -e F4_SERVER_ADDR="$sip:5868" "$server" sh -c '/f4.test -test.run "^TestBuiltInSCTPTLSRestartWireServer$" -test.v -test.timeout 20s >/tmp/server.log 2>&1 & pid=$!; echo "$pid" >/tmp/server.pid; wait "$pid"; echo "$?" >/tmp/server.exit' sleep .5 docker exec -d -e F4_SERVER_ADDR="$sip:5868" -e F4_CLIENT_ADDR="$cip:43869" -e F4_PHASE=initial "$client" sh -c '/f4.test -test.run "^TestBuiltInSCTPTLSRestartWireClient$" -test.v -test.timeout 20s >/tmp/initial.log 2>&1 & pid=$!; echo "$pid" >/tmp/initial.pid; wait "$pid"; echo "$?" >/tmp/initial.exit' @@ -62,18 +42,6 @@ docker exec "$server" test -s /tmp/server.exit docker cp "$server:/tmp/server.exit" "$out/server.exit" >/dev/null result=$(cat "$out/server.exit") printf '%s\n' "$client_result" > "$out/restart.exit" -docker exec "$server" iptables -nvxL INPUT > "$out/iptables.txt" -docker exec "$server" uname -a > "$out/kernel.txt" -docker inspect "$server" --format '{{.Image}}' > "$out/image-id.txt" -cleanup -trap - EXIT -cat "$out/server-server.log" "$out/restart.log" -if [[ $client_result != 0 ]]; then exit "$client_result"; fi -set +e -python3 "$harness/analyze.py" "$out" > "$out/assertions.txt" 2>&1 -analysis_result=$? -set -e -printf '%s\n' "$analysis_result" > "$out/assertions.exit" -cat "$out/assertions.txt" -if [[ $result == 0 ]]; then result=$analysis_result; fi -exit "$result" +} + +wire_run diff --git a/diam/peer/testdata/sctprestart/Dockerfile b/scripts/sctp-wire/Dockerfile similarity index 100% rename from diam/peer/testdata/sctprestart/Dockerfile rename to scripts/sctp-wire/Dockerfile diff --git a/scripts/sctp-wire/harness.sh b/scripts/sctp-wire/harness.sh new file mode 100644 index 0000000..c74a144 --- /dev/null +++ b/scripts/sctp-wire/harness.sh @@ -0,0 +1,183 @@ +#!/usr/bin/env bash +# Shared lifecycle for the package-local SCTP wire scenarios. +# The caller supplies build arguments, source hashes, ports and wire_scenario. +# Variables are shared with the sourcing scenario. +# shellcheck disable=SC2034,SC2154 + +wire_wait() { + local peer=$1 child=$2 attempts=$3 i + for ((i=0; i/dev/null 2>&1; then return 0; fi + sleep .1 + done + docker exec "$peer" test -s "/tmp/$child.exit" +} + +wire_stop() { + local peer=$1 child=$2 signal=$3 attempts=$4 + if docker exec "$peer" test -s "/tmp/$child.pid" >/dev/null 2>&1; then + if ! docker exec "$peer" test -s "/tmp/$child.exit" >/dev/null 2>&1; then + docker exec "$peer" sh -c 'kill -"$1" "$(cat "/tmp/$2.pid")"' sh "$signal" "$child" + wire_wait "$peer" "$child" "$attempts" + fi + fi +} + +wire_cleanup() { + local peer child signal file destination resource + for peer in "$client" "$server"; do + # Never stop or remove a resource owned by another run. + [[ $(docker inspect -f '{{index .Config.Labels "harness"}}' "$peer" 2>/dev/null) == "$run" ]] || continue + for child in capture initial server alternate; do + signal=TERM + if [[ $child == capture ]]; then signal=INT; fi + if [[ $child == initial ]]; then signal=KILL; fi + wire_stop "$peer" "$child" "$signal" "$cleanup_attempts" >/dev/null 2>&1 || true + done + for file in wire.pcapng capture.log capture.exit server.log server.exit initial.log initial.exit alternate.log alternate.exit; do + destination=$out/${peer##*-}-$file + docker cp "$peer:/tmp/$file" "$destination" >/dev/null 2>&1 || true + done + done + for resource in "$client" "$server" "$builder"; do + if [[ $(docker inspect -f '{{index .Config.Labels "harness"}}' "$resource" 2>/dev/null) == "$run" ]]; then + docker rm -f -v "$resource" >/dev/null 2>&1 || true + fi + done + if [[ $(docker network inspect -f '{{index .Labels "harness"}}' "$network" 2>/dev/null) == "$run" ]]; then + docker network rm "$network" >/dev/null 2>&1 || true + fi + # This also reaps a tshark export interrupted during analysis. + while IFS= read -r resource; do + [[ -n $resource ]] || continue + docker rm -f -v "$resource" >/dev/null 2>&1 || true + done < <(docker ps -aq --filter "label=harness=$run") + docker ps -a --filter "label=harness=$run" --format '{{.Names}}' > "$out/cleanup-containers.txt" + docker volume ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-volumes.txt" + docker network ls --filter "label=harness=$run" --format '{{.Name}}' > "$out/cleanup-networks.txt" + docker ps -a --filter "name=$run" --format '{{.Names}}' > "$out/cleanup-name.txt" + docker volume ls --filter "name=$run" --format '{{.Name}}' > "$out/cleanup-name-volumes.txt" + docker network ls --filter "name=$run" --format '{{.Name}}' > "$out/cleanup-name-networks.txt" +} + +wire_exit() { + wire_cleanup + rm -rf "$rig" +} + +wire_build() { + local tools=$root/scripts/sctp-wire image=${SCTP_WIRE_IMAGE:-sctp-wire-tools:go1.27.2} + # Builder state stays in the run's scratch directory; the caller's Docker + # context and credentials stay in effect for every command. + mkdir -p "$rig/buildx" "$rig/out" + if ! docker image inspect "$image" > "$out/image-build.log" 2>&1; then + BUILDX_CONFIG=$rig/buildx docker build -t "$image" "$tools" >> "$out/image-build.log" 2>&1 + fi + image_id=$(docker image inspect -f '{{.Id}}' "$image") + printf '%s\n' "$image_id" > "$out/image-id.txt" + printf '%s\n' "$image" > "$out/image-ref.txt" + # Compile in the container's writable layer; only read-only source is bound. + # docker cp returns caller-owned artifacts, leaving no root-owned host files. + docker create --name "$builder" --label "harness=$run" \ + --mount "type=bind,src=$source,dst=/source,readonly" \ + --mount "type=bind,src=$harness,dst=/harness,readonly" \ + "$image_id" sh -eu -c ' + fixture=$1; hash_count=$2; binary=$3; shift 3 + cp -a /source /tmp/build + cd /tmp/build + if test -n "$fixture"; then mkdir -p "$(dirname "$fixture")"; cp /harness/main.go "$fixture"; fi + mkdir /out + while test "$hash_count" -gt 0; do + if test -f "$1"; then sha256sum "$1"; fi + shift; hash_count=$((hash_count-1)) + done > /out/source-hashes.txt + go "$@" + go version > /out/go-version.txt + go version -m "/out/$binary" > /out/binary-build.txt + ' sh "$fixture" "${#hashes[@]}" "$binary" "${hashes[@]}" "${build[@]}" > "$out/build.log" 2>&1 + docker start -a "$builder" >> "$out/build.log" 2>&1 + docker cp "$builder:/out/." "$rig/out" >/dev/null + cp "$rig/out/source-hashes.txt" "$rig/out/go-version.txt" "$rig/out/binary-build.txt" "$out/" +} + +wire_start() { + local peer + docker network create --label "harness=$run" "$network" >/dev/null + for peer in "$server" "$client"; do + docker run -d --privileged --label "harness=$run" --name "$peer" --network "$network" \ + --mount "type=bind,src=$rig/out/$binary,dst=$wire_path,readonly" "$image_id" sleep infinity >/dev/null + done + sip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$server") + cip=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$client") + # RFC 9260 ยง5.2.4.1 (Errata 7387) describes the retained-association restart. + # Receiver INPUT keeps the stale ABORT visible in both namespace captures. + docker exec "$server" iptables -I INPUT -p sctp -s "$cip" --sport "$client_port" --dport "$server_port" -m sctp --chunk-types any ABORT -j DROP + for peer in "$server" "$client"; do + docker exec -d "$peer" sh -c 'tshark -i eth0 -f sctp -w /tmp/wire.pcapng >/tmp/capture.log 2>&1 & pid=$!; printf "%s\n" "$pid" >/tmp/capture.pid; wait "$pid"; printf "%s\n" "$?" >/tmp/capture.exit' + done + sleep 1 +} + +wire_collect() { + local peer + for peer in "$server" "$client"; do + wire_stop "$peer" capture INT 30 + docker exec "$peer" test -s /tmp/capture.exit + docker cp "$peer:/tmp/wire.pcapng" "$out/${peer##*-}$capture_suffix.pcapng" >/dev/null + docker cp "$peer:/tmp/capture.log" "$out/${peer##*-}-capture.log" >/dev/null + docker cp "$peer:/tmp/capture.exit" "$out/${peer##*-}-capture.exit" >/dev/null + done + docker cp "$server:/tmp/server.log" "$out/server.log" >/dev/null + docker cp "$client:/tmp/initial.log" "$out/initial.log" >/dev/null + docker cp "$client:/tmp/initial.exit" "$out/initial.exit" >/dev/null + if [[ $scenario == peer ]]; then docker cp "$client:/tmp/alternate.log" "$out/alternate.log" >/dev/null; fi + docker exec "$server" iptables -nvxL INPUT > "$out/iptables.txt" + docker exec "$server" uname -a > "$out/kernel.txt" + docker exec "$server" tshark --version > "$out/tshark-version.txt" + wire_stop "$server" server TERM 30 + docker exec "$server" test -s /tmp/server.exit +} + +wire_analyze() { + local analysis_result=0 + if [[ $scenario == tls ]]; then + cat "$out/server-server.log" "$out/restart.log" + if [[ $client_result != 0 ]]; then result=$client_result; return; fi + else + printf '%s\n' "run=$run restart_exit=$result output=$out" + cat "$out/restarted.log" + fi + if [[ $result == 0 || $analyze_on_failure == true ]]; then + python3 -B "$harness/analyze.py" "$out" > "$out/assertions.txt" 2>&1 || analysis_result=$? + cat "$out/assertions.txt" + if [[ $scenario == tls ]]; then printf '%s\n' "$analysis_result" > "$out/assertions.exit"; fi + if [[ $result == 0 || $scenario == peer ]]; then + if [[ $analysis_result != 0 ]]; then result=$analysis_result; fi + fi + fi + if [[ $scenario == peer ]]; then sha256sum "$out"/*.pcapng > "$out/capture-sha256.txt"; fi +} + +wire_run() { + out=$(realpath -m "$output") + mkdir -p "$out" + rig=$(mktemp -d "${TMPDIR:-/tmp}/sctp-wire-build.XXXXXXXXXX") + run=sctp-wire-${scenario}-${variant}-${rig##*.} + network=$run-net + server=$run-server + client=$run-client + builder=$run-build + trap wire_exit EXIT + trap 'exit 130' INT TERM + printf '%s\n' "$run" > "$out/run-id" + wire_build + wire_start + wire_scenario + wire_collect + wire_cleanup + # Keep the trap through analysis so interruption also reaps tshark containers. + wire_analyze + wire_exit + trap - EXIT + exit "$result" +} diff --git a/scripts/sctp-wire/harness_test.py b/scripts/sctp-wire/harness_test.py new file mode 100644 index 0000000..facaad8 --- /dev/null +++ b/scripts/sctp-wire/harness_test.py @@ -0,0 +1,72 @@ +"""Exercise cleanup's Docker boundary with current and foreign resources.""" +import os +import pathlib +import subprocess +import tempfile +import unittest + + +DOCKER = '''#!/usr/bin/env python3 +import os, pathlib, sys +args = sys.argv[1:] +with pathlib.Path(os.environ['DOCKER_TRACE']).open('a') as trace: + trace.write(' '.join(args) + '\\n') +if args[0] == 'inspect': + if args[-1] in ('wire-server', 'wire-client'): + print(os.environ['RESOURCE_OWNER']) + else: + sys.exit(1) +elif args[0] == 'exec': + sys.exit(1) +elif args[0] == 'cp': + if args[1] == 'wire-server:/tmp/server.log': + pathlib.Path(args[2]).write_text('current server log') + else: + sys.exit(1) +elif args[:2] == ['network', 'inspect']: + sys.exit(1) +''' + + +class CleanupTest(unittest.TestCase): + def cleanup(self, directory, owner): + root = pathlib.Path(directory) + docker = root / 'docker' + docker.write_text(DOCKER) + docker.chmod(0o755) + trace = root / 'docker.trace' + harness = pathlib.Path(__file__).with_name('harness.sh').resolve() + env = dict(os.environ, PATH=f'{root}:{os.environ["PATH"]}', + DOCKER_TRACE=str(trace), RESOURCE_OWNER=owner) + subprocess.run(['bash', '-euc', ''' +source "$1" +run=wire +client=wire-client +server=wire-server +builder=wire-build +network=wire-net +out=$2 +cleanup_attempts=1 +wire_cleanup +''', 'bash', str(harness), str(root)], env=env, check=True) + return trace.read_text() + + def test_refreshes_server_log_in_reused_output_directory(self): + with tempfile.TemporaryDirectory() as directory: + log = pathlib.Path(directory) / 'server-server.log' + log.write_text('stale server log') + self.cleanup(directory, 'wire') + self.assertEqual(log.read_text(), 'current server log') + + def test_leaves_foreign_resources_and_artifacts_untouched(self): + with tempfile.TemporaryDirectory() as directory: + log = pathlib.Path(directory) / 'server-server.log' + log.write_text('previous evidence') + trace = self.cleanup(directory, 'another-run') + self.assertEqual(log.read_text(), 'previous evidence') + for command in ('exec ', 'cp ', 'rm '): + self.assertNotIn(command, trace) + + +if __name__ == '__main__': + unittest.main() diff --git a/scripts/sctp-wire/tshark.py b/scripts/sctp-wire/tshark.py new file mode 100644 index 0000000..3a63d94 --- /dev/null +++ b/scripts/sctp-wire/tshark.py @@ -0,0 +1,16 @@ +"""Run tshark against a capture using the tools image recorded by its runner.""" +import pathlib +import subprocess +import uuid + + +def tshark(out, capture, *arguments): + out = pathlib.Path(out).resolve() + run = (out / 'run-id').read_text().strip() + image_file = out / 'image-id.txt' + image = image_file.read_text().strip() if image_file.exists() else 'sctp-wire-tools:go1.27.2' + command = ['docker', 'run', '--rm', '--name', f'{run}-tshark-{uuid.uuid4().hex}', + '--label', f'harness={run}', '--mount', + f'type=bind,src={out},dst=/evidence,readonly', image, + 'tshark', '-r', f'/evidence/{capture}', *arguments] + return subprocess.run(command, text=True, capture_output=True, check=True) diff --git a/scripts/sctp-wire/tshark_test.py b/scripts/sctp-wire/tshark_test.py new file mode 100644 index 0000000..46d0956 --- /dev/null +++ b/scripts/sctp-wire/tshark_test.py @@ -0,0 +1,38 @@ +"""Check the tshark process boundary without requiring packet capture.""" +import pathlib +import runpy +import subprocess +import tempfile +import unittest +from unittest.mock import patch + + +class TsharkExportTest(unittest.TestCase): + def test_uses_recorded_image_and_read_only_evidence(self): + export = runpy.run_path(str(pathlib.Path(__file__).with_name('tshark.py')))['tshark'] + with tempfile.TemporaryDirectory() as directory: + out = pathlib.Path(directory) + (out / 'run-id').write_text('sctp-wire-synthetic') + (out / 'image-id.txt').write_text('sha256:synthetic-image\n') + completed = subprocess.CompletedProcess([], 0, 'packets', 'diagnostic') + with patch('subprocess.run', return_value=completed) as run: + self.assertIs(export(out, 'client.pcapng', '-Y', 'sctp', '-T', 'fields'), completed) + args = run.call_args.args[0] + self.assertIn('sha256:synthetic-image', args) + self.assertIn(f'type=bind,src={out},dst=/evidence,readonly', args) + self.assertIn('harness=sctp-wire-synthetic', args) + self.assertEqual(args[-6:], ['-r', '/evidence/client.pcapng', '-Y', 'sctp', '-T', 'fields']) + self.assertTrue(run.call_args.kwargs['check']) + + def test_standalone_analysis_defaults_to_shared_image(self): + export = runpy.run_path(str(pathlib.Path(__file__).with_name('tshark.py')))['tshark'] + with tempfile.TemporaryDirectory() as directory: + out = pathlib.Path(directory) + (out / 'run-id').write_text('sctp-wire-synthetic') + with patch('subprocess.run') as run: + export(out, 'server.pcapng', '-T', 'jsonraw') + self.assertIn('sctp-wire-tools:go1.27.2', run.call_args.args[0]) + + +if __name__ == '__main__': + unittest.main()