diff --git a/.github/workflows/lint.yaml b/.github/workflows/lint.yaml index 7c6bf7db..1402fa6b 100644 --- a/.github/workflows/lint.yaml +++ b/.github/workflows/lint.yaml @@ -62,6 +62,7 @@ jobs: cd typhoon && cargo fmt --check cd ../evaluation/protocols/typhoon && cargo fmt --check cd ../wireguard_daita/machines-printer && cargo fmt --check + cd ../../quic/quic-eval && cargo fmt --check - name: Run clippy πŸ“Ž working-directory: typhoon @@ -104,12 +105,15 @@ jobs: - name: Setup Go 🦫 uses: actions/setup-go@v5 with: - go-version: "1.22" + go-version: "1.23" + + - name: Install staticcheck πŸ”§ + run: go install honnef.co/go/tools/cmd/staticcheck@latest - name: Lint Go (wireguard_daita) πŸ” working-directory: evaluation/protocols/wireguard_daita/wg-daita run: | git clone --depth=1 --branch mullvad https://github.com/mullvad/wireguard-go ../wireguard-go - go install honnef.co/go/tools/cmd/staticcheck@latest go mod tidy + go vet -tags daita ./... staticcheck -tags daita ./... diff --git a/.gitignore b/.gitignore index 6db9f718..d0389ca6 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,12 @@ typhoon/.test_keys # Evaluation: +evaluation/transport/target +evaluation/transport/Cargo.lock + +evaluation/protocols/quic/quic-eval/target +evaluation/protocols/quic/quic-eval/Cargo.lock + evaluation/protocols/wireguard_daita/wg-daita/go.sum evaluation/protocols/wireguard_daita/wireguard-go diff --git a/PROTOCOL.md b/PROTOCOL.md index 07e4c374..e9d7005e 100644 --- a/PROTOCOL.md +++ b/PROTOCOL.md @@ -1472,6 +1472,27 @@ Flushing too eagerly (on every packet) eliminates the syscall reduction; flushin The queue must also account for the fact that `sendmmsg` batches datagrams for a single socket, while the server may manage multiple sockets simultaneously (one per flow on SO_REUSEPORT platforms), meaning coalescing must happen per-socket. The right flush policy β€” fixed threshold, fixed deadline, or adaptive β€” requires empirical benchmarking under realistic load patterns. +#### Multi-job send/receive queue routine + +The batching described above coalesces the wire packets of a _single_ user message, but the dominant cost under sustained bulk transfer is different: profiling a saturated flow shows the process spending the large majority of its on-CPU time inside the per-datagram `sendto`/`recvfrom` syscalls, with symmetric crypto a rounding error by comparison. +The reason is architectural rather than cryptographic β€” the current data path issues one syscall per wire packet and, on the sender side, the caller `await`s each `send_bytes` through the full flow-manager stack before the next packet is prepared, so throughput is bounded by _per-packet round-trips through the executor_ rather than by how fast bytes can be encrypted. +A multi-job queue routine decouples the application-visible send/receive calls from the syscalls that move bytes, so that many packets amortise into few kernel crossings and the crypto work spreads across cores. + +**Send path.** +Each socket owns a bounded multi-producer queue of ready-to-send jobs. +A `send_bytes` call splits its user message into wire packets, performs the trailer/crypto work β€” optionally handed to a small crypto worker pool so several messages encrypt in parallel β€” and enqueues the finished `(wire_bytes, addr)` jobs, returning as soon as they are accepted rather than after the datagrams have left the host. +A dedicated per-socket flusher job drains the queue and issues one `sendmmsg` per drained batch, bounded by a maximum batch width (the `sendmmsg` vector limit) and a flush policy that fires on whichever of a fill threshold or a short deadline is reached first. +Decoy emission feeds the same queue, so real and decoy datagrams interleave in the batched syscall exactly as they would on the wire and no separate send path can be timed apart from the data path. + +**Receive path.** +A dedicated per-socket reader job pulls a batch of datagrams in a single `recvmmsg` call and hands the raw buffers to a decrypt worker pool; each worker locates and verifies the trailer, strips fake header/body padding, and routes the recovered payload to the owning session's receive queue keyed by identity. +Batched receive matters most on the server, where a single listening socket may carry datagrams for many concurrent sessions in one `recvmmsg` sweep; per-session ordering is preserved because a single reader assigns arrival order before fan-out, and the worker pool only parallelises the per-packet crypto, not the enqueue. + +**The challenge**: the queue routine reintroduces, at a larger granularity, every tension the single-message batching already faces β€” flush eagerness versus syscall amortisation, and per-socket rather than per-process coalescing β€” but adds three of its own. +First, ordering: parallel crypto workers may finish out of order, so the design must either restore per-session sequence before delivery or rely on the existing trailer sequencing to tolerate reordering, and must not let a slow worker head-of-line-block an unrelated session. +Second, backpressure: when a queue fills β€” a stalled link, a slow consumer β€” `send_bytes` must block or shed rather than grow memory unboundedly, and that backpressure has to surface to the caller without deadlocking the flusher that drains the same queue. +Third, timing fidelity: coalescing must not distort the very inter-packet timing the flow shaper works to produce, so the flush deadline has to stay well below the smallest inter-arrival gap the shaper and health-check scheduler rely on, or the batching becomes a fingerprint of its own. + ### Per-deployment randomisation seed In the current design, every TYPHOON deployment draws `FlowConfig` and decoy parameters from the same global distributions defined in `TYPHOON_*` settings keys. diff --git a/README.md b/README.md index e03b1f4f..8f452ab1 100644 --- a/README.md +++ b/README.md @@ -246,7 +246,7 @@ At default sizes, both TYPHOON _and_ other UDP-native services (WireGuard, OpenV The [`evaluation/`](evaluation/) directory contains a Docker-based traffic capture and analysis harness, organised into three independent parts: 1. **TYPHOON self-comparison** β€” measure run-to-run and scenario-to-scenario variability of TYPHOON's own traffic profile. -2. **Operational comparison** β€” capture all 16 protocols (TYPHOON + 15 comparators) under a controlled Docker network and compare throughput, overhead, goodput efficiency, byte entropy, burstiness, and handshake metrics. _Operational, not detectability._ +2. **Operational comparison** β€” run a per-packet latency ping against all 16 protocols (TYPHOON + 15 comparators) under a controlled Docker network and compare **delivery** and the **round-trip latency distribution** (clean and under netem loss/jitter), plus wire-shape metrics (overhead, entropy, burstiness, handshake). _Operational, not detectability._ 3. **Background-blending evaluation** β€” generate a corpus of natural UDP traffic (QUIC HTTPS, DNS, RTP voice/video, gaming, control plane), run TYPHOON alongside, and measure how often a passive classifier mistakes TYPHOON for benign traffic. See [evaluation/README.md](evaluation/README.md) for requirements, install steps, CLI reference, and instructions for reading the results. diff --git a/evaluation/README.md b/evaluation/README.md index 5095f741..b1a0d04b 100644 --- a/evaluation/README.md +++ b/evaluation/README.md @@ -8,7 +8,8 @@ For the protocol itself, see [PROTOCOL.md](../PROTOCOL.md). ```text evaluation/ -β”œβ”€β”€ protocols/ # Dockerfiles for the 15 comparison protocols +β”œβ”€β”€ protocols/ # Dockerfiles for the 16 comparison protocols +β”œβ”€β”€ transport/ # shared Rust crate: the senders/sinks every protocol runs β”œβ”€β”€ background/ # Dockerfiles for the 8 UDP-traffic generators + open-set unknown β”œβ”€β”€ compose/, observer/ # docker-compose stacks and the tcpdump observer container β”œβ”€β”€ chaos/ # tc/netem sidecar for latency / jitter / loss @@ -30,7 +31,11 @@ Four independent parts; each answers one question. ### Part 2 β€” Operational comparison -*Where does TYPHOON sit on throughput / overhead / handshake-cost relative to 15 other UDP/TCP secure-transport protocols?* Captures each protocol once under the same Docker network and emits direct operational metrics. **No classifiers** β€” a closed-world classifier across these 16 protocols would score near 100 % because each protocol has a distinct wire footprint by design, which says nothing about Part 3's question. +*How does TYPHOON compare on reliability and cost to 15 other UDP/TCP secure-transport protocols?* +Runs a **per-packet latency ping** against each: a spaced sequence of small equal-sized probes (500 Γ— 256 B @ 20 ms) that the sink echoes, measuring per-packet **round-trip time** in tunnel-like conditions. +It reports **delivery** (what fraction the receiver gets / echoes back) and the **RTT distribution**. +See [Operational metrics](#operational-metrics-delivery--latency) for what actually discriminates the protocols (spoiler: the tail, not the median). +**No classifiers** β€” a closed-world classifier across these 16 protocols would score near 100 % because each protocol has a distinct wire footprint by design, which says nothing about Part 3's question. Protocols compared: `raw_udp`, `raw_tcp`, `tls`, `wireguard`, `quic`, `obfs4` (Γ—3 IAT modes), `amneziawg`, `hysteria2`, `shadowsocks`, `tor`, `vless_reality`, `openvpn`, `wireguard_daita`, `typhoon`. @@ -88,13 +93,13 @@ poe plot --example heavy_traffic --out-dir out/ # per-flow packet-structure SV ### Part 2 β€” Operational comparison (CLI) ```shell -poe capture --all # capture all 16 protocols (default: bulk, 10 MB) -poe capture --all --chaos # …with latency + jitter + loss -poe analyze # parse pcaps β†’ stats.json +poe capture --all # per-packet latency ping across all 16 protocols +poe capture --all --chaos # …under netem (2% loss, 100 msΒ±30 ms delay) +poe analyze # parse pcaps β†’ stats.json (detectability metrics) poe proto-compare # plots + comparison table for the latest run ``` -Useful flags: `--protocol ` (single protocol), `--scenario {bulk,interactive,streaming,burst,echo,idle}`, `--run YYYYMMDD_HHMMSS` (target an earlier run). +Useful `capture` flags: `--protocol ` (single protocol); `--chaos` + `--loss-pct `; `--seed ` (reproducible). `--profile ` selects the TYPHOON *settings* profile (fake-body mode, decoys); `bulk_upload` is the operational default, the rest are the Part-3 mimicry profiles. `analyze`/`proto-compare` take `--run YYYYMMDD_HHMMSS` to target an earlier run. ### Part 3 β€” Background-blending (CLI) @@ -155,8 +160,8 @@ Outputs are split into two trees: results/ β”œβ”€β”€ captures/run_/ β”‚ β”œβ”€β”€ .pcap # raw capture (handshake + data) -β”‚ β”œβ”€β”€ stats.json # per-pcap metrics β€” see below -β”‚ β”œβ”€β”€ metadata.json # transfer_bytes, scenario, timing +β”‚ β”œβ”€β”€ stats.json # per-pcap detectability metrics β€” see below +β”‚ β”œβ”€β”€ metadata.json # delivery %, per-packet RTT distribution (rtt_*) β”‚ └── logs// # client + server + observer container logs └── background/pipeline_/run_*/ # Part 3 per-run pcaps + metadata @@ -175,6 +180,28 @@ artifacts/pipeline_/ └── distplot/ # per-pair size/IAT overlays PDFs + JSON ``` +### Operational metrics (delivery / latency) + +Part 2's numbers come from the two endpoints (recorded in `metadata.json`), not the pcap: + +| Metric | Meaning | +| --- | --- | +| `delivery_pct` | probes the sink received Γ· sent (one-way); `roundtrip_delivery_pct` is the client's echoes-received Γ· sent | +| `rtt_min / p50 / p95 / p99_ms`, `rtt_jitter_ms` | per-packet round-trip distribution | + +**Read the tail, not the median.** +The p50 RTT is dominated by the shared path β€” the observer hop in clean mode, the netem delay under chaos β€” and is ~identical across protocols, so it is *not* a discriminator. +The signal lives in two places: + +- **Delivery** separates best-effort from reliable: under chaos, UDP-family transports settle at the raw ~2 % one-way loss (they *drop* it) while TCP/QUIC/proxies stay 100 % (they *retransmit*). +- **The latency tail (p95/p99)** exposes the *cost* of a protocol's design that the median hides: + - clean β€” traffic-shapers pay a visible tail (wireguard_daita p95 ~45 ms, obfs4 IAT-mode ~24 ms) from their padding / inter-arrival machines, while plain transports sit at ~7 ms; + - chaos β€” reliable transports pay a retransmit spike (p99 ~600–850 ms) that best-effort UDP does not (p99 β‰ˆ p95). + +So the metric answers *"how reliable is it, and what does its obfuscation / loss-recovery cost in latency"* β€” e.g. TYPHOON delivers like the other UDP transports yet keeps a flat tail (its decoys / fake-headers add no measurable latency penalty), unlike DAITA. +It is deliberately **not** a throughput benchmark: the shared capture point and best-effort UDP's lack of congestion control make bulk throughput an artifact *across* transport classes (an earlier bulk-transfer design was abandoned for exactly this reason β€” see the git history). +For the TYPHOON implementation's raw speed on loopback, free of these network artifacts, see **Part 4** (`cargo bench`). + ### Per-pcap metrics (`stats.json`) Computed separately per direction (`c2s`, `s2c`, `all`). Packet sizes are **transport-payload bytes** (UDP payload or TCP segment data) β€” IP/UDP/TCP header bytes are excluded so transport overhead doesn't leak into protocol stats. @@ -201,7 +228,7 @@ Computed separately per direction (`c2s`, `s2c`, `all`). Packet sizes are **tran ### Part 2 plots (under `artifacts//proto_compare/`) -- `run__proto_compare.pdf` β€” six panels: (A) size CDF, (B) IAT CDF, (C) throughput vs goodput-efficiency scatter, (D) overhead bars, (E) byte entropy by phase, (F) normalised heatmap. +- `run__proto_compare.pdf` β€” six panels: (A) size CDF, (B) IAT CDF, (C) per-packet RTT vs delivery scatter, (D) overhead bars, (E) byte entropy by phase, (F) normalised heatmap. - `run__handshake.pdf` β€” handshake duration / packet count / byte fraction across protocols. - `run__compare_table.md` β€” one row per protocol; quick-glance ranking by any column. diff --git a/evaluation/compose/docker-compose.build.yml b/evaluation/compose/docker-compose.build.yml index 3ad524a4..992df166 100644 --- a/evaluation/compose/docker-compose.build.yml +++ b/evaluation/compose/docker-compose.build.yml @@ -15,6 +15,12 @@ name: typhoon-eval-build services: + # ── Shared transport binaries (built once, COPY --from into each protocol) ── + transport: + build: + context: ../transport + image: typhoon-eval-transport + # ── Observer ────────────────────────────────────────────────────────────── observer: build: @@ -32,12 +38,16 @@ services: build: context: ../protocols/raw_udp dockerfile: Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-raw-udp-server raw-udp-client: build: context: ../protocols/raw_udp dockerfile: Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-raw-udp-client # ── raw_tcp ─────────────────────────────────────────────────────────────── @@ -45,12 +55,16 @@ services: build: context: ../protocols dockerfile: raw_tcp/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-raw-tcp-server raw-tcp-client: build: context: ../protocols dockerfile: raw_tcp/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-raw-tcp-client # ── tls ─────────────────────────────────────────────────────────────────── @@ -58,12 +72,16 @@ services: build: context: ../protocols/tls dockerfile: Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-tls-server tls-client: build: context: ../protocols dockerfile: tls/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-tls-client # ── wireguard ───────────────────────────────────────────────────────────── @@ -71,12 +89,16 @@ services: build: context: ../protocols dockerfile: wireguard/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-wireguard-server wireguard-client: build: context: ../protocols dockerfile: wireguard/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-wireguard-client # ── quic ────────────────────────────────────────────────────────────────── @@ -97,12 +119,16 @@ services: build: context: ../protocols dockerfile: obfs4/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-obfs4-server obfs4-client: build: context: ../protocols dockerfile: obfs4/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-obfs4-client # ── shadowsocks ─────────────────────────────────────────────────────────── @@ -110,12 +136,16 @@ services: build: context: ../protocols dockerfile: shadowsocks/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-shadowsocks-server shadowsocks-client: build: context: ../protocols dockerfile: shadowsocks/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-shadowsocks-client # ── tor ─────────────────────────────────────────────────────────────────── @@ -123,12 +153,16 @@ services: build: context: ../protocols/tor dockerfile: Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-tor-server tor-client: build: context: ../protocols dockerfile: tor/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-tor-client # ── vless_reality ───────────────────────────────────────────────────────── @@ -136,12 +170,16 @@ services: build: context: ../protocols dockerfile: vless_reality/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-vless-reality-server vless-reality-client: build: context: ../protocols dockerfile: vless_reality/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-vless-reality-client # ── amneziawg ───────────────────────────────────────────────────────────── @@ -149,12 +187,16 @@ services: build: context: ../protocols dockerfile: amneziawg/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-amneziawg-server amneziawg-client: build: context: ../protocols dockerfile: amneziawg/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-amneziawg-client # ── hysteria2 ───────────────────────────────────────────────────────────── @@ -162,12 +204,16 @@ services: build: context: ../protocols dockerfile: hysteria2/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-hysteria2-server hysteria2-client: build: context: ../protocols dockerfile: hysteria2/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-hysteria2-client # ── openvpn ─────────────────────────────────────────────────────────────── @@ -175,12 +221,16 @@ services: build: context: ../protocols dockerfile: openvpn/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-openvpn-server openvpn-client: build: context: ../protocols dockerfile: openvpn/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-openvpn-client # ── wireguard_daita ─────────────────────────────────────────────────────── @@ -188,12 +238,16 @@ services: build: context: ../protocols dockerfile: wireguard_daita/Dockerfile.server + additional_contexts: + transport: service:transport image: typhoon-eval-wireguard-daita-server wireguard-daita-client: build: context: ../protocols dockerfile: wireguard_daita/Dockerfile.client + additional_contexts: + transport: service:transport image: typhoon-eval-wireguard-daita-client # ── typhoon ─────────────────────────────────────────────────────────────── diff --git a/evaluation/compose/docker-compose.yml b/evaluation/compose/docker-compose.yml index 56a9f9da..a1edc59d 100644 --- a/evaluation/compose/docker-compose.yml +++ b/evaluation/compose/docker-compose.yml @@ -27,6 +27,10 @@ x-profile-env: &profile-env PROFILE_BURST_IDLE_S: ${PROFILE_BURST_IDLE_S:-0} PROFILE_DECOYS_ENABLED: ${PROFILE_DECOYS_ENABLED:-1} PROFILE_ID_LENGTH: ${PROFILE_ID_LENGTH:-} + LAT_COUNT: ${LAT_COUNT:-} + LAT_INTERVAL_MS: ${LAT_INTERVAL_MS:-} + LAT_SIZE: ${LAT_SIZE:-} + LAT_RECV_TIMEOUT_MS: ${LAT_RECV_TIMEOUT_MS:-} volumes: eval_keys: {} @@ -88,15 +92,11 @@ services: - net.ipv6.conf.all.disable_ipv6=1 environment: <<: *profile-env - # Observer's net_right IP β€” server adds return route via this gateway OBSERVER_GW: 172.21.0.2 - # Server's own externally-reachable IP β€” embedded in the TYPHOON certificate - # so the client knows where to connect (0.0.0.0 is unroutable as a peer address) CERT_HOST: 172.21.0.10 - # obfs4_iat: when set, obfs4proxy uses the given IAT mode (0=off, 1=on, 2=paranoid) OBFS4_IAT_MODE: ${OBFS4_IAT_MODE:-} + IDLE_TIMEOUT_S: ${IDLE_TIMEOUT_S:-30} RUST_LOG: ${RUST_LOG:-} - # TYPHOON crate settings overrides (read by SettingsBuilder via env; empty for non-typhoon protocols so they're not affected). TYPHOON_DRAIN_CHANNEL_CAPACITY: ${TYPHOON_DRAIN_CHANNEL_CAPACITY:-} TYPHOON_RECEIVE_BUFFER_SIZE: ${TYPHOON_RECEIVE_BUFFER_SIZE:-} TYPHOON_DECOY_HEAVY_BASE_RATE: ${TYPHOON_DECOY_HEAVY_BASE_RATE:-} @@ -124,9 +124,8 @@ services: environment: <<: *profile-env SERVER_HOST: 172.21.0.10 - # Observer's net_left IP β€” client adds forward route via this gateway OBSERVER_GW: 172.20.0.2 - INTER_PACKET_DELAY_MS: ${INTER_PACKET_DELAY_MS:-40} + INTER_PACKET_DELAY_MS: ${INTER_PACKET_DELAY_MS:-0} DELAY_EVERY_N: ${DELAY_EVERY_N:-10} TRAFFIC_SCENARIO: ${TRAFFIC_SCENARIO:-bulk} QUIC_WAIT_TIMEOUT_S: ${QUIC_WAIT_TIMEOUT_S:-240} diff --git a/evaluation/protocols/amneziawg/Dockerfile.client b/evaluation/protocols/amneziawg/Dockerfile.client index 77bce259..372c9348 100644 --- a/evaluation/protocols/amneziawg/Dockerfile.client +++ b/evaluation/protocols/amneziawg/Dockerfile.client @@ -19,13 +19,11 @@ RUN make ## Final image FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 python3 python3-pip libmnl0 \ - && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir --break-system-packages PySocks + iproute2 libmnl0 \ + && rm -rf /var/lib/apt/lists/* COPY --from=go-builder /build/amneziawg-go /usr/local/bin/amneziawg-go COPY --from=tools-builder /src/src/wg /usr/local/bin/awg COPY amneziawg/client-entrypoint.sh /entrypoint.sh -COPY common/udp_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/udp-sender /usr/local/bin/udp-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/amneziawg/Dockerfile.server b/evaluation/protocols/amneziawg/Dockerfile.server index e0675502..8b2da17e 100644 --- a/evaluation/protocols/amneziawg/Dockerfile.server +++ b/evaluation/protocols/amneziawg/Dockerfile.server @@ -19,12 +19,12 @@ RUN make ## Final image FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 python3 libmnl0 \ + iproute2 libmnl0 \ && rm -rf /var/lib/apt/lists/* COPY --from=go-builder /build/amneziawg-go /usr/local/bin/amneziawg-go COPY --from=tools-builder /src/src/wg /usr/local/bin/awg COPY amneziawg/server-entrypoint.sh /entrypoint.sh -COPY common/udp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/udp-sink /usr/local/bin/udp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=15s \ CMD ip link show awg0 >/dev/null 2>&1 && ss -uln | grep -q ':9000' diff --git a/evaluation/protocols/amneziawg/client-entrypoint.sh b/evaluation/protocols/amneziawg/client-entrypoint.sh index e95cc85d..209b1210 100755 --- a/evaluation/protocols/amneziawg/client-entrypoint.sh +++ b/evaluation/protocols/amneziawg/client-entrypoint.sh @@ -37,4 +37,4 @@ for i in {1..30}; do sleep 1 done -SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec python3 /app/client.py +SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec udp-sender diff --git a/evaluation/protocols/amneziawg/server-entrypoint.sh b/evaluation/protocols/amneziawg/server-entrypoint.sh index 16224894..5fe06243 100755 --- a/evaluation/protocols/amneziawg/server-entrypoint.sh +++ b/evaluation/protocols/amneziawg/server-entrypoint.sh @@ -25,7 +25,7 @@ ip link set awg0 up PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +udp-sink & SINK_PID=$! trap 'kill -TERM "${SINK_PID}" 2>/dev/null; wait "${SINK_PID}"; exit' SIGTERM SIGINT diff --git a/evaluation/protocols/common/_profile.py b/evaluation/protocols/common/_profile.py deleted file mode 100644 index 1ae9d36e..00000000 --- a/evaluation/protocols/common/_profile.py +++ /dev/null @@ -1,213 +0,0 @@ -#!/usr/bin/env python3 -"""Profile-driven traffic execution for non-TYPHOON evaluation senders. - -Reads PROFILE_* env vars (written by the evaluation orchestrator) to drive -the c2s portion of any profile from `shared/profiles.py`. Non-TYPHOON -protocols cannot drive bidirectional s2c traffic without changes to the -underlying server image, so the s2c portion of any profile is silently -ignored on this side. - -Two flavours: - * `run_profile(send_fn)` β€” sync, for protocols whose data path is a - blocking call (TCP `sendall`, SOCKS5 wrapper, raw UDP `send`). - * `run_profile_async(send_fn)` β€” async, for protocols whose data path - is asyncio-based (QUIC). `send_fn` may be sync or async; if it - returns an awaitable, the loop awaits it. - -Both honour the same `PROFILE_*` knobs (chunk, IAT, bytes, duration, -bursty, INTER_PACKET_DELAY_MS, DELAY_EVERY_N). -""" - -from asyncio import sleep as asleep -from collections.abc import Awaitable, Callable -from inspect import isawaitable -from os import environ, urandom -from time import monotonic, sleep -from typing import NamedTuple - - -class _ProfileConfig(NamedTuple): - chunk_c2s: int - iat_c2s_ms: float - bytes_c2s: int - duration_s: float - bursty: bool - burst_count: int - burst_idle_s: float - inter_batch_delay_ms: float - batch_size: int - - -def _env_int(key: str, default: int) -> int: - raw = environ.get(key) - return int(raw) if raw is not None and raw.strip() else default - - -def _env_float(key: str, default: float) -> float: - raw = environ.get(key) - return float(raw) if raw is not None and raw.strip() else default - - -def _read_config() -> _ProfileConfig: - return _ProfileConfig( - chunk_c2s=max(1, _env_int("PROFILE_CHUNK_C2S", 500)), - iat_c2s_ms=_env_float("PROFILE_IAT_C2S_MS", 0.0), - bytes_c2s=_env_int("PROFILE_BYTES_C2S", 10_485_760), - duration_s=_env_float("PROFILE_DURATION_S", 60.0), - bursty=_env_int("PROFILE_BURSTY", 0) != 0, - burst_count=max(1, _env_int("PROFILE_BURST_COUNT", 1)), - burst_idle_s=_env_float("PROFILE_BURST_IDLE_S", 0.0), - inter_batch_delay_ms=_env_float("INTER_PACKET_DELAY_MS", 40.0), - batch_size=max(1, _env_int("DELAY_EVERY_N", 10)), - ) - - -# Fill the chunk buffer with random bytes once so subsequent slices look like -# compressed/encrypted application data; sending all-zero payloads would let a -# passive observer split flows by trivial byte-entropy alone. -def _build_chunk(chunk_c2s: int) -> bytes: - return urandom(chunk_c2s) - - -# ── Sync API ───────────────────────────────────────────────────────────────── - - -def run_profile(send_fn: Callable[[bytes], None]) -> tuple[int, float]: - """Execute the c2s portion of the active profile using *send_fn*. - - Returns (bytes_sent, total_sleep_s). - """ - cfg = _read_config() - if cfg.bytes_c2s <= 0: - return 0, 0.0 - - start = monotonic() - deadline = start + cfg.duration_s - chunk = _build_chunk(cfg.chunk_c2s) - delay_s = max(0.0, cfg.iat_c2s_ms) / 1000.0 - batch_delay_s = max(0.0, cfg.inter_batch_delay_ms) / 1000.0 - sent = 0 - total_sleep = 0.0 - - if cfg.bursty and cfg.burst_count > 1: - bytes_per_burst = cfg.bytes_c2s // cfg.burst_count - for i in range(cfg.burst_count): - target = sent + bytes_per_burst - sent, total_sleep = _send_until(send_fn, chunk, sent, target, delay_s, deadline, total_sleep, cfg.batch_size, batch_delay_s) - if sent >= cfg.bytes_c2s or monotonic() >= deadline: - break - if i + 1 < cfg.burst_count and cfg.burst_idle_s > 0: - sleep(cfg.burst_idle_s) - total_sleep += cfg.burst_idle_s - else: - sent, total_sleep = _send_until(send_fn, chunk, sent, cfg.bytes_c2s, delay_s, deadline, total_sleep, cfg.batch_size, batch_delay_s) - - return sent, total_sleep - - -def _send_until( - send_fn: Callable[[bytes], None], - chunk: bytes, - sent: int, - target: int, - delay_s: float, - deadline: float, - total_sleep: float, - batch_size: int, - batch_delay_s: float, -) -> tuple[int, float]: - """Drive `send_fn` toward *target* bytes. - - Pacing layers (applied independently): - * `delay_s` β€” per-packet inter-arrival time - (`PROFILE_IAT_C2S_MS`). - * `batch_delay_s` β€” extra sleep every `batch_size` packets - (`INTER_PACKET_DELAY_MS` / `DELAY_EVERY_N`), - the receiver-safe rate cap. - """ - chunk_size = len(chunk) - packets_in_batch = 0 - while sent < target and monotonic() < deadline: - n = min(chunk_size, target - sent) - send_fn(chunk[:n]) - sent += n - packets_in_batch += 1 - if delay_s > 0.0: - sleep(delay_s) - total_sleep += delay_s - if batch_delay_s > 0.0 and packets_in_batch >= batch_size: - sleep(batch_delay_s) - total_sleep += batch_delay_s - packets_in_batch = 0 - return sent, total_sleep - - -# ── Async API ──────────────────────────────────────────────────────────────── - - -async def run_profile_async( - send_fn: Callable[[bytes], Awaitable[None] | None], -) -> tuple[int, float]: - """Async version of `run_profile` for asyncio-based senders (QUIC). - - `send_fn` may be sync (returns `None`) or async (returns an awaitable); - if the call returns an awaitable, the loop awaits it before pacing the - next packet. Returns (bytes_sent, total_sleep_s). - """ - cfg = _read_config() - if cfg.bytes_c2s <= 0: - return 0, 0.0 - - start = monotonic() - deadline = start + cfg.duration_s - chunk = _build_chunk(cfg.chunk_c2s) - delay_s = max(0.0, cfg.iat_c2s_ms) / 1000.0 - batch_delay_s = max(0.0, cfg.inter_batch_delay_ms) / 1000.0 - sent = 0 - total_sleep = 0.0 - - if cfg.bursty and cfg.burst_count > 1: - bytes_per_burst = cfg.bytes_c2s // cfg.burst_count - for i in range(cfg.burst_count): - target = sent + bytes_per_burst - sent, total_sleep = await _send_until_async(send_fn, chunk, sent, target, delay_s, deadline, total_sleep, cfg.batch_size, batch_delay_s) - if sent >= cfg.bytes_c2s or monotonic() >= deadline: - break - if i + 1 < cfg.burst_count and cfg.burst_idle_s > 0: - await asleep(cfg.burst_idle_s) - total_sleep += cfg.burst_idle_s - else: - sent, total_sleep = await _send_until_async(send_fn, chunk, sent, cfg.bytes_c2s, delay_s, deadline, total_sleep, cfg.batch_size, batch_delay_s) - - return sent, total_sleep - - -async def _send_until_async( - send_fn: Callable[[bytes], Awaitable[None] | None], - chunk: bytes, - sent: int, - target: int, - delay_s: float, - deadline: float, - total_sleep: float, - batch_size: int, - batch_delay_s: float, -) -> tuple[int, float]: - """Async mirror of `_send_until`. See that function for the pacing model.""" - chunk_size = len(chunk) - packets_in_batch = 0 - while sent < target and monotonic() < deadline: - n = min(chunk_size, target - sent) - result = send_fn(chunk[:n]) - if isawaitable(result): - await result - sent += n - packets_in_batch += 1 - if delay_s > 0.0: - await asleep(delay_s) - total_sleep += delay_s - if batch_delay_s > 0.0 and packets_in_batch >= batch_size: - await asleep(batch_delay_s) - total_sleep += batch_delay_s - packets_in_batch = 0 - return sent, total_sleep diff --git a/evaluation/protocols/common/socks5_sender.py b/evaluation/protocols/common/socks5_sender.py deleted file mode 100644 index 788710bd..00000000 --- a/evaluation/protocols/common/socks5_sender.py +++ /dev/null @@ -1,75 +0,0 @@ -#!/usr/bin/env python3 -""" -SOCKS5 sender β€” connects to SERVER_HOST through a local SOCKS5 proxy, runs the -c2s portion of the active TRAFFIC_PROFILE, exits 0. - -For standard proxies (Shadowsocks, Tor, VLESS): no auth. -For obfs4proxy PT SOCKS5: set SOCKS5_USERNAME to the PT arg string, e.g. - cert=;iat-mode=0 - -Env vars: - SERVER_HOST final destination IP (required) - SERVER_PORT final destination port (default 9000) - OBSERVER_GW gateway for route add - FORWARD_SUBNET subnet to route (default 172.21.0.0/24) - SOCKS5_HOST proxy host (default 127.0.0.1) - SOCKS5_PORT proxy port (default 1080) - SOCKS5_USERNAME PT args for obfs4 auth; omit for standard SOCKS5 - CONNECT_RETRIES attempts before giving up (default 30) - TRAFFIC_PROFILE profile name (informational) - PROFILE_CHUNK_C2S, PROFILE_IAT_C2S_MS, PROFILE_BYTES_C2S, PROFILE_DURATION_S, - PROFILE_BURSTY, PROFILE_BURST_COUNT, PROFILE_BURST_IDLE_S -""" - -from os import environ -from socket import SHUT_WR -from subprocess import run -from sys import exit -from time import monotonic, sleep - -from _profile import run_profile -from socks import SOCKS5, socksocket # PySocks - -observer_gw = environ.get("OBSERVER_GW") -forward_subnet = environ.get("FORWARD_SUBNET", "172.21.0.0/24") -server_host = environ["SERVER_HOST"] -server_port = int(environ.get("SERVER_PORT", 9000)) -socks5_host = environ.get("SOCKS5_HOST", "127.0.0.1") -socks5_port = int(environ.get("SOCKS5_PORT", 1080)) -socks5_user = environ.get("SOCKS5_USERNAME") -retries = int(environ.get("CONNECT_RETRIES", 30)) - -if observer_gw: - run(["ip", "route", "add", forward_subnet, "via", observer_gw], check=False, capture_output=True) - -for attempt in range(retries): - try: - s = socksocket() - if socks5_user: - s.set_proxy(SOCKS5, socks5_host, socks5_port, username=socks5_user, password="\x00") - else: - s.set_proxy(SOCKS5, socks5_host, socks5_port) - s.settimeout(10) - s.connect((server_host, server_port)) - s.settimeout(None) - transfer_start = monotonic() - sent, total_sleep = run_profile(s.sendall) - transfer_time_s = monotonic() - transfer_start - total_sleep - try: - s.shutdown(SHUT_WR) - s.settimeout(120) - while s.recv(65536): - pass - except OSError: - pass - s.close() - print(f"sent {sent} bytes via SOCKS5", flush=True) - print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - exit(0) - except Exception as exc: - print(f"attempt {attempt + 1}/{retries}: {exc}", flush=True) - if attempt < retries - 1: - sleep(2) - -print("failed to send via SOCKS5", flush=True) -exit(1) diff --git a/evaluation/protocols/common/tcp_sender.py b/evaluation/protocols/common/tcp_sender.py deleted file mode 100644 index 31039389..00000000 --- a/evaluation/protocols/common/tcp_sender.py +++ /dev/null @@ -1,57 +0,0 @@ -#!/usr/bin/env python3 -""" -TCP sender β€” connects to SERVER_HOST:LISTEN_PORT, runs the c2s portion of the -active TRAFFIC_PROFILE, exits 0. - -Env vars: - SERVER_HOST destination IP or hostname (required) - OBSERVER_GW gateway IP for the route add - FORWARD_SUBNET subnet to route (default 172.21.0.0/24) - LISTEN_PORT destination port (default 9000) - CONNECT_RETRIES times to retry on connection refused (default 30) - TRAFFIC_PROFILE profile name (informational) - PROFILE_CHUNK_C2S, PROFILE_IAT_C2S_MS, PROFILE_BYTES_C2S, PROFILE_DURATION_S, - PROFILE_BURSTY, PROFILE_BURST_COUNT, PROFILE_BURST_IDLE_S -""" - -from os import environ -from socket import SHUT_WR, create_connection -from subprocess import run -from sys import exit -from time import monotonic, sleep - -from _profile import run_profile - -observer_gw = environ.get("OBSERVER_GW") -forward_subnet = environ.get("FORWARD_SUBNET", "172.21.0.0/24") -server_host = environ["SERVER_HOST"] -port = int(environ.get("LISTEN_PORT", 9000)) -retries = int(environ.get("CONNECT_RETRIES", 30)) - -if observer_gw: - run(["ip", "route", "add", forward_subnet, "via", observer_gw], check=False, capture_output=True) - -for attempt in range(retries): - try: - with create_connection((server_host, port), timeout=5) as s: - s.settimeout(None) - transfer_start = monotonic() - sent, total_sleep = run_profile(s.sendall) - transfer_time_s = monotonic() - transfer_start - total_sleep - try: - s.shutdown(SHUT_WR) - s.settimeout(120) - while s.recv(65536): - pass - except OSError: - pass - - print(f"sent {sent} bytes", flush=True) - print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - exit(0) - except (ConnectionRefusedError, OSError): - if attempt < retries - 1: - sleep(1) - -print("failed to connect after retries", flush=True) -exit(1) diff --git a/evaluation/protocols/common/tcp_sink.py b/evaluation/protocols/common/tcp_sink.py deleted file mode 100644 index 07c6c82e..00000000 --- a/evaluation/protocols/common/tcp_sink.py +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env python3 -""" -TCP sink β€” accepts one connection, receives PROFILE_BYTES_C2S, exits 0. - -Env vars: - OBSERVER_GW gateway IP to route the opposite /24 subnet through - RETURN_SUBNET subnet to add a route to (default 172.20.0.0/24) - PROFILE_BYTES_C2S bytes to receive before exiting (default 100 MB) - LISTEN_PORT TCP port to bind (default 9000) -""" - -from contextlib import suppress -from os import environ -from signal import SIGTERM, signal -from socket import AF_INET, SO_REUSEADDR, SOCK_STREAM, SOL_SOCKET, socket -from subprocess import run -from sys import exit -from time import monotonic -from types import FrameType -from typing import NoReturn - -observer_gw = environ.get("OBSERVER_GW") -return_subnet = environ.get("RETURN_SUBNET", "172.20.0.0/24") -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -port = int(environ.get("LISTEN_PORT", 9000)) - -if observer_gw: - run(["ip", "route", "add", return_subnet, "via", observer_gw], check=False, capture_output=True) - -idle_timeout = int(environ.get("IDLE_TIMEOUT_S", 120)) - -received = 0 - - -def _sigterm(signum: int, frame: FrameType | None) -> NoReturn: - pct = received / transfer_bytes * 100 - print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) - exit(0) - - -signal(SIGTERM, _sigterm) - -srv = socket(AF_INET, SOCK_STREAM) -srv.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) -srv.bind(("0.0.0.0", port)) -srv.listen(1) -print(f"TCP sink ready on :{port}", flush=True) - -conn, _ = srv.accept() -conn.settimeout(idle_timeout) - -first_byte_time = None -last_byte_time = None -with suppress(TimeoutError, OSError): - while received < transfer_bytes: - data = conn.recv(65536) - if not data: - break - if first_byte_time is None: - first_byte_time = monotonic() - received += len(data) - last_byte_time = monotonic() -with suppress(OSError): - conn.close() - -pct = received / transfer_bytes * 100 -print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) -if first_byte_time is not None and last_byte_time is not None: - print(f"recv_time_s={last_byte_time - first_byte_time:.3f}", flush=True) -exit(0) diff --git a/evaluation/protocols/common/udp_sender.py b/evaluation/protocols/common/udp_sender.py deleted file mode 100644 index 7d5a7dda..00000000 --- a/evaluation/protocols/common/udp_sender.py +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env python3 -""" -UDP sender β€” sends the c2s portion of the active TRAFFIC_PROFILE to SERVER_HOST:9000, exits 0. - -Env vars: - SERVER_HOST destination IP or hostname (required) - TRAFFIC_PROFILE profile name (informational) - PROFILE_CHUNK_C2S, PROFILE_IAT_C2S_MS, PROFILE_BYTES_C2S, PROFILE_DURATION_S, - PROFILE_BURSTY, PROFILE_BURST_COUNT, PROFILE_BURST_IDLE_S -""" - -from os import environ -from signal import SIGTERM, signal -from socket import AF_INET, SOCK_DGRAM, socket -from sys import exit -from time import monotonic, sleep - -from _profile import run_profile - -signal(SIGTERM, lambda *_: exit(0)) - -server_host = environ["SERVER_HOST"] -port = 9000 - -sock = socket(AF_INET, SOCK_DGRAM) -sock.connect((server_host, port)) - -# Wait 200 ms so the server socket is bound before the first packet arrives. -sleep(0.2) - -transfer_start = monotonic() -sent, total_sleep = run_profile(sock.send) -transfer_time_s = monotonic() - transfer_start - total_sleep - -print(f"sent {sent} bytes", flush=True) -print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - -sock.send(b"DONE") -sleep(0.5) -exit(0) diff --git a/evaluation/protocols/common/udp_sink.py b/evaluation/protocols/common/udp_sink.py deleted file mode 100644 index 0107c689..00000000 --- a/evaluation/protocols/common/udp_sink.py +++ /dev/null @@ -1,53 +0,0 @@ -#!/usr/bin/env python3 -from os import environ -from signal import SIGTERM, signal -from socket import AF_INET, SOCK_DGRAM, socket -from sys import exit -from time import monotonic -from types import FrameType -from typing import NoReturn - -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -initial_timeout = int(environ.get("INITIAL_TIMEOUT_S", 60)) -idle_timeout = int(environ.get("IDLE_TIMEOUT_S", 30)) -port = 9000 - -received = 0 - - -def _sigterm(signum: int, frame: FrameType | None) -> NoReturn: - pct = received / transfer_bytes * 100 - print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) - exit(0) - - -signal(SIGTERM, _sigterm) - -sock = socket(AF_INET, SOCK_DGRAM) -sock.bind(("0.0.0.0", port)) -sock.settimeout(initial_timeout) -print(f"UDP sink ready on :{port}", flush=True) - -first_byte_time = None -last_byte_time = None -first = True -while received < transfer_bytes: - try: - data, _ = sock.recvfrom(65536) - except TimeoutError: - break - if first: - first = False - sock.settimeout(idle_timeout) - if data == b"DONE": - break - if first_byte_time is None: - first_byte_time = monotonic() - received += len(data) - last_byte_time = monotonic() - -pct = received / transfer_bytes * 100 -print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) -if first_byte_time is not None and last_byte_time is not None: - print(f"recv_time_s={last_byte_time - first_byte_time:.3f}", flush=True) -exit(0) diff --git a/evaluation/protocols/hysteria2/Dockerfile.client b/evaluation/protocols/hysteria2/Dockerfile.client index 1b5430c3..adeaa4ec 100644 --- a/evaluation/protocols/hysteria2/Dockerfile.client +++ b/evaluation/protocols/hysteria2/Dockerfile.client @@ -1,14 +1,12 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - curl ca-certificates iproute2 python3 python3-pip \ + curl ca-certificates iproute2 \ && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir --break-system-packages PySocks \ && curl -fsSL \ https://github.com/apernet/hysteria/releases/download/app%2Fv2.6.1/hysteria-linux-amd64 \ -o /usr/local/bin/hysteria \ && chmod +x /usr/local/bin/hysteria COPY hysteria2/client-entrypoint.sh /entrypoint.sh -COPY common/socks5_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/socks5-sender /usr/local/bin/socks5-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/hysteria2/Dockerfile.server b/evaluation/protocols/hysteria2/Dockerfile.server index de889336..280ff3c2 100644 --- a/evaluation/protocols/hysteria2/Dockerfile.server +++ b/evaluation/protocols/hysteria2/Dockerfile.server @@ -1,13 +1,13 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - curl ca-certificates iproute2 python3 openssl \ + curl ca-certificates iproute2 openssl \ && rm -rf /var/lib/apt/lists/* \ && curl -fsSL \ https://github.com/apernet/hysteria/releases/download/app%2Fv2.6.1/hysteria-linux-amd64 \ -o /usr/local/bin/hysteria \ && chmod +x /usr/local/bin/hysteria COPY hysteria2/server-entrypoint.sh /entrypoint.sh -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/tcp-sink /usr/local/bin/tcp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=2s --timeout=5s --retries=30 --start-period=15s \ CMD ss -ulnp | grep -q ':443' && test -f /keys/hysteria_ready diff --git a/evaluation/protocols/hysteria2/client-entrypoint.sh b/evaluation/protocols/hysteria2/client-entrypoint.sh index 57e7603b..3801c11b 100755 --- a/evaluation/protocols/hysteria2/client-entrypoint.sh +++ b/evaluation/protocols/hysteria2/client-entrypoint.sh @@ -34,6 +34,6 @@ done SERVER_HOST=127.0.0.1 \ SERVER_PORT=9000 \ OBSERVER_GW="" \ -python3 /app/client.py +socks5-sender kill "${CLIENT_PID}" 2>/dev/null || true diff --git a/evaluation/protocols/hysteria2/server-entrypoint.sh b/evaluation/protocols/hysteria2/server-entrypoint.sh index 72417e38..c08a42ae 100755 --- a/evaluation/protocols/hysteria2/server-entrypoint.sh +++ b/evaluation/protocols/hysteria2/server-entrypoint.sh @@ -30,7 +30,7 @@ EOF # tcp_sink receives the data proxied through Hysteria2 PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +tcp-sink & SINK_PID=$! hysteria server -c /tmp/server.yaml & diff --git a/evaluation/protocols/obfs4/Dockerfile.client b/evaluation/protocols/obfs4/Dockerfile.client index 654bcb23..eebf14b6 100644 --- a/evaluation/protocols/obfs4/Dockerfile.client +++ b/evaluation/protocols/obfs4/Dockerfile.client @@ -1,10 +1,8 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - obfs4proxy iproute2 python3 python3-pip \ - && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir --break-system-packages PySocks + obfs4proxy iproute2 \ + && rm -rf /var/lib/apt/lists/* COPY obfs4/client-entrypoint.sh /entrypoint.sh -COPY common/socks5_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/socks5-sender /usr/local/bin/socks5-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/obfs4/Dockerfile.server b/evaluation/protocols/obfs4/Dockerfile.server index d550bdce..04aca536 100644 --- a/evaluation/protocols/obfs4/Dockerfile.server +++ b/evaluation/protocols/obfs4/Dockerfile.server @@ -1,9 +1,9 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - obfs4proxy iproute2 python3 \ + obfs4proxy iproute2 \ && rm -rf /var/lib/apt/lists/* COPY obfs4/server-entrypoint.sh /entrypoint.sh -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/tcp-sink /usr/local/bin/tcp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=3s --retries=30 --start-period=5s \ CMD ss -tln | grep -q ':9000' && test -f /keys/obfs4_args.txt diff --git a/evaluation/protocols/obfs4/client-entrypoint.sh b/evaluation/protocols/obfs4/client-entrypoint.sh index a50834fa..7cfd15aa 100644 --- a/evaluation/protocols/obfs4/client-entrypoint.sh +++ b/evaluation/protocols/obfs4/client-entrypoint.sh @@ -51,6 +51,6 @@ SOCKS5_USERNAME="${SOCKS5_USERNAME}" \ SERVER_HOST="${SERVER_HOST:-172.21.0.10}" \ SERVER_PORT=9000 \ OBSERVER_GW="" \ -python3 /app/client.py +socks5-sender kill "${PT_PID}" 2>/dev/null || true diff --git a/evaluation/protocols/obfs4/server-entrypoint.sh b/evaluation/protocols/obfs4/server-entrypoint.sh index 8d39545b..ba855849 100644 --- a/evaluation/protocols/obfs4/server-entrypoint.sh +++ b/evaluation/protocols/obfs4/server-entrypoint.sh @@ -8,7 +8,7 @@ ip route add 172.20.0.0/24 via "${OBSERVER_GW}" || true PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ LISTEN_PORT=9001 \ -python3 /app/server.py & +tcp-sink & SINK_PID=$! mkdir -p /state diff --git a/evaluation/protocols/openvpn/Dockerfile.client b/evaluation/protocols/openvpn/Dockerfile.client index c635ce68..8d7d9a4b 100644 --- a/evaluation/protocols/openvpn/Dockerfile.client +++ b/evaluation/protocols/openvpn/Dockerfile.client @@ -1,9 +1,8 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - openvpn iproute2 python3 \ + openvpn iproute2 \ && rm -rf /var/lib/apt/lists/* COPY openvpn/client-entrypoint.sh /entrypoint.sh -COPY common/udp_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/udp-sender /usr/local/bin/udp-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/openvpn/Dockerfile.server b/evaluation/protocols/openvpn/Dockerfile.server index 8ee52658..a3550bde 100644 --- a/evaluation/protocols/openvpn/Dockerfile.server +++ b/evaluation/protocols/openvpn/Dockerfile.server @@ -1,9 +1,9 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - openvpn openssl iproute2 python3 \ + openvpn openssl iproute2 \ && rm -rf /var/lib/apt/lists/* COPY openvpn/server-entrypoint.sh /entrypoint.sh -COPY common/udp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/udp-sink /usr/local/bin/udp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=5s \ CMD ip link show tun0 >/dev/null 2>&1 && ss -uln | grep -q ':9000' diff --git a/evaluation/protocols/openvpn/client-entrypoint.sh b/evaluation/protocols/openvpn/client-entrypoint.sh index a75a9903..86791778 100644 --- a/evaluation/protocols/openvpn/client-entrypoint.sh +++ b/evaluation/protocols/openvpn/client-entrypoint.sh @@ -41,6 +41,6 @@ done SERVER_HOST=10.200.0.1 \ SERVER_PORT=9000 \ OBSERVER_GW="" \ -python3 /app/client.py +udp-sender kill "${OVP_PID}" 2>/dev/null || true diff --git a/evaluation/protocols/openvpn/server-entrypoint.sh b/evaluation/protocols/openvpn/server-entrypoint.sh index 4ac9cd16..8e14d4fb 100644 --- a/evaluation/protocols/openvpn/server-entrypoint.sh +++ b/evaluation/protocols/openvpn/server-entrypoint.sh @@ -50,7 +50,7 @@ EOF PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +udp-sink & SINK_PID=$! openvpn --config /tmp/server.conf & diff --git a/evaluation/protocols/quic/Dockerfile.client b/evaluation/protocols/quic/Dockerfile.client index 8f1e78d5..eac6c1a7 100644 --- a/evaluation/protocols/quic/Dockerfile.client +++ b/evaluation/protocols/quic/Dockerfile.client @@ -1,8 +1,16 @@ -FROM python:3.12-slim +## Stage 1: build the native Rust QUIC binary (quinn + BBR) +## Context is ../protocols, so the crate lives at quic/quic-eval. +FROM rust:1-bookworm AS build +WORKDIR /build/quic-eval +COPY quic/quic-eval/ ./ +RUN cargo build --release + +## Stage 2: runtime image +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ - && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir aioquic -COPY quic/client.py /app/client.py -COPY common/_profile.py /app/_profile.py -CMD ["python3", "/app/client.py"] + && rm -rf /var/lib/apt/lists/* + +COPY --from=build /build/quic-eval/target/release/quic-eval /usr/local/bin/quic-eval + +CMD ["quic-eval", "client"] diff --git a/evaluation/protocols/quic/Dockerfile.server b/evaluation/protocols/quic/Dockerfile.server index 74f0baf3..08327843 100644 --- a/evaluation/protocols/quic/Dockerfile.server +++ b/evaluation/protocols/quic/Dockerfile.server @@ -1,10 +1,19 @@ -FROM python:3.12-slim +## Stage 1: build the native Rust QUIC binary (quinn + BBR) +FROM rust:1-bookworm AS build +WORKDIR /build/quic-eval +COPY quic-eval/ ./ +RUN cargo build --release + +## Stage 2: runtime image +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 openssl \ - && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir aioquic -COPY server.py /app/server.py -# UDP port 9000 bound AND cert file present + iproute2 \ + && rm -rf /var/lib/apt/lists/* + +COPY --from=build /build/quic-eval/target/release/quic-eval /usr/local/bin/quic-eval + +# UDP port 9000 bound AND self-signed cert written (client readiness gate) HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=5s \ CMD ss -u -l -n | grep -q ':9000' && test -f /keys/quic_cert.pem -CMD ["python3", "/app/server.py"] + +CMD ["quic-eval", "server"] diff --git a/evaluation/protocols/quic/client.py b/evaluation/protocols/quic/client.py deleted file mode 100644 index 8cbeb431..00000000 --- a/evaluation/protocols/quic/client.py +++ /dev/null @@ -1,90 +0,0 @@ -#!/usr/bin/env python3 -""" -QUIC sender β€” runs the c2s portion of the active TRAFFIC_PROFILE over a single -QUIC stream. Uses `_profile.run_profile_async` so PROFILE_DURATION_S, batch -pacing, and IAT pacing are honoured uniformly with the other senders without -blocking the aioquic event loop on a sync `time.sleep`. -""" - -from asyncio import TimeoutError, run, wait_for -from os import environ, path -from ssl import CERT_NONE -from subprocess import run as subprocess_run -from sys import exit -from time import monotonic, sleep -from traceback import print_exc - -from _profile import run_profile_async -from aioquic.asyncio import connect -from aioquic.asyncio.protocol import QuicConnectionProtocol -from aioquic.quic.configuration import QuicConfiguration - -observer_gw = environ.get("OBSERVER_GW") -server_host = environ["SERVER_HOST"] -PORT = 9000 -wait_timeout = int(environ.get("QUIC_WAIT_TIMEOUT_S", 240)) - -if observer_gw: - subprocess_run( - ["ip", "route", "add", "172.21.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -for _ in range(30): - if path.exists("/keys/quic_cert.pem"): - break - sleep(1) -else: - print("quic_cert.pem never appeared", flush=True) - exit(1) - - -async def main() -> None: - config = QuicConfiguration(is_client=True, alpn_protocols=["eval"]) - config.verify_mode = CERT_NONE - config.server_name = "quic-eval" - config.idle_timeout = 300.0 - config.max_stream_data = 128 * 1024 * 1024 - config.max_data = 256 * 1024 * 1024 - config.congestion_control_algorithm = "cubic" - - print("Connecting...", flush=True) - async with connect( - server_host, - PORT, - configuration=config, - create_protocol=QuicConnectionProtocol, - ) as proto: - print("Connected, sending data...", flush=True) - stream_id = proto._quic.get_next_available_stream_id() - - def send_chunk(data: bytes) -> None: - """Enqueue *data* on the eval stream and trigger transmission.""" - proto._quic.send_stream_data(stream_id, data, end_stream=False) - proto.transmit() - - transfer_start = monotonic() - sent_bytes, total_sleep = await run_profile_async(send_chunk) - transfer_time_s = monotonic() - transfer_start - total_sleep - - # Close the stream cleanly; aioquic requires a final send to flip FIN. - proto._quic.send_stream_data(stream_id, b"", end_stream=True) - proto.transmit() - - print(f"All {sent_bytes} bytes enqueued, waiting for server close...", flush=True) - try: - await wait_for(proto.wait_closed(), timeout=wait_timeout) - except TimeoutError: - print(f"wait_closed timed out after {wait_timeout}s", flush=True) - - print(f"sent {sent_bytes} bytes via QUIC", flush=True) - print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - - -try: - run(main()) -except Exception: - print_exc() - exit(1) -exit(0) diff --git a/evaluation/protocols/quic/quic-eval/Cargo.toml b/evaluation/protocols/quic/quic-eval/Cargo.toml new file mode 100644 index 00000000..f4288bf2 --- /dev/null +++ b/evaluation/protocols/quic/quic-eval/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "quic-eval" +version = "0.1.0" +edition = "2021" + +# Native Rust QUIC sender/receiver for the protocol comparison, using quinn with +# its BBR congestion controller β€” the loss-tolerant CC that quic-go lacks, so +# QUIC no longer collapses under chaos (2% loss + 100 ms RTT). + +[dependencies] +quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring", "log"] } +rustls = { version = "0.23", default-features = false, features = ["ring", "std"] } +rcgen = { version = "0.13", default-features = false, features = ["ring", "pem"] } +tokio = { version = "1", features = ["rt-multi-thread", "macros", "time", "io-util"] } +rand = "0.8" +libc = "0.2" + +[[bin]] +name = "quic-eval" +path = "src/main.rs" diff --git a/evaluation/protocols/quic/quic-eval/src/main.rs b/evaluation/protocols/quic/quic-eval/src/main.rs new file mode 100644 index 00000000..3033c364 --- /dev/null +++ b/evaluation/protocols/quic/quic-eval/src/main.rs @@ -0,0 +1,309 @@ +//! quic-eval β€” native Rust QUIC sender/receiver for the TYPHOON protocol +//! comparison, using `quinn` with its **BBR** congestion controller. +//! +//! History: the Python (aioquic) client stalled under chaos; the Go (quic-go) +//! rewrite completed but collapsed to ~300 s under 2% loss because quic-go has +//! no BBR β€” only NewReno. quinn ships a BBR controller, which is loss-tolerant, +//! so this build stays fast on lossy links. +//! +//! One binary, two roles selected by argv[1] (`client` | `server`). The server +//! writes a self-signed cert to /keys/quic_cert.pem (client readiness gate) and +//! echoes each probe on the client-opened bidi stream; the client pings probes +//! and prints the `rtt_*` / delivery contract the harness parses (see `lat`). + +use std::env::var; +use std::error::Error; +use std::fs::write; +use std::net::{Ipv4Addr, SocketAddr}; +use std::path::Path; +use std::process::exit; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +use quinn::congestion::BbrConfig; +use quinn::crypto::rustls::QuicClientConfig; +use quinn::{ClientConfig, Endpoint, ServerConfig, TransportConfig}; +use rustls::pki_types::{CertificateDer, PrivateKeyDer, PrivatePkcs8KeyDer, ServerName, UnixTime}; + +const PORT: u16 = 9000; +const CERT_PATH: &str = "/keys/quic_cert.pem"; + +type BoxErr = Box; + +#[tokio::main] +async fn main() { + // rustls 0.23 requires a process-wide crypto provider before any config build. + let _ = rustls::crypto::ring::default_provider().install_default(); + + let role = std::env::args().nth(1).unwrap_or_default(); + let result = match role.as_str() { + "server" => run_server().await, + "client" => run_client().await, + other => { + eprintln!("usage: quic-eval (got {other:?})"); + exit(2); + } + }; + if let Err(e) = result { + eprintln!("{e}"); + exit(1); + } +} + +/// Host-wide monotonic clock in nanoseconds. Docker containers share the kernel +/// clock (no time namespace by default), so client `send_start/end` and server +/// `recv_first/last` are directly comparable β€” the cross-endpoint transfer base. +fn monotonic_ns() -> u128 { + let mut ts = libc::timespec { + tv_sec: 0, + tv_nsec: 0, + }; + unsafe { + libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts); + } + (ts.tv_sec as u128) * 1_000_000_000 + (ts.tv_nsec as u128) +} + +/// Best-effort forward route to the opposite /24 via the observer tap. +fn add_route(subnet: &str) { + if let Ok(gw) = var("OBSERVER_GW") { + if !gw.is_empty() { + let _ = std::process::Command::new("ip") + .args(["route", "add", subnet, "via", &gw]) + .status(); + } + } +} + +/// Shared transport config: BBR congestion control + generous windows/idle so a +/// single bulk stream is bounded by the link, not by flow control. +fn transport() -> Arc { + let mut t = TransportConfig::default(); + t.congestion_controller_factory(Arc::new(BbrConfig::default())); + t.max_idle_timeout(Some(Duration::from_secs(300).try_into().unwrap())); + t.stream_receive_window((128u32 << 20).into()); + t.receive_window((256u32 << 20).into()); + Arc::new(t) +} + +// ── Latency mode: per-packet round-trip ping over a bidi stream ────────────── +// A duplicate of the shared `eval-transport::latency` contract (the eval crates +// don't share a dependency). QUIC is reliable, so echoes always return and RTT +// grows under loss (like TCP) rather than dropping. +mod lat { + use std::env::var; + use std::time::Duration; + + pub const HEADER: usize = 20; // seq(4) + send_ns(16) + + pub fn count() -> u32 { + envu("LAT_COUNT", 500) + } + pub fn interval() -> Duration { + Duration::from_secs_f64(envf("LAT_INTERVAL_MS", 20.0) / 1000.0) + } + pub fn size() -> usize { + (envu("LAT_SIZE", 256) as usize).max(HEADER) + } + pub fn recv_timeout() -> Duration { + Duration::from_secs_f64(envf("LAT_RECV_TIMEOUT_MS", 5000.0) / 1000.0) + } + pub fn pack(seq: u32, send_ns: u128, size: usize) -> Vec { + let mut m = vec![0u8; size]; + m[0..4].copy_from_slice(&seq.to_be_bytes()); + m[4..HEADER].copy_from_slice(&send_ns.to_be_bytes()); + m + } + pub fn send_ns_of(msg: &[u8]) -> u128 { + let mut b = [0u8; 16]; + b.copy_from_slice(&msg[4..HEADER]); + u128::from_be_bytes(b) + } + pub fn report(rtts: &mut [f64], count: u32) { + println!("sent {count} packets"); + let delivery = rtts.len() as f64 / count.max(1) as f64 * 100.0; + println!("roundtrip_delivery_pct={delivery:.1}"); + if !rtts.is_empty() { + rtts.sort_by(|a, b| a.partial_cmp(b).unwrap()); + let pct = |p: f64| rtts[(((rtts.len() - 1) as f64) * p).round() as usize]; + let p50 = pct(0.50); + let p95 = pct(0.95); + println!("rtt_min_ms={:.3}", rtts[0]); + println!("rtt_p50_ms={p50:.3}"); + println!("rtt_p95_ms={p95:.3}"); + println!("rtt_p99_ms={:.3}", pct(0.99)); + println!("rtt_jitter_ms={:.3}", p95 - p50); + } + } + fn envu(k: &str, d: u32) -> u32 { + var(k).ok().and_then(|v| v.parse().ok()).unwrap_or(d) + } + fn envf(k: &str, d: f64) -> f64 { + var(k).ok().and_then(|v| v.parse().ok()).unwrap_or(d) + } +} + +// ── Server ─────────────────────────────────────────────────────────────────── + +async fn run_server() -> Result<(), BoxErr> { + add_route("172.20.0.0/24"); + + // Self-signed cert; write the PEM last so its presence gates the client. + let cert = rcgen::generate_simple_self_signed(vec!["quic-eval".to_string()])?; + let cert_der = cert.cert.der().clone(); + let key_der = PrivateKeyDer::Pkcs8(PrivatePkcs8KeyDer::from(cert.key_pair.serialize_der())); + write(CERT_PATH, cert.cert.pem())?; + + let mut server_config = ServerConfig::with_single_cert(vec![cert_der], key_der)?; + server_config.transport_config(transport()); + + let addr = SocketAddr::new(Ipv4Addr::UNSPECIFIED.into(), PORT); + let endpoint = Endpoint::server(server_config, addr)?; + println!("QUIC sink ready on :{PORT}"); + + let connecting = endpoint + .accept() + .await + .ok_or("endpoint closed before a connection arrived")?; + let connection = connecting.await?; + println!("connection accepted"); + + // Echo each probe back on the client-opened bidi stream (see `lat`). + let (mut send, mut recv) = connection.accept_bi().await?; + let cnt = lat::count(); + let mut buf = vec![0u8; lat::size()]; + let mut received = 0u32; + while received < cnt { + match recv.read_exact(&mut buf).await { + Ok(()) => { + send.write_all(&buf).await?; + received += 1; + } + Err(_) => break, + } + } + let pct = received as f64 / cnt.max(1) as f64 * 100.0; + println!("received {received}/{cnt} packets ({pct:.1}%)"); + connection.close(0u32.into(), b"done"); + endpoint.wait_idle().await; + Ok(()) +} + +// ── Client ─────────────────────────────────────────────────────────────────── + +async fn run_client() -> Result<(), BoxErr> { + add_route("172.21.0.0/24"); + let server_host = var("SERVER_HOST").map_err(|_| "SERVER_HOST not set")?; + + // Gate on the server's cert file, mirroring the prior implementations. + let mut ready = false; + for _ in 0..30 { + if Path::new(CERT_PATH).exists() { + ready = true; + break; + } + tokio::time::sleep(Duration::from_secs(1)).await; + } + if !ready { + return Err(format!("{CERT_PATH} never appeared").into()); + } + + let crypto = rustls::ClientConfig::builder() + .dangerous() + .with_custom_certificate_verifier(Arc::new(SkipServerVerification::new())) + .with_no_client_auth(); + let mut client_config = ClientConfig::new(Arc::new(QuicClientConfig::try_from(crypto)?)); + client_config.transport_config(transport()); + + let mut endpoint = Endpoint::client(SocketAddr::new(Ipv4Addr::UNSPECIFIED.into(), 0))?; + endpoint.set_default_client_config(client_config); + + let server_addr = SocketAddr::new(server_host.parse()?, PORT); + println!("Connecting..."); + let connection = endpoint.connect(server_addr, "quic-eval")?.await?; + println!("Connected, sending data..."); + + // Ping probes over a bidi stream, timing round-trips (see `lat`). + let (mut send, mut recv) = connection.open_bi().await?; + let cnt = lat::count(); + let interval = lat::interval(); + let rto = lat::recv_timeout(); + let mut rbuf = vec![0u8; lat::size()]; + let mut rtts: Vec = Vec::with_capacity(cnt as usize); + for seq in 0..cnt { + let send_ns = monotonic_ns(); + let msg = lat::pack(seq, send_ns, lat::size()); + let t0 = Instant::now(); + send.write_all(&msg).await?; + match tokio::time::timeout(rto, recv.read_exact(&mut rbuf)).await { + Ok(Ok(())) => { + let rtt = (monotonic_ns().saturating_sub(lat::send_ns_of(&rbuf))) as f64 / 1e6; + rtts.push(rtt); + } + _ => break, // reliable stream: timeout/err means the peer is gone + } + let el = t0.elapsed(); + if el < interval { + tokio::time::sleep(interval - el).await; + } + } + lat::report(&mut rtts, cnt); + connection.close(0u32.into(), b"done"); + endpoint.wait_idle().await; + Ok(()) +} +// ── TLS: accept the eval's self-signed cert (verification is out of scope) ───── + +#[derive(Debug)] +struct SkipServerVerification(Arc); + +impl SkipServerVerification { + fn new() -> Self { + Self(Arc::new(rustls::crypto::ring::default_provider())) + } +} + +impl rustls::client::danger::ServerCertVerifier for SkipServerVerification { + fn verify_server_cert( + &self, + _end_entity: &CertificateDer<'_>, + _intermediates: &[CertificateDer<'_>], + _server_name: &ServerName<'_>, + _ocsp: &[u8], + _now: UnixTime, + ) -> Result { + Ok(rustls::client::danger::ServerCertVerified::assertion()) + } + + fn verify_tls12_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &rustls::DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls12_signature( + message, + cert, + dss, + &self.0.signature_verification_algorithms, + ) + } + + fn verify_tls13_signature( + &self, + message: &[u8], + cert: &CertificateDer<'_>, + dss: &rustls::DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls13_signature( + message, + cert, + dss, + &self.0.signature_verification_algorithms, + ) + } + + fn supported_verify_schemes(&self) -> Vec { + self.0.signature_verification_algorithms.supported_schemes() + } +} diff --git a/evaluation/protocols/quic/server.py b/evaluation/protocols/quic/server.py deleted file mode 100644 index edb8ac4b..00000000 --- a/evaluation/protocols/quic/server.py +++ /dev/null @@ -1,94 +0,0 @@ -#!/usr/bin/env python3 -from asyncio import Event, run, sleep -from os import environ -from subprocess import run as subprocess_run -from sys import exit - -from aioquic.asyncio import serve -from aioquic.asyncio.protocol import QuicConnectionProtocol -from aioquic.quic.configuration import QuicConfiguration -from aioquic.quic.events import ConnectionTerminated, QuicEvent, StreamDataReceived - -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -observer_gw = environ.get("OBSERVER_GW") -PORT = 9000 - -if observer_gw: - subprocess_run( - ["ip", "route", "add", "172.20.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -subprocess_run( - [ - "openssl", - "req", - "-x509", - "-newkey", - "rsa:2048", - "-keyout", - "/tmp/quic_key.pem", - "-out", - "/keys/quic_cert.pem", - "-days", - "1", - "-nodes", - "-subj", - "/CN=quic-eval", - "-addext", - "subjectAltName=DNS:quic-eval", - ], - check=True, - capture_output=True, -) - - -class SinkProtocol(QuicConnectionProtocol): - def __init__(self, *args: object, **kwargs: object) -> None: - super().__init__(*args, **kwargs) - self._received = 0 - self._done = Event() - - def quic_event_received(self, event: QuicEvent) -> None: - if isinstance(event, StreamDataReceived): - self._received += len(event.data) - if event.end_stream or self._received >= transfer_bytes: - self._done.set() - elif isinstance(event, ConnectionTerminated): - print(f"connection terminated (received so far: {self._received})", flush=True) - self._done.set() - - async def wait_done(self) -> None: - await self._done.wait() - pct = self._received / transfer_bytes * 100 - print(f"received {self._received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) - - -async def main() -> None: - config = QuicConfiguration(is_client=False, alpn_protocols=["eval"]) - config.idle_timeout = 300.0 - config.max_stream_data = 128 * 1024 * 1024 - config.max_data = 256 * 1024 * 1024 - config.congestion_control_algorithm = "cubic" - config.load_cert_chain("/keys/quic_cert.pem", "/tmp/quic_key.pem") - - protocols: list[SinkProtocol] = [] - - def factory(*args: object, **kwargs: object) -> SinkProtocol: - p = SinkProtocol(*args, **kwargs) - protocols.append(p) - print(f"connection accepted (total: {len(protocols)})", flush=True) - return p - - print(f"QUIC sink ready on :{PORT}", flush=True) - server = await serve("0.0.0.0", PORT, configuration=config, create_protocol=factory) - - while not protocols: - await sleep(0.1) - await protocols[0].wait_done() - server.close() - - -run(main()) -exit(0) diff --git a/evaluation/protocols/raw_tcp/Dockerfile.client b/evaluation/protocols/raw_tcp/Dockerfile.client index d4c29267..8efae74e 100644 --- a/evaluation/protocols/raw_tcp/Dockerfile.client +++ b/evaluation/protocols/raw_tcp/Dockerfile.client @@ -1,7 +1,6 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY common/tcp_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py -CMD ["python3", "/app/client.py"] +COPY --from=transport /usr/local/bin/tcp-sender /usr/local/bin/tcp-sender +CMD ["tcp-sender"] diff --git a/evaluation/protocols/raw_tcp/Dockerfile.server b/evaluation/protocols/raw_tcp/Dockerfile.server index 459f5287..0ba6ccb4 100644 --- a/evaluation/protocols/raw_tcp/Dockerfile.server +++ b/evaluation/protocols/raw_tcp/Dockerfile.server @@ -1,8 +1,8 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/tcp-sink /usr/local/bin/tcp-sink HEALTHCHECK --interval=1s --timeout=3s --retries=15 --start-period=2s \ CMD ss -tln | grep -q ':9000' -CMD ["python3", "/app/server.py"] +CMD ["tcp-sink"] diff --git a/evaluation/protocols/raw_udp/Dockerfile.client b/evaluation/protocols/raw_udp/Dockerfile.client index 71981b3f..58269533 100644 --- a/evaluation/protocols/raw_udp/Dockerfile.client +++ b/evaluation/protocols/raw_udp/Dockerfile.client @@ -1,8 +1,8 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY client.py /app/client.py +COPY --from=transport /usr/local/bin/udp-sender /usr/local/bin/udp-sender COPY client-entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/raw_udp/Dockerfile.server b/evaluation/protocols/raw_udp/Dockerfile.server index dcce55c5..25165c16 100644 --- a/evaluation/protocols/raw_udp/Dockerfile.server +++ b/evaluation/protocols/raw_udp/Dockerfile.server @@ -1,11 +1,10 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY server.py /app/server.py +COPY --from=transport /usr/local/bin/udp-sink /usr/local/bin/udp-sink COPY server-entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh -# Health check: UDP port 9000 appears in the bound-socket table HEALTHCHECK --interval=1s --timeout=3s --retries=15 --start-period=2s \ CMD ss -u -l -n | grep -q ':9000' ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/raw_udp/client-entrypoint.sh b/evaluation/protocols/raw_udp/client-entrypoint.sh index 812e1f03..4d8accdf 100644 --- a/evaluation/protocols/raw_udp/client-entrypoint.sh +++ b/evaluation/protocols/raw_udp/client-entrypoint.sh @@ -11,4 +11,4 @@ if [[ -n "${OBSERVER_GW}" ]]; then || echo "[client] WARNING: route add failed (already exists?)" fi -exec python3 /app/client.py +exec udp-sender diff --git a/evaluation/protocols/raw_udp/client.py b/evaluation/protocols/raw_udp/client.py deleted file mode 100644 index ec56f48c..00000000 --- a/evaluation/protocols/raw_udp/client.py +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env python3 -from os import environ -from socket import AF_INET, SOCK_DGRAM, socket -from subprocess import run -from sys import exit -from time import monotonic, sleep - -observer_gw = environ.get("OBSERVER_GW") -server_host = environ["SERVER_HOST"] -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -port = 9000 -chunk_size = 500 # small payload so padding protocols show distinct wire-size distributions - -delay_ms = float(environ.get("INTER_PACKET_DELAY_MS", 0)) -delay_every = int(environ.get("DELAY_EVERY_N", 1)) - -if observer_gw: - run( - ["ip", "route", "add", "172.21.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -sock = socket(AF_INET, SOCK_DGRAM) -sock.connect((server_host, port)) - -# Brief pause so the server socket is definitely bound before first packet. -sleep(0.2) - -chunk = bytes(chunk_size) -sent = 0 -packets = 0 -total_sleep = 0.0 -transfer_start = monotonic() -while sent < transfer_bytes: - n = min(chunk_size, transfer_bytes - sent) - sock.send(chunk[:n]) - sent += n - packets += 1 - if delay_ms > 0 and packets % delay_every == 0: - sleep(delay_ms / 1000) - total_sleep += delay_ms / 1000 -transfer_time_s = monotonic() - transfer_start - total_sleep - -print(f"sent {sent} bytes", flush=True) -print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - -sock.send(b"DONE") -exit(0) diff --git a/evaluation/protocols/raw_udp/server-entrypoint.sh b/evaluation/protocols/raw_udp/server-entrypoint.sh index 231b82ab..e4b8f02e 100644 --- a/evaluation/protocols/raw_udp/server-entrypoint.sh +++ b/evaluation/protocols/raw_udp/server-entrypoint.sh @@ -10,4 +10,4 @@ if [[ -n "${OBSERVER_GW}" ]]; then || echo "[server] WARNING: route add failed (already exists?)" fi -exec python3 /app/server.py +exec udp-sink diff --git a/evaluation/protocols/raw_udp/server.py b/evaluation/protocols/raw_udp/server.py deleted file mode 100644 index 01660071..00000000 --- a/evaluation/protocols/raw_udp/server.py +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env python3 -from os import environ -from socket import AF_INET, SOCK_DGRAM, socket -from subprocess import run -from sys import exit - -observer_gw = environ.get("OBSERVER_GW") -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -initial_timeout = int(environ.get("INITIAL_TIMEOUT_S", 60)) -idle_timeout = int(environ.get("IDLE_TIMEOUT_S", 30)) -port = 9000 - -if observer_gw: - run( - ["ip", "route", "add", "172.20.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -sock = socket(AF_INET, SOCK_DGRAM) -sock.bind(("0.0.0.0", port)) -sock.settimeout(initial_timeout) -print(f"UDP sink ready on :{port}", flush=True) - -received = 0 -first = True -while received < transfer_bytes: - try: - data, _ = sock.recvfrom(65536) - except TimeoutError: - break - if first: - first = False - sock.settimeout(idle_timeout) - if data == b"DONE": - break - received += len(data) - -pct = received / transfer_bytes * 100 -print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) -exit(0) diff --git a/evaluation/protocols/shadowsocks/Dockerfile.client b/evaluation/protocols/shadowsocks/Dockerfile.client index 21db33e8..3cc5bfd7 100644 --- a/evaluation/protocols/shadowsocks/Dockerfile.client +++ b/evaluation/protocols/shadowsocks/Dockerfile.client @@ -1,10 +1,8 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - shadowsocks-libev iproute2 python3 python3-pip \ - && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir --break-system-packages PySocks + shadowsocks-libev iproute2 \ + && rm -rf /var/lib/apt/lists/* COPY shadowsocks/client-entrypoint.sh /entrypoint.sh -COPY common/socks5_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/socks5-sender /usr/local/bin/socks5-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/shadowsocks/Dockerfile.server b/evaluation/protocols/shadowsocks/Dockerfile.server index 3ba94518..0a60060c 100644 --- a/evaluation/protocols/shadowsocks/Dockerfile.server +++ b/evaluation/protocols/shadowsocks/Dockerfile.server @@ -1,12 +1,12 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - shadowsocks-libev iproute2 python3 \ + shadowsocks-libev iproute2 \ && rm -rf /var/lib/apt/lists/* COPY shadowsocks/server.json /etc/shadowsocks-libev/config.json -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/tcp-sink /usr/local/bin/tcp-sink CMD sh -c 'ip route add 172.20.0.0/24 via "$OBSERVER_GW" 2>/dev/null || true; \ PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" OBSERVER_GW="" \ - python3 /app/server.py & SINK=$!; \ + tcp-sink & SINK=$!; \ ss-server -c /etc/shadowsocks-libev/config.json & SS=$!; \ wait $SINK' HEALTHCHECK --interval=1s --timeout=3s --retries=15 --start-period=3s \ diff --git a/evaluation/protocols/shadowsocks/client-entrypoint.sh b/evaluation/protocols/shadowsocks/client-entrypoint.sh index 3817fc14..32536644 100644 --- a/evaluation/protocols/shadowsocks/client-entrypoint.sh +++ b/evaluation/protocols/shadowsocks/client-entrypoint.sh @@ -20,6 +20,6 @@ done SERVER_HOST=127.0.0.1 \ SERVER_PORT=9000 \ OBSERVER_GW="" \ -python3 /app/client.py +socks5-sender kill "${SSLOCAL_PID}" 2>/dev/null || true diff --git a/evaluation/protocols/tls/Dockerfile.client b/evaluation/protocols/tls/Dockerfile.client index 02d5af53..19b098cf 100644 --- a/evaluation/protocols/tls/Dockerfile.client +++ b/evaluation/protocols/tls/Dockerfile.client @@ -1,7 +1,6 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY tls/client.py /app/client.py -COPY common/_profile.py /app/_profile.py -CMD ["python3", "/app/client.py"] +COPY --from=transport /usr/local/bin/tls-sender /usr/local/bin/tls-sender +CMD ["tls-sender"] diff --git a/evaluation/protocols/tls/Dockerfile.server b/evaluation/protocols/tls/Dockerfile.server index 2a0388cc..64ea518d 100644 --- a/evaluation/protocols/tls/Dockerfile.server +++ b/evaluation/protocols/tls/Dockerfile.server @@ -1,8 +1,8 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 openssl \ + iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY server.py /app/server.py +COPY --from=transport /usr/local/bin/tls-sink /usr/local/bin/tls-sink HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=3s \ CMD ss -tln | grep -q ':9000' && test -f /keys/tls_cert.pem -CMD ["python3", "/app/server.py"] +CMD ["tls-sink"] diff --git a/evaluation/protocols/tls/client.py b/evaluation/protocols/tls/client.py deleted file mode 100644 index 4fc6567d..00000000 --- a/evaluation/protocols/tls/client.py +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/env python3 -""" -TLS 1.3 sender β€” waits for /keys/tls_cert.pem from the server, then runs the -c2s portion of the active TRAFFIC_PROFILE over TLS using that cert as the -trusted CA. -""" - -from contextlib import suppress -from os import environ, path -from socket import SHUT_WR, create_connection -from ssl import PROTOCOL_TLS_CLIENT, SSLContext, SSLError, TLSVersion -from subprocess import run -from sys import exit -from time import monotonic, sleep - -from _profile import run_profile - -observer_gw = environ.get("OBSERVER_GW") -server_host = environ["SERVER_HOST"] -port = 9000 -retries = 30 - -if observer_gw: - run( - ["ip", "route", "add", "172.21.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -# Wait for server cert -for _ in range(retries): - if path.exists("/keys/tls_cert.pem"): - break - sleep(1) -else: - print("tls_cert.pem never appeared", flush=True) - exit(1) - -ctx = SSLContext(PROTOCOL_TLS_CLIENT) -ctx.minimum_version = TLSVersion.TLSv1_3 -ctx.load_verify_locations("/keys/tls_cert.pem") -ctx.check_hostname = False - -for _attempt in range(retries): - try: - raw = create_connection((server_host, port), timeout=5) - raw.settimeout(None) - tls = ctx.wrap_socket(raw, server_hostname="tls-eval") - transfer_start = monotonic() - sent, total_sleep = run_profile(tls.sendall) - transfer_time_s = monotonic() - transfer_start - total_sleep - try: - raw2 = tls.unwrap() - except (SSLError, OSError): - raw2 = raw - with suppress(OSError): - raw2.shutdown(SHUT_WR) - raw2.settimeout(120) - while raw2.recv(65536): - pass - with suppress(OSError): - raw2.close() - print(f"sent {sent} bytes", flush=True) - print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - exit(0) - except (ConnectionRefusedError, OSError, SSLError): - sleep(1) - -exit(1) diff --git a/evaluation/protocols/tls/server.py b/evaluation/protocols/tls/server.py deleted file mode 100644 index 4b59dc50..00000000 --- a/evaluation/protocols/tls/server.py +++ /dev/null @@ -1,77 +0,0 @@ -#!/usr/bin/env python3 -""" -TLS 1.3 sink β€” generates a self-signed cert, writes it to /keys/tls_cert.pem -so the client can trust it, then receives PROFILE_BYTES_C2S over TLS. -""" - -from contextlib import suppress -from os import environ -from socket import AF_INET, SO_REUSEADDR, SOCK_STREAM, SOL_SOCKET, socket -from ssl import PROTOCOL_TLS_SERVER, SSLContext, SSLError, TLSVersion -from subprocess import run -from sys import exit - -observer_gw = environ.get("OBSERVER_GW") -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -idle_timeout = int(environ.get("IDLE_TIMEOUT_S", 120)) -port = 9000 - -if observer_gw: - run( - ["ip", "route", "add", "172.20.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -run( - [ - "openssl", - "req", - "-x509", - "-newkey", - "rsa:2048", - "-keyout", - "/tmp/tls_key.pem", - "-out", - "/keys/tls_cert.pem", - "-days", - "1", - "-nodes", - "-subj", - "/CN=tls-eval", - ], - check=True, - capture_output=True, -) - -ctx = SSLContext(PROTOCOL_TLS_SERVER) -ctx.minimum_version = TLSVersion.TLSv1_3 -ctx.load_cert_chain("/keys/tls_cert.pem", "/tmp/tls_key.pem") - -raw = socket(AF_INET, SOCK_STREAM) -raw.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) -raw.bind(("0.0.0.0", port)) -raw.listen(1) -print(f"TLS sink ready on :{port}", flush=True) - -conn, _ = raw.accept() -try: - tls = ctx.wrap_socket(conn, server_side=True) -except SSLError as e: - print(f"TLS handshake failed: {e}", flush=True) - exit(1) - -tls.settimeout(idle_timeout) -received = 0 -with suppress(SSLError, OSError, TimeoutError): - while received < transfer_bytes: - data = tls.recv(65536) - if not data: - break - received += len(data) -with suppress(OSError): - tls.close() - -pct = received / transfer_bytes * 100 -print(f"received {received}/{transfer_bytes} bytes ({pct:.1f}%)", flush=True) -exit(0) diff --git a/evaluation/protocols/tor/Dockerfile.client b/evaluation/protocols/tor/Dockerfile.client index fb0c0652..c07dee25 100644 --- a/evaluation/protocols/tor/Dockerfile.client +++ b/evaluation/protocols/tor/Dockerfile.client @@ -1,7 +1,6 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY tor/client.py /app/client.py -COPY common/_profile.py /app/_profile.py -CMD ["python3", "/app/client.py"] +COPY --from=transport /usr/local/bin/tor-sender /usr/local/bin/tor-sender +CMD ["tor-sender"] diff --git a/evaluation/protocols/tor/Dockerfile.server b/evaluation/protocols/tor/Dockerfile.server index 372790bb..bd105421 100644 --- a/evaluation/protocols/tor/Dockerfile.server +++ b/evaluation/protocols/tor/Dockerfile.server @@ -1,10 +1,10 @@ -FROM python:3.12-slim +FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 netcat-openbsd openssl \ + iproute2 \ && rm -rf /var/lib/apt/lists/* -COPY server.py /app/server.py +COPY --from=transport /usr/local/bin/tor-sink /usr/local/bin/tor-sink # Health check: OR port listening (ss doesn't connect, so it won't trigger TLS) # AND cert written so the client can load it before connecting. HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=3s \ CMD ss -t -l -n | grep -q ':9001' && test -f /keys/tor_cert.pem -CMD ["python3", "/app/server.py"] +CMD ["tor-sink"] diff --git a/evaluation/protocols/tor/client.py b/evaluation/protocols/tor/client.py deleted file mode 100644 index b2bf6550..00000000 --- a/evaluation/protocols/tor/client.py +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env python3 -""" -Tor link-protocol-v4 sender β€” wraps the c2s portion of the active -TRAFFIC_PROFILE in 514-byte RELAY cells over TLS. - -Each chunk produced by `_profile.run_profile` is split into one or more -DATA_PER_CELL-byte segments and emitted as RELAY cells, so PROFILE_CHUNK_C2S -controls the application data unit while the wire continues to use Tor's -fixed 514-byte cell size. -""" - -from collections.abc import Callable -from contextlib import suppress -from os import environ, path, urandom -from socket import SHUT_WR, create_connection -from ssl import PROTOCOL_TLS_CLIENT, SSLContext, SSLError, SSLSocket, TLSVersion -from struct import pack -from subprocess import run -from sys import exit -from time import monotonic, sleep - -from _profile import run_profile - -CELL = 514 -DATA_PER_CELL = 498 # usable bytes per RELAY cell -PORT = 9001 - - -def make_relay_cell(data: bytes) -> bytes: - """Build a 514-byte Tor link-protocol-v4 RELAY cell.""" - # header: circid(4) + CMD_RELAY(1) = 5 bytes - header = pack("!IB", 1, 3) - # relay body: relay_cmd(1) + recognized(2) + stream_id(2) + digest(4) + length(2) + data(498) = 509 - body = ( - b"\x02" # relay_cmd = RELAY_DATA - + b"\x00\x00" # recognized - + b"\x00\x01" # stream_id - + urandom(4) # digest (random for realism) - + pack("!H", len(data)) # length - + data.ljust(DATA_PER_CELL, b"\x00") - ) - return header + body # 5 + 509 = 514 - - -observer_gw = environ.get("OBSERVER_GW") -server_host = environ["SERVER_HOST"] -retries = 30 - -if observer_gw: - run( - ["ip", "route", "add", "172.21.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -for _ in range(retries): - if path.exists("/keys/tor_cert.pem"): - break - sleep(1) -else: - print("tor_cert.pem never appeared", flush=True) - exit(1) - -ctx = SSLContext(PROTOCOL_TLS_CLIENT) -ctx.minimum_version = TLSVersion.TLSv1_3 -ctx.load_verify_locations("/keys/tor_cert.pem") -ctx.check_hostname = False - -for attempt in range(retries): - try: - raw = create_connection((server_host, PORT), timeout=5) - raw.settimeout(None) - tls = ctx.wrap_socket(raw, server_hostname="tor-eval") - - def _create_send_function(tls: SSLSocket) -> Callable[[bytes], None]: - """Return a function that sends *data* in RELAY cells over *tls*.""" - def send_in_cells(data: bytes) -> None: - """Split *data* into ≀DATA_PER_CELL chunks and emit RELAY cells.""" - offset = 0 - while offset < len(data): - n = min(DATA_PER_CELL, len(data) - offset) - tls.sendall(make_relay_cell(data[offset:offset + n])) - offset += n - return send_in_cells - - transfer_start = monotonic() - sent, total_sleep = run_profile(_create_send_function(tls)) - transfer_time_s = monotonic() - transfer_start - total_sleep - - try: - raw2 = tls.unwrap() - except (SSLError, OSError): - raw2 = raw - with suppress(OSError): - raw2.shutdown(SHUT_WR) - raw2.settimeout(120) - while raw2.recv(65536): - pass - with suppress(OSError): - raw2.close() - print(f"sent {sent} data bytes in cells", flush=True) - print(f"transfer_time_s={transfer_time_s:.3f}", flush=True) - exit(0) - except (ConnectionRefusedError, OSError, SSLError) as exc: - print(f"attempt {attempt + 1}: {exc}", flush=True) - sleep(1) - -exit(1) diff --git a/evaluation/protocols/tor/server.py b/evaluation/protocols/tor/server.py deleted file mode 100644 index fbb46442..00000000 --- a/evaluation/protocols/tor/server.py +++ /dev/null @@ -1,93 +0,0 @@ -#!/usr/bin/env python3 -from contextlib import suppress -from os import environ -from socket import AF_INET, SO_REUSEADDR, SOCK_STREAM, SOL_SOCKET, socket -from ssl import PROTOCOL_TLS_SERVER, SSLContext, SSLError, TLSVersion -from struct import unpack -from subprocess import run -from sys import exit - -CELL = 514 -PORT = 9001 # conventional Tor ORPort -LENGTH_OFFSET = 14 - - -def recv_exact(sock: socket, n: int) -> bytes: - buf = bytearray() - while len(buf) < n: - chunk = sock.recv(n - len(buf)) - if not chunk: - return bytes(buf) - buf += chunk - return bytes(buf) - - -observer_gw = environ.get("OBSERVER_GW") -transfer_bytes = int(environ.get("PROFILE_BYTES_C2S", 104_857_600)) -idle_timeout = int(environ.get("IDLE_TIMEOUT_S", 120)) - -if observer_gw: - run( - ["ip", "route", "add", "172.20.0.0/24", "via", observer_gw], - check=False, - capture_output=True, - ) - -run( - [ - "openssl", - "req", - "-x509", - "-newkey", - "rsa:2048", - "-keyout", - "/tmp/tor_key.pem", - "-out", - "/keys/tor_cert.pem", - "-days", - "1", - "-nodes", - "-subj", - "/CN=tor-eval", - ], - check=True, - capture_output=True, -) - -ctx = SSLContext(PROTOCOL_TLS_SERVER) -ctx.minimum_version = TLSVersion.TLSv1_3 -ctx.load_cert_chain("/keys/tor_cert.pem", "/tmp/tor_key.pem") - -raw = socket(AF_INET, SOCK_STREAM) -raw.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) -raw.bind(("0.0.0.0", PORT)) -raw.listen(1) -print(f"Tor OR simulator listening on :{PORT}", flush=True) - -received_data = 0 -DATA_PER_CELL = 498 # usable payload bytes per RELAY cell - -conn, addr = raw.accept() -try: - tls = ctx.wrap_socket(conn, server_side=True) -except SSLError as e: - print(f"TLS handshake failed: {e}", flush=True) - exit(1) - -print(f"TLS connection from {addr}", flush=True) -tls.settimeout(idle_timeout) - -with suppress(SSLError, OSError, TimeoutError): - while received_data < transfer_bytes: - cell = recv_exact(tls, CELL) - if len(cell) < CELL: - break - (length,) = unpack("!H", cell[LENGTH_OFFSET:LENGTH_OFFSET + 2]) - received_data += min(length, DATA_PER_CELL) - -with suppress(OSError): - tls.close() - -pct = received_data / transfer_bytes * 100 -print(f"received ~{received_data} bytes in cells ({pct:.1f}%)", flush=True) -exit(0) diff --git a/evaluation/protocols/typhoon/Cargo.toml b/evaluation/protocols/typhoon/Cargo.toml index 0d3fcf35..5526c3d2 100644 --- a/evaluation/protocols/typhoon/Cargo.toml +++ b/evaluation/protocols/typhoon/Cargo.toml @@ -9,6 +9,7 @@ tokio = { version = "^1", features = ["rt-multi-thread", "macros", "time", env_logger = "^0.11" log = "^0.4" rand = "^0.8" +libc = "^0.2" [lib] path = "src/lib.rs" diff --git a/evaluation/protocols/typhoon/src/bin/eval_client.rs b/evaluation/protocols/typhoon/src/bin/eval_client.rs index 47bc407a..2cbb0ded 100644 --- a/evaluation/protocols/typhoon/src/bin/eval_client.rs +++ b/evaluation/protocols/typhoon/src/bin/eval_client.rs @@ -1,11 +1,10 @@ //! TYPHOON evaluation client. //! //! Polls `/keys/typhoon.cert` until the server writes it, loads the certificate, -//! connects to the server, and runs the traffic profile selected by the -//! `TRAFFIC_PROFILE` env variable. Profile parameters (chunk sizes, IATs, byte -//! budgets, FlowConfig overrides) are read from the `PROFILE_*` env vars; both -//! the client and server containers receive identical values from the same -//! per-run env file written by the orchestrator. +//! connects, and runs the per-packet latency ping (see `typhoon_eval::latency`). +//! `TRAFFIC_PROFILE` / `PROFILE_*` still select the TYPHOON flow settings (decoys, +//! fake headers, FlowConfig overrides) so the ping runs over a realistically +//! shaped flow; the profile's transfer fields (chunk/bytes/IAT) are unused here. use std::env::var; use std::path::Path; @@ -39,6 +38,8 @@ use typhoon::settings::keys::{ }; use typhoon::socket::ClientSocketBuilder; use typhoon_eval::identity::ShortIdentity; +use typhoon_eval::latency; +use typhoon_eval::monotonic_ns; use typhoon_eval::profile::TrafficProfile; // ── Eval-side overrides (skipped for `raw_default`) ────────────────────────── @@ -281,191 +282,37 @@ async fn main() { println!("Connected to server"); let socket = Arc::new(socket); - let transfer_start = Instant::now(); - let deadline = transfer_start + profile.duration(); - - // Concurrent c2s send / s2c receive loops bounded by deadline and byte budgets. - let send_handle = if profile.has_c2s_traffic() { - Some(tokio::spawn(run_c2s_send( - socket.clone(), - profile.clone(), - deadline, - ))) - } else { - None - }; - let recv_handle = if profile.has_s2c_traffic() { - Some(tokio::spawn(run_s2c_recv( - socket.clone(), - profile.clone(), - deadline, - ))) - } else { - None - }; - - let had_send = send_handle.is_some(); - let had_recv = recv_handle.is_some(); - let (sent, total_sleep_s) = match send_handle { - Some(h) => h.await.expect("c2s join"), - None => (0, 0.0), - }; - let received = match recv_handle { - Some(h) => h.await.expect("s2c join"), - None => 0, - }; - - if !had_send && !had_recv { - let now = Instant::now(); - if deadline > now { - sleep(deadline - now).await; - } - } - - let elapsed_s = transfer_start.elapsed().as_secs_f64(); - let transfer_time_s = (elapsed_s - total_sleep_s).max(0.0); - println!("Sent {sent} bytes c2s, received {received} bytes s2c β€” done"); - println!("transfer_time_s={transfer_time_s:.3}"); -} - -/// Drive the c2s send loop, respecting `bytes_c2s`, `chunk_c2s`, IAT, and bursty mode. -async fn run_c2s_send( - socket: Arc< - typhoon::socket::ClientSocket< - ShortIdentity, - DefaultExecutor, - DefaultClientConnectionHandler, - >, - >, - profile: TrafficProfile, - deadline: Instant, -) -> (usize, f64) { - let buf_size = profile.chunk_c2s.max(profile.chunk_c2s_max); - let chunk = vec![0u8; buf_size]; - let mut sent: usize = 0; - let mut total_sleep_s: f64 = 0.0; - - if profile.bursty && profile.burst_count > 1 { - let bytes_per_burst = profile.bytes_c2s / profile.burst_count.max(1); - for i in 0..profile.burst_count { - let (s, slept) = - send_until(&socket, &chunk, &profile, deadline, sent + bytes_per_burst).await; - sent += s; - total_sleep_s += slept; - if sent >= profile.bytes_c2s || Instant::now() >= deadline { - break; - } - if i + 1 < profile.burst_count { - let burst_idle = profile.burst_idle(); - let idle_until = Instant::now() + burst_idle; - if idle_until > deadline { - break; - } - sleep(burst_idle).await; - total_sleep_s += burst_idle.as_secs_f64(); - } - } - } else { - let (s, slept) = send_until(&socket, &chunk, &profile, deadline, profile.bytes_c2s).await; - sent += s; - total_sleep_s += slept; - } - (sent, total_sleep_s) + run_latency(&socket).await; } -async fn send_until( +/// Sequential ping β€” send a probe, await its echo, record RTT. +async fn run_latency( socket: &typhoon::socket::ClientSocket< ShortIdentity, DefaultExecutor, DefaultClientConnectionHandler, >, - chunk: &[u8], - profile: &TrafficProfile, - deadline: Instant, - target: usize, -) -> (usize, f64) { - let mut sent = 0; - let mut packets = 0; - let mut total_sleep_s: f64 = 0.0; - let fixed_delay = profile.c2s_delay(); - let randomise = profile.is_unrestricted(); - let inter_batch_delay_ms: f64 = var("INTER_PACKET_DELAY_MS") - .ok() - .and_then(|v| v.parse().ok()) - .unwrap_or(40.0); - let batch_size: u64 = var("DELAY_EVERY_N") - .ok() - .and_then(|v| v.parse().ok()) - .unwrap_or(10) - .max(1); - let batch_delay = Duration::from_micros((inter_batch_delay_ms * 1000.0) as u64); - let mut packets_in_batch: u64 = 0; - - let mut rng = StdRng::from_entropy(); - while sent < target && Instant::now() < deadline && packets < profile.max_packets() { - let pkt_size = if randomise { - profile.sample_chunk_c2s(&mut rng) - } else { - profile.chunk_c2s - }; - let n = pkt_size.min(target - sent).min(chunk.len()); - if n == 0 { +) { + let cfg = latency::Config::from_env(); + let mut rtts: Vec = Vec::with_capacity(cfg.count as usize); + for seq in 0..cfg.count { + let send_ns = monotonic_ns(); + let msg = latency::pack(seq, send_ns, cfg.size); + let t0 = Instant::now(); + if socket.send_bytes(&msg).await.is_err() { break; } - if socket.send_bytes(&chunk[..n]).await.is_err() { - break; - } - sent += n; - packets += 1; - packets_in_batch += 1; - let delay = if randomise { - profile.sample_c2s_delay(&mut rng) - } else { - fixed_delay - }; - if !delay.is_zero() { - sleep(delay).await; - total_sleep_s += delay.as_secs_f64(); - } - if !batch_delay.is_zero() && packets_in_batch >= batch_size { - sleep(batch_delay).await; - total_sleep_s += batch_delay.as_secs_f64(); - packets_in_batch = 0; + match timeout(cfg.recv_timeout, socket.receive_bytes()).await { + Ok(Ok(echo)) if echo.len() >= latency::HEADER => { + let rtt = (monotonic_ns().saturating_sub(latency::send_ns_of(&echo))) as f64 / 1e6; + rtts.push(rtt); + } + _ => {} // lost echo β€” keep pinging } - } - (sent, total_sleep_s) -} - -/// Drive the s2c receive loop until the byte budget or deadline is met. -async fn run_s2c_recv( - socket: Arc< - typhoon::socket::ClientSocket< - ShortIdentity, - DefaultExecutor, - DefaultClientConnectionHandler, - >, - >, - profile: TrafficProfile, - deadline: Instant, -) -> usize { - let mut received: usize = 0; - let now = Instant::now(); - let mut remaining = if deadline > now { - deadline - now - } else { - Duration::from_secs(0) - }; - while received < profile.bytes_s2c && !remaining.is_zero() { - match timeout(remaining, socket.receive_bytes()).await { - Ok(Ok(data)) => received += data.len(), - _ => break, + let elapsed = t0.elapsed(); + if elapsed < cfg.interval { + sleep(cfg.interval - elapsed).await; } - let now = Instant::now(); - remaining = if deadline > now { - deadline - now - } else { - Duration::from_secs(0) - }; } - received + latency::print_client_report(&mut rtts, cfg.count); } diff --git a/evaluation/protocols/typhoon/src/bin/eval_server.rs b/evaluation/protocols/typhoon/src/bin/eval_server.rs index e9305450..12ded883 100644 --- a/evaluation/protocols/typhoon/src/bin/eval_server.rs +++ b/evaluation/protocols/typhoon/src/bin/eval_server.rs @@ -1,32 +1,24 @@ //! TYPHOON evaluation server. //! //! Generates a `ServerKeyPair`, saves the `ClientCertificate` to -//! `/keys/typhoon.cert` (so the client container can load it from the -//! shared `eval_keys` volume), then accepts one connection and runs the -//! traffic profile selected by the `TRAFFIC_PROFILE` env variable. -//! -//! Profile parameters are read from the same `PROFILE_*` env vars that the -//! client receives, so both ends drive matching c2s/s2c loops without any -//! in-band negotiation. +//! `/keys/typhoon.cert` (so the client container can load it from the shared +//! `eval_keys` volume), then accepts one connection and echoes the client's +//! latency-ping probes (see `typhoon_eval::latency`). `TRAFFIC_PROFILE` / +//! `PROFILE_*` select the TYPHOON flow settings, matching the client's, so the +//! ping runs over a realistically shaped flow. use std::env::var; use std::net::SocketAddr; use std::process::Command; use std::process::exit; use std::sync::Arc; -use std::sync::atomic::{AtomicUsize, Ordering}; -use std::time::Instant; use env_logger::{Builder, Env}; use log::info; -use rand::SeedableRng; -use rand::rngs::StdRng; -use tokio::signal::unix::{SignalKind, signal}; -use tokio::sync::Notify; -use tokio::time::{Duration, sleep, timeout}; +use tokio::time::{Duration, timeout}; -const SIGTERM_DRAIN_GRACE: Duration = Duration::from_secs(3); use typhoon_eval::identity::{EvalServerConnectionHandler, ShortIdentity}; +use typhoon_eval::latency; use typhoon_eval::profile::TrafficProfile; use typhoon::certificate::ServerKeyPair; @@ -283,200 +275,31 @@ async fn main() { let client = Arc::new(listener.accept().await.expect("accept")); println!("Client connected"); - let session_start = Instant::now(); - let deadline = session_start + profile.duration(); - - let received_counter: Arc = Arc::new(AtomicUsize::new(0)); - let sent_counter: Arc = Arc::new(AtomicUsize::new(0)); - - let terminate_signal: Arc = Arc::new(Notify::new()); - { - let terminate_signal = terminate_signal.clone(); - tokio::spawn(async move { - let mut sigterm = signal(SignalKind::terminate()).expect("install SIGTERM handler"); - sigterm.recv().await; - terminate_signal.notify_one(); - }); - } - - // Concurrent c2s receive and s2c send loops bounded by deadline and byte budgets. - let recv_handle = if profile.has_c2s_traffic() { - Some(tokio::spawn(run_c2s_recv( - client.clone(), - profile.clone(), - deadline, - received_counter.clone(), - terminate_signal.clone(), - ))) - } else { - None - }; - let send_handle = if profile.has_s2c_traffic() { - Some(tokio::spawn(run_s2c_send( - client.clone(), - profile.clone(), - deadline, - sent_counter.clone(), - ))) - } else { - None - }; - - let had_recv = recv_handle.is_some(); - let had_send = send_handle.is_some(); - let received = match recv_handle { - Some(h) => h.await.expect("c2s join"), - None => 0, - }; - let sent = match send_handle { - Some(h) => h.await.expect("s2c join"), - None => 0, - }; - - if !had_recv && !had_send { - let now = Instant::now(); - if deadline > now { - sleep(deadline - now).await; - } - } - - let pct_c2s = if profile.bytes_c2s > 0 { - received as f64 / profile.bytes_c2s as f64 * 100.0 - } else { - 100.0 - }; - println!( - "Received {received}/{} bytes c2s ({pct_c2s:.1}%); sent {sent}/{} bytes s2c β€” done", - profile.bytes_c2s, profile.bytes_s2c - ); - println!("recv_time_s={:.3}", session_start.elapsed().as_secs_f64()); + run_latency_echo(&client).await; exit(0); } -/// Drive the c2s receive loop until the byte budget or deadline is met. -async fn run_c2s_recv( - client: Arc>, - profile: TrafficProfile, - deadline: Instant, - counter: Arc, - terminate_signal: Arc, -) -> usize { - let mut received: usize = 0; - let mut effective_deadline = deadline; - let mut signalled = false; - - while received < profile.bytes_c2s && Instant::now() < effective_deadline { - let remaining = effective_deadline.saturating_duration_since(Instant::now()); - if remaining.is_zero() { - break; - } - tokio::select! { - biased; - // Listen for SIGTERM only while we haven't already shortened the deadline. - _ = terminate_signal.notified(), if !signalled => { - effective_deadline = (Instant::now() + SIGTERM_DRAIN_GRACE).min(deadline); - signalled = true; - } - result = timeout(remaining, client.receive_bytes()) => { - match result { - Ok(Ok(data)) => { - received += data.len(); - counter.store(received, Ordering::Relaxed); - } - _ => break, - } - } - } - } - received -} - -/// Drive the s2c send loop, respecting `bytes_s2c`, `chunk_s2c`, IAT, and bursty mode. -async fn run_s2c_send( - client: Arc>, - profile: TrafficProfile, - deadline: Instant, - counter: Arc, -) -> usize { - let buf_size = profile.chunk_s2c.max(profile.chunk_s2c_max); - let chunk = vec![0u8; buf_size]; - let mut sent: usize = 0; - - if profile.bursty && profile.burst_count > 1 { - let bytes_per_burst = profile.bytes_s2c / profile.burst_count.max(1); - for i in 0..profile.burst_count { - sent += send_until_s( - &client, - &chunk, - &profile, - deadline, - sent + bytes_per_burst, - counter.clone(), - ) - .await; - if sent >= profile.bytes_s2c || Instant::now() >= deadline { - break; - } - if i + 1 < profile.burst_count { - let idle_until = Instant::now() + profile.burst_idle(); - if idle_until > deadline { +/// Echo each probe back to the client until `count` seen or idle. +async fn run_latency_echo(client: &ClientHandle) { + let cfg = latency::Config::from_env(); + let idle = Duration::from_secs( + var("IDLE_TIMEOUT_S") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(30), + ); + let mut received = 0u32; + while received < cfg.count { + match timeout(idle, client.receive_bytes()).await { + Ok(Ok(data)) => { + if client.send_bytes(&data).await.is_err() { break; } - sleep(profile.burst_idle()).await; + received += 1; } - } - } else { - sent += send_until_s( - &client, - &chunk, - &profile, - deadline, - profile.bytes_s2c, - counter, - ) - .await; - } - sent -} - -async fn send_until_s( - client: &ClientHandle, - chunk: &[u8], - profile: &TrafficProfile, - deadline: Instant, - target: usize, - counter: Arc, -) -> usize { - let mut sent = 0; - let mut packets = 0; - let fixed_delay = profile.s2c_delay(); - let randomise = profile.is_unrestricted(); - // `StdRng::from_entropy()` is `Send` β€” see eval_client.rs for rationale. - let mut rng = StdRng::from_entropy(); - while sent < target && Instant::now() < deadline && packets < profile.max_packets() { - let pkt_size = if randomise { - profile.sample_chunk_s2c(&mut rng) - } else { - profile.chunk_s2c - }; - let n = pkt_size.min(target - sent).min(chunk.len()); - if n == 0 { - break; - } - if client.send_bytes(&chunk[..n]).await.is_err() { - break; - } - sent += n; - counter.store(sent, Ordering::Relaxed); - packets += 1; - let delay = if randomise { - profile.sample_s2c_delay(&mut rng) - } else { - fixed_delay - }; - if !delay.is_zero() { - sleep(delay).await; + _ => break, // idle / error ends the run } } - sent + let delivery = received as f64 / cfg.count.max(1) as f64 * 100.0; + println!("received {received}/{} packets ({delivery:.1}%)", cfg.count); } diff --git a/evaluation/protocols/typhoon/src/lib.rs b/evaluation/protocols/typhoon/src/lib.rs index dfa08bb2..651843d9 100644 --- a/evaluation/protocols/typhoon/src/lib.rs +++ b/evaluation/protocols/typhoon/src/lib.rs @@ -7,3 +7,96 @@ pub mod identity; pub mod profile; + +/// Host-wide monotonic clock in nanoseconds. Docker containers share the kernel +/// clock (no time namespace by default), so the client's `send_start/end` and +/// the server's `recv_first/last` readings are directly comparable β€” the basis +/// for the cross-endpoint transfer timings the analysis derives. +pub fn monotonic_ns() -> u128 { + let mut ts = libc::timespec { + tv_sec: 0, + tv_nsec: 0, + }; + unsafe { + libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts); + } + (ts.tv_sec as u128) * 1_000_000_000 + (ts.tv_nsec as u128) +} + +/// Latency-mode helpers (a duplicate of the shared `eval-transport::latency` +/// contract, since the two eval crates don't share a dependency). The client +/// pings small equal-sized probes that the server echoes; RTT is timed on the +/// client. See the transport crate for the rationale. +pub mod latency { + use std::env::var; + use std::time::Duration; + + pub const HEADER: usize = 20; // seq(4) + send_ns(16) + + /// Ping parameters: probe count, spacing, fixed size, and per-echo timeout. + pub struct Config { + pub count: u32, + pub interval: Duration, + pub size: usize, + pub recv_timeout: Duration, + } + + impl Config { + /// Read the config from the `LAT_*` env vars (harness-set). + pub fn from_env() -> Self { + Self { + count: env_u32("LAT_COUNT", 500), + interval: Duration::from_secs_f64(env_f64("LAT_INTERVAL_MS", 20.0) / 1000.0), + size: (env_u32("LAT_SIZE", 256) as usize).max(HEADER), + recv_timeout: Duration::from_secs_f64( + env_f64("LAT_RECV_TIMEOUT_MS", 5000.0) / 1000.0, + ), + } + } + } + + pub fn pack(seq: u32, send_ns: u128, size: usize) -> Vec { + let mut m = vec![0u8; size]; + m[0..4].copy_from_slice(&seq.to_be_bytes()); + m[4..HEADER].copy_from_slice(&send_ns.to_be_bytes()); + m + } + + pub fn send_ns_of(msg: &[u8]) -> u128 { + let mut b = [0u8; 16]; + b.copy_from_slice(&msg[4..HEADER]); + u128::from_be_bytes(b) + } + + /// Print the client `sent` / `roundtrip_delivery_pct` / `rtt_*` contract. + pub fn print_client_report(rtts: &mut [f64], count: u32) { + println!("sent {count} packets"); + let delivery = rtts.len() as f64 / count.max(1) as f64 * 100.0; + println!("roundtrip_delivery_pct={delivery:.1}"); + if !rtts.is_empty() { + rtts.sort_by(|a, b| a.partial_cmp(b).unwrap()); + let pct = |p: f64| rtts[(((rtts.len() - 1) as f64) * p).round() as usize]; + let p50 = pct(0.50); + let p95 = pct(0.95); + println!("rtt_min_ms={:.3}", rtts[0]); + println!("rtt_p50_ms={p50:.3}"); + println!("rtt_p95_ms={p95:.3}"); + println!("rtt_p99_ms={:.3}", pct(0.99)); + println!("rtt_jitter_ms={:.3}", p95 - p50); + } + } + + fn env_u32(key: &str, default: u32) -> u32 { + var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) + } + + fn env_f64(key: &str, default: f64) -> f64 { + var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) + } +} diff --git a/evaluation/protocols/vless_reality/Dockerfile.client b/evaluation/protocols/vless_reality/Dockerfile.client index b52f22b7..729a196e 100644 --- a/evaluation/protocols/vless_reality/Dockerfile.client +++ b/evaluation/protocols/vless_reality/Dockerfile.client @@ -3,12 +3,10 @@ FROM ghcr.io/xtls/xray-core AS xray FROM debian:bookworm-slim COPY --from=xray /usr/local/bin/xray /usr/local/bin/xray RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 python3 python3-pip \ + iproute2 \ && rm -rf /var/lib/apt/lists/* \ - && pip install --no-cache-dir --break-system-packages PySocks \ && mkdir -p /etc/xray COPY vless_reality/client-entrypoint.sh /entrypoint.sh -COPY common/socks5_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/socks5-sender /usr/local/bin/socks5-sender RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/vless_reality/Dockerfile.server b/evaluation/protocols/vless_reality/Dockerfile.server index 2e62d33b..e6693a1b 100644 --- a/evaluation/protocols/vless_reality/Dockerfile.server +++ b/evaluation/protocols/vless_reality/Dockerfile.server @@ -3,11 +3,11 @@ FROM ghcr.io/xtls/xray-core AS xray FROM debian:bookworm-slim COPY --from=xray /usr/local/bin/xray /usr/local/bin/xray RUN apt-get update && apt-get install -y --no-install-recommends \ - iproute2 openssl python3 \ + iproute2 openssl \ && rm -rf /var/lib/apt/lists/* \ && mkdir -p /etc/xray COPY vless_reality/server-entrypoint.sh /entrypoint.sh -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/tcp-sink /usr/local/bin/tcp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=5s --retries=30 --start-period=8s \ CMD ss -tln | grep -q ':443' && test -f /keys/vless_public_key diff --git a/evaluation/protocols/vless_reality/client-entrypoint.sh b/evaluation/protocols/vless_reality/client-entrypoint.sh index e3b62acd..96e6b496 100644 --- a/evaluation/protocols/vless_reality/client-entrypoint.sh +++ b/evaluation/protocols/vless_reality/client-entrypoint.sh @@ -60,6 +60,6 @@ done SERVER_HOST=127.0.0.1 \ SERVER_PORT=9000 \ OBSERVER_GW="" \ -python3 /app/client.py +socks5-sender kill "${XRAY_PID}" 2>/dev/null || true diff --git a/evaluation/protocols/vless_reality/server-entrypoint.sh b/evaluation/protocols/vless_reality/server-entrypoint.sh index 7e783467..5a32c23b 100644 --- a/evaluation/protocols/vless_reality/server-entrypoint.sh +++ b/evaluation/protocols/vless_reality/server-entrypoint.sh @@ -53,7 +53,7 @@ EOF PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +tcp-sink & SINK_PID=$! xray run -config /etc/xray/config.json & diff --git a/evaluation/protocols/wireguard/Dockerfile.client b/evaluation/protocols/wireguard/Dockerfile.client index 6fcb09b2..f32cffe8 100644 --- a/evaluation/protocols/wireguard/Dockerfile.client +++ b/evaluation/protocols/wireguard/Dockerfile.client @@ -1,11 +1,9 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - wireguard-tools iproute2 python3 \ + wireguard-tools iproute2 \ && rm -rf /var/lib/apt/lists/* COPY wireguard/client-entrypoint.sh /entrypoint.sh -COPY common/udp_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/udp-sender /usr/local/bin/udp-sender RUN chmod +x /entrypoint.sh -# Client connects to the server's WireGuard IP 10.100.0.1:9000 ENV SERVER_HOST=10.100.0.1 ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/wireguard/Dockerfile.server b/evaluation/protocols/wireguard/Dockerfile.server index da2451f2..a027b57a 100644 --- a/evaluation/protocols/wireguard/Dockerfile.server +++ b/evaluation/protocols/wireguard/Dockerfile.server @@ -1,9 +1,9 @@ FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ - wireguard-tools iproute2 python3 \ + wireguard-tools iproute2 \ && rm -rf /var/lib/apt/lists/* COPY wireguard/server-entrypoint.sh /entrypoint.sh -COPY common/udp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/udp-sink /usr/local/bin/udp-sink RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=3s --retries=20 --start-period=5s \ CMD ip link show wg0 >/dev/null 2>&1 && ss -uln | grep -q ':9000' diff --git a/evaluation/protocols/wireguard/client-entrypoint.sh b/evaluation/protocols/wireguard/client-entrypoint.sh index bf9f2fe4..0c0f3617 100644 --- a/evaluation/protocols/wireguard/client-entrypoint.sh +++ b/evaluation/protocols/wireguard/client-entrypoint.sh @@ -30,4 +30,4 @@ for i in {1..30}; do sleep 1 done -SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec python3 /app/client.py +SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec udp-sender diff --git a/evaluation/protocols/wireguard/server-entrypoint.sh b/evaluation/protocols/wireguard/server-entrypoint.sh index a4bfa3ff..3f06d305 100644 --- a/evaluation/protocols/wireguard/server-entrypoint.sh +++ b/evaluation/protocols/wireguard/server-entrypoint.sh @@ -12,7 +12,7 @@ ip link set wg0 up PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +udp-sink & SINK_PID=$! ( diff --git a/evaluation/protocols/wireguard_daita/Dockerfile.client b/evaluation/protocols/wireguard_daita/Dockerfile.client index f1604b57..f18a8cb8 100644 --- a/evaluation/protocols/wireguard_daita/Dockerfile.client +++ b/evaluation/protocols/wireguard_daita/Dockerfile.client @@ -36,8 +36,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ COPY --from=go-build /usr/local/bin/wg-daita /usr/local/bin/wg-daita COPY --from=rust-build /build/machines-client.txt /etc/daita/machines.txt -COPY common/tcp_sender.py /app/client.py -COPY common/_profile.py /app/_profile.py +COPY --from=transport /usr/local/bin/udp-sender /usr/local/bin/udp-sender COPY wireguard_daita/client-entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh diff --git a/evaluation/protocols/wireguard_daita/Dockerfile.server b/evaluation/protocols/wireguard_daita/Dockerfile.server index f9322fb3..8bbd77ae 100644 --- a/evaluation/protocols/wireguard_daita/Dockerfile.server +++ b/evaluation/protocols/wireguard_daita/Dockerfile.server @@ -40,11 +40,11 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ COPY --from=go-build /usr/local/bin/wg-daita /usr/local/bin/wg-daita COPY --from=rust-build /build/machines-server.txt /etc/daita/machines.txt -COPY common/tcp_sink.py /app/server.py +COPY --from=transport /usr/local/bin/udp-sink /usr/local/bin/udp-sink COPY wireguard_daita/server-entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh HEALTHCHECK --interval=1s --timeout=3s --retries=30 --start-period=10s \ - CMD ss -tln | grep -q ':9000' + CMD ss -uln | grep -q ':9000' ENTRYPOINT ["/entrypoint.sh"] diff --git a/evaluation/protocols/wireguard_daita/client-entrypoint.sh b/evaluation/protocols/wireguard_daita/client-entrypoint.sh index aa1a229e..bcf7a827 100644 --- a/evaluation/protocols/wireguard_daita/client-entrypoint.sh +++ b/evaluation/protocols/wireguard_daita/client-entrypoint.sh @@ -62,4 +62,4 @@ for i in {1..30}; do sleep 1 done -SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec python3 /app/client.py +SERVER_HOST=10.100.0.1 SERVER_PORT=9000 OBSERVER_GW="" exec udp-sender diff --git a/evaluation/protocols/wireguard_daita/server-entrypoint.sh b/evaluation/protocols/wireguard_daita/server-entrypoint.sh index 34b20f3d..421457d4 100644 --- a/evaluation/protocols/wireguard_daita/server-entrypoint.sh +++ b/evaluation/protocols/wireguard_daita/server-entrypoint.sh @@ -25,7 +25,7 @@ print(binascii.hexlify(base64.b64decode(b64)).decode()) # Start tcp_sink PROFILE_BYTES_C2S="${PROFILE_BYTES_C2S:-104857600}" \ OBSERVER_GW="" \ -python3 /app/server.py & +udp-sink & SINK_PID=$! # Wait for client public key (written as base64 by client) diff --git a/evaluation/pyproject.toml b/evaluation/pyproject.toml index 4a18663f..4ed7f404 100644 --- a/evaluation/pyproject.toml +++ b/evaluation/pyproject.toml @@ -71,6 +71,7 @@ numpy = "np" [tool.poe.tasks.build] cmd = "docker compose -f compose/docker-compose.build.yml build" help = "Build all protocol Docker images" + env = { DOCKER_BUILDKIT = "1", COMPOSE_DOCKER_CLI_BUILD = "1" } [tool.poe.tasks.capture] cmd = "python -m typhoon_eval.shared.orchestrator" @@ -114,6 +115,7 @@ numpy = "np" [tool.poe.tasks.background-build] cmd = "docker compose -f background/compose/docker-compose.build.yml build" help = "Build all background generator Docker images (DNS, gaming, RTP, QUIC, control plane, WG idle)" + env = { DOCKER_BUILDKIT = "1", COMPOSE_DOCKER_CLI_BUILD = "1" } [tool.poe.tasks.background-corpus] cmd = "python -m typhoon_eval.background.corpus" diff --git a/evaluation/src/typhoon_eval/background/corpus.py b/evaluation/src/typhoon_eval/background/corpus.py index f9d498ec..0d882e3c 100644 --- a/evaluation/src/typhoon_eval/background/corpus.py +++ b/evaluation/src/typhoon_eval/background/corpus.py @@ -39,6 +39,7 @@ class independently (each still draws its own random shape per run). from jinja2 import Environment, FileSystemLoader, StrictUndefined from typhoon_eval.shared.console import console +from typhoon_eval.shared.docker_utils import BUILDKIT_ENV from typhoon_eval.shared.profiles import ( BACKGROUND_PROFILES, CHAOS_DUPLICATE_PCT, @@ -89,6 +90,7 @@ def _build_images() -> None: result = run( ["docker", "compose", "-f", str(compose_file), "build"], stdin=DEVNULL, + env=BUILDKIT_ENV, ) if result.returncode != 0: console.print(f"[red]docker compose build failed[/red] for {compose_file}") diff --git a/evaluation/src/typhoon_eval/pipeline.py b/evaluation/src/typhoon_eval/pipeline.py index dde5ec61..992201d6 100644 --- a/evaluation/src/typhoon_eval/pipeline.py +++ b/evaluation/src/typhoon_eval/pipeline.py @@ -45,6 +45,7 @@ from typhoon_eval.shared.analysis import CAPTURES_ROOT, _latest_run from typhoon_eval.shared.analysis import main as _analysis_main from typhoon_eval.shared.console import console +from typhoon_eval.shared.docker_utils import BUILDKIT_ENV from typhoon_eval.shared.orchestrator import main as _orchestrator_main from typhoon_eval.shared.pcap_flow_plot import main as _pcap_flow_plot_main @@ -106,9 +107,9 @@ def _shell(label: str, cmd: list[str], log_path: Path | None = None, quiet: bool lf.write(f"\n$ {' '.join(cmd)}\n") if quiet and log_path: with log_path.open("a") as lf: - result = run(cmd, stdin=DEVNULL, stdout=lf, stderr=STDOUT) + result = run(cmd, stdin=DEVNULL, stdout=lf, stderr=STDOUT, env=BUILDKIT_ENV) else: - result = run(cmd, stdin=DEVNULL) + result = run(cmd, stdin=DEVNULL, env=BUILDKIT_ENV) if log_path: with log_path.open("a") as lf: lf.write(f"exit_code={result.returncode}\n") diff --git a/evaluation/src/typhoon_eval/protocols_op/proto_compare_plots.py b/evaluation/src/typhoon_eval/protocols_op/proto_compare_plots.py index 84ba25c5..9b404ddc 100644 --- a/evaluation/src/typhoon_eval/protocols_op/proto_compare_plots.py +++ b/evaluation/src/typhoon_eval/protocols_op/proto_compare_plots.py @@ -3,14 +3,13 @@ Reads pcaps from a capture-run directory and produces: {run}_proto_compare.pdf β€” six panels: - A) packet-size CDF, B) IAT CDF (log x), C) throughput vs goodput-eff scatter, + A) packet-size CDF, B) IAT CDF (log x), C) per-packet RTT vs delivery scatter, D) protocol-overhead bars, E) byte entropy by phase, F) operational heatmap. {run}_handshake.pdf β€” three panels (duration, packet count, byte fraction). {run}_compare_table.md β€” markdown comparison table. """ from json import loads -from math import isnan from pathlib import Path from sys import exit @@ -38,8 +37,14 @@ def _compute_metrics( s2c: list[tuple[float, int, bytes]], proto: Protocol | None, transfer_bytes: int | None, + meta: dict, ) -> dict: - """Compute operational metrics for one protocol from raw packet records.""" + """Compute operational metrics for one protocol from raw packet records. + + Detectability (sizes, IAT, entropy, overhead) comes from the pcap; the + operational metrics (per-packet RTT, delivery) come from `meta` (the run's + endpoint-reported values), not the wire span. + """ all_recs = sorted(c2s + s2c, key=lambda r: r[0]) if not all_recs: return {} @@ -68,9 +73,6 @@ def _compute_metrics( overhead_ratio = (total_bytes - transfer_bytes) / transfer_bytes goodput_efficiency = transfer_bytes / total_bytes - tx_time_s = float(all_ts.max() - all_ts.min()) if len(all_ts) > 1 else 0.0 - throughput_mbps = (total_bytes * 8 / tx_time_s / 1e6) if tx_time_s > 0 else 0.0 - iat_mean = float(iats_ms.mean()) if len(iats_ms) else 0.0 iat_std = float(iats_ms.std()) if len(iats_ms) else 0.0 burstiness = iat_std / iat_mean if iat_mean > 0 else 0.0 @@ -88,7 +90,9 @@ def _compute_metrics( "entropy_data": _entropy(data_payload) if data_payload else None, "overhead_ratio": overhead_ratio, "goodput_efficiency": goodput_efficiency, - "throughput_mbps": throughput_mbps, + "rtt_p50_ms": meta.get("rtt_p50_ms"), + "rtt_p95_ms": meta.get("rtt_p95_ms"), + "delivery_pct": meta.get("delivery_pct"), "burstiness": burstiness, "direction_asymmetry": direction_asymmetry, "hs_duration_s": (hs_end - all_ts.min()) if hs_end and len(all_ts) > 0 else None, @@ -135,25 +139,23 @@ def _plot_main(metrics: list[dict], run_name: str, out_dir: Path) -> None: ax_iat_cdf.set_title("B Inter-arrival-time CDF", fontweight="bold") ax_iat_cdf.grid(True, alpha=0.3, which="both") - # C β€” throughput vs goodput efficiency scatter + # C β€” per-packet RTT (p50) vs delivery scatter valid = [ - (m["throughput_mbps"], m["goodput_efficiency"], colors[i], m["label"]) + (m["rtt_p50_ms"], m["delivery_pct"], colors[i], m["label"]) for i, m in enumerate(metrics) - if m.get("goodput_efficiency") is not None and not isnan(m["goodput_efficiency"]) + if m.get("rtt_p50_ms") is not None and m.get("delivery_pct") is not None ] if valid: - tps, effs, clrs, lbls = zip(*valid, strict=True) - ax_thru.scatter(tps, effs, c=clrs, s=80, zorder=3, edgecolors="white", linewidth=0.5) - for tp, eff, lbl in zip(tps, effs, lbls, strict=True): - ax_thru.annotate(lbl, (tp, eff), fontsize=6, textcoords="offset points", xytext=(4, 2)) - ax_thru.axhline(1.0, color="gray", linestyle="--", linewidth=0.8, label="perfect efficiency") - ax_thru.legend(fontsize=8, loc="lower right") + rtts, dels, clrs, lbls = zip(*valid, strict=True) + ax_thru.scatter(rtts, dels, c=clrs, s=80, zorder=3, edgecolors="white", linewidth=0.5) + for rt, dl, lbl in zip(rtts, dels, lbls, strict=True): + ax_thru.annotate(lbl, (rt, dl), fontsize=6, textcoords="offset points", xytext=(4, 2)) else: - ax_thru.text(0.5, 0.5, "No transfer_bytes data available", ha="center", va="center", + ax_thru.text(0.5, 0.5, "No latency data available", ha="center", va="center", transform=ax_thru.transAxes, fontsize=9, color="gray") - ax_thru.set_xlabel("Throughput (Mbps)") - ax_thru.set_ylabel("Goodput efficiency (payload / total bytes)") - ax_thru.set_title("C Throughput vs. efficiency", fontweight="bold") + ax_thru.set_xlabel("Median RTT (ms)") + ax_thru.set_ylabel("Delivery (%)") + ax_thru.set_title("C Per-packet RTT vs. delivery", fontweight="bold") ax_thru.grid(True, alpha=0.3) # D β€” overhead bars @@ -188,24 +190,24 @@ def _plot_main(metrics: list[dict], run_name: str, out_dir: Path) -> None: ax_ent.set_title("E Byte entropy by phase", fontweight="bold") ax_ent.legend(fontsize=8) - # F β€” operational metric heatmap (normalised) - metric_names = ["Throughput", "Goodput eff.", "Data entropy", "Burstiness", "HS duration", "HS byte frac"] + # F β€” operational metric heatmap (normalised; green = better) + metric_names = ["RTT p50 (↓)", "Delivery", "Data entropy", "Burstiness", "HS duration", "HS byte frac"] def _norm(vals: list) -> ndarray: arr = array(vals, dtype=float) lo, hi = nanmin(arr), nanmax(arr) return (arr - lo) / (hi - lo) if hi > lo else zeros_like(arr) - throughputs = [m["throughput_mbps"] for m in metrics] - effs = [m["goodput_efficiency"] if m["goodput_efficiency"] is not None else float("nan") for m in metrics] + rtts = [m["rtt_p50_ms"] if m["rtt_p50_ms"] is not None else float("nan") for m in metrics] + dels = [m["delivery_pct"] if m["delivery_pct"] is not None else float("nan") for m in metrics] data_ents = [m["entropy_data"] or 0.0 for m in metrics] bursts = [m["burstiness"] for m in metrics] hs_durs = [m["hs_duration_s"] if m["hs_duration_s"] is not None else float("nan") for m in metrics] hs_fracs = [m["hs_byte_frac"] for m in metrics] heat = vstack([ - _norm(throughputs), - _norm(effs), + 1.0 - _norm(rtts), # lower RTT is better β†’ green + _norm(dels), _norm(data_ents), _norm(bursts), _norm(hs_durs), @@ -273,20 +275,22 @@ def _write_table(metrics: list[dict], run_name: str, out_dir: Path) -> None: lines = [ f"# Operational comparison β€” `{run_name}`", "", - "| Protocol | Throughput (Mbps) | Bytes (MB) | Overhead | Goodput | Data entropy (bits) | Burstiness | HS duration (s) | HS pkts | HS byte % | c2s/s2c |", + "| Protocol | Delivery | RTT p50 (ms) | RTT p95 (ms) | Overhead | Data entropy (bits) | Burstiness | HS duration (s) | HS pkts | HS byte % | c2s/s2c |", "| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |", ] for m in metrics: oh = f"{m['overhead_ratio']:.1%}" if m["overhead_ratio"] is not None else "β€”" - eff = f"{m['goodput_efficiency']:.1%}" if m["goodput_efficiency"] is not None else "β€”" hs_dur = f"{m['hs_duration_s']:.3f}" if m["hs_duration_s"] is not None else "β€”" ent_data = f"{m['entropy_data']:.3f}" if m["entropy_data"] is not None else "β€”" + delivery = f"{m['delivery_pct']:.1f}%" if m["delivery_pct"] is not None else "β€”" + rtt50 = f"{m['rtt_p50_ms']:.2f}" if m["rtt_p50_ms"] is not None else "β€”" + rtt95 = f"{m['rtt_p95_ms']:.2f}" if m["rtt_p95_ms"] is not None else "β€”" lines.append( f"| {m['label']} | " - f"{m['throughput_mbps']:.1f} | " - f"{m['total_bytes'] / 1e6:.1f} | " + f"{delivery} | " + f"{rtt50} | " + f"{rtt95} | " f"{oh} | " - f"{eff} | " f"{ent_data} | " f"{m['burstiness']:.2f} | " f"{hs_dur} | " @@ -336,7 +340,7 @@ def main(run_id: str | None, out_dir: str) -> None: if not c2s and not s2c: continue - m = _compute_metrics(name, c2s, s2c, proto, transfer_bytes) + m = _compute_metrics(name, c2s, s2c, proto, transfer_bytes, metadata.get(proto_key, {})) if m: metrics_list.append(m) diff --git a/evaluation/src/typhoon_eval/shared/analysis.py b/evaluation/src/typhoon_eval/shared/analysis.py index 1ec10b56..d1111aac 100644 --- a/evaluation/src/typhoon_eval/shared/analysis.py +++ b/evaluation/src/typhoon_eval/shared/analysis.py @@ -98,7 +98,11 @@ def main(run_id: str | None) -> None: transfer_bytes: int | None = metadata.get(proto_key, {}).get("transfer_bytes") sniffer = BY_NAME[proto_key].handshake_sniffer if proto_key in BY_NAME else None - stats = analyze_pcap(pcap, transfer_bytes=transfer_bytes, handshake_sniffer=sniffer) + stats = analyze_pcap( + pcap, + transfer_bytes=transfer_bytes, + handshake_sniffer=sniffer, + ) all_stats[name] = stats progress.update(task, total=1, completed=1, @@ -132,9 +136,17 @@ def _fmt_pct(v: float | None) -> str: return f"[{color}]{v:+.1%}[/{color}]" +def _fmt_rtt(meta: dict) -> str: + """`p50 / p95 / jit ms` from the client's per-packet RTT distribution.""" + p50, p95, jit = meta.get("rtt_p50_ms"), meta.get("rtt_p95_ms"), meta.get("rtt_jitter_ms") + if p50 is None or p95 is None: + return "[dim]β€”[/dim]" + jit_s = f"{jit:.2f}" if jit is not None else "β€”" + return f"{p50:.2f} / {p95:.2f} / {jit_s}" + + def _print_summary(all_stats: dict[str, dict], metadata: dict, cfg: dict) -> None: chaos = cfg.get("chaos", False) - injected_delay_s: float = cfg.get("injected_delay_s", 0.0) if chaos: title = "Analysis summary (chaos mode)" @@ -152,8 +164,8 @@ def _print_summary(all_stats: dict[str, dict], metadata: dict, cfg: dict) -> Non table.add_column("Capture", style="cyan", no_wrap=True) table.add_column("Dir", style="dim") table.add_column("Pkts", justify="right", style="dim") - table.add_column("Eff.Time (s)", justify="right") - table.add_column("Throughput", justify="right") + table.add_column("Delivery%", justify="right") + table.add_column("RTT p50/p95/jit (ms)", justify="right") table.add_column("Overhead", justify="right") table.add_column("Size p5/p50/p95 (B)", justify="right") table.add_column("Burst / Reg / Eff", justify="right") @@ -176,8 +188,6 @@ def _print_summary(all_stats: dict[str, dict], metadata: dict, cfg: dict) -> Non ip50 = f"{iat.get('p50', 0):.2f}" if iat else "β€”" ip95 = f"{iat.get('p95', 0):.2f}" if iat else "β€”" - byte_count = s.get("byte_count", 0) - if chaos: delivery_pct: float | None = metadata.get(proto_key, {}).get("delivery_pct") ent_str = ( @@ -198,24 +208,6 @@ def _print_summary(all_stats: dict[str, dict], metadata: dict, cfg: dict) -> Non ) else: meta = metadata.get(proto_key, {}) - if meta.get("transfer_time_s") is not None: - eff_s = float(meta["transfer_time_s"]) - elif meta.get("recv_time_s") is not None: - eff_s = float(meta["recv_time_s"]) - elif meta.get("effective_time_s") is not None: - eff_s = float(meta["effective_time_s"]) - else: - t_s = s.get("transmission_time_s", 0) - eff_s = max(t_s - injected_delay_s, 0.0) if injected_delay_s > 0 else t_s - transfer_bytes_meta = meta.get("transfer_bytes") - if eff_s > 0 and direction == "c2s" and transfer_bytes_meta: - mbps = transfer_bytes_meta * 8 / eff_s / 1_000_000 - throughput = f"{mbps:.1f} Mbps" - elif eff_s > 0 and direction == "c2s" and byte_count > 0: - mbps = byte_count * 8 / eff_s / 1_000_000 - throughput = f"{mbps:.1f} Mbps" - else: - throughput = "[dim]β€”[/dim]" overhead = _fmt_pct(s.get("overhead_ratio")) if direction == "all": burst = s.get("burstiness") @@ -231,8 +223,8 @@ def _print_summary(all_stats: dict[str, dict], metadata: dict, cfg: dict) -> Non name if first else "", direction, str(s.get("packet_count", 0)), - f"{eff_s:.1f}" if eff_s > 0 else "[dim]β€”[/dim]", - throughput, + _fmt_delivery(meta.get("delivery_pct")) if first else "", + _fmt_rtt(meta) if first else "", overhead, f"{p5} / {p50} / {p95}" if ps else "[dim]β€”[/dim]", fingerprint, diff --git a/evaluation/src/typhoon_eval/shared/docker_utils.py b/evaluation/src/typhoon_eval/shared/docker_utils.py index 53ca602d..8d50996f 100644 --- a/evaluation/src/typhoon_eval/shared/docker_utils.py +++ b/evaluation/src/typhoon_eval/shared/docker_utils.py @@ -29,6 +29,8 @@ COMPOSE_DIR = Path(__file__).parent.parent.parent.parent / "compose" BASE_COMPOSE = COMPOSE_DIR / "docker-compose.yml" +TEARDOWN_GRACE_S = 15 +BUILDKIT_ENV = {**environ, "DOCKER_BUILDKIT": "1", "COMPOSE_DOCKER_CLI_BUILD": "1"} def _project_name(protocol_name: str) -> str: @@ -65,50 +67,47 @@ def _overlay_env(extra: dict[str, str]) -> Generator[None, None, None]: environ[k] = v -def _parse_delivery(dc: DockerClient, protocol_name: str) -> float | None: - """Extract the delivery percentage from the server container's logs.""" - server_name = f"{_project_name(protocol_name)}-server-1" - try: - logs: str = dc.container.logs(server_name) - for line in reversed(logs.splitlines()): - m = search(r"\((\d+(?:\.\d+)?)%\)", line) - if m: - return float(m.group(1)) - except Exception: - pass - return None +# Endpoint values reported by the sender (client log) and sink (server log). +_CLIENT_FIELDS = { + "rtt_min_ms": (r"rtt_min_ms=([\d.]+)", float), + "rtt_p50_ms": (r"rtt_p50_ms=([\d.]+)", float), + "rtt_p95_ms": (r"rtt_p95_ms=([\d.]+)", float), + "rtt_p99_ms": (r"rtt_p99_ms=([\d.]+)", float), + "rtt_jitter_ms": (r"rtt_jitter_ms=([\d.]+)", float), + "roundtrip_delivery_pct": (r"roundtrip_delivery_pct=([\d.]+)", float), +} +_SERVER_FIELDS = { + "delivery_pct": (r"\((\d+(?:\.\d+)?)%\)", float), + "received_packets": (r"received (\d+)/\d+ packets", int), +} -def _parse_timing(dc: DockerClient, protocol_name: str) -> tuple[float | None, float | None]: - """ - Extract transfer_time_s from the client log and recv_time_s from the server log. - Both are printed as 'transfer_time_s=' / 'recv_time_s='. - """ - client_name = f"{_project_name(protocol_name)}-client-1" - server_name = f"{_project_name(protocol_name)}-server-1" +def _parse_endpoint_stats(dc: DockerClient, protocol_name: str) -> dict: + """Extract all endpoint-reported values from the client + server logs. - transfer_time_s: float | None = None - recv_time_s: float | None = None - - try: - logs: str = dc.container.logs(client_name) - for line in logs.splitlines(): - m = search(r"transfer_time_s=([\d.]+)", line) - if m: - transfer_time_s = float(m.group(1)) - except Exception: - pass + Returns a dict with the sink's delivery_pct + received_packets and the + client's per-packet rtt_* / roundtrip_delivery_pct. Missing keys are None. + The last match on each line-scan wins. + """ + stats: dict = dict.fromkeys( + ("delivery_pct", "received_packets", "rtt_min_ms", "rtt_p50_ms", + "rtt_p95_ms", "rtt_p99_ms", "rtt_jitter_ms", "roundtrip_delivery_pct") + ) - try: - logs = dc.container.logs(server_name) + def scan(name: str, fields: dict) -> None: + try: + logs: str = dc.container.logs(name) + except Exception: + return for line in logs.splitlines(): - m = search(r"recv_time_s=([\d.]+)", line) - if m: - recv_time_s = float(m.group(1)) - except Exception: - pass + for key, (rx, cast) in fields.items(): + m = search(rx, line) + if m: + stats[key] = cast(m.group(1)) - return transfer_time_s, recv_time_s + scan(f"{_project_name(protocol_name)}-client-1", _CLIENT_FIELDS) + scan(f"{_project_name(protocol_name)}-server-1", _SERVER_FIELDS) + return stats def _make_client(protocol_name: str, env_file: Path, chaos: bool) -> DockerClient: @@ -152,17 +151,16 @@ def _docker_op(fn: Callable[[], None], timeout_s: int = 60) -> None: t.join(timeout=timeout_s) -def compose_up(protocol_name: str, env_file: Path, extra_env: dict[str, str], chaos: bool, timeout: int, log_dir: Path | None = None) -> tuple[bool, float | None, float | None, float | None]: +def compose_up(protocol_name: str, env_file: Path, extra_env: dict[str, str], chaos: bool, timeout: int, log_dir: Path | None = None) -> tuple[bool, dict]: """ Run `docker compose up` for a protocol capture. - Non-chaos: blocks until the client container exits or timeout fires, then - tears down. Returns True iff the client exited with code 0. - - Chaos: starts the stack detached so that the client can exit while the - chaos service keeps draining the netem queue to the server. Polls until - the server exits (it finishes or hits idle-timeout), then tears down. - Returns True iff the client exited with code 0. + Starts the stack detached (both clean and chaos) and waits for the SERVER + (sink) container to exit β€” it is authoritative for delivery and finishes on + target / DONE / idle-timeout. The client may exit much earlier (an unpaced + UDP sender empties into the tunnel tx-queue and returns at once), so the + server, not the client, is what we wait on. Returns True iff the server + exited with code 0; timeout fires the same teardown. Ctrl-C (KeyboardInterrupt) is propagated to the caller; cleanup (stop + down) still runs via the finally block so containers and networks are @@ -180,52 +178,46 @@ def compose_up(protocol_name: str, env_file: Path, extra_env: dict[str, str], ch _docker_op(lambda: dc.compose.down(volumes=True, remove_orphans=True, quiet=True)) try: - if not chaos: - def _run_up() -> None: - with suppress(DockerException): - dc.compose.up(abort_on_container_exit=True, no_build=True, quiet=True) + # Run detached and wait for the SERVER (echo sink) to finish: it is + # authoritative for delivery and exits on count / idle-timeout. + try: + dc.compose.up(detach=True, no_build=True, quiet=True) + except DockerException: + timed_out = True + + if not timed_out: + server_name = f"{_project_name(protocol_name)}-server-1" - _up_thread = Thread(target=_run_up, daemon=True) + def _wait_server() -> None: + with suppress(Exception): + dc.container.wait(server_name) + + _up_thread = Thread(target=_wait_server, daemon=True) _up_thread.start() _up_thread.join(timeout=timeout) if _up_thread.is_alive(): timed_out = True - _docker_op(lambda: dc.compose.stop(), timeout_s=30) - _up_thread.join(timeout=30) - - else: - try: - dc.compose.up(detach=True, no_build=True, quiet=True) - except DockerException: - timed_out = True + else: + client_name = f"{_project_name(protocol_name)}-client-1" - if not timed_out: - server_name = f"{_project_name(protocol_name)}-server-1" - - def _wait_server() -> None: + def _wait_client() -> None: with suppress(Exception): - dc.container.wait(server_name) - - _up_thread = Thread(target=_wait_server, daemon=True) - _up_thread.start() - _up_thread.join(timeout=timeout) + dc.container.wait(client_name) - if _up_thread.is_alive(): - timed_out = True + _client_thread = Thread(target=_wait_client, daemon=True) + _client_thread.start() + _client_thread.join(timeout=TEARDOWN_GRACE_S) - _docker_op(lambda: dc.compose.stop(), timeout_s=30) - # Brief pause so tcpdump flushes its write buffer before down. - sleep(2) + _docker_op(lambda: dc.compose.stop(), timeout_s=30) + # Brief pause so tcpdump flushes its write buffer before down. + sleep(2) if not timed_out: - # In chaos mode the client is killed by SIGTERM after the server exits - # naturally, so its exit code is 143 (not meaningful). Use server exit. - target = "server" if chaos else "client" try: for container in dc.compose.ps(all=True): service = container.config.labels.get("com.docker.compose.service", "") - if service == target: + if service == "server": success = container.state.exit_code == 0 break except Exception: @@ -235,10 +227,9 @@ def _wait_server() -> None: _docker_op(lambda: dc.compose.stop(), timeout_s=30) if _up_thread is not None: _up_thread.join(timeout=10) - delivery_pct = _parse_delivery(dc, protocol_name) - transfer_time_s, recv_time_s = _parse_timing(dc, protocol_name) + stats = _parse_endpoint_stats(dc, protocol_name) if log_dir is not None: _save_logs(dc, protocol_name, log_dir) _docker_op(lambda: dc.compose.down(volumes=True, remove_orphans=True, quiet=True), timeout_s=60) - return success, delivery_pct, transfer_time_s, recv_time_s + return success, stats diff --git a/evaluation/src/typhoon_eval/shared/orchestrator.py b/evaluation/src/typhoon_eval/shared/orchestrator.py index 9e0ee004..1b734827 100644 --- a/evaluation/src/typhoon_eval/shared/orchestrator.py +++ b/evaluation/src/typhoon_eval/shared/orchestrator.py @@ -25,13 +25,16 @@ RESULTS_DIR = Path(__file__).parent.parent.parent.parent / "results" -# Minimum pcap size to consider non-empty / capture-worthy (bytes). Below this -# the file is essentially just the libpcap global header (24 B) plus a stub. MIN_PCAP_SIZE_B = 100 -# Delivery-rate colour bands β€” mirrors `shared/analysis.py`. DELIVERY_GREEN_PCT = 99.9 DELIVERY_YELLOW_PCT = 80.0 ENV_DIR = COMPOSE_DIR / "env" +TYPHOON_DRAIN_CHANNEL_CAPACITY = 81_920 +IDLE_TIMEOUT_S = 30 +LATENCY_COUNT = 500 +LATENCY_INTERVAL_MS = 20 +LATENCY_SIZE = 256 +LATENCY_RECV_TIMEOUT_MS = 5000 def _run_one( @@ -43,14 +46,14 @@ def _run_one( bw_mbps: float, captures_dir: Path, log_dir: Path, -) -> tuple[bool, str, float | None, float | None, float | None]: +) -> tuple[bool, str, dict]: """ Run a single protocol capture. - Returns (success, error_message, delivery_pct, transfer_time_s, recv_time_s). + Returns (success, error_message, endpoint_stats) β€” see _parse_endpoint_stats. """ env_file = ENV_DIR / f".env.{protocol.name}" if not env_file.exists(): - return False, f"missing env file: {env_file}", None, None, None + return False, f"missing env file: {env_file}", {} suffix = "_chaos" if chaos else "" pumba_target = f"re2:typhoon-eval-{protocol.name.replace('_', '-')}-client-1" @@ -72,10 +75,16 @@ def _run_one( "PUMBA_TARGET": pumba_target, "CHAOS_LOSS_PCT": str(loss_pct), "CHAOS_BW_MBPS": str(bw_mbps), + "TYPHOON_DRAIN_CHANNEL_CAPACITY": str(TYPHOON_DRAIN_CHANNEL_CAPACITY), + "IDLE_TIMEOUT_S": str(IDLE_TIMEOUT_S), + "LAT_COUNT": str(LATENCY_COUNT), + "LAT_INTERVAL_MS": str(LATENCY_INTERVAL_MS), + "LAT_SIZE": str(LATENCY_SIZE), + "LAT_RECV_TIMEOUT_MS": str(LATENCY_RECV_TIMEOUT_MS), } extra_env.update(profile_env) - success, delivery_pct, transfer_time_s, recv_time_s = compose_up( + success, stats = compose_up( protocol_name=protocol.name, env_file=env_file, extra_env=extra_env, @@ -87,11 +96,11 @@ def _run_one( pcap = captures_dir / f"{protocol.name}{suffix}.pcap" if success: if not pcap.exists(): - return False, "observer did not write pcap", None, None, None + return False, "observer did not write pcap", stats if pcap.stat().st_size < MIN_PCAP_SIZE_B: - return False, f"pcap is empty ({pcap.stat().st_size} bytes)", None, None, None + return False, f"pcap is empty ({pcap.stat().st_size} bytes)", stats - return success, "" if success else "non-zero exit or timeout", delivery_pct, transfer_time_s, recv_time_s + return success, "" if success else "non-zero exit or timeout", stats @command(context_settings={"help_option_names": ["-h", "--help"]}) @@ -106,9 +115,9 @@ def _run_one( @option("--chaos", is_flag=True, default=False, help="Enable pumba chaos overlay (latency + jitter).") @option( "--timeout", - default=900, + default=300, show_default=True, - help="Per-protocol timeout in seconds before the run is killed. Sized to fit `bulk_upload.duration_s` (600 s) plus setup/teardown headroom under chaos.", + help="Per-protocol timeout in seconds before the run is killed. The latency ping is short in clean mode; under chaos its sequential RTTs stretch it, so leave headroom.", ) @option( "--profile", @@ -158,13 +167,12 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr rng = Random(seed) profile_obj = PROFILES[profile] profile_env = profile_to_env(profile_obj, rng) - transfer_bytes = int(profile_env["PROFILE_BYTES_C2S"]) console.print(f"\n[bold]TYPHOON evaluation{chaos_note}[/bold]") console.print(f" Protocols : {', '.join(p.name for p in protocols)}") console.print(f" Timeout : {timeout}s per run (env file may override per protocol)") + console.print(f" Latency : {LATENCY_COUNT} Γ— {LATENCY_SIZE} B probes @ {LATENCY_INTERVAL_MS} ms") console.print(f" Profile : {profile} ({profile_obj.description})") - console.print(f" Transfer : c2s={int(profile_env['PROFILE_BYTES_C2S']) / 1_048_576:.1f} MB / s2c={int(profile_env['PROFILE_BYTES_S2C']) / 1_048_576:.1f} MB") if chaos and (loss_pct > 0 or bw_mbps > 0): console.print(f" Chaos : loss={loss_pct}% bw={bw_mbps or 'unlimited'} Mbps") console.print() @@ -174,6 +182,7 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr captures_dir.mkdir(parents=True, exist_ok=True) console.print(f" Run ID : [dim]{run_id}[/dim]\n") + transfer_bytes = LATENCY_COUNT * LATENCY_SIZE config: dict = { "run_id": run_id, "started_at": datetime.now(UTC).isoformat(), @@ -185,6 +194,11 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr "transfer_bytes": transfer_bytes, "loss_pct": loss_pct, "bw_mbps": bw_mbps, + "latency": { + "count": LATENCY_COUNT, + "interval_ms": LATENCY_INTERVAL_MS, + "size_b": LATENCY_SIZE, + }, } (captures_dir / "config.json").write_text(dumps(config, indent=2)) @@ -205,7 +219,7 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr ) started_at = datetime.now(UTC) - success, error, delivery_pct, transfer_time_s, recv_time_s = _run_one( + success, error, stats = _run_one( protocol=protocol, chaos=chaos, timeout=timeout, @@ -221,12 +235,10 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr "success": success, "error": error, "elapsed_s": round(elapsed, 1), - "transfer_time_s": round(transfer_time_s, 3) if transfer_time_s is not None else None, - "recv_time_s": round(recv_time_s, 3) if recv_time_s is not None else None, "chaos": chaos, "transfer_bytes": transfer_bytes, - "delivery_pct": delivery_pct, "timestamp": datetime.now(UTC).isoformat(), + **stats, } icon = "[green]βœ“[/green]" if success else "[red]βœ—[/red]" @@ -266,7 +278,7 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr table.add_column("Status") table.add_column("Delivery", justify="right") table.add_column("Elapsed (s)", justify="right", style="dim") - table.add_column("Eff.Time (s)", justify="right") + table.add_column("RTT p50/p95 (ms)", justify="right") ok = 0 for p in protocols: @@ -281,9 +293,9 @@ def main(run_all: bool, protocol_name: str | None, chaos: bool, timeout: int, pr delivery = f"[yellow]{pct:.1f}%[/yellow]" else: delivery = f"[red]{pct:.1f}%[/red]" - eff = r.get("transfer_time_s") - table.add_row(p.name, p.transport, status, delivery, str(r["elapsed_s"]), - f"{eff:.3f}" if eff is not None else "[dim]β€”[/dim]") + p50, p95 = r.get("rtt_p50_ms"), r.get("rtt_p95_ms") + perf = f"{p50:.2f} / {p95:.2f}" if p50 is not None and p95 is not None else "[dim]β€”[/dim]" + table.add_row(p.name, p.transport, status, delivery, str(r["elapsed_s"]), perf) if r["success"]: ok += 1 diff --git a/evaluation/src/typhoon_eval/shared/pcap_stats.py b/evaluation/src/typhoon_eval/shared/pcap_stats.py index dd699cd9..7bc2e1e7 100644 --- a/evaluation/src/typhoon_eval/shared/pcap_stats.py +++ b/evaluation/src/typhoon_eval/shared/pcap_stats.py @@ -208,6 +208,10 @@ def analyze_pcap( Each value is a dict of computed metrics (empty dict if no packets in that direction). transfer_bytes is used only for the overhead_ratio field. handshake_sniffer overrides the default time-window handshake detection. + + This produces only the packet-shape / entropy statistics used for + detectability; the operational metrics (delivery, per-packet RTT) come from + the endpoints, not the pcap. """ c2s, s2c = parse_pcap(path) diff --git a/evaluation/src/typhoon_eval/shared/profiles.py b/evaluation/src/typhoon_eval/shared/profiles.py index 4914f9dc..402564c5 100644 --- a/evaluation/src/typhoon_eval/shared/profiles.py +++ b/evaluation/src/typhoon_eval/shared/profiles.py @@ -194,7 +194,7 @@ class Profile: description="Operational-comparison default for `poe capture --all`", chunk_c2s=IntRange(1100, 1200), chunk_s2c=IntRange(0, 0), - iat_c2s_ms=Range(4.0, 4.0), + iat_c2s_ms=Range(0.0, 0.0), iat_s2c_ms=Range(0.0, 0.0), bytes_c2s=IntRange(10_000_000, 10_000_000), bytes_s2c=IntRange(0, 0), @@ -463,9 +463,9 @@ class BackgroundProfile: # Jitter is sampled relative to the per-run latency (see corpus._sample_chaos); # this Range is the *fraction* of latency used for the jitter ceiling. CHAOS_JITTER_FRACTION: Final[Range] = Range(0.0, 0.5) -# Loss capped at 1.0 % β€” aioquic destabilises under sustained higher loss. -# Real residential / mobile loss rates (Paxson 1999; Bauer 2009) sit in -# 0.05–1 % range, so this also matches measured wild-internet rates. +# Loss capped at 1.0 %. Real residential / mobile loss rates (Paxson 1999; +# Bauer 2009) sit in the 0.05–1 % range, so this matches measured wild-internet +# rates. CHAOS_LOSS_PCT: Final[Range] = Range(0.0, 1.0) # Duplicate rate. Bellardo & Savage IMC 2005 measure 0.001–0.1 % duplicates # on real transit links; we extend slightly to 0.2 % so chaos can probe the diff --git a/evaluation/src/typhoon_eval/shared/protocols.py b/evaluation/src/typhoon_eval/shared/protocols.py index e9090b22..466e9aa3 100644 --- a/evaluation/src/typhoon_eval/shared/protocols.py +++ b/evaluation/src/typhoon_eval/shared/protocols.py @@ -91,8 +91,8 @@ def _img(slug: str, role: str) -> str: Protocol("obfs4_iat","OBFS4 (IAT=1)", "tcp", _img("obfs4", "client"), _img("obfs4", "server"), _SN_OBFS4), Protocol("obfs4_iat2","OBFS4 (IAT=2)", "tcp", _img("obfs4", "client"), _img("obfs4", "server"), _SN_OBFS4), Protocol("amneziawg","AmneziaWG", "udp", _img("amneziawg", "client"), _img("amneziawg", "server"), _SN_WIREGUARD), - Protocol("hysteria2","Hysteria2+Brutal", "udp", _img("hysteria2", "client"), _img("hysteria2", "server"), _SN_HYSTERIA2), - Protocol("shadowsocks","Shadowsocks", "tcp/udp", _img("shadowsocks", "client"), _img("shadowsocks", "server"), _SN_SHADOWSOCKS), + Protocol("hysteria2","Hysteria2 (BBR)", "udp", _img("hysteria2", "client"), _img("hysteria2", "server"), _SN_HYSTERIA2), + Protocol("shadowsocks","Shadowsocks", "tcp", _img("shadowsocks", "client"), _img("shadowsocks", "server"), _SN_SHADOWSOCKS), Protocol("tor", "Tor", "tcp", _img("tor", "client"), _img("tor", "server"), _SN_TOR), Protocol("vless_reality","VLESS REALITY","tcp", _img("vless-reality", "client"), _img("vless-reality", "server"), _SN_VLESS), Protocol("openvpn", "OpenVPN", "udp", _img("openvpn", "client"), _img("openvpn", "server"), _SN_OPENVPN), diff --git a/evaluation/transport/Cargo.toml b/evaluation/transport/Cargo.toml new file mode 100644 index 00000000..b55fc783 --- /dev/null +++ b/evaluation/transport/Cargo.toml @@ -0,0 +1,50 @@ +[package] +name = "eval-transport" +version = "0.1.0" +edition = "2021" + +[dependencies] +rand = "0.8" +socks = "0.3" +libc = "0.2" +rustls = { version = "0.23", default-features = false, features = ["ring", "std"] } +rcgen = { version = "0.13", default-features = false, features = ["ring", "pem"] } + +[lib] +path = "src/lib.rs" + +[[bin]] +name = "tcp-sender" +path = "src/bin/tcp_sender.rs" + +[[bin]] +name = "tcp-sink" +path = "src/bin/tcp_sink.rs" + +[[bin]] +name = "udp-sender" +path = "src/bin/udp_sender.rs" + +[[bin]] +name = "udp-sink" +path = "src/bin/udp_sink.rs" + +[[bin]] +name = "socks5-sender" +path = "src/bin/socks5_sender.rs" + +[[bin]] +name = "tls-sender" +path = "src/bin/tls_sender.rs" + +[[bin]] +name = "tls-sink" +path = "src/bin/tls_sink.rs" + +[[bin]] +name = "tor-sender" +path = "src/bin/tor_sender.rs" + +[[bin]] +name = "tor-sink" +path = "src/bin/tor_sink.rs" diff --git a/evaluation/transport/Dockerfile b/evaluation/transport/Dockerfile new file mode 100644 index 00000000..10f73a14 --- /dev/null +++ b/evaluation/transport/Dockerfile @@ -0,0 +1,19 @@ +# Shared builder image for the eval transport binaries. Built ONCE (compose +# `transport` service); every protocol image then COPY --from=transport the +# binaries it needs, so the Rust crate compiles once and doesn't bloat the +# per-protocol build cache. +FROM rust:1-bookworm AS build +WORKDIR /build/transport +COPY . ./ +RUN cargo build --release + +FROM debian:bookworm-slim +COPY --from=build /build/transport/target/release/tcp-sender /usr/local/bin/tcp-sender +COPY --from=build /build/transport/target/release/tcp-sink /usr/local/bin/tcp-sink +COPY --from=build /build/transport/target/release/udp-sender /usr/local/bin/udp-sender +COPY --from=build /build/transport/target/release/udp-sink /usr/local/bin/udp-sink +COPY --from=build /build/transport/target/release/socks5-sender /usr/local/bin/socks5-sender +COPY --from=build /build/transport/target/release/tls-sender /usr/local/bin/tls-sender +COPY --from=build /build/transport/target/release/tls-sink /usr/local/bin/tls-sink +COPY --from=build /build/transport/target/release/tor-sender /usr/local/bin/tor-sender +COPY --from=build /build/transport/target/release/tor-sink /usr/local/bin/tor-sink diff --git a/evaluation/transport/src/bin/socks5_sender.rs b/evaluation/transport/src/bin/socks5_sender.rs new file mode 100644 index 00000000..87c1796e --- /dev/null +++ b/evaluation/transport/src/bin/socks5_sender.rs @@ -0,0 +1,71 @@ +//! SOCKS5 client β€” connects to SERVER_HOST:SERVER_PORT through a local SOCKS5 +//! proxy (Shadowsocks / Tor / VLESS / obfs4 / Hysteria2) and pings small +//! equal-sized messages the sink echoes, timing per-packet round-trips through +//! the proxy (see `eval_transport::latency`). +//! +//! obfs4's pluggable-transport args go in SOCKS5_USERNAME (password is a NUL). + +use std::env::var; +use std::io::{Read, Write}; +use std::process::exit; +use std::thread::sleep; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::latency::{self, Config}; +use socks::Socks5Stream; + +fn main() { + add_route("172.21.0.0/24"); + let server_host = var("SERVER_HOST").expect("SERVER_HOST not set"); + let server_port: u16 = var("SERVER_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(9000); + let socks_host = var("SOCKS5_HOST").unwrap_or_else(|_| "127.0.0.1".to_string()); + let socks_port: u16 = var("SOCKS5_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(1080); + let user = var("SOCKS5_USERNAME").ok(); + + let proxy = (socks_host.as_str(), socks_port); + let target = (server_host.as_str(), server_port); + + let mut stream = None; + for attempt in 0..30 { + let res = match &user { + Some(u) => Socks5Stream::connect_with_password(proxy, target, u, "\x00"), + None => Socks5Stream::connect(proxy, target), + }; + match res { + Ok(s) => { + stream = Some(s); + break; + } + Err(e) => { + println!("attempt {}/30: {e}", attempt + 1); + sleep(Duration::from_secs(2)); + } + } + } + let Some(mut stream) = stream else { + eprintln!("failed to connect via SOCKS5"); + exit(1); + }; + + let cfg = Config::from_env(); + stream + .get_ref() + .set_read_timeout(Some(cfg.recv_timeout)) + .ok(); + let mut rbuf = vec![0u8; cfg.size]; + latency::ping_loop( + |m| { + stream.write_all(m)?; + stream.read_exact(&mut rbuf)?; + Ok(Some(rbuf.clone())) + }, + &cfg, + ); +} diff --git a/evaluation/transport/src/bin/tcp_sender.rs b/evaluation/transport/src/bin/tcp_sender.rs new file mode 100644 index 00000000..aea49b09 --- /dev/null +++ b/evaluation/transport/src/bin/tcp_sender.rs @@ -0,0 +1,51 @@ +//! TCP client β€” a spaced ping of small equal-sized messages echoed by the sink, +//! measuring per-packet round-trip time (see `eval_transport::latency`). Over a +//! reliable stream every echo returns, so delivery stays 100% and loss shows up +//! as higher RTT (retransmit) β€” the honest TCP behaviour. + +use std::env::var; +use std::io::{Read, Write}; +use std::net::TcpStream; +use std::process::exit; +use std::thread::sleep; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::latency::{self, Config}; + +fn main() { + add_route("172.21.0.0/24"); + let host = var("SERVER_HOST").expect("SERVER_HOST not set"); + let port: u16 = var("LISTEN_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(9000); + + let mut stream = None; + for _ in 0..30 { + match TcpStream::connect((host.as_str(), port)) { + Ok(s) => { + stream = Some(s); + break; + } + Err(_) => sleep(Duration::from_secs(1)), + } + } + let Some(mut stream) = stream else { + eprintln!("failed to connect to {host}:{port}"); + exit(1); + }; + stream.set_nodelay(true).ok(); + + let cfg = Config::from_env(); + stream.set_read_timeout(Some(cfg.recv_timeout)).ok(); + let mut rbuf = vec![0u8; cfg.size]; + latency::ping_loop( + |m| { + stream.write_all(m)?; + stream.read_exact(&mut rbuf)?; // reliable: echo always arrives, or Err on close + Ok(Some(rbuf.clone())) + }, + &cfg, + ); +} diff --git a/evaluation/transport/src/bin/tcp_sink.rs b/evaluation/transport/src/bin/tcp_sink.rs new file mode 100644 index 00000000..75dfde59 --- /dev/null +++ b/evaluation/transport/src/bin/tcp_sink.rs @@ -0,0 +1,43 @@ +//! TCP sink β€” an echo server: reads each fixed-size probe and writes it straight +//! back (see `eval_transport::latency`). Reports one-way (c2s) delivery. + +use std::env::var; +use std::io::{Read, Write}; +use std::net::TcpListener; +use std::process::exit; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::idle_timeout_s; +use eval_transport::latency::{self, Config}; + +fn main() { + add_route("172.20.0.0/24"); + // Honour LISTEN_PORT so a proxy fronting the sink can own :9000 and forward + // to the sink on another port (e.g. obfs4proxy β†’ ORPORT 9001). + let port: u16 = var("LISTEN_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(9000); + + let listener = TcpListener::bind(("0.0.0.0", port)).expect("bind"); + println!("TCP sink ready on :{port}"); + + let (mut conn, _) = listener.accept().expect("accept"); + conn.set_read_timeout(Some(Duration::from_secs(idle_timeout_s()))) + .ok(); + + let cfg = Config::from_env(); + let mut rbuf = vec![0u8; cfg.size]; + latency::echo_loop( + || match conn.read_exact(&mut rbuf) { + Ok(()) => { + conn.write_all(&rbuf)?; + Ok(true) + } + Err(_) => Ok(false), // EOF / idle / reset ends the run + }, + &cfg, + ); + exit(0); +} diff --git a/evaluation/transport/src/bin/tls_sender.rs b/evaluation/transport/src/bin/tls_sender.rs new file mode 100644 index 00000000..45a8a946 --- /dev/null +++ b/evaluation/transport/src/bin/tls_sender.rs @@ -0,0 +1,70 @@ +//! TLS 1.3 client β€” a spaced ping of small equal-sized messages echoed by the +//! sink, timing per-packet round-trips over the TLS session +//! (see `eval_transport::latency`). + +use std::env::var; +use std::io::{Read, Write}; +use std::net::TcpStream; +use std::path::Path; +use std::process::exit; +use std::thread::sleep; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::latency::{self, Config}; +use eval_transport::tls::{client_config, install_provider}; +use rustls::pki_types::ServerName; +use rustls::{ClientConnection, StreamOwned}; + +const CERT_PATH: &str = "/keys/tls_cert.pem"; + +fn main() { + install_provider(); + add_route("172.21.0.0/24"); + let host = var("SERVER_HOST").expect("SERVER_HOST not set"); + let port: u16 = var("LISTEN_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(9000); + + for _ in 0..30 { + if Path::new(CERT_PATH).exists() { + break; + } + sleep(Duration::from_secs(1)); + } + + let config = client_config(); + + let mut sock = None; + for _ in 0..30 { + match TcpStream::connect((host.as_str(), port)) { + Ok(s) => { + sock = Some(s); + break; + } + Err(_) => sleep(Duration::from_secs(1)), + } + } + let Some(sock) = sock else { + eprintln!("failed to connect to {host}:{port}"); + exit(1); + }; + sock.set_nodelay(true).ok(); + + let cfg = Config::from_env(); + sock.set_read_timeout(Some(cfg.recv_timeout)).ok(); + let server_name = ServerName::try_from("tls-eval").unwrap(); + let conn = ClientConnection::new(config, server_name).expect("client conn"); + let mut tls = StreamOwned::new(conn, sock); + + let mut rbuf = vec![0u8; cfg.size]; + latency::ping_loop( + |m| { + tls.write_all(m)?; + tls.read_exact(&mut rbuf)?; + Ok(Some(rbuf.clone())) + }, + &cfg, + ); +} diff --git a/evaluation/transport/src/bin/tls_sink.rs b/evaluation/transport/src/bin/tls_sink.rs new file mode 100644 index 00000000..6ddf8221 --- /dev/null +++ b/evaluation/transport/src/bin/tls_sink.rs @@ -0,0 +1,48 @@ +//! TLS 1.3 sink β€” self-signed cert to /keys/tls_cert.pem (client readiness gate), +//! accepts one TLS connection and echoes each fixed-size probe +//! (see `eval_transport::latency`). Reports one-way (c2s) delivery. + +use std::fs::write; +use std::io::{Read, Write}; +use std::net::TcpListener; +use std::process::exit; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::idle_timeout_s; +use eval_transport::latency::{self, Config}; +use eval_transport::tls::{install_provider, server_config}; +use rustls::{ServerConnection, StreamOwned}; + +const PORT: u16 = 9000; +const CERT_PATH: &str = "/keys/tls_cert.pem"; + +fn main() { + install_provider(); + add_route("172.20.0.0/24"); + + let (config, cert_pem) = server_config("tls-eval"); + write(CERT_PATH, cert_pem).expect("write cert"); + + let listener = TcpListener::bind(("0.0.0.0", PORT)).expect("bind"); + println!("TLS sink ready on :{PORT}"); + let (sock, _) = listener.accept().expect("accept"); + sock.set_read_timeout(Some(Duration::from_secs(idle_timeout_s()))) + .ok(); + let conn = ServerConnection::new(config).expect("server conn"); + let mut tls = StreamOwned::new(conn, sock); + + let cfg = Config::from_env(); + let mut rbuf = vec![0u8; cfg.size]; + latency::echo_loop( + || match tls.read_exact(&mut rbuf) { + Ok(()) => { + tls.write_all(&rbuf)?; + Ok(true) + } + Err(_) => Ok(false), + }, + &cfg, + ); + exit(0); +} diff --git a/evaluation/transport/src/bin/tor_sender.rs b/evaluation/transport/src/bin/tor_sender.rs new file mode 100644 index 00000000..e93b5865 --- /dev/null +++ b/evaluation/transport/src/bin/tor_sender.rs @@ -0,0 +1,91 @@ +//! Tor link-protocol-v4 client β€” wraps traffic in fixed 514-byte RELAY cells +//! over TLS 1.3 (the same on-wire shape as a Tor ORPort). Pings small probes, +//! one per cell, that the sink echoes as cells, timing per-packet round-trips +//! (see `eval_transport::latency`). + +use std::env::var; +use std::io::{Read, Write}; +use std::net::TcpStream; +use std::path::Path; +use std::process::exit; +use std::thread::sleep; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::latency::{self, Config}; +use eval_transport::tls::{client_config, install_provider}; +use rand::RngCore; +use rustls::pki_types::ServerName; +use rustls::{ClientConnection, StreamOwned}; + +const PORT: u16 = 9001; +const CELL: usize = 514; +const DATA_PER_CELL: usize = 498; +const CERT_PATH: &str = "/keys/tor_cert.pem"; + +/// Build a 514-byte Tor link-protocol-v4 RELAY_DATA cell around `data` (≀498 B). +fn make_relay_cell(data: &[u8]) -> [u8; CELL] { + let mut cell = [0u8; CELL]; + cell[0..4].copy_from_slice(&1u32.to_be_bytes()); // circid + cell[4] = 3; // CMD_RELAY + cell[5] = 2; // relay_cmd = RELAY_DATA + cell[6..8].copy_from_slice(&[0, 0]); // recognized + cell[8..10].copy_from_slice(&[0, 1]); // stream_id + rand::thread_rng().fill_bytes(&mut cell[10..14]); // digest (random for realism) + cell[14..16].copy_from_slice(&(data.len() as u16).to_be_bytes()); // length + cell[16..16 + data.len()].copy_from_slice(data); // payload, zero-padded + cell +} + +/// Extract the RELAY_DATA payload from a 514-byte cell. +fn cell_data(cell: &[u8]) -> Vec { + let len = (u16::from_be_bytes([cell[14], cell[15]]) as usize).min(DATA_PER_CELL); + cell[16..16 + len].to_vec() +} + +fn main() { + install_provider(); + add_route("172.21.0.0/24"); + let host = var("SERVER_HOST").expect("SERVER_HOST not set"); + + for _ in 0..30 { + if Path::new(CERT_PATH).exists() { + break; + } + sleep(Duration::from_secs(1)); + } + + let config = client_config(); + + let mut sock = None; + for _ in 0..30 { + match TcpStream::connect((host.as_str(), PORT)) { + Ok(s) => { + sock = Some(s); + break; + } + Err(_) => sleep(Duration::from_secs(1)), + } + } + let Some(sock) = sock else { + eprintln!("failed to connect to {host}:{PORT}"); + exit(1); + }; + sock.set_nodelay(true).ok(); + + let cfg = Config::from_env(); + sock.set_read_timeout(Some(cfg.recv_timeout)).ok(); + let server_name = ServerName::try_from("tor-eval").unwrap(); + let conn = ClientConnection::new(config, server_name).expect("client conn"); + let mut tls = StreamOwned::new(conn, sock); + + let mut rbuf = [0u8; CELL]; + latency::ping_loop( + |m| { + tls.write_all(&make_relay_cell(m))?; // probe ≀ 498 B fits one cell + tls.read_exact(&mut rbuf)?; + Ok(Some(cell_data(&rbuf))) + }, + &cfg, + ); +} diff --git a/evaluation/transport/src/bin/tor_sink.rs b/evaluation/transport/src/bin/tor_sink.rs new file mode 100644 index 00000000..6f1c51b9 --- /dev/null +++ b/evaluation/transport/src/bin/tor_sink.rs @@ -0,0 +1,69 @@ +//! Tor OR simulator sink β€” self-signed cert to /keys/tor_cert.pem (client +//! readiness gate), accepts one TLS connection on the conventional ORPort 9001 +//! and echoes each probe as a fresh RELAY cell (see `eval_transport::latency`). +//! Reports one-way (c2s) delivery. + +use std::fs::write; +use std::io::{Read, Write}; +use std::net::TcpListener; +use std::process::exit; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::idle_timeout_s; +use eval_transport::latency::{self, Config}; +use eval_transport::tls::{install_provider, server_config}; +use rand::RngCore; +use rustls::{ServerConnection, StreamOwned}; + +const PORT: u16 = 9001; +const CELL: usize = 514; +const DATA_PER_CELL: usize = 498; +const LENGTH_OFFSET: usize = 14; +const CERT_PATH: &str = "/keys/tor_cert.pem"; + +/// Build a 514-byte RELAY_DATA cell around `data` (≀498 B) β€” mirror of the sender. +fn make_relay_cell(data: &[u8]) -> [u8; CELL] { + let mut cell = [0u8; CELL]; + cell[0..4].copy_from_slice(&1u32.to_be_bytes()); + cell[4] = 3; + cell[5] = 2; + cell[8..10].copy_from_slice(&[0, 1]); + rand::thread_rng().fill_bytes(&mut cell[10..14]); + cell[LENGTH_OFFSET..16].copy_from_slice(&(data.len() as u16).to_be_bytes()); + cell[16..16 + data.len()].copy_from_slice(data); + cell +} + +fn main() { + install_provider(); + add_route("172.20.0.0/24"); + + let (config, cert_pem) = server_config("tor-eval"); + write(CERT_PATH, cert_pem).expect("write cert"); + + let listener = TcpListener::bind(("0.0.0.0", PORT)).expect("bind"); + println!("Tor OR simulator listening on :{PORT}"); + let (sock, _) = listener.accept().expect("accept"); + sock.set_read_timeout(Some(Duration::from_secs(idle_timeout_s()))) + .ok(); + let conn = ServerConnection::new(config).expect("server conn"); + let mut tls = StreamOwned::new(conn, sock); + + let cfg = Config::from_env(); + let mut rbuf = [0u8; CELL]; + latency::echo_loop( + || match tls.read_exact(&mut rbuf) { + Ok(()) => { + let len = (u16::from_be_bytes([rbuf[LENGTH_OFFSET], rbuf[LENGTH_OFFSET + 1]]) + as usize) + .min(DATA_PER_CELL); + tls.write_all(&make_relay_cell(&rbuf[16..16 + len]))?; + Ok(true) + } + Err(_) => Ok(false), + }, + &cfg, + ); + exit(0); +} diff --git a/evaluation/transport/src/bin/udp_sender.rs b/evaluation/transport/src/bin/udp_sender.rs new file mode 100644 index 00000000..8e8bf3a8 --- /dev/null +++ b/evaluation/transport/src/bin/udp_sender.rs @@ -0,0 +1,42 @@ +//! UDP client β€” a spaced ping of small equal-sized datagrams echoed by the sink, +//! measuring per-packet round-trip time in tunnel-like conditions +//! (see `eval_transport::latency`). + +use std::env::var; +use std::io::ErrorKind; +use std::net::UdpSocket; +use std::thread::sleep; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::latency::{self, Config}; + +fn main() { + add_route("172.21.0.0/24"); + let host = var("SERVER_HOST").expect("SERVER_HOST not set"); + let port: u16 = var("LISTEN_PORT") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(9000); + + let sock = UdpSocket::bind("0.0.0.0:0").expect("bind"); + sock.connect((host.as_str(), port)).expect("connect"); + sleep(Duration::from_millis(200)); // let the sink bind before the first probe + + let cfg = Config::from_env(); + sock.set_read_timeout(Some(cfg.recv_timeout)).ok(); + let mut buf = vec![0u8; cfg.size.max(2048)]; + latency::ping_loop( + |m| { + sock.send(m)?; + match sock.recv(&mut buf) { + Ok(n) => Ok(Some(buf[..n].to_vec())), + Err(e) if e.kind() == ErrorKind::WouldBlock || e.kind() == ErrorKind::TimedOut => { + Ok(None) // lost echo β€” keep pinging + } + Err(e) => Err(e), + } + }, + &cfg, + ); +} diff --git a/evaluation/transport/src/bin/udp_sink.rs b/evaluation/transport/src/bin/udp_sink.rs new file mode 100644 index 00000000..5f11b9cc --- /dev/null +++ b/evaluation/transport/src/bin/udp_sink.rs @@ -0,0 +1,53 @@ +//! UDP sink β€” an echo server: reflects each probe back to its sender so the +//! client can time round-trips (see `eval_transport::latency`). Reports one-way +//! (c2s) delivery. + +use std::net::UdpSocket; +use std::os::unix::io::AsRawFd; +use std::process::exit; +use std::time::Duration; + +use eval_transport::add_route; +use eval_transport::idle_timeout_s; +use eval_transport::latency::{self, Config}; + +const PORT: u16 = 9000; +const RCVBUF_BYTES: libc::c_int = 64 * 1024 * 1024; + +fn main() { + add_route("172.20.0.0/24"); + + let sock = UdpSocket::bind(("0.0.0.0", PORT)).expect("bind"); + set_rcvbuf(&sock); + sock.set_read_timeout(Some(Duration::from_secs(idle_timeout_s()))) + .ok(); + println!("UDP sink ready on :{PORT}"); + + let cfg = Config::from_env(); + let mut buf = vec![0u8; cfg.size.max(2048)]; + latency::echo_loop( + || match sock.recv_from(&mut buf) { + Ok((n, from)) => { + sock.send_to(&buf[..n], from)?; + Ok(true) + } + Err(_) => Ok(false), // idle timeout ends the run + }, + &cfg, + ); + exit(0); +} + +/// SO_RCVBUFFORCE bypasses net.core.rmem_max with CAP_NET_ADMIN; fall back to +/// SO_RCVBUF (capped) if that fails. +fn set_rcvbuf(sock: &UdpSocket) { + let fd = sock.as_raw_fd(); + let sz = RCVBUF_BYTES; + let len = std::mem::size_of::() as libc::socklen_t; + let ptr = std::ptr::addr_of!(sz).cast(); + unsafe { + if libc::setsockopt(fd, libc::SOL_SOCKET, libc::SO_RCVBUFFORCE, ptr, len) != 0 { + libc::setsockopt(fd, libc::SOL_SOCKET, libc::SO_RCVBUF, ptr, len); + } + } +} diff --git a/evaluation/transport/src/lib.rs b/evaluation/transport/src/lib.rs new file mode 100644 index 00000000..c269bb26 --- /dev/null +++ b/evaluation/transport/src/lib.rs @@ -0,0 +1,278 @@ +//! Shared helpers for the eval transport binaries: the per-packet latency ping +//! (`latency` module), the cross-endpoint monotonic clock, forward routing, and +//! (in the `tls` module) the rustls plumbing used by the TLS and Tor binaries. +//! Blocking (std) β€” a sequential ping needs no async runtime. + +use std::env::var; + +/// Best-effort forward route to the opposite /24 via the observer tap. +pub fn add_route(default_subnet: &str) { + if let Ok(gw) = var("OBSERVER_GW") { + if !gw.is_empty() { + let subnet = var("FORWARD_SUBNET").unwrap_or_else(|_| default_subnet.to_string()); + let _ = std::process::Command::new("ip") + .args(["route", "add", &subnet, "via", &gw]) + .status(); + } + } +} + +/// Idle timeout (s) after which an echo sink gives up waiting for more probes. +pub fn idle_timeout_s() -> u64 { + var("IDLE_TIMEOUT_S") + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(30) +} + +/// Host-wide monotonic clock in nanoseconds. Docker containers share the kernel +/// clock (no time namespace by default), so client and server readings are +/// directly comparable β€” the basis for the cross-endpoint transfer timings. +pub fn monotonic_ns() -> u128 { + let mut ts = libc::timespec { + tv_sec: 0, + tv_nsec: 0, + }; + unsafe { + libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts); + } + (ts.tv_sec as u128) * 1_000_000_000 + (ts.tv_nsec as u128) +} + +/// Round-trip per-packet latency probe β€” the operational default. A spaced ping +/// of small, equal-sized packets that the sink echoes verbatim, so we measure +/// each protocol's inherent per-packet cost (client processing + wire + server +/// processing, both ways) in tunnel-like conditions, without the bulk pressure +/// that best-effort transports are *designed* to shed. Sequential (one packet +/// outstanding) so it works over any transport, including a single TLS session, +/// with no threading: a lost UDP echo simply times out and counts as loss, while +/// a reliable transport always echoes (its RTT just grows under retransmit). +pub mod latency { + use std::env::var; + use std::io; + use std::thread::sleep; + use std::time::{Duration, Instant}; + + use rand::RngCore; + + use super::monotonic_ns; + + /// Probe header: `seq: u32 BE` + `send_ns: u128 BE`. + pub const HEADER: usize = 20; + + /// Ping parameters: how many probes, their spacing, their fixed size, and + /// how long to wait for one echo before treating it as lost. + pub struct Config { + pub count: u32, + pub interval: Duration, + pub size: usize, + pub recv_timeout: Duration, + } + + impl Config { + /// Read the config from the `LAT_*` env vars (harness-set; defaults match + /// the orchestrator). + pub fn from_env() -> Self { + let count = env_u32("LAT_COUNT", 500); + let interval = Duration::from_secs_f64(env_f64("LAT_INTERVAL_MS", 20.0) / 1000.0); + let size = (env_u32("LAT_SIZE", 256) as usize).max(HEADER); + let recv_timeout = + Duration::from_secs_f64(env_f64("LAT_RECV_TIMEOUT_MS", 5000.0) / 1000.0); + Self { + count, + interval, + size, + recv_timeout, + } + } + } + + /// Build a `size`-byte probe: seq + monotonic send timestamp + random pad + /// (random so payload entropy matches a real encrypted packet). + pub fn pack(seq: u32, send_ns: u128, size: usize, pad: &[u8]) -> Vec { + let mut m = vec![0u8; size]; + m[0..4].copy_from_slice(&seq.to_be_bytes()); + m[4..HEADER].copy_from_slice(&send_ns.to_be_bytes()); + let n = size.saturating_sub(HEADER).min(pad.len()); + m[HEADER..HEADER + n].copy_from_slice(&pad[..n]); + m + } + + /// Recover the send timestamp an echo carries. + pub fn send_ns_of(msg: &[u8]) -> u128 { + let mut b = [0u8; 16]; + b.copy_from_slice(&msg[4..HEADER]); + u128::from_be_bytes(b) + } + + /// Client side: for each probe, `probe(msg)` sends it and returns its echo β€” + /// `Ok(Some(echo))` (RTT recorded), `Ok(None)` for a UDP-style lost echo + /// (miss, keep going), or `Err` for a dead/EOF stream (stop). A single closure + /// (send-then-receive) so it can hold the one `&mut` a stream needs. Prints + /// the `sent`, round-trip delivery, and `rtt_*` contract. + pub fn ping_loop(mut probe: impl FnMut(&[u8]) -> io::Result>>, cfg: &Config) { + let mut pad = vec![0u8; cfg.size]; + rand::thread_rng().fill_bytes(&mut pad); + let mut rtts: Vec = Vec::with_capacity(cfg.count as usize); + + for seq in 0..cfg.count { + let send_ns = monotonic_ns(); + let msg = pack(seq, send_ns, cfg.size, &pad); + let t_send = Instant::now(); + match probe(&msg) { + Ok(Some(echo)) if echo.len() >= HEADER => { + let rtt_ms = (monotonic_ns().saturating_sub(send_ns_of(&echo))) as f64 / 1e6; + rtts.push(rtt_ms); + } + Ok(_) => {} // lost echo (UDP) β€” count as a miss, continue + Err(_) => break, // stream dead / peer gone + } + let elapsed = t_send.elapsed(); + if elapsed < cfg.interval { + sleep(cfg.interval - elapsed); + } + } + + let echoed = rtts.len() as u32; + println!("sent {} packets", cfg.count); + let delivery = echoed as f64 / cfg.count.max(1) as f64 * 100.0; + println!("roundtrip_delivery_pct={delivery:.1}"); + if !rtts.is_empty() { + rtts.sort_by(|a, b| a.partial_cmp(b).unwrap()); + let pct = |p: f64| rtts[(((rtts.len() - 1) as f64) * p).round() as usize]; + let p50 = pct(0.50); + let p95 = pct(0.95); + println!("rtt_min_ms={:.3}", rtts[0]); + println!("rtt_p50_ms={p50:.3}"); + println!("rtt_p95_ms={p95:.3}"); + println!("rtt_p99_ms={:.3}", pct(0.99)); + println!("rtt_jitter_ms={:.3}", p95 - p50); + } + } + + /// Server side: `serve()` receives one probe and echoes it, returning + /// `Ok(true)` (handled), `Ok(false)` (idle/EOF β€” stop), or `Err` (stop). One + /// closure (receive-then-send) for the same `&mut` reason as `ping_loop`. + /// Prints one-way (c2s) delivery in the shared `received N/M packets (P%)` + /// form the harness parses. + pub fn echo_loop(mut serve: impl FnMut() -> io::Result, cfg: &Config) { + let mut received = 0u32; + while received < cfg.count { + match serve() { + Ok(true) => received += 1, + _ => break, // idle / EOF / error + } + } + let delivery = received as f64 / cfg.count.max(1) as f64 * 100.0; + println!("received {received}/{} packets ({delivery:.1}%)", cfg.count); + } + + fn env_u32(key: &str, default: u32) -> u32 { + var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) + } + + fn env_f64(key: &str, default: f64) -> f64 { + var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) + } +} + +/// Shared rustls plumbing for the TLS-based binaries (tls, tor): a self-signed +/// server config and a verification-skipping client config. The eval measures +/// transport behaviour, not a real PKI, so cert trust is intentionally bypassed. +pub mod tls { + use std::sync::Arc; + + use rustls::client::danger::{HandshakeSignatureValid, ServerCertVerified, ServerCertVerifier}; + use rustls::pki_types::{ + CertificateDer, PrivateKeyDer, PrivatePkcs8KeyDer, ServerName, UnixTime, + }; + use rustls::{ClientConfig, DigitallySignedStruct, ServerConfig, SignatureScheme}; + + /// Install the ring crypto provider (idempotent); call once at startup. + pub fn install_provider() { + let _ = rustls::crypto::ring::default_provider().install_default(); + } + + /// Self-signed server config for CN=`name`; returns the config plus the cert + /// PEM to write where the sender's readiness gate looks. + pub fn server_config(name: &str) -> (Arc, String) { + let cert = rcgen::generate_simple_self_signed(vec![name.to_string()]).expect("cert"); + let pem = cert.cert.pem(); + let key = PrivateKeyDer::Pkcs8(PrivatePkcs8KeyDer::from(cert.key_pair.serialize_der())); + let config = ServerConfig::builder() + .with_no_client_auth() + .with_single_cert(vec![cert.cert.der().clone()], key) + .expect("server config"); + (Arc::new(config), pem) + } + + /// Client config that accepts any server certificate (self-signed eval certs). + pub fn client_config() -> Arc { + Arc::new( + ClientConfig::builder() + .dangerous() + .with_custom_certificate_verifier(Arc::new(SkipVerify::new())) + .with_no_client_auth(), + ) + } + + #[derive(Debug)] + struct SkipVerify(Arc); + + impl SkipVerify { + fn new() -> Self { + Self(Arc::new(rustls::crypto::ring::default_provider())) + } + } + + impl ServerCertVerifier for SkipVerify { + fn verify_server_cert( + &self, + _e: &CertificateDer<'_>, + _i: &[CertificateDer<'_>], + _n: &ServerName<'_>, + _o: &[u8], + _t: UnixTime, + ) -> Result { + Ok(ServerCertVerified::assertion()) + } + + fn verify_tls12_signature( + &self, + m: &[u8], + c: &CertificateDer<'_>, + d: &DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls12_signature( + m, + c, + d, + &self.0.signature_verification_algorithms, + ) + } + + fn verify_tls13_signature( + &self, + m: &[u8], + c: &CertificateDer<'_>, + d: &DigitallySignedStruct, + ) -> Result { + rustls::crypto::verify_tls13_signature( + m, + c, + d, + &self.0.signature_verification_algorithms, + ) + } + + fn supported_verify_schemes(&self) -> Vec { + self.0.signature_verification_algorithms.supported_schemes() + } + } +}