sidebar — a private intercom for a few kids

A private hardware voice intercom for a small circle of known children. Each kid has an ESP32-S3 touch-screen unit (ONX2432G028) that rings a friend by name; calls are ring-to-answer and travel through a family-run relay on the internet. Devices connect outbound only — control over TLS WebSockets, audio as AES-256-GCM encrypted UDP — so no household ever opens a port. There are no public rooms, searchable users, recordings, free-text contacts, or strangers; the roster and join code live on the parent-run server, and firmware updates arrive over the air with on-screen consent.

Tap a friend's name and their unit rings. Tap Everyone and they all ring. That is the whole product.

Running this yourself: every name, hostname and address in this repository is a placeholder (alex, sam, riley, relay.example.com). Substitute your own roster and relay host. Read SECURITY.md before putting it in front of children — particularly the section on what the monitoring dashboard deliberately cannot do.

The enclosure in enclosure/ — MainTalkie.stl, MainLid.stl and MainStand.stl — was designed in FreeCAD by Skanda (age 11), measured off the real board with a ruler and vernier callipers, and fitted on the first print. The .scad file beside it is an earlier, separate parametric concept.

Supported environments

• Linux development workstation for build, flash, serial monitoring, tests, and an isolated same-LAN relay.
• Debian/Ubuntu Linux VPS for the production relay.
• ESP-IDF 5.4.4 targeting ESP32-S3.
• Five ONX2432G028 boards (one per child) with Nextion Dual MIC and BOX Speaker.

Windows is not part of the supported workflow. Keep this repository at a Linux path without spaces, for example ~/src/sidebar-intercom.

Start here

Follow these in order:

1. ESP-IDF beginner guide - understand the toolchain, install ESP-IDF 5.4.4, build a vendor demo, compile, flash, monitor, configure local testing, and diagnose common failures.
2. Linux development - canonical commands for tests, same-LAN relay setup, and development builds.
3. Hardware bring-up - prove display/touch, microphone/speaker, and Wi-Fi with official vendor examples before the intercom application.
4. VPS deployment - deploy public WSS through Nginx on port 443 with the relay restricted to loopback.
5. Release checklist - pass every release gate before child use.
6. Security provisioning - only after normal firmware and signed rollback pass, provision Secure Boot/flash/NVS encryption one unit at a time.

The authoritative wire format and security requirements are in the protocol and SECURITY.md.

System architecture

flowchart LR
    subgraph DEV["Linux development workstation"]
        SRC["Repository and Python tests"]
        IDF["ESP-IDF 5.4.4 build, flash, monitor"]
        LOCAL["Local debug relay on LAN :8443"]
        CLIENT["Synthetic test clients and WAV capture"]
        SRC --> IDF
        SRC --> CLIENT
        CLIENT <--> LOCAL
    end

    subgraph HOMES["Children's homes - outbound connections only"]
        S["Kid unit 1<br/>LVGL touch UI<br/>NVS identity/token<br/>PDM mic + I2S speaker"]
        A["Kid unit 2<br/>LVGL touch UI<br/>NVS identity/token<br/>PDM mic + I2S speaker"]
        M["Kid unit 3<br/>LVGL touch UI<br/>NVS identity/token<br/>PDM mic + I2S speaker"]
    end

    subgraph VPS["Linux VPS"]
        NGINX["Nginx :443<br/>Cloudflare-proxied TLS<br/>WSS and HTTPS OTA"]
        RELAY["Python relay<br/>127.0.0.1:8443<br/>auth, presence, Call objects,<br/>control and audio routing"]
        ALLOW["allowlist.json mode 0600<br/>join-code hash and device token hashes"]
        OTA["Signed firmware files<br/>/var/lib/intercom/ota"]
        NGINX <--> RELAY
        RELAY <--> ALLOW
        OTA --> NGINX
    end

    IDF -->|"USB flash and serial monitor"| S
    IDF -->|"USB flash and serial monitor"| A
    IDF -->|"USB flash and serial monitor"| M
    LOCAL <-->|"trusted-LAN ws:// debug only"| S
    LOCAL <-->|"trusted-LAN ws:// debug only"| A
    LOCAL <-->|"trusted-LAN ws:// debug only"| M
    S <-->|"outbound WSS :443 control<br/>+ AES-GCM UDP :8444 audio"| NGINX
    A <-->|"outbound WSS :443 control<br/>+ AES-GCM UDP :8444 audio"| NGINX
    M <-->|"outbound WSS :443 control<br/>+ AES-GCM UDP :8444 audio"| NGINX

Loading

Each board maintains one outbound WSS control connection (through Cloudflare and Nginx to the loopback-only relay) plus an outbound UDP audio flow straight to the relay host — datagrams are AES-256-GCM sealed with per-session keys handed out inside the TLS welcome, so the proxyless path exposes only ciphertext. If UDP is unreachable, audio falls back to the websocket automatically. The relay does not mix or record audio: it forwards each sender's frames tagged with the sender name, and receiving boards keep a jitter buffer per speaker and mix locally.

Calls, presence, and joining a call already in progress

A call is a relay-side Call object with a random call_id and a member set. It is not a public room and is visible only to authenticated family devices. The relay includes active calls in every presence update and in the welcome sent when a device reconnects.

sequenceDiagram
    participant S as Kid 1 unit
    participant R as Relay
    participant A as Kid 2 unit
    participant M as Kid 3 unit

    S->>R: call targets [kid2]
    R->>A: incoming from kid1, call_id C
    Note over A: Rings: "kid1 is calling!"<br/>Answer / No
    A->>R: join call_id C (tapped Answer)
    R->>S: call_started C, peers [kid2]
    R->>A: call_started C, peers [kid1]
    R-->>M: presence calls [{call_id: C, members: [kid1, kid2]}]
    Note over M: Idle screen shows<br/>"kid1 + kid2 - Join"
    M->>R: join call_id C
    R->>S: call_started C, peers [jordan, casey]
    R->>A: call_started C, peers [casey, alex]
    R->>M: call_started C, peers [jordan, alex]
    R-->>S: presence with updated members
    R-->>A: presence with updated members
    R-->>M: presence with updated members
    Note over S,M: All three now exchange audio<br/>through the same Call object

Loading

There are two late-join cases:

1. Online but not selected: the device receives the new presence update, notices a call whose members do not include itself, and displays the Join banner.
2. Offline when the call began: after reconnect and authentication, its welcome contains the still-active call. The same Join banner appears.

An offline or powered-off board cannot be alerted in real time because it has no connection. Its notification is therefore the Join banner shown immediately after it reconnects, provided at least two members are still in the call.

Tapping Join sends {"type":"join","call_id":"C"} — the same verb that accepting a ring uses. The relay verifies the device was invited or the call still has at least two members, adds it, sends the updated peer list to every member, and broadcasts new presence. If the call ended before the tap, the relay returns no_such_call and the stale banner disappears with the next presence update. An invited device rings with Answer/No until someone answers, the caller cancels, or the ring times out (missed-call notice).

Repository map

• relay/ - relay, synthetic audio client, tests, and systemd unit.
• relay/stats.py - metadata-only usage log (names, call ids, timestamps). Never touched by the audio paths.
• relay/monitor.py - parent dashboard: who is online, who is talking, and for how long. A separate process that opens the log read-only, so it cannot eavesdrop; see SECURITY.md.
• firmware/ - ESP-IDF application and board components.
• firmware/main/app_main.c - app lifecycle, Wi-Fi/websocket/audio tasks, and the shared call/roster/OTA state.
• firmware/main/ui.c / ui.h / ui_shared.h - the LVGL screen manager (lobby, menu, clock, missed calls, settings, call/ring screens) split out of app_main.c in v1.3.1; ui.h is the cross-boundary interface.
• firmware/vendor/nextion/ - vendor reference examples. Not redistributed here — download from https://nextion.tech/wiki/onx2432g028/ and unpack there to run the bring-up demos. Every vendor file used is linked in firmware-lineage.md.
• docs/getting-started-esp-idf.md - beginner-oriented Linux/ESP-IDF setup, build, flash, monitor, and troubleshooting guide.
• docs/firmware-lineage.md - vendor provenance, copied components, and the exact local modifications in the active firmware tree.
• tools/make_join_code.py - interactive enrollment-code hash generator.
• tools/provision_board.sh - one-command new-board bring-up (backup, flash, open enrollment).
• tools/start_bench_relay.sh - one-command bench relay with the firewall and Wi-Fi power-save gotchas handled.
• tools/backup_factory.sh - chunked factory-firmware backup that survives the CH340's lossy reads.
• tools/provision_nvs.py - Linux bench escape hatch if touch provisioning needs to be bypassed temporarily during hardware debugging.
• relay/nginx/e-ham.conf - the deployed production nginx site.
• docs/ - Linux, hardware, VPS, protocol, and release runbooks.

Quick Linux verification

cd ~/src/sidebar-intercom
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r relay/requirements-dev.txt
pytest -q

Network modes

Development on one trusted LAN:

boards -- ws://LINUX_WORKSTATION_IP:8443 --> local Python relay

This requires the firmware's explicit insecure-local build option and must never be port-forwarded or released.

Production:

boards -- wss://relay.example.com:443 --> Cloudflare edge (proxied)
                                               |
                                               +--> Nginx :443 (CF origin cert)
                                                      |
                                                      +--> 127.0.0.1:8443 relay

The public firewall exposes 443, not 8443. Nginx forwards the real client IP only to the loopback relay so enrollment rate limits remain meaningful.

Status (2026-08-06)

Production is live on v1.3.1. Five units, one per child; three are in daily use and the remaining two are being brought up. The relay advertises v1.3.1 and devices take it through the consent-based OTA flow. Units run against a family relay on a Hetzner VPS: control plane on wss://relay.example.com (Cloudflare-proxied), audio as AES-256-GCM encrypted UDP datagrams straight to the relay host with per-session keys, replay protection, and automatic websocket fallback so mixed firmware versions interoperate.

Verified on hardware across more than a dozen OTA releases:

• Touch provisioning with free-text names, one-time enrollment behind a parent-held join code, and a Wi-Fi repair flow that fixes changed credentials without losing the device identity.

• Ring-to-answer calls: Answer/No screen with ring tone, decline, caller cancel, ring timeout, and missed-call notices. Direct, group, and broadcast calls plus late-join of a live call; calls survive brief Wi-Fi drops via a server-side seat-hold and silent rejoin.

• Two-way audio at 16 kHz PCM: frame bundling for 2.4 GHz airtime, ~200 ms WAN jitter buffer with underrun concealment, per-speaker buffers with local mixing, and a persisted in-call volume slider.

• The OTA pipeline itself: on-screen offer with Install/Later, release notes, download progress bar, digest verification, and a health-gated rollback.

• A home menu (v1.2) reachable from the lobby: an always-on clock, a persisted missed-call list with timestamps, and a settings screen (volume, Wi-Fi repair, version). Calling stays one tap from idle.

• A reworked lobby (v1.3): menu button top-left, clock top-right, status bar along the bottom, and a scrollable friend list that holds up to eight online kids so the circle can grow past three units. The clock screen now shows the year alongside the time. The in-call screen gains a mute button that drops mic capture at the source (local-only, no protocol change).

• A parent monitoring dashboard (relay/monitor.py): who is online, calls in progress with running durations, call history and per-child talk time. It is built so it cannot eavesdrop — separate read-only process, metadata-only schema; see SECURITY.md.

• An enclosure that exists: three printed parts, designed by an eleven-year-old, fitted first try. See enclosure/.

The kid roster lives only in the server's allowlist; devices hardcode no names. The relay has 39 automated tests.

Known not working: acoustic echo. You hear your own voice return at the end of a sentence — the far end's speaker feeds its own microphone and the round trip comes back to you. Moving the mic and speaker apart in the enclosure reduced it but did not fix it. See the Roadmap.

Roadmap

Next:

• Echo suppression. The one thing that visibly is not right. Enclosure separation helped but did not fix it; the plan is half-duplex gating — close the microphone while the speaker is active, with a short hangover to cover the room's reverb tail, which is how real ham radio behaves anyway. Mechanical isolation (soft-mounting the speaker, gasketing the mic ports) is the parallel fix, since vibration travels through the shell as well as the air. Full-duplex would need proper acoustic echo cancellation.
• Signed firmware. Sign images so a device only accepts firmware signed by the maintainer's key. Deliberately the signed-apps configuration (SECURE_SIGNED_APPS_NO_SECURE_BOOT), which needs no eFuse burn and stays reversible — the current weakest link is that OTA integrity rests on TLS plus a digest, with no authenticity check. Full Secure Boot comes later.
• Bring up the last two units and hand them over: enroll against the relay, print an enclosure each. The v1.3 scrollable lobby already handles up to eight children online at once, so the roster has room to grow.
• Remove bench instrumentation (mic_probe, ui heartbeat) now that the audio path is stable.
• Validate call quality at more houses; watch the RSSI diagnostics for placement problems.

Later:

• Battery operation. The boards charge over USB-C and have a BAT connector, but no cell is fitted yet; the enclosure has room. Needs charge and thermal validation before it goes near a child.
• Full production hardening per SECURITY.md: Secure Boot v2, flash encryption, encrypted NVS (see docs/security-provisioning.md — burns eFuses, do last and one unit at a time).
• Opus or ADPCM if uncompressed PCM becomes a bandwidth limit.
• Voicemail via the SD card slot, so a call to an offline unit leaves something behind.
• Relay observability: uptime alerting and automated allowlist backup. The monitoring dashboard covers presence and usage; neither of these yet.

Done:

• UI refactor (v1.3.1) — LVGL screens and handlers moved out of app_main.c into ui.c (1563 → 587 lines), interface in ui.h, shared macros and types in ui_shared.h. Behaviour-preserving.
• Enclosure — three printed parts, board landscape, USB reachable without opening the case, mic and speaker separated on the face. The corded hand-mic idea was dropped: the microphone's flat flex cable cannot be coiled without fatiguing it, so the mic is built into the front panel.
• Parent monitoring — presence, live calls, durations and history, with eavesdropping ruled out by construction rather than by policy.
• In-call mute (v1.3.0) — drops microphone capture at the source.

Considered and not committed:

• Opportunistic peer-to-peer media (relay-signalled, 1:1 only) as a latency and cost optimisation. Real gains are modest and NAT traversal is the hard part.
• An off-grid LoRa text variant (915 MHz, ISED-legal in Canada). Pin-mux conflicts with the PDM microphone make it a separate device, not a mode.
• Putting the devices on a WireGuard/Tailscale network. Audio and control are already encrypted to the relay, so it would mostly re-encrypt encrypted traffic; the real win — no public relay endpoint — costs a per-device key and an extra failure mode on constrained hardware.