Evdev on Kandelo - /dev/input keyboard and pointer, plus an espeak-ng speech demo - #698
Evdev on Kandelo - /dev/input keyboard and pointer, plus an espeak-ng speech demo#698mho22 wants to merge 9 commits into
Conversation
Phase B-1 matrix build status —
|
| Package | Arch | Status | Sha |
|---|---|---|---|
| libcurl | wasm32 | built | 05aae726 |
| libcxx | wasm32 | built | 7924fbe0 |
| libcxx | wasm64 | built | cfe94678 |
| libpng | wasm32 | built | e11caae0 |
| libxml2 | wasm32 | built | 6db06345 |
| libxml2 | wasm64 | built | bfabb4ff |
| openssl | wasm32 | built | 2708b58d |
| openssl | wasm64 | built | a1be8942 |
| sqlite | wasm32 | built | 0affae3a |
| sqlite | wasm64 | built | 2638b6ad |
| zlib | wasm32 | built | a74da2ed |
| zlib | wasm64 | built | 76101599 |
| bc | wasm32 | built | 46973bfb |
| bzip2 | wasm32 | built | ffa5f4c4 |
| coreutils | wasm32 | built | 4d8239d1 |
| curl | wasm32 | built | dca90723 |
| dash | wasm32 | built | ad59e65d |
| diffutils | wasm32 | built | d67cb386 |
| dinit | wasm32 | built | 6780ca3d |
| fbdoom | wasm32 | built | a4dcc46f |
| file | wasm32 | built | d2312c00 |
| findutils | wasm32 | built | 3767b4e1 |
| gawk | wasm32 | built | faedad8e |
| git | wasm32 | built | f50d2974 |
| grep | wasm32 | built | b9b6b1e6 |
| gzip | wasm32 | built | 539fee04 |
| kandelo-sdk | wasm32 | built | bdddb4fe |
| kernel | wasm32 | built | 51062af5 |
| less | wasm32 | built | 5180eac9 |
| lsof | wasm32 | built | 96fa5be8 |
| m4 | wasm32 | built | 0aa7ce01 |
| make | wasm32 | built | 5cdd9bd8 |
| mariadb | wasm32 | built | 4c185e36 |
| mariadb | wasm64 | built | ec3ea5f8 |
| msmtpd | wasm32 | built | aa3de664 |
| nano | wasm32 | built | 4a52b067 |
| ncurses | wasm32 | built | ff1ca5a3 |
| netcat | wasm32 | built | 20b49fdf |
| nginx | wasm32 | built | 5c15a366 |
| php | wasm32 | built | f1bfba4b |
| posix-utils-lite | wasm32 | built | df2ce6e9 |
| sed | wasm32 | built | 80e62002 |
| spidermonkey | wasm32 | built | 0b3b3066 |
| tar | wasm32 | built | 0532dd66 |
| tcl | wasm32 | built | eba6fe94 |
| unzip | wasm32 | built | b4f32850 |
| userspace | wasm32 | built | 1d4c6a6f |
| vim | wasm32 | built | 9895fd65 |
| wget | wasm32 | built | 6c82e2c4 |
| xz | wasm32 | built | 7bf9b496 |
| zip | wasm32 | built | 261d8148 |
| zstd | wasm32 | built | 0c16fdb0 |
| bash | wasm32 | built | 15fc009b |
| mariadb-test | wasm32 | built | 884a67b1 |
| mariadb-vfs | wasm32 | built | dd1e248f |
| mariadb-vfs | wasm64 | built | f3286762 |
| nethack | wasm32 | built | 73993e86 |
| node | wasm32 | built | 2e1b58a3 |
| spidermonkey-node | wasm32 | built | 22cfbf7c |
| vim-browser-bundle | wasm32 | built | 0deeb2d2 |
| nethack-browser-bundle | wasm32 | built | 8561a152 |
| rootfs | wasm32 | built | 74c566e1 |
| shell | wasm32 | failed | — |
| lamp | wasm32 | failed | — |
| node-vfs | wasm32 | failed | — |
| wordpress | wasm32 | failed | — |
Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.
brandonpayton
left a comment
brandonpayton
left a comment
There was a problem hiding this comment.
Codex review
This feedback is from Codex (OpenAI), reviewing at Brandon's request.
Summary
I do not recommend adopting this kernel ALSA surface as Kandelo's sound API for the stated SDL goal. The browser AudioWorklet, shared-ring transport, dual-host AudioDriver abstraction, and end-to-end eSpeak demonstration are useful engineering work. However, the demo proves a Kandelo-specific pcaudiolib backend and the host playback path; it does not demonstrate compatibility with SDL's ALSA backend, alsa-lib, or ordinary ALSA applications.
ALSA is a large Linux kernel/userspace contract, not simply a PCM node plus several ioctl numbers. Implementing /dev/snd/* usefully requires the control plane, device/card enumeration, hardware and software parameter negotiation, state and XRUN behavior, poll semantics, read/write and mmap paths, and the ABI details expected by alsa-lib. Carrying a small custom subset under ALSA names risks freezing an interface that looks compatible but is not.
For Kandelo's immediate objective, I recommend a smaller implementation-neutral PCM core with a correct OSS-compatible /dev/dsp frontend.
Completeness concerns
-
This does not establish SDL ALSA compatibility. SDL calls a broad
libasoundAPI; it does not normally issue these raw kernel ioctls itself. This PR does not port or testalsa-lib, and eSpeak uses a new Kandelo-specific pcaudiolib backend instead. The resulting demo is valuable, but it should not be treated as evidence that SDL or other ALSA software will run unmodified. -
The control device is intentionally a stub. The PR description says
/dev/snd/controlC0opens but all ioctls fail. A node that passes an existence probe but cannot enumerate or describe a card is not a complete ALSA control implementation and can cause consumers to fail later in less obvious ways. -
The advertised PCM ABI is only a narrow subset. Capture is absent, the full control/device discovery contract is absent, and compatibility with real
alsa-libread/write and mmap behavior has not been demonstrated. Unsupported functionality should not be advertised under standard ALSA device names unless the supported conformance boundary is explicit and tested. -
The proof is application-specific. The custom eSpeak backend is coupled to exactly the implemented subset, so it cannot reveal missing behavior that a general ALSA client would exercise. At minimum, a claim of ALSA compatibility would require an unmodified
alsa-libplus representative clients and negative/state-transition tests. -
Required verification is incomplete. The PR says the musl libc-test and Open POSIX Test Suite were not run. Repository policy requires both before completion, even for an additive change. The full Vitest suite also needs to run, not only selected audio and package-system files.
-
The size cost is not isolated. This PR is merged with large DRI/input work, so its source diff and final kernel artifact do not show the incremental cost of ALSA. Before choosing this API, measure the audio-only delta in raw and compressed kernel Wasm, host/worklet JavaScript, steady-state ring memory, and userspace artifacts such as libraries and test/demo binaries.
Direction Brandon is exploring
Brandon is exploring deeper /dev/dsp support on a clean branch, with the goal of satisfying SDL without implementing the Linux ALSA kernel ABI.
There are strong reasons for that direction:
- Upstream SDL2 and SDL3 already contain an OSS
dspbackend. SDL2 primarily relies on blockingwrite()for pacing; SDL3 additionally usesSNDCTL_DSP_GETOSPACE. - Kandelo already has
/dev/dsp, its SDK header, and an existing non-SDL consumer, so this is an incremental compatibility improvement rather than a second sound ABI. - FreeBSD's native PCM framework exposes an OSS-compatible
/dev/dspAPI rather than kernel ALSA. FreeBSD supplies ALSA compatibility in userspace over OSS, which is also a viable later path for Kandelo if Linux-specific applications become important. - POSIX does not standardize audio. A small Unix device extension does not make Kandelo less POSIX, while ALSA does not make it more POSIX.
- SDL performs application-side conversion and resampling, so the kernel does not need ALSA's full negotiation and plugin model merely to provide useful playback.
The /dev/dsp work should be source/API compatible rather than claiming host binary compatibility. Kandelo should own and ABI-snapshot its wasm32 ioctl values and fixed-width structures such as audio_buf_info.
For SDL-quality playback, the current /dev/dsp implementation still needs real backpressure: blocking writes, O_NONBLOCK/EAGAIN, truthful POLLOUT with wakeups, GETOSPACE, actual drain/reset behavior, audio-clock-driven consumption, and deterministic open-file-description ownership. It must stop dropping the oldest queued audio when full. Playback-only is a reasonable first boundary, but capture opens should then fail rather than advertise a recording device that returns EOF.
Suggested disposition
I would treat this PR as a valuable audio-transport exploration and reuse the sound parts selectively:
- retain the
AudioDriverseparation and dual-host wiring; - retain the AudioWorklet-driven browser consumption concept;
- retain deterministic Node pacing and host instrumentation where useful;
- move those pieces behind a generic PCM stream rather than ALSA state/mmap structures;
- do not merge
/dev/sndand the partial ALSA UAPI as Kandelo's public sound contract solely for SDL.
If broad compatibility with unmodified Linux ALSA applications later becomes an explicit product requirement, evaluate a userspace alsa-lib OSS/native PCM adapter first. A kernel ALSA implementation should only be undertaken with a defined compatibility target and conformance suite.
— Codex (OpenAI), at Brandon's request
d1b1156 to
e439b86
Compare
…ontract Codex's review of #698 recommended against adopting `/dev/snd` as Kandelo's sound API: ALSA is a large kernel/userspace contract, this was a narrow subset of it, and carrying it under standard ALSA names freezes an interface that looks compatible but is not. The reviewer's alternative — a correct OSS-compatible `/dev/dsp` — is already on main. `crates/kernel/src/audio/oss.rs` was byte-identical to main's `crates/kernel/src/audio.rs`; this branch only renamed it. Main's version already answers every item on the review's SDL-quality list: blocking `write`, `set_nonblock`/EAGAIN, `poll_writable`, `output_space` (GETOSPACE), `output_delay`, `output_pointer`, `sync`, `reset_stream`, `claim_transport`/`clock_update` for audio-clock-driven consumption, and `open_stream`/`preflight_close` for deterministic OFD ownership. So the file moves back and the ALSA siblings go. Removed: `audio/{pcm_ioctl,mmap,sab,tick,wait}.rs`, the `/dev/snd` devfs directory with `controlC0` + `pcmC0D0p`, the `AlsaPcm` / `AlsaControl` virtual devices, the `pcmC0D0c` disabled-node path, the `audio` / `audio_ctl` OFD sidecars and their fork/exec serialisation, the `kernel_audio_{init_sab,period_tick,get_appl_ptr}` exports, the `shared::audio` ALSA ABI, and `sound/asound.h`. Host side loses the ALSA-shaped driver — `audio/{audio-driver, browser-audio-driver,node-audio-driver,instrumented-audio-driver}.ts` and `wpk-audio-worklet.js` — plus `attachAudioDriver` and the three ring/tick/appl-ptr protocol messages on both hosts. Main's PCM stack (`audio/{pcm-driver,browser-pcm-driver,node-pcm-driver,pcm-transport}.ts` + `pcm-audio-worklet.js`) already serves `/dev/dsp` and stays. The espeak demo now rides that path instead of attaching its own driver. Kernel: 1635 tests pass. Shared: 53 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mho22
changed the title
e439b86 to
f498fc4
Compare
…ontract Codex's review of #698 recommended against adopting `/dev/snd` as Kandelo's sound API: ALSA is a large kernel/userspace contract, this was a narrow subset of it, and carrying it under standard ALSA names freezes an interface that looks compatible but is not. The reviewer's alternative — a correct OSS-compatible `/dev/dsp` — is already on main. `crates/kernel/src/audio/oss.rs` was byte-identical to main's `crates/kernel/src/audio.rs`; this branch only renamed it. Main's version already answers every item on the review's SDL-quality list: blocking `write`, `set_nonblock`/EAGAIN, `poll_writable`, `output_space` (GETOSPACE), `output_delay`, `output_pointer`, `sync`, `reset_stream`, `claim_transport`/`clock_update` for audio-clock-driven consumption, and `open_stream`/`preflight_close` for deterministic OFD ownership. So the file moves back and the ALSA siblings go. Removed: `audio/{pcm_ioctl,mmap,sab,tick,wait}.rs`, the `/dev/snd` devfs directory with `controlC0` + `pcmC0D0p`, the `AlsaPcm` / `AlsaControl` virtual devices, the `pcmC0D0c` disabled-node path, the `audio` / `audio_ctl` OFD sidecars and their fork/exec serialisation, the `kernel_audio_{init_sab,period_tick,get_appl_ptr}` exports, the `shared::audio` ALSA ABI, and `sound/asound.h`. Host side loses the ALSA-shaped driver — `audio/{audio-driver, browser-audio-driver,node-audio-driver,instrumented-audio-driver}.ts` and `wpk-audio-worklet.js` — plus `attachAudioDriver` and the three ring/tick/appl-ptr protocol messages on both hosts. Main's PCM stack (`audio/{pcm-driver,browser-pcm-driver,node-pcm-driver,pcm-transport}.ts` + `pcm-audio-worklet.js`) already serves `/dev/dsp` and stays. The espeak demo now rides that path instead of attaching its own driver. Kernel: 1635 tests pass. Shared: 53 pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f498fc4 to
71654e9
Compare
`struct input_event`, the `EV_*`/`KEY_*`/`BTN_*`/`REL_*`/`ABS_*` codes and the `EVIOC*` request numbers are the contract three sides have to agree on: the kernel that fills the ring, the libc header a C client includes, and the host that injects events. They live in `shared` so a single definition generates all three, and `dump_abi` emits them into the snapshot. `EVIOCGNAME`, `EVIOCGBIT` and friends encode their buffer length in the request word, so the request a client sends is not a constant the kernel can match on. `family_request_contract()` decodes direction, size, magic and number, then resolves the family — which keeps the ioctl table finite instead of one entry per possible length.
The two device nodes are the keyboard and the pointer. Each open file description carries its own `InputFdState`, because two readers of the same device are independent: one draining events must not consume the other's, and each needs its own position in the ring. Keeping the state on the OFD rather than the fd is what makes `dup()` and fork-inherited descriptors share one reader, which is the behaviour a real evdev client relies on.
`kernel_input_event` is the host's way in. It fans one event out to every open reader of the target device, so a demo and a compositor both see the same keystroke. A reader whose ring is full loses its oldest events and is handed a `SYN_DROPPED` first, which is how evdev tells a client its stream has a hole and state must be re-read. `sys_read` drains whole events only — a short buffer returns what fits and leaves the rest queued, never a partial record. `sys_poll` reports POLLIN from the same queue, so a client that selects before reading is never woken for an empty ring. The `EVIOCG*` handlers answer through the family table and populate the capability bitmaps a client probes at open time.
An `InputFdState` is per-OFD, so fork must carry it to the child with the descriptor rather than leave the child reading a device with no reader state. Serialising it alongside the other OFD kinds keeps a forked client's stream position intact. The matching release on close returns the ring, which otherwise leaks for every client that opens a device and exits.
`InputSource` is the seam between "where events come from" and "how they reach the kernel". A source emits records; the host injects them. `BrowserInputSource` captures DOM keyboard and pointer events and translates them. Keys map through `codeToKey` on `KeyboardEvent.code`, the physical key, so a Dvorak or AZERTY layout produces the keycode the guest's own keymap then interprets — translating `.key` instead would apply the layout twice. Pointer motion becomes `REL_X`/`REL_Y` deltas and buttons become `BTN_*`, each batched behind the `SYN_REPORT` that closes an evdev packet. `NodeInputSource` emits nothing. Headless tests drive `injectInputEvent` directly, and a null source lets them exercise the same attach path without a DOM.
`attachInputSource` sets the canvas dimensions, starts the source, and funnels every record it emits into an `input_event_inject` message; the kernel worker turns that into a `kernel_input_event` call. Both hosts get the same method, the same message and the same worker handling, so a page written against one runs unchanged on the other. Injection enters through the kernel entry gate rather than calling the export directly. Input arrives from a DOM callback, which can land while a syscall is mid-flight, and only the gate serialises the two.
A C client includes `<linux/input.h>`; the sysroot had neither it nor `<linux/input-event-codes.h>`. Both are vendored from the shared ABI so the header a program compiles against and the record the kernel writes cannot drift apart silently. `evdev_demo` is the only in-tree consumer of the header, and its `_Static_assert` fails the build if the two layouts ever diverge. `input-evdev-smoke` deliberately inlines its own structs and ioctl numbers instead: it is the fixture the end-to-end test drives, so its ABI check has to be independent of the header under test.
espeak-ng links upstream pcaudiolib built with only its OSS backend, so `create_audio_device_object` falls through to `/dev/dsp` — the PCM contract the kernel already serves. That is what lets one binary invocation produce audible speech with no host-side audio pipeline of its own. The build also runs a native espeak-ng first, because generating the voice data requires executing the compiler's own output. `evdev-demo` gives `evdev_demo.wasm` a registry owner so the browser demo resolves it like any other package artifact rather than depending on a loose build output.
`/?demo=evdev` attaches a `BrowserInputSource` to the window and runs `evdev_demo` from bash, so the user's own keystrokes and pointer motion come back as an event log in the Shell pane. `/?demo=espeak` speaks a line through espeak-ng. Both stage their binary into the image before the worker takes exclusive ownership of the VFS, because each runs its program by path.
|
Folded into #948. The stack was rebased onto
Review the work in #948. The pre-rebase tip |
71654e9 to
6c28ee0
Compare
|
Retracting the comment above. This PR was closed in error and is reopened. The stack was rebased onto The branch is force-pushed to The pre-rebase tip is kept locally as |
2 participants
Why
Kandelo had no input devices: nothing existed under
/dev/input, so evdev consumers — SDL2 (#709), Wayland (#948), Hyprland (#976) up the stack — had no way to read a keyboard or a pointer. This PR is the base layer of the six-PR DRI stack (#698 → #709 → #948 → #976 → #1318 → #1241). It adds a kernel evdev subsystem fed by host input sources on both hosts, and proves the audio path end-to-end by porting espeak-ng against main's existing OSS/dev/dspdevice.What changed
crates/kernel/src/input/):/dev/input/event0(keyboard) and/dev/input/event1(pointer) devfs nodes with per-OFDInputFdState;EVIOCG*ioctl dispatch, with the length-encodedEVIOCioctls resolved through a family table; akernel_input_eventexport that fans events out to every open fd, with ring overflow handled bySYN_DROPPEDresync;sys_readdrains the ring andsys_pollgatesPOLLIN; state is released on close and serialised across fork/execstruct input_eventplusEV_*/KEY_*/BTN_*/REL_*/ABS_*/EVIOC*constants;linux/input.handlinux/input-event-codes.hvendored into the musl overlay;abi/snapshot.json,host/src/generated/abi.ts, andlibc/glue/abi_constants.hregenerated — additive-only, noABI_VERSIONbumpInputSourceinterface;BrowserInputSourcecaptures DOM keyboard/pointer events and translates them through a key-code table;NodeInputSourceis the headless null source so vitest exercises the same path; ingress routes through the kernel entry gatepackages/registry/espeak-ng/): cross-build recipe with a cmake toolchain file, linked against upstream pcaudiolib 1.3 built with only its OSS backend — that backend opens/dev/dsp, so no pcaudiolib source is patched and no kernel audio code is added; the English-only voice data ships as a resolver-published runtime file; the image builder stages/usr/bin/espeak-ng,/usr/share/espeak-ng-data, and/usr/local/bin/evdev_demoat image-compose timekandelo-evdev.spec.ts/kandelo-espeak.spec.ts;programs/evdev_demo.candprograms/input-evdev-smoke.c; anevdev-demopackage owningevdev_demo.wasmNotes
Earlier revisions of this PR carried a kernel ALSA subsystem (
/dev/snd/{pcmC0D0p,controlC0}, an mmap ring, threekernel_audio_*exports) and a parallel host ALSA driver set, then deleted both again once the PR converged on/dev/dsp. The history is now rebuilt and never adds any of it. main'saudio.rsalready answers the OSS quality checklist — blocking write,EAGAIN,GETOSPACE, audio-clock-driven consumption — andhost/src/audio/is unchanged versus main, so the PCM drivers the demo plays through (BrowserPcmDriver,NodePcmDriver) are main's. The net kernel and host audio delta versus main is zero.Both hosts
Browser:
BrowserInputSource, wired throughbrowser-kernel-host.ts,browser-kernel-protocol.ts,browser-kernel-worker-entry.ts. Node:NodeInputSource, wired through the node counterparts in the same commits. Shared:kernel-worker.tscarries the input fan-in for both. Vitest covers the Node path end-to-end (input-evdev.test.ts: key + pointer + ring overflow) and the browser translation layer (browser-input-source.test.ts); the Playwright specs cover the browser path.Rebased and rebuilt
This branch sits on
mainat888e628d5. The history was rebuilt from 27 commits to 9: the added-then-deleted ALSA work is gone, as are three process commits and two rebase-reconnect commits.Against the previous tip, the only files that differ outside the 836 that
mainitself moved are four, and every change in them is a comment:host/test/input-attach-source.test.ts,host/test/input-evdev.test.ts,programs/evdev_demo.c,programs/input-evdev-smoke.c. They dropped plan labels (B4,B5,C1,Phase C) that no longer resolve to anything in the tree; each label was replaced by the fact it stood for. No code changed.Verification
Run at the pre-rebase stack tip
ab7ab2b63(explore-dri-omarchy), which contained all 9 commits here:cargo test -p kandelo --target aarch64-apple-darwin --lib→ 1665 passed, 0 failedbash scripts/check-abi-version.sh→ exit 0. This layer is additive at ABI 43; Toolkit on Kandelo — the C/C++ library stack behind foot, Waybar and mako #1318 up-stack bumps to 44.scripts/run-libc-tests.sh→ 302 passed, 20 XFAIL, 0 build failures, and 1 unexpected failure:regression/raise-race. That test forks 100 children inside a signal handler; 5 waits returnECHILD, reproducibly.scripts/run-libc-tests.sh:163skips it wheneverCI=true, so it does not gate this PR, and whether it regressed or predates the stack is not yet established.scripts/run-posix-tests.sh→ 174 passed, 0 FAIL, 3 XFAIL, 2 SKIPBoth suites need
WASM_POSIX_RESOLUTION_POLICY=source-only-v1andWASM_POSIX_SOURCE_ONLY_BINARY_ROOT=<repo>/local-binaries/source-only-v1.binaries-abi-v44/index.tomldoes not exist until the toolkit tier merges, so thebinaries/provenance tier cannot be materialized locally. The full host vitest suite needs both provenance tiers resolvable at once, which only CI's prepared workspace provides — it is the gate for that suite and for the browser demos.The stack has since been rebased onto
mainat888e628d5. Two gates were re-run at the new stack tipccc7c6214(explore-dri-omarchy):bash scripts/check-abi-version.sh→ exit 0, snapshot in sync and the 43 → 44 bump consistent;scripts/ci-run-test-suite.sh vitest exact-abi-source→ 3049 passed, 0 failed, with two files failing to collect on paths #1321 moved tocrates/runtime-core/. That collection failure is amaindefect and #1337 fixes it.