Add cc PC backend, browser VNC, RAM framebuffer and PE linking APIs

.github/workflows/test.yml

@@ -18,6 +18,11 @@ jobs:
         with:
           go-version-file: go.mod
           cache: true
+      - name: Enable KVM for the runner
+        run: |
+          sudo chown "$(id -u):$(id -g)" /dev/kvm
+          sudo chmod 660 /dev/kvm
+          test -r /dev/kvm && test -w /dev/kvm
       - run: go test ./...
       - run: go vet ./...
       - run: go build ./cmd/trex

docs/cc-pc.md

@@ -0,0 +1,121 @@
+# CrumbleCracker PC backend
+
+`cc.backend()` implements `vmm.Backend` in process on Linux/amd64 with KVM.
+It runs an existing BIOS-bootable disk supplied through the VMM block interface.
+Guest image construction and operating-system setup belong to the caller.
+Go constructs the BIOS data areas and supplies interrupt services through
+small OUT/IRET firmware entrypoints. No firmware binary or external process
+is used.
+
+The PC contains one CPU, KVM PIC/PIT, a CMOS RTC with periodic IRQ8, one primary
+ATA PIO disk, standard planar VGA and i8042 keyboard/mouse. The disk uses the
+portable block interface; snapshot attachments use bounded memory overlays.
+The VGA aperture is unmapped RAM and generates MMIO exits. Guest RAM, disk
+and firmware are never transferred through temporary host files.
+
+```starlark
+vm = vmm.start(vmm.machine(
+    architecture = "i386",
+    memory = 64 << 20,
+    disks = [vmm.disk(disk, bus = "ide", unit = 0,
+                      chs = (520, 16, 63), snapshot = True)],
+    display = vmm.display("capturable"),
+    start_paused = True,
+), cc.backend())
+vm.resume()
+# Use @stdlib//vmm:automation.star wait_duration for bounded event waits.
+vm.pause()
+frame = vm.screenshot()
+vm.resume()
+vm.chord(["control", "alt", "delete"])
+vm.stop()
+result = vm.wait(timeout = 2)
+vm.close()
+```
+
+TinyRangeX supplies the NT 3.1 image recipes, guest drivers and integration
+smokes in [tinyrangex](https://github.com/tinyrange/tinyrangex). Those are
+consumer-level validation of this backend, not image-building features of trex.
+
+## Inspection and limitations
+
+`vm.extension("cc.v1").state()` returns a serialized observation containing
+PC, CR0/CR3, last BIOS service, ATA task registers and command count, VGA
+access count, PIC/PIT state, and bounded keyboard/CMOS transaction tails.
+It also exposes integer registers and the IDT base. `read_physical(address,
+size)` returns an owned snapshot of at most 64 KiB of RAM, including the display
+aperture. `breakpoint(address)` installs a debugger-owned execution breakpoint;
+zero disables it. A hit pauses execution and `resume()` skips that occurrence.
+Reading a screenshot or state serializes with execution. Closing the VM
+releases KVM and RAM and is idempotent; stopping preserves inspection until
+close. Events support `debug.select` and the portable automation helpers.
+
+This initial platform supports one IDE disk, BIOS modes 03h/12h, PNG capture,
+keyboard transitions, chords and relative PS/2 pointer input. Network, text injection,
+interactive host windows, ACPI powerdown, reset, guest snapshots and debugger
+channels are not advertised capabilities. ARM64 UEFI continuation remains
+available through its existing emulator interface. Other host platforms return
+an explicit unsupported-backend error for cc PC execution.
+
+`vm.extension("cc.v1").disk()` takes an immutable, in-memory snapshot of a
+snapshot-attached disk for inspection with trex filesystem readers. It preserves
+the blocks at that instant; it does not quiesce guest filesystem transactions or
+capture CPU/RAM state. This allows guest crash logs to be inspected without
+extracting a disk image to a host file.
+
+## Native execution contract
+
+CC's `hypervisor.X86` accepts architectural state and RAM mappings, with no
+host paths or image-building policy. `MapRAMRegions` allows one RAM backing
+allocation to leave MMIO holes. Calls are serialized except for `Cancel`.
+`CompleteIO` completes the pending operation without executing a further
+instruction before the firmware reads or modifies CPU state. This follows the
+[KVM I/O completion contract](https://docs.kernel.org/virt/kvm/api.html#the-kvm-run-structure).
+
+Two firmware details are required by NT 3.1 and covered by regression tests:
+
+- IRQ0 must invoke the INT 1Ch user timer hook; NTLDR uses it for calibration.
+- The AT CMOS configuration checksum covers bytes 10h–2Dh and is stored
+  big-endian at 2Eh–2Fh. NT's HAL uses its validity to choose century register
+  32h; an invalid checksum selects 37h and produces an invalid system time.
+
+NTDETECT also needs INT 15h C2h mouse reset/identification services to discover
+the PS/2 auxiliary device. Without them, the kernel leaves IRQ12 masked and the
+mouse disabled even though the BIOS equipment word advertises a pointing device.
+The BIOS detection regression and PS/2 packet tests cover this path. See
+[browser display](vnc.md) for the WebSocket/VNC frontend.
+
+### VGA performance
+
+KVM executes ordinary guest instructions in hardware, but the planar VGA
+aperture generates device exits. NT 3.1 uses masked set/reset write mode 3
+extensively (about 245,000 writes in one interactive-logon boot); these writes
+combine CPU-supplied masks, VGA registers and read latches. Mapping the aperture
+as ordinary RAM would lose those semantics. A RAM-backed linear framebuffer
+needs a matching guest display driver.
+
+The native x86 run path uses cancellation callbacks instead of starting and
+joining a goroutine at every IO/MMIO exit. On an i7-1165G7, the isolated exit
+benchmark improved from 8.6–8.9 microseconds to 3.7–3.8 microseconds. Four fresh
+NT desktop dialog measurements, without a browser, improved from 2.1–2.4 seconds
+to 1.0–1.1 seconds for roughly 305,000 VGA byte accesses. These timings include
+input and screenshot caption checks, and are not general application speedups.
+
+`go test ./vmm/cc -run '^$' -bench BenchmarkVGAExit` exercises the native
+aperture path. VGA tests cover masked writes and latches; native CPU tests check
+repeated cancellation followed by resumed IO.
+
+### RAM framebuffer protocol
+
+The host maps an additional 8 MiB of ordinary RAM at physical `0xe0000000`,
+outside the RAM advertised by the BIOS. Six little-endian words describe the
+TRF1 protocol: magic `0x31465254`, active, width, height, stride, and format
+(`1` = BGRX). Pixels begin at offset 4096. A guest driver activates a mode
+and renders into this allocation. Captures validate its dimensions and stride,
+copy pixels while the guest CPU is stopped, and fall back to emulated VGA
+before mode activation. The protocol and decoder live in `vmm/ramfb`.
+
+Guest-specific driver implementations and installation policy belong to the
+image recipe. TinyRangeX provides the NT 3.1 implementation. Trex supplies
+`windows.pe32_link` for in-memory i386 object linking and stdcall adapters,
+and `windows.pe(...).with_resources` for PE resource construction.

docs/starlark/namespaces/reference.md

@@ -2541,6 +2541,12 @@ Creates a bounded writable copy-on-write layer over a base block device. Writes
 
 Returns a read-only file view of a live block device. Later device changes remain visible through the view; it is not an immutable snapshot.
 
+### `cc.backend`
+
+`cc.backend() -> vmm_backend`
+
+Creates an in-process CrumbleCracker PC backend for Linux/amd64 KVM. The initial i386 platform provides Go BIOS services, one ATA disk, VGA capture and keyboard input. vmm.start creates the guest from portable memory and disk intent.
+
 ### `clock.monotonic`
 
 `clock.monotonic() -> elapsed seconds`
@@ -3581,6 +3587,12 @@ Creates a lazy inspection object for a PE32 or PE32+ file. Metadata and data sha
 
 Links a labeled section, fixups and optional imports into a minimal PE32 executable. The caller supplies instruction bytes and policy; the builder lays out headers, RVAs and imports and checks fixup bounds.
 
+### `windows.pe32_link`
+
+`windows.pe32_link(object, imports, exports, callbacks=None, entry='', subsystem=3, version_major=3, version_minor=10, image_base=0x62000000) -> bytes`
+
+Links an in-memory ELF32/i386 cdecl object into a PE32 DLL or native driver. Imports map DLLs to symbol/stack-word dictionaries; exports and callbacks map symbols to stack-word counts. Explicit stdcall adapters bridge the object ABI. Supports R_386_32 and R_386_PC32 relocations; rejects unresolved symbols and unsupported relocations. No OS runtime is linked.
+
 ### `windows.pe_sign`
 
 `windows.pe_sign(file, identity, replace=False, page_hashes=False) -> bytes`
@@ -4148,7 +4160,7 @@ Methods and attributes: `kind`, `name`, `rva`.
 
 A lazy PE32/PE32+ inspection value. Metadata attributes expose headers, sections, imports, exports, resources, strings and debug/type-library data from an owned snapshot. read/disasm use RVAs; patch returns modified file bytes without changing the original. data provides the shared immutable source snapshot.
 
-Methods and attributes: `codeview`, `data`, `disasm(rva, size=256)`, `exports`, `imports`, `info`, `messages`, `patch(rva, data, update_checksum=True)`, `pointer_string_tables(suffix='', minimum=2, maximum=260)`, `read(rva, size)`, `resources`, `sections`, `typelibs`, `version`.
+Methods and attributes: `codeview`, `data`, `disasm(rva, size=256)`, `exports`, `imports`, `info`, `messages`, `patch(rva, data, update_checksum=True)`, `pointer_string_tables(suffix='', minimum=2, maximum=260)`, `read(rva, size)`, `resources`, `sections`, `typelibs`, `version`, `with_resources(resources)`.
 
 ### `windows.signing_identity` value
 

docs/vnc.md

@@ -0,0 +1,52 @@
+# Browser VM display
+
+The native `-serve` frontend accepts a script whose `main(args)` returns a VM
+with the portable `vmm.DisplaySource` capability. cc supplies owned RGBA frames
+and serialized keyboard/pointer input. A callable result still serves a regular
+Starlark web application.
+
+From a trex checkout, pass a recipe whose `main(args)` returns a running VM:
+
+```sh
+go run ./cmd/trex -serve 127.0.0.1:8080 /path/to/vm.star
+```
+
+The recipe supplies the guest disk and backend configuration. Trex does not
+construct or configure an operating system through this frontend. TinyRangeX
+provides an NT 3.1 browser recipe and guest drivers in
+[tinyrangex](https://github.com/tinyrange/tinyrangex).
+
+Open the complete session URL printed in the terminal. Closing the browser
+leaves the VM running. Disk persistence follows the recipe's attachment policy.
+Only one browser can control the VM at a time. Disconnect to hand over control.
+
+Click the canvas to capture keyboard and mouse; Esc releases mouse capture.
+The toolbar sends Ctrl+Alt+Delete without invoking the browser/host shortcut.
+Keyboard input uses US physical key positions. Clipboard, wheel, touch, audio,
+and file transfer are not supported. PS/2 input is relative, so mouse capture
+avoids confusing host/guest cursor alignment. Absolute
+pointer devices require additional guest/backend support.
+
+The Go RFB 3.8 implementation is in `vmm/rfb`, over `channel.ByteChannel`.
+It supports RGB888 raw rectangles, incremental requests and DesktopSize.
+`vmm/vncweb` implements RFC 6455 binary framing and embeds the original
+JavaScript client; there are no external client libraries or CDN resources.
+The client requests updates at up to 20 Hz; incremental updates send the bounding
+rectangle of changed pixels, and unchanged frames send no pixels.
+Relative pointer positions wrap modulo 65536 in the included client/server.
+
+The HTTP adapter requires a random session token and matching Origin for the
+WebSocket connection, rejects unmasked/oversized input, and bounds stalled I/O.
+The URL fragment is removed from browser history after loading; the token stays
+in that tab's session storage for reloads. Plain HTTP is intended for loopback use;
+remote deployment needs HTTPS to protect the bearer token and guest traffic.
+
+Validation:
+
+```sh
+go test -race ./vmm/rfb ./vmm/vncweb ./vmm/cc ./vmm/star
+node --test vmm/vncweb/client_test.mjs
+```
+
+Protocol references: [RFB 3.8](https://www.rfc-editor.org/rfc/rfc6143) and
+[WebSocket](https://www.rfc-editor.org/rfc/rfc6455).

frontend/starlark/cli.go

@@ -28,7 +28,7 @@ func RunCLI(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
 	fs.SetOutput(stderr)
 	webAddr := fs.String("web", "", "serve a web archive browser on this address")
 	webPublic := fs.Bool("web-public", false, "allow the archive browser to bind a non-loopback address")
-	serveAddr := fs.String("serve", "", "serve a Starlark web application on this address")
+	serveAddr := fs.String("serve", "", "serve a Starlark web application or VM browser display on this address")
 	stdlibDocs := fs.Bool("stdlib-docs", false, "write embedded Starlark API documentation to stdout")
 	replMode := fs.Bool("repl", false, "start an interactive Starlark session without a script")
 	cpuProfile := fs.String("cpuprofile", "", "write a Go CPU profile to this path")

frontend/starlark/namespace_test.go

@@ -7,7 +7,7 @@ import (
 
 func TestPublicNamespaces(t *testing.T) {
 	environment := predeclared()
-	for _, name := range []string{"archive", "binary", "block", "crypto", "debug", "filesystem", "firmware", "qemu", "vmm", "windows"} {
+	for _, name := range []string{"archive", "binary", "block", "cc", "crypto", "debug", "filesystem", "firmware", "qemu", "vmm", "windows"} {
 		if _, ok := environment[name]; !ok {
 			t.Errorf("missing public namespace %q", name)
 		}

frontend/starlark/runtime.go

@@ -73,6 +73,7 @@ import (
 	starcrypto "github.com/tinyrange/trex/script/crypto"
 	starjson "github.com/tinyrange/trex/script/json"
 	storagenative "github.com/tinyrange/trex/storage/native"
+	ccapi "github.com/tinyrange/trex/vmm/cc"
 	qemuapi "github.com/tinyrange/trex/vmm/qemu"
 	vmmstar "github.com/tinyrange/trex/vmm/star"
 	webstar "github.com/tinyrange/trex/web/star"
@@ -83,6 +84,9 @@ import (
 
 func predeclared() starlark.StringDict {
 	nativeIO := storagenative.Builtins()
+	if ccapi.Available() {
+		vmmstar.RegisterBackend("cc.v1", ccapi.Capabilities())
+	}
 	if qemuapi.Available() {
 		vmmstar.RegisterBackend("qemu.v1", qemuapi.Capabilities())
 	}
@@ -183,6 +187,7 @@ func predeclared() starlark.StringDict {
 		},
 		"firmware": namespace{name: "firmware", attrs: acpistar.Builtins()},
 		"image":    namespace{name: "image", attrs: imagestar.Builtins()},
+		"cc":       namespace{name: "cc", attrs: ccapi.Builtins()},
 		"qemu":     namespace{name: "qemu", attrs: qemuapi.Builtins()},
 		"runtime":  runtimeNamespace(),
 		"vmm":      namespace{name: "vmm", attrs: vmmstar.Builtins()},

frontend/starlark/starlark_docs.go

@@ -128,7 +128,7 @@ var nativeStarlarkTypes = map[string][]string{
 	"vmm_network":              {"kind", "name", "required"},
 	"windows.kd":               {"breakin()", "breakpoint(address, timeout=30)", "close()", "context(timeout=30)", "continue(status=0x10002, timeout=30)", "file_io(handler=None)", "next_event(timeout=-1)", "packet(kind, payload, packet_id=None, timeout=30)", "read_physical(address, size, timeout=30)", "read_virtual(address, size, timeout=30)", "request(api, processor=-1, arguments={}, data=b'', timeout=30)", "set_context(raw, edi=None, esi=None, ebx=None, edx=None, ecx=None, eax=None, ebp=None, eip=None, eflags=None, esp=None, timeout=30)", "write_physical(address, data, timeout=30)", "write_virtual(address, data, timeout=30)"},
 	"windows.kd_breakpoint":    {"address", "handle", "remove(timeout=30)", "removed"},
-	"windows.pe":               {"codeview", "data", "disasm(rva, size=256)", "exports", "imports", "info", "messages", "patch(rva, data, update_checksum=True)", "pointer_string_tables(suffix='', minimum=2, maximum=260)", "read(rva, size)", "resources", "sections", "typelibs", "version"},
+	"windows.pe":               {"codeview", "data", "disasm(rva, size=256)", "exports", "imports", "info", "messages", "patch(rva, data, update_checksum=True)", "pointer_string_tables(suffix='', minimum=2, maximum=260)", "read(rva, size)", "resources", "sections", "typelibs", "version", "with_resources(resources)"},
 	"windows.pdb":              {"age", "dbi_age", "guid", "nearest(rva)", "signature", "symbols"},
 	"windows.pdb_symbol":       {"kind", "name", "rva"},
 	"windows.memory_image":     {"read(address, size)", "probe(address, size)", "view(address, size)", "ranges", "address_space(directory_table_base, pae=False)"},
@@ -240,6 +240,7 @@ var nativeStarlarkSignatures = map[string]string{
 	"windows.win9x_vxd_library":         "windows.win9x_vxd_library(base, members, exclude=[]) -> file",
 	"windows.win9x_vxd_library_members": "windows.win9x_vxd_library_members(file) -> list[string]",
 	"qemu.acpi_table":                   "qemu.acpi_table(file)",
+	"cc.backend":                        "cc.backend() -> vmm_backend",
 	"qemu.backend":                      "qemu.backend(binary='', machine='pc', machine_properties={}, accelerator='auto', firmware='bios', display_frontend='auto', display_zoom_to_fit=False, block_transport='auto', overlay_limit=256MiB, stderr_limit=1MiB, devices=[], netdevs=[], chardevs=[], options=[], acpi_tables=[])",
 	"qemu.chardev":                      "qemu.chardev(name, **properties)",
 	"qemu.device":                       "qemu.device(name, **properties)",