Skip to content

Add a view-mode selector to the Roku Device View (WebRTC playback) - #833

Draft
chrisdp wants to merge 3 commits into
masterfrom
feature/device-view-modes
Draft

Add a view-mode selector to the Roku Device View (WebRTC playback)#833
chrisdp wants to merge 3 commits into
masterfrom
feature/device-view-modes

Conversation

@chrisdp

@chrisdp chrisdp commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Adds a source-mode selector to the Roku Device View. A dropdown switches the panel between Screenshot (default, unchanged) and live stream/capture modes; more can be added later. Everything runs in the webview, with no extension host changes. The selected mode and a per-mode last-used URL persist in webview storage.

Modes:

  • Screenshot: the existing ECP screenshot view.
  • WebRTC (WHEP): POSTs a recvonly SDP offer, applies the answer, renders the track in a <video>, and DELETEs the session on Stop.
  • Video URL: native <video src> for progressive files.
  • MJPEG: native <img> multipart/x-mixed-replace.

Capability detection

Webview media support varies by platform and Electron build, so on load the panel probes what actually works and only offers supported modes (WebRTC gated on RTCPeerConnection; Video URL kept only if a native container plays). Native <video> support is measured with a real load-probe (load a tiny embedded clip, watch for loadeddata vs error), because canPlayType reports the codec allowlist without knowing whether the container demuxer is present. The Video URL mode shows the detected support inline.

VS Code webview media support (what we measured)

  • VS Code's bundled ffmpeg (Chromium 148 / Electron 42) has the MP4 demuxer but no Matroska/WebM demuxer, identically on macOS, Linux, and Windows. Native WebM file playback fails everywhere, regardless of codec.
  • H.264/AAC mp4 plays (AAC via the OS platform decoder).
  • WebRTC plays both H.264 and VP8/VP9, because it uses RTP plus its own bundled decoders and never the container demuxer. This is the portable path for a live device view.
  • Reference: Webms with supported format not previewable in VSCode microsoft/vscode#195758 (closed as upstream/electron).

Follow-ups

  • Cursor-location node inspection overlay (hover highlight + node info from Screenshot mode) available in all modes
  • Additional source modes as needed
  • Configurable ICE servers (custom STUN/TURN) for WebRTC streams that can't connect over the default public STUN alone
  • Auth support for WHEP endpoints that require it (e.g. Authorization: Bearer <streamKey>)
  • Auto-reconnect on dropped WebRTC connections
  • Audio starts muted for autoplay reliability; add an unmute affordance and remember the preference
  • Recent-URL history instead of a single remembered URL

@chrisdp chrisdp self-assigned this Jul 16, 2026
FrankSAURET added a commit to FrankSAURET/kablix that referenced this pull request Jul 26, 2026
Cause enfin identifiée (PR rokucommunity/vscode-brightscript-language#833) :
l'Electron de VS Code n'embarque PAS le démuxeur Matroska. Aucun .webm n'y est
lisible — webview, aperçu Markdown ou éditeur vidéo — quel que soit le codec.
canPlayType répondait « probably » (Chromium sait décoder VP9) sans révéler que
le conteneur n'est pas démuxable : d'où le lecteur inerte des v190/191.

- 3 démos réencodées en MP4/H.264 (ftyp + avc1 + faststart, yuv420p, 720p,
  4 im/s) : 2,03 Mo à trois (CRF 27, retenu après comparaison 24/27/30).
- Guides FR + EN : <video src="../../media/*.mp4"> (le HTML brut de v192 reste,
  il ne manquait que le bon conteneur).
- WebM banni : les 3 media/*.webm supprimés, media/*.webm exclu du vsix, raison
  écrite dans .vscodeignore pour ne pas y revenir.
- verify:docs 23 contrôles : en-tête ftyp/avc1, moov avant mdat, < 1,5 Mo, non
  exclus, aucun .webm référencé ni resté dans media/, type="video/mp4" au rendu.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant