Add MQTT command to trigger stored keyboard macros - #1518
Open
markfrancisonly wants to merge 1 commit into
Open
Conversation
Exposes stored keyboard macros over MQTT: publishing a macro's ID (or case-insensitive name) to a new keyboard_macro/set command topic runs it, gated on the existing EnableActions flag like the other MQTT commands. The macro is actuated through the same session-independent rpcExecuteKeyboardMacro path the browser uses, so automation (Home Assistant, scripts) can trigger a macro while a WebRTC session stays connected. The key/modifier name -> HID code tables are ported from the web UI (ui/src/keyboardMappings.ts) into a new internal/keyboard package; macros are layout-independent because they are authored as key names, not characters. Home Assistant discovery publishes one button per stored macro, slot-indexed (macro_1 .. macro_<MaxMacrosPerDevice>) so retained configs for deleted or renamed macros are always cleared, even across reboots. setKeyboardMacros republishes discovery so edits appear immediately. rpcExecuteKeyboardMacro previously assumed a single serial caller (the WebRTC session HID queue); MQTT adds concurrent callers, so it now cancels any running macro and serializes execution to prevent overlapping macros from interleaving keystrokes on the shared gadget. Keyboard commands are subscribed at QoS 0 and retained messages dropped, so a macro is never replayed from the broker session or retained state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1517
Summary
keyboard_macro/setMQTT command that triggers a stored keyboard macroby ID (or case-insensitive name), gated on the existing
EnableActionsflag.The macro runs through the same session-independent
rpcExecuteKeyboardMacropath the browser uses, so automation (Home Assistant, scripts) can fire a
stored macro while a WebRTC session stays connected. Home Assistant discovery
adds one button per stored macro.
Checklist
make test_e2elocally and passed (see below)Closes #<issue-number>)E2E Testing Results:
make check(go vet+go test ./...) passed, and bothbuild_devfirmware builds (baseline + release) succeeded.make test_e2e DEVICE_IP=<ip>ran the full Playwright suite against a physical device:uiproject: 14/14 passed.remote-agentproject: skipped — these require a controlled host wired tothe KVM (HDMI + USB) running the remote agent (
JETKVM_REMOTE_HOST), which Idon't have set up.
no HDMI video source for the video-dimension check, and OTA release
availability. Notably, the
ota-upgrade-to-signedtest installed thisbranch's firmware, rebooted, and reconnected successfully before the
video-signal step.
keyboard_macro/setend-to-end against a device (macrotypes into the attached host over MQTT while a browser session stays
connected). This is a backend-only MQTT change and doesn't touch the
HID-injection/video/OTA paths those remaining tests cover.
What changed
internal/keyboard(keymap.go): the key-name→HID-usage andmodifier-name→mask tables, ported from
ui/src/keyboardMappings.tsso thedevice agrees byte-for-byte with the names the web UI stores into macros.
HIDSteps([]MacroStep)converts a stored macro's steps into the wire-levelhidrpc.KeyboardMacroSteps, mirroring the web UI'sexecuteMacroRemote(
ui/src/hooks/useKeyboard.ts): each step becomes a press report held brieflyplus an all-zero release report carrying the step's delay. The explicit
release is required —
rpcDoExecuteKeyboardMacroplays reports as-is and onlyauto-releases on cancellation, so without it keys stay held on the host.
Macros are layout-independent because they are authored as key names, not
characters.
mqtt_commands.go—handleKeyboardMacroCommand, registered underkeyboard_macro/set, following the existinghandleJigglerCommandshape(
actionsAllowed()gate → trim → log → act). It is subscribed at QoS 0(unlike the other commands): the client uses a persistent session
(
CleanSession(false)), so a QoS 1 subscription would make the broker queuecommands published while the device is offline and replay them on reconnect —
a macro must never be replayed. For the same reason retained messages are
dropped. Execution is dispatched to a goroutine because paho serializes
callbacks and a macro can sleep for seconds.
mqtt_discovery.go— one Home Assistantbuttonper stored macro,published only when actions are enabled. Buttons are slot-indexed
(
macro_1..macro_<MaxMacrosPerDevice>): each pass publishes occupiedslots and clears empty ones, so retained configs for deleted or renamed macros
are always removed — including ones published before a reboot — without
tracking any state.
payload_presscarries the macro ID, so a buttonalways resolves against current config.
setKeyboardMacrosrepublishesdiscovery so macro edits appear in HA immediately, and teardown sweeps the
slots.
jsonrpc.go—rpcExecuteKeyboardMacrowas only ever called from thesingle WebRTC session's HID queue, so callers were inherently serial. MQTT
adds genuinely concurrent callers (a goroutine per command, and a browser
paste can run at the same time), so it now cancels any running macro and
serializes execution — overlapping macros run one after another instead of
interleaving reports on the shared gadget. This also hardens the pre-existing
browser-paste-vs-paste path.
How it works (session-independent HID path)
rpcExecuteKeyboardMacronil-checkscurrentSessioneverywhere, so no WebRTCsession is required and a connected browser session is not evicted.
How to test
With MQTT configured,
EnableActionson, and a macro stored with IDlogin-root(base topic defaults tojetkvm/<device-id>):Keep the web UI open in a browser while doing this — the macro types into the
attached host and the browser session is not kicked. You can also publish the
macro's name (case-insensitive). In Home Assistant, the device shows a
"Macro: <name>" button per stored macro that publishes the same command.
Negative checks: unknown macro → warning logged, nothing typed;
EnableActionsoff → command rejected and buttons removed; retained publish → ignored.
Security considerations
config.KeyboardMacros; it cannot define arbitrary keystrokes over the wire.Triggering is bounded to sequences the user already authored via the editor,
and to the existing
config.Validatelimits.EnableActionsflag, exactly like thepower/media/reboot commands. Flagging honestly that this flag currently
defaults on; if you'd prefer keyboard triggering behind a dedicated default-off
flag, that's an additive change — happy to do it.
equivalent to typing on the console keyboard; the broker should be
authenticated and ideally TLS-protected (a deployment concern the firmware
can't enforce).
replayed from the broker session or retained state. (Residual: a broker that
also queues QoS 0 for offline persistent sessions could still replay; the
common default does not.)
Backwards compatibility
No behavior change when MQTT is disabled or actions are off. No config schema
changes — reuses the existing
KeyboardMacro/KeyboardMacroSteptypes and theexisting macro-execution path. The new topic and buttons only exist when MQTT is
enabled, and the command is rejected (and buttons removed) unless
EnableActionsis set. No UI changes.
Follow-ups
keyboard_text) and ad-hoc hotkey (keyboard_keys) injection overMQTT were deliberately left out: arbitrary keystroke injection is a larger
security surface than a curated macro selector, and free-text additionally
needs on-device keyboard-layout tables (today the layout mapping lives only in
the frontend,
ui/src/keyboardLayouts/*). Either could be added later behind adedicated default-off flag, and free-text could source its layout data from the
existing UI tables via codegen rather than hand-maintained duplicates.