Skip to content

feat(cua-driver-rs)(macos): enable Chromium/Electron AX trees (AXManualAccessibility) for get_window_state#1756

Draft
f-trycua wants to merge 1 commit into
mainfrom
feat/ax-chromium-enablement
Draft

feat(cua-driver-rs)(macos): enable Chromium/Electron AX trees (AXManualAccessibility) for get_window_state#1756
f-trycua wants to merge 1 commit into
mainfrom
feat/ax-chromium-enablement

Conversation

@f-trycua
Copy link
Copy Markdown
Collaborator

Problem

Chromium-based apps (Arc, VS Code, Electron shells) ship their web-content accessibility tree off and only build it once an assistive client requests it. Without enablement, the first get_window_state AX walk of such an app returns an empty / title-bar-only tree — the macOS/AX analog of the empty-tree symptom tracked in #1616 (VS Code exposing only its title-bar elements). With enablement the full tree materializes.

Change

In walk_tree's app-setup path (right after AXUIElementCreateApplication(pid)):

  1. Enable — set AXManualAccessibility = true on the app root (the modern, side-effect-free opt-in). If the app reports the attribute unsupported (kAXErrorAttributeUnsupported / -25205), fall back to AXEnhancedUserInterface = true (the legacy flag some Electron builds expose instead). Native Cocoa apps (Finder, Calculator, TextEdit) reject both — they take no further action.
  2. Settle — the tree is built asynchronously over IPC once the app detects the client, so a walk that starts immediately still sees only the chrome. When the flip actually took, pump the run loop ~500ms (reusing the existing pump_run_loop_briefly helper) so the tree has time to materialize before the walk reads it.
  3. Per-pid cache — a process-global HashSet<pid> (guarded by a Mutex in a OnceLock) records pids we've already enabled + settled. Repeat snapshots of the same app skip the settle; the tree is already built and stays built for the life of the process.

The settle delay fires only when enablement actually flips an app from unsupported→supported, so apps that never needed it (every native Cocoa app) pay zero added latency.

Hard dependency

This is intentionally higher-risk and must land after the Phase 1 messaging-timeout PR. Enablement makes the tree bigger (fully materialized instead of empty/partial) — which is exactly why it depends on the per-element AX messaging timeout (so a wedged element fails fast rather than stalling the now-larger walk) plus the node cap to keep the materialized tree bounded. Shipping this without those bounds would trade "empty/partial" for "complete but potentially huge."

Test

  • cargo build --release -p cua-driver passes.
  • cargo test --no-run -p platform-macos compiles cleanly (the crate's test binary has a pre-existing, unrelated libswift_Concurrency.dylib rpath issue at the run step only).
  • macOS-only change; platform-windows / platform-linux are untouched.

Refs #1616

🤖 Generated with Claude Code

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview May 29, 2026 4:38pm

Request Review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 29, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: abe4a1c9-6d29-45e2-bc43-d4e15ec1e093

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/ax-chromium-enablement

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

…_window_state

Chromium/Electron apps (Arc, VS Code, Electron shells) ship their web-content
accessibility tree off and only build it once an assistive client requests it.
Without enablement the first AX walk returns an empty/title-bar-only tree.

Flip AXManualAccessibility (modern, side-effect-free) on the application root,
falling back to AXEnhancedUserInterface when the modern attribute is
unsupported. When the flip actually takes, let the asynchronously-built tree
settle (~500ms run-loop pump) before walking. Cache per-pid so repeat snapshots
skip the settle. Native Cocoa apps reject the attribute and pay no cost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@f-trycua f-trycua force-pushed the feat/ax-chromium-enablement branch from df5ddd5 to d9491d3 Compare May 29, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant