Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions Package.swift
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
// swift-tools-version: 6.0
import PackageDescription

// Root shim: re-exports CuaDriverCore and CuaDriverServer so Swift packages
// Root shim: re-exports cua-driver Swift package products so Swift packages
// can consume them directly from the trycua/cua monorepo without knowing the
// internal layout. Sources live in libs/cua-driver/swift/Sources/; this file
// uses path: to forward there.
//
// CuaDriverCore and CuaDriverServer are the legacy Swift implementation.
// CuaDriverEmbedded is the Swift wrapper for the Rust embedded driver C ABI.
// SPM does not build Rust crates by itself, so apps using CuaDriverEmbedded
// must also link libcua_driver_embedded.a/.dylib or a packaged xcframework.
//
// IMPORTANT — SPM version resolution:
// SPM's `from:` / `upToNextMajor` only recognises semver tags ("0.1.0",
// "v0.1.0"). This repo uses "cua-driver-v*" tags for the CLI releases, which
// SPM cannot parse. Until plain semver tags are published, pin by revision:
//
// .package(url: "https://github.com/trycua/cua.git", .revision("cua-driver-v0.1.0"))
// .package(url: "https://github.com/trycua/cua.git", .revision("cua-driver-v0.2.18"))
//
// When the repo starts publishing semver tags alongside the CLI tags, use:
//
Expand All @@ -21,6 +26,7 @@ import PackageDescription
//
// .product(name: "CuaDriverCore", package: "cua") // AX, input, capture, recording
// .product(name: "CuaDriverServer", package: "cua") // MCP tool handlers + daemon layer
// .product(name: "CuaDriverEmbedded", package: "cua") // Rust embedded MCP wrapper

let package = Package(
name: "cua",
Expand All @@ -33,6 +39,10 @@ let package = Package(
// MCP tool handlers and daemon server built on top of CuaDriverCore.
// Depends on modelcontextprotocol/swift-sdk for the MCP protocol types.
.library(name: "CuaDriverServer", targets: ["CuaDriverServer"]),

// Thin Swift wrapper over the Rust embedded driver's C ABI.
// The host app must link the Rust static library, dylib, or xcframework.
.library(name: "CuaDriverEmbedded", targets: ["CuaDriverEmbedded"]),
],
dependencies: [
.package(
Expand All @@ -56,5 +66,9 @@ let package = Package(
],
path: "libs/cua-driver/swift/Sources/CuaDriverServer"
),
.target(
name: "CuaDriverEmbedded",
path: "libs/cua-driver/swift/Sources/CuaDriverEmbedded"
),
]
)
182 changes: 182 additions & 0 deletions docs/content/docs/cua-driver/guide/getting-started/embedded-mcp.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
---
title: Embedded MCP
description: Run cua-driver inside your app process and handle MCP requests without a separate driver executable
---

import { Callout } from 'fumadocs-ui/components/callout';

`cua-driver-embedded` is the Rust-native entry point for hosting cua-driver inside another application process. It builds the same platform tool registry used by `cua-driver mcp`, but it does not read from stdio, start the daemon proxy, or launch `CuaDriver.app`.

Use this when your application wants to expose the cua-driver MCP tools internally and keep macOS TCC authorization attached to your own app bundle.

<Callout type="info">
This is the supported direction for embedded integrations. The older Swift integration page documents the legacy Swift package products; new embedding work should use the Rust driver core and add a thin host-language wrapper where needed.
</Callout>

## Why embedding helps on macOS

macOS Privacy and Security grants are attributed to the responsible app or process that calls protected APIs. If a Swift, Objective-C, Electron, or Rust app links cua-driver and calls the driver in-process, Accessibility and Screen Recording prompts are for the host app.

That avoids the common two-prompt failure mode:

```text
YourApp.app
links cua-driver-embedded
calls AX / ScreenCaptureKit in-process
TCC grant: YourApp.app
```

instead of:

```text
YourApp.app
spawns cua-driver mcp
launches CuaDriver.app or a shell binary
TCC grant: CuaDriver.app or the spawning terminal
```

The rule is simple: protected API calls must stay in the host process if you want the host app's existing authorization to apply. Do not spawn `cua-driver mcp`, `cua-driver serve`, or another helper process for the actual AX and screen-capture work.

## Rust usage

Add the crate from the workspace:

```toml
[dependencies]
cua-driver-embedded = { path = "libs/cua-driver/rust/crates/cua-driver-embedded" }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
```

Create a driver and call tools directly:

```rust
use cua_driver_embedded::EmbeddedDriver;
use serde_json::json;

let driver = EmbeddedDriver::new();

let tools = driver.tools_list();
let apps = driver.call_tool("list_apps", json!({})).await;
```

Or pass MCP JSON-RPC messages through an in-process transport:

```rust
use cua_driver_embedded::{EmbeddedDriver, EmbeddedOptions};
use serde_json::json;

let driver = EmbeddedDriver::with_options(EmbeddedOptions {
claude_code_compat: true,
});

let response = driver
.handle_mcp_request_value(json!({
"jsonrpc": "2.0",
"id": 1,
"method": "tools/list"
}))
.await;
```

`handle_mcp_request_value` and `handle_mcp_request_json` return `None` for JSON-RPC notifications, matching the stdio MCP server.

## C ABI for non-Rust hosts

Non-Rust apps can use the C ABI exported by the same crate instead of shelling out to the standalone driver.

Header:

```c
#include "cua_driver_embedded.h"

CuaDriver *driver = cua_driver_embedded_new(false);

char *response = cua_driver_embedded_handle_mcp_json(
driver,
"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/list\"}"
);

if (response != NULL) {
// Parse JSON response in the host language.
cua_driver_embedded_string_free(response);
}

cua_driver_embedded_free(driver);
```

The header lives at:

```text
libs/cua-driver/rust/crates/cua-driver-embedded/include/cua_driver_embedded.h
```

Build the Rust crate as a static library or cdylib:

```bash
cd libs/cua-driver/rust
cargo build -p cua-driver-embedded --release
```

The build emits `libcua_driver_embedded.a` and `libcua_driver_embedded.dylib` under `target/release`. Link one of them into the host app and sign only the host app. The ABI is intentionally JSON-in/JSON-out: Rust structs, MCP protocol structs, and tool result internals do not become part of the Swift or C ABI.

## Swift Package usage

Swift projects can import the thin Swift wrapper from the root package:

```swift
dependencies: [
.package(url: "https://github.com/trycua/cua.git", .revision("cua-driver-v0.2.18")),
]
```

Then add the product to your app target:

```swift
.product(name: "CuaDriverEmbedded", package: "cua")
```

The Swift product declares the wrapper API. It does not build Rust for you. Your app target must also link the Rust `libcua_driver_embedded` static library, dylib, or a packaged xcframework.

Use it from Swift:

```swift
import CuaDriverEmbedded

let driver = try CuaDriverEmbedded()

let response = driver.handleMCPJSON(
#"{"jsonrpc":"2.0","id":1,"method":"tools/list"}"#
)

if let response {
// Decode JSON response.
}
```

In a GUI app, do not call `handleMCPJSON` from the main thread for arbitrary tool calls. Route MCP requests through a worker queue and keep the app's main run loop alive for AppKit. This mirrors how a normal Swift or Objective-C app should host blocking native work.

## Current behavior

- Builds the same platform registry as `cua-driver mcp`.
- Supports direct tool calls with `call_tool`.
- Supports MCP `initialize`, `tools/list`, and `tools/call` through JSON-RPC request handlers.
- Exports a C ABI for non-Rust hosts.
- Exposes a root Swift Package product named `CuaDriverEmbedded`.
- Registers the same recording screenshot and click-marker callbacks as the standalone driver.
- Does not start the daemon proxy, stdio server, or `CuaDriver.app`.
- Does not initialize the visual cursor overlay. Host apps own their main thread and AppKit run loop.

## Constraints

- The host app still needs Accessibility and Screen Recording grants. Embedding changes which bundle gets authorized; it does not bypass TCC.
- Keep one embedded driver registry per process unless you have a reason to isolate state. Recording callbacks are process-global.
- Sandboxed Mac App Store apps are not the primary target. Some automation APIs are fundamentally constrained by sandboxing and user consent.
- If a tool launches a browser automation bridge or another external process, that subprocess has its own system-level behavior. The core AX and screen-capture operations should remain in-process for TCC attribution.
- On macOS today, `list_apps` still uses `osascript`/System Events internally and can require Automation permission from the host app. This is separate from the embedded driver process model and should be replaced with a fully native implementation before treating every tool as helper-free.

## See also

- App-bundle smoke test: `libs/cua-driver/rust/crates/cua-driver-embedded/examples/macos-app-smoke/run.sh`
- [MCP process model](./process-model) - when the standalone CLI stays in-process vs daemon-proxy mode.
- [Swift Integration](./swift-integration) - legacy Swift package products.
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
"description": "Get up and running with Cua Driver",
"icon": "Rocket",
"defaultOpen": true,
"pages": ["introduction", "installation", "quickstart", "windows-ssh", "linux", "autostart", "integrations", "swift-integration", "process-model", "comparison", "faq"]
"pages": ["introduction", "installation", "quickstart", "windows-ssh", "linux", "autostart", "integrations", "embedded-mcp", "swift-integration", "process-model", "comparison", "faq"]
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ import { Callout } from 'fumadocs-ui/components/callout';

`CuaDriverCore` and `CuaDriverServer` are available as Swift library products directly from the `trycua/cua` repository. Use this when you want to embed accessibility automation, window capture, or MCP tool handling into your own macOS Swift app or package — without shelling out to the `cua-driver` CLI.

<Callout type="warn">
This page documents the legacy Swift implementation. The maintained driver is now the Rust implementation. For new embedded integrations, use the Rust `cua-driver-embedded` crate and add a thin Swift or C ABI wrapper around its JSON-RPC MCP surface.
</Callout>

## Add the dependency

In your `Package.swift`:
Expand All @@ -15,7 +19,7 @@ In your `Package.swift`:
dependencies: [
.package(
url: "https://github.com/trycua/cua.git",
from: "cua-driver-v0.1.0"
.revision("cua-driver-v0.2.18")
),
],
```
Expand Down
13 changes: 13 additions & 0 deletions libs/cua-driver/rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions libs/cua-driver/rust/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
resolver = "2"
members = [
"crates/cua-driver",
"crates/cua-driver-embedded",
"crates/cua-driver-uia",
"crates/cua-driver-core",
"crates/platform-macos",
Expand Down
42 changes: 29 additions & 13 deletions libs/cua-driver/rust/crates/cua-driver-core/src/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,23 +30,22 @@ pub async fn run(registry: Arc<ToolRegistry>) -> anyhow::Result<()> {
}
debug!(raw = trimmed, "→ request");

let response = match serde_json::from_str::<Request>(trimmed) {
let Some(response) = (match serde_json::from_str::<Request>(trimmed) {
Err(e) => {
error!("JSON parse error: {e}");
Response::parse_error()
}
Ok(req) if req.is_notification() => {
// Notifications are silently dropped.
continue;
}
Ok(req) => {
let id = req.id.clone().unwrap_or(serde_json::Value::Null);
handle_request(req, id, &registry).await
Some(Response::parse_error())
}
Ok(req) => handle_request(req, &registry).await,
}) else {
// Notifications are silently dropped.
continue;
};

let serialized = serde_json::to_string(&response)
.unwrap_or_else(|e| format!(r#"{{"jsonrpc":"2.0","id":null,"error":{{"code":-32603,"message":"serialize error: {e}"}}}}"#));
let serialized = serde_json::to_string(&response).unwrap_or_else(|e| {
format!(
r#"{{"jsonrpc":"2.0","id":null,"error":{{"code":-32603,"message":"serialize error: {e}"}}}}"#
)
});
debug!(raw = %serialized, "← response");

writer.write_all(serialized.as_bytes()).await?;
Expand All @@ -57,7 +56,24 @@ pub async fn run(registry: Arc<ToolRegistry>) -> anyhow::Result<()> {
Ok(())
}

async fn handle_request(req: Request, id: serde_json::Value, registry: &Arc<ToolRegistry>) -> Response {
/// Handle one parsed MCP JSON-RPC request against a registry.
///
/// Returns `None` for notifications, matching the stdio server behavior.
/// Embedders can use this to expose cua-driver over an in-process transport
/// without going through stdin/stdout or launching the standalone driver.
pub async fn handle_request(req: Request, registry: &Arc<ToolRegistry>) -> Option<Response> {
if req.is_notification() {
return None;
}
let id = req.id.clone().unwrap_or(serde_json::Value::Null);
Some(dispatch_request(req, id, registry).await)
}

async fn dispatch_request(
req: Request,
id: serde_json::Value,
registry: &Arc<ToolRegistry>,
) -> Response {
match req.method.as_str() {
"initialize" => Response::ok(id, initialize_result()),

Expand Down
27 changes: 27 additions & 0 deletions libs/cua-driver/rust/crates/cua-driver-embedded/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "cua-driver-embedded"
version.workspace = true
edition.workspace = true
authors.workspace = true
license.workspace = true
repository.workspace = true

[lib]
crate-type = ["rlib", "staticlib", "cdylib"]

[dependencies]
cua-driver-core = { path = "../cua-driver-core" }
serde_json = { workspace = true }
tokio = { workspace = true }

[target.'cfg(target_os = "macos")'.dependencies]
platform-macos = { path = "../platform-macos" }

[target.'cfg(any(target_os = "windows", target_os = "linux"))'.dependencies]
cursor-overlay = { path = "../cursor-overlay" }

[target.'cfg(target_os = "windows")'.dependencies]
platform-windows = { path = "../platform-windows" }

[target.'cfg(target_os = "linux")'.dependencies]
platform-linux = { path = "../platform-linux" }
Loading
Loading