# Computer use (opt-in) Cluster Build can observe your desktop and drive it — clicking, scrolling, typing, and pressing keys — through a single native `computer` tool, available in interactive sessions on Linux GNOME Wayland and on macOS 26 or newer. It's off by default and requires two separate consent prompts before it can see or touch anything. Computer use is not browser control — the built-in Playwright provider still owns web pages. The `computer` tool is for native applications that can't be reached through the browser. ## Enabling it [#enabling-it] ```sh cluster config set computer_enabled true ``` or set it directly in `~/.cluster/config.toml`: ```toml [computer] enabled = true ``` The interactive TUI only advertises `computer` when this is enabled **and** a supported native backend is available. Headless surfaces — `cluster exec`, cron runs, local child agents, and app-server sessions — never advertise it, regardless of this setting. ### Linux setup [#linux-setup] The Linux backend targets an already logged-in GNOME Wayland session. It needs `gnome-screenshot` for capture and a readable, writable `/dev/uinput` for pointer input, which a one-time machine setup installs: ```bash version="$(cluster --version | awk '{print $2}')" base="https://assets.cluster.app/serve/cstatic-assets/releases/cli/${version}" curl -fLO "${base}/setup-linux-computer.sh" curl -fLO "${base}/70-cluster-uinput.rules" chmod +x setup-linux-computer.sh ./setup-linux-computer.sh ``` Keep both files together — the script installs the udev `uaccess` rule next to it. It currently supports Debian/Ubuntu. Capture is prompt-free: no XDG portal picker is opened, and nothing is shared with the compositor beyond the session Cluster is already running in. Without the helper or the input-device grant, the backend stays unavailable and the tool is never advertised. ### macOS setup [#macos-setup] macOS attributes Screen Recording and Accessibility to an application launched through Launch Services, not reliably to a terminal child process. The release therefore ships the `cluster` executable inside a background-only `Cluster.app`, with `~/.local/bin/cluster` symlinked to it — one binary, not a second CLI. The foreground CLI launches that same binary through Launch Services as a short-lived local broker and talks to it over a private, owner-scoped Unix socket under `~/.cluster/computer/`. Running a `cluster` binary from outside the app bundle leaves computer use unavailable. Two macOS privacy grants are required, both in **System Settings → Privacy & Security**: * **Screen & System Audio Recording** — for `observe` and every post-action screenshot. * **Accessibility** — for `click`, `scroll`, `type`, and `key`. They're reported as separate errors, so a missing grant names the one to fix. Grant it and restart `cluster`. The desktop session must be unlocked — Cluster holds the display awake for the run but never bypasses the login window. ## What's implemented [#whats-implemented] * **`observe`** captures the desktop and returns a screenshot plus a `frame_id`. Linux captures the full desktop; macOS captures the primary display, and secondary displays aren't supported yet. * **`click`** performs a left click at image coordinates grounded against a specific `frame_id`, then returns a fresh screenshot. An optional `click_count` of `1` (default) or `2` requests a single or double click; other values are rejected. * **`scroll`** scrolls at a grounded point, up to 10 wheel steps per axis. * **`type`** types literal printable ASCII text (tab and newline included, up to 1000 characters) into the application focused in that frame. * **`key`** presses one bounded key — a letter, digit, `f1`–`f12`, or a named navigation key such as `enter`, `escape`, or `arrow_up` — optionally with `ctrl`, `alt`, `shift`, or `meta` held. * **`wait`** delays up to 10 seconds (`duration_ms` in `1..=10000`) before capturing a fresh screenshot, with no input performed. It's for applications whose visual state changes asynchronously — not a background task or an unbounded polling primitive. Cancelling a `wait` mid-delay performs no capture; the next action must observe again. Drag and free pointer movement are not exposed. Windows has no backend. Every action is validated against the frame it was grounded on: a stale `frame_id`, out-of-bounds coordinates, or display geometry that changed since the observation is rejected rather than acted on. ## Consent [#consent] Two prompts gate this feature independently, each scoped to the current process — separate from the OS-level grants above, which are machine-level and persist: * **Capture** — approves sending a screenshot of the desktop to the model. Required before the first `observe`. * **Control** — approves sending pointer and keyboard input to the desktop. Required before the first `click`, `scroll`, `type`, or `key`, separately from capture. `wait` only requires capture consent, like `observe`. Approving one does not approve the other, and neither persists past the process — restarting `cluster` asks again. Set `computer_unattended` to treat your machine-level setup plus that explicit opt-in as standing authorization and skip both prompts; leave it off for interactive use. Screenshots and action results follow the same private, local session storage as other tool results; nothing is uploaded except as model request content for the active turn. ## Plan mode [#plan-mode] `observe` is read-only and allowed in [plan mode](/docs/build/interactive#plan-mode). `wait` is likewise allowed in plan mode. `click`, `scroll`, `type`, and `key` are mutating actions and are denied in plan mode like any other mutating tool. ## Next steps [#next-steps] * **[Advanced configuration](/docs/build/config/advanced)** — the rest of `config.toml`. * **[Interactive sessions](/docs/build/interactive)** — plan mode and workspace trust, which govern when tools may run at all.