claude-hidden-toolkit/tool-catalog/hidden-tools/computer-use-tools.md

Computer Use Tools (mcp__computer-use__*)

Status: The computer tool exists under mcp__Claude_in_Chrome__computer in Dispatch sessions (verified via ToolSearch and Chrome tab context). The standalone mcp__computer-use__* namespace was not found in deferred tools — it requires an explicit computer-use MCP connection. The Chrome-based computer tool includes all 13 actions documented below.

Verified: 2026-03-19. mcp__Claude_in_Chrome__computer screenshot action tested and working in Dispatch.

Full desktop automation via screenshot-then-act loop. Registered as a dynamic MCP server — tools appear as mcp__computer-use__* in the deferred tools list only when the computer-use MCP is connected.

Priority order (from system prompt):

1. Dedicated MCP for the app (Slack, Gmail, etc.) — fastest, most precise
2. Chrome MCP (mcp__Claude in Chrome__*) — DOM-aware, faster than pixels
3. Computer use — only when neither above fits (native desktop apps, cross-app workflows)

---

computer

The single tool that handles all desktop interactions through an action parameter.

Parameters

Param	Type	Required	Description
action	string (enum)	yes	One of 13 actions (see below)
coordinate	[number, number]	varies	(x, y) pixel coordinates from top-left. Required for click/scroll actions. For left_click_drag, this is the end position.
text	string	varies	Text to type (for type) or key(s) to press (for key). For key: space-separated, supports modifier combos like "cmd+a" or "ctrl+a".
duration	number	for wait	Seconds to wait. Max 5 seconds.
scroll_direction	string (enum)	for scroll	"up", "down", "left", "right"
scroll_amount	number	no	Scroll wheel ticks (1-10). Default: 3.
start_coordinate	[number, number]	for left_click_drag	Starting position for drag.
region	[number, number, number, number]	for zoom	(x0, y0, x1, y1) rectangle to capture.
repeat	number	no	Times to repeat key sequence (1-100). Default: 1. Only for key action.
ref	string	for scroll_to	Element reference ID from read_page or find tools (e.g., "ref_1"). Can substitute for coordinate in click actions.
modifiers	string	no	Modifier keys for clicks: "ctrl", "shift", "alt", "cmd"/"meta", "win"/"windows". Combine with "+" (e.g., "ctrl+shift").
tabId	number	yes	Tab ID — must be in current group. Use tabs_context_mcp first.
save_to_disk	boolean	no	For screenshot/zoom: save image to disk so it can be attached to a message. Returns saved path.

Actions

Action	Description	Required Params
left_click	Left mouse click	coordinate or ref, tabId
right_click	Right mouse click (context menus)	coordinate or ref, tabId
double_click	Double left click	coordinate or ref, tabId
triple_click	Triple left click	coordinate or ref, tabId
type	Type text string	text, tabId
screenshot	Take screenshot of the screen	tabId
wait	Wait N seconds	duration, tabId
scroll	Scroll at coordinates	coordinate, scroll_direction, tabId
key	Press keyboard key(s)	text, tabId
left_click_drag	Drag from start to end	start_coordinate, coordinate, tabId
zoom	Capture region for close inspection	region, tabId
scroll_to	Scroll element into view by ref ID	ref, tabId
hover	Move cursor without clicking (tooltips, dropdowns, hover states)	coordinate or ref, tabId

Notes

• Every action requires tabId — call tabs_context_mcp first
• Key combos use platform modifier: "cmd" on Mac, "ctrl" on Windows/Linux
• save_to_disk returns a file path — only use when sharing the screenshot with the user
• Click positioning tip from system prompt: "cursor tip in the center of the element, don't click edges"

---

Access Flow: request_access

Before any computer-use action, request_access must be called with the list of applications needed. This is a permission handler invoked as computer:request_access — it functions like a tool call but goes through the permission system.

Input

Param	Type	Description
apps	array	List of applications to request access to

Tiered App Access

App Category	Tier	What's Allowed
Browsers (Safari, Chrome, Firefox, Edge, Arc, etc.)	read	Visible in screenshots, but clicks and typing blocked. Use Chrome MCP instead.
Terminals & IDEs (Terminal, iTerm, VS Code, JetBrains, etc.)	click	Visible and left-clickable, but typing/key presses/right-click restricted.
All other apps	full	Full access — screenshots, clicks, typing, all interactions.

Returns

Field	Type	Description
granted	array	Apps that were approved with their bundleId and tier
denied	array	Apps that were denied with bundleId and reason ("user_denied")
flags	object	Permission flags

Notes

• User approves each application explicitly via a dialog
• May need to call again mid-task if another app is needed
• Browser tier (read) is a hard restriction — use Chrome MCP for browser interaction
• Terminal tier (click) blocks typing — use bash tools for terminal interaction

---

Teach Mode: request_teach_access + teach_step

An interactive walkthrough system for showing users how to do something on their screen. This is the system behind the chicago codename (see feature-flags-and-codenames.md).

Trigger: When a user wants to be "walked through" or "shown how" to do something, the model offers a choice between (1) interactive walkthrough and (2) text explanation.

Flow

1. Call request_teach_access — activates teach mode for the session
2. Call teach_step repeatedly — each step shows the user what to do
3. The user follows along; teach mode tracks their progress
4. Session ends when user exits or all steps complete

Internal state

• teachModeActive (boolean) — whether teach mode is running
• teachModeEnteredAt (timestamp) — when teach mode started
• cuLockAcquiredAt (timestamp) — when computer use lock was acquired

Notes

• These are not standard tool registrations — they're handled by the teach mode subsystem
• teach_step returns a Promise that resolves when the user takes action ({action: "exit"} etc.)
• Only one pending teach step at a time — a new one auto-resolves the previous as "exit"
• The isChicagoEnabled feature flag forces teach mode enabled regardless of GrowthBook config
• chicagoAutoUnhide (default: true) and chicagoUserDeniedBundleIds (default: []) control teach mode UX

---

Computer Use Lock System

Computer use has a session-level lock to prevent multiple CU sessions from conflicting:

• Only one session can hold the CU lock at a time
• Lock tracks duration held and whether teach mode was active
• Release triggers include: session end, explicit release, teach mode exit