Mcp MCP Server

mandala-computer-mcp

An MCP server for Mandala Computer — cloud desktops for AI agents.

Point Claude Code, Claude Desktop, or anything else that speaks MCP at a real Linux desktop it can see and drive. Screenshots come back as images, so the model looks at the screen and clicks what it sees.

Status: alpha. The tool surface is settling; expect breaking changes before 1.0. Tracks the platform's /api/v1, which is itself still moving.

Install

MCP requires Node 20.3 or newer. The TypeScript CLI login setup requires Node 22 or newer (mandala-computer on npm). Sign in once, then add the MCP server:

npm install -g mandala-computer
mandala login
claude mcp add mandala -- npx -y mandala-computer-mcp

On Node 20, MCP can use an existing saved profile or an environment key. The TypeScript CLI guides you through browser approval and saves an API key in ~/.mandala/credentials.json. MCP only reads that file; it never issues or approves a device, writes credentials, or starts login automatically. The Python distribution also provides a mandala command; use the TypeScript CLI for login.

To select a separate profile and workspace:

mandala login --profile Work --workspace Research
claude mcp add mandala -- npx -y mandala-computer-mcp --profile Work

--profile takes precedence over MANDALA_PROFILE, then the saved default. Names are case-sensitive. An explicit key or nonempty MANDALA_API_KEY takes precedence over every profile and avoids accessing the credential store. Empty explicit keys fail; an empty or whitespace-only environment key is absent. You can still use a key from Settings → Credentials → API keys with MANDALA_API_KEY.

Saved credentials require POSIX protection: a real directory owned by you with mode 0700, and an owned regular file with mode 0600 and one link. Symlinks, hardlinks, unsafe permissions, malformed files and unsupported protection are refused before any API request. Windows file loading is unsupported; explicit or environment keys remain available. Only your home directory's store is read.

A saved profile also binds its API base URL. --base-url or MANDALA_BASE_URL must match that stored base after canonicalization, including the entire path prefix and port. An explicitly empty base flag is invalid with saved credentials. Each stdio session resolves once: restart to pick up another saved key. Revoking the key in Settings → Credentials → API keys makes later calls fail; MCP does not switch profiles, reread the file, or retry a refused action. Run login explicitly when new credentials are needed.

A saved profile can also carry a default workspace, set with mandala workspaces use <workspace> (and removed with --clear); the CLI keeps it in ~/.mandala/defaults.json, beside credentials.json, which it never changes. Started from that profile, the secret tools (list_secrets, get_secret, create_secret, set_secret, replace_secret, delete_secret), create_computer and list_computers use it when a call leaves out workspace_id, as mandala computers create and computers list do; an explicit workspace_id always wins. Their descriptions say so and name the workspace, list_secrets, list_computers and create_computer say when the default chose the workspace, and startup prints it on stderr. There is no "every workspace" value: to list elsewhere, pass another workspace_id (or 'unassigned'), or start the server from a profile with no default. MCP only reads that file, once at startup, with the same checks as credentials.json. It is not applied when the profile's key is confined to a workspace (that key's own scope applies), when the default was saved for another account than the profile is logged in to now, or with an explicit or environment key. A file that cannot be read (not valid JSON, another version, or readable by others) is noted on stderr: list_secrets, get_secret and list_computers then use account-wide, but create_secret, set_secret, replace_secret, delete_secret and create_computer refuse a call without workspace_id, sending nothing, rather than act account-wide; pass workspace_id, or fix or delete the file and restart the server. The hosted HTTP server never reads it, and its tools are unchanged.

Claude Code — as a plugin, which installs the server and a skill together:

# Run mandala login first, or export MANDALA_API_KEY in this shell.
/plugin marketplace add mandalacomputer/mcp
/plugin install mandala-computer@mandala

The skill — plugin/skills/mandala-computer/SKILL.md — is the part the tools cannot say for themselves: when a cloud desktop is the right answer at all, that it costs money until it is suspended or stopped, that run_agent is usually the right level and a screenshot per click is not, and which refusals are worth a second try. It is a description of when and how, not a second client; once the server is installed it stays out of the way. MANDALA_MODEL_KEY, if exported alongside, is passed through and turns on run_agent when the configured filters permit it. Without it, run_agent runs on the account's AI credits when they can pay (see Agents on AI credits).

To use an environment key instead of a saved profile:

claude mcp add mandala -e MANDALA_API_KEY=com_… -- npx -y mandala-computer-mcp

Claude Desktop — in claude_desktop_config.json:

{
  "mcpServers": {
    "mandala": {
      "command": "npx",
      "args": ["-y", "mandala-computer-mcp"]
    }
  }
}

Cursor, Windsurf and the rest take the same three fields — command, args, env — in whichever file they keep their MCP servers in.

Your MCP client starts this as a subprocess. It uses the saved profile’s API base, or https://app.mandala.computer/api/v1 by default with an environment key. Programmatic local hosts can call runStdio({ profile: 'Work' }); the exported StdioConfig allows a local key or profile. createServer still requires an explicit API key, and HttpConfig has no local credential options.

Use

Ask for what you want. A first session usually goes:

Create a computer from the base template, open example.com, and show me what's on the screen.

Under that, the model is doing roughly this:

create_computer(template="base")     → builds it and selects it for the session
wait_for_computer(until="guest")     → the desktop inside is answering, not just the VM
open_url(url="https://example.com")  → puts the page on the screen
screenshot()                         → an image the model can point at
click(x=640, y=400)                  → clicks what it saw
screenshot()                         → looks again

create_computer(size="large") is the fast path: the named shapes from list_sizes are the ones the platform keeps pre-booted, so one of those is usually answered in about a second where a custom cpu/ram_mb/disk_gb shape boots cold. A size sets the template and the numbers together, so send it alone or the explicit fields alone.

use_computer binds a machine to the session, so every later call can leave computer_id out. Pass computer_id explicitly on any call to override it without changing the binding, which is how you drive two machines at once.

Set MANDALA_COMPUTER_ID to bind one at startup and skip use_computer entirely.

The tools

This is the unfiltered inventory. The filters below can withhold tools from both listing and calling; workflows in this README apply only when the needed tools are available. The offline fixture checks exercised operation coverage; verification against the live publication is a separate required CI gate. Parameter and response-mode support remains a separate contract.

Choosing a machine — list_templates, list_sizes, list_computers, get_computer, use_computer, wait_for_computer, get_desktop_url

Lifecycle — create_computer, start_computer, stop_computer, suspend_computer, restart_computer, update_computer, clone_computer, delete_computer, move_computer_to_workspace, move_computer, list_moves, get_operation, list_operations, wait_for_operation — see Lifecycle operations

move_computer_to_workspace changes workspace membership and who can access a computer. It requires an account-wide member or owner credential and explicit computer_id and workspace_id arguments (null means unassigned). The selected computer and default workspace are not used. The computer must be stopped, with no saved session, snapshots or snapshot history copies, secret bindings, or proxy credentials. Refusals leave those resources intact; the tool never removes them to force a move. This is separate from move_computer, which relocates a computer to different hardware.

Driving the desktop — screenshot, click, type_text, paste_text, press_key, scroll, drag, move_mouse, mouse_button, cursor_position, wait

Every one of these that acts — all but screenshot and cursor_position — takes context: true and then answers the desktop as it stands just after the action: the windows list_windows would list and the one with focus, without a second call. They are read once, straight after the action, so a window still opening may not be listed yet; when they cannot be read the action still happened, and the answer says why. When the focused window is Chromium the answer also carries page: the page's URL and title and one line per interactive element visible in it — what it is, what it says (and its accessible name when that is something else), and the point to click. Firefox, the default browser, has no page; then page_not_read gives the platform's reason beside the windows.

Inside the guest — exec, exec_poll, exec_kill, get_execution, read_execution_output, open_url, list_windows, window_action, read_clipboard, write_clipboard, read_file, write_file, list_directory

write_file replaces a file already at the path. With overwrite: false it creates the file only if nothing is there, and a path that is taken is refused without that attempt writing anything (Linux computers only). Incomplete contents are never published at the path, but a failure while publishing or answering can leave the complete file there, so after any error read the path before retrying or overwriting. read_file and write_file take no_wake: true to refuse (409) rather than resume a computer that is not running. For embedders, the Api raises that refusal as FileExistsError, and a create-only 409 whose reason could not be read as CreateOnlyConflictError, which says nothing about the path; isTransient is false for both.

Retained versions — retain_execution_output, get_result, read_result_output, delete_result, publish_artifact, get_artifact, read_artifact, delete_artifact

Passive metadata — list_activities, get_activity, get_activity_results, read_signals

Being told rather than asking — wait_for_event, poll_events, wait_for_file_change

Snapshots — list_snapshots, snapshot_holdings, create_snapshot, restore_snapshot, clone_snapshot, snapshot_schedule, get_retention, delete_snapshot

Your own templates — get_template_schema, check_template, publish_template, get_template, retire_template

Building one — build_template, list_builds, get_build, watch_build

Account quota — get_account

Who you are — whoami, list_api_keys, list_workspaces, get_workspace, list_workspace_members, create_workspace, rename_workspace, delete_workspace. Minting and revoking API keys are deliberately not tools — see Who you are, and API keys.

Deliberately not tools — five API operations, each because it would put a credential in a model's context: minting and revoking API keys (above), and the three browser-connection routes (create, renew and revoke). A browser connection is a raw capability to drive the guest's browser; the TypeScript and Python SDKs' browser drivers create, renew and revoke it and keep it to themselves. Every other API operation has a tool.

Every tool refuses an argument it does not declare. A misspelt or unknown key is an input error naming the key, and nothing is sent — so a misspelt flag can never be dropped and the call go ahead without it.

Spending — get_usage, get_credits

Being told somewhere else — list_webhooks, create_webhook, get_webhook, update_webhook, rotate_webhook_secret, test_webhook, list_webhook_deliveries, delete_webhook

SSH access — list_ssh_keys, add_ssh_key, remove_ssh_key, get_computer_ssh, set_computer_ssh. Keys belong to the person the API key was issued to, not to the account. A key added with add_ssh_key (an API key or connected app) is accepted only by computers with SSH on in the account that credential acts on, and is removed when the credential is revoked; a key added from the dashboard is accepted on every account where that person is an owner or member. list_ssh_keys marks each key with where it works from the account the API key acts on: every account, this account only, or another account only (refused here; which account is not said). A key is registered once, so a key bound to another account cannot be added again here. remove_ssh_key removes only a key marked this account only: one added from the dashboard (every account) or bound to another account answers as if there were no such key, and the person removes it from the dashboard. A workspace-scoped key can read keys but not add or remove them.

Showing an app on the internet — list_public_urls, create_public_url, update_public_url, delete_public_url. A port inside a computer served at https://<label>.<base_domain> (the domain is in the list_public_urls answer, e.g. mandala.dev), on 443 with no port in the address. Public by default: anyone with the link can open it; access: "private" requires an API key of this account as Authorization: Bearer, which never reaches the app. The app must listen on 0.0.0.0, not 127.0.0.1. Labels that read as the platform's or another company's address are refused. A deleted URL's label stays with the account: no other account can ever take it, and the account may use it again unless the platform has blocked it. A computer over its plan's limit serves only its oldest URLs, and the listing marks the rest.

A browser opening a new account's URL may first see a one-time notice page. It goes to any request that looks like a browser opening a page — a navigation by its fetch metadata, or, without fetch metadata, Accept: text/html from a Mozilla/ user agent — so a program sending browser headers sees it too. A client that sets headers skips it with mandala-skip-warning: 1; in the computer's own browser, where open_url cannot send headers, click Continue.

Secrets — list_secrets, get_secret, create_secret, set_secret, replace_secret, delete_secret for the account's secret store, and get_computer_secrets, set_computer_secrets and create_computer's secrets for which of them a computer receives. A value goes in through create_secret, set_secret (create the name, or replace its value if the scope holds it — names match ignoring ASCII case) or replace_secret, and never comes back out. No route answers one, and a store tool's result holds only the decoded documented fields (success) or the status, the reason word and a sentence of its own (refusal). It never includes the platform's response text, and the Api keeps none for these routes. The bindings carry only secret ids, revisions and names. A computer receives a secret as an environment variable (env) or as a file under /run/mandala-secrets/user/files (file). A set replaces the whole binding list ([] removes every binding) and reaches the guest at the computer's next start or restart; send the version a read answered to have it refused with 409 if the list changed since. A replaced value also reaches a running computer: a file binding's file is rewritten in place, and on an image that supports it an env binding reaches new shells, and every exec, plain or with desktop: true, runs with the bound variables as they are at that moment (programs already running keep the old value until a restart). replace_secret and delete_secret need the current revision_id, and a stale one is a 409. delete_secret also needs confirm: true: a computer still bound to a deleted secret cannot start again until that binding is removed. A bound computer runs, and its guest answers, a few seconds before its secrets land, so wait_for_computer(until="guest") on one also waits until they have; get_computer shows the gap as "its secrets are still on their way in".

A proxy for the browsers — browser_proxy: {server, bypass} on create_computer and update_computer sends a Linux computer's browsers (Chromium, Chrome, Firefox) through a proxy; nothing else on the computer uses it. On update_computer it goes alone, replaces the setting whole, and null removes it. Which proxies are accepted is the platform's rule, and its refusal comes back as it is. A running computer has a change within seconds: wait_for_computer(until="guest") waits until its browsers have it, and get_computer shows the gap as "its browser proxy is still being applied".

A proxy for all outbound traffic — egress_proxy: {server, credentials_secret_id} on create_computer and update_computer sends ALL of a computer's outbound TCP (exec, terminals, package managers and browsers alike) through a proxy, taken on its host so nothing inside the computer can opt out. The server is http://, https:// or socks5:// with an explicit port; there is no bypass list. credentials_secret_id names a secret holding user:password that is not bound to the computer and never reaches it: the computer's host signs in to the proxy with it. It fails closed (proxy down or refusing, or credentials not on the host yet: the connection fails, nothing goes direct), drops UDP to the internet and ICMP, and does not proxy DNS lookups. On update_computer it goes alone — beside any other field it is refused before a request is sent — replaces the setting whole, and null removes it. get_computer names the proxy and, while the host waits for its credentials, says "its egress proxy is waiting for credentials; connections are closed until they arrive".

Delegating — run_agent, run_agent_chat, registered when a model key is present: MANDALA_MODEL_KEY on stdio, or the caller's own X-Model-Key header over HTTP. Without a key, run_agent alone is registered when the account's AI credits can pay for a run (see Agents on AI credits). Both must also survive the configured filters.

Tool filters

Set MANDALA_READ_ONLY=1 to register only tools whose existing annotations.readOnlyHint is exactly true. It accepts 1, true, yes, on for on and 0, false, no, off, empty or unset for off, ignoring surrounding whitespace and letter case. Any other value fails at startup. Safety is not inferred from a tool's name or HTTP method: read_file and cursor_position can wake and bill a computer, so they are withheld. So are mixed read/write tools such as wait_for_computer and snapshot_schedule. screenshot, read_clipboard and list_windows retain their read-only hints.

Set MANDALA_TAGS=input,guest to select a union of named tool groups. Names are lowercase only, comma-separated, trimmed and deduplicated. Empty or unset means no tag filter; empty comma-separated entries are ignored. An unknown nonempty tag fails before stdio connects or HTTP starts listening, with an error listing all valid tags.

The selected tags form a union, then intersect with read-only and the existing lifecycle and model-key restrictions. MANDALA_NO_LIFECYCLE=1 still withholds its five tools even when their tags are selected; selecting agent cannot enable run_agent_chat without a per-session model key, nor run_agent without one or credits that can pay. Both agent tools are withheld by read-only, but remain available with lifecycle disabled alone. MANDALA_TAGS=files MANDALA_READ_ONLY=1 exposes only list_directory. An agent selection with read-only is a valid empty tool list. activities,results includes get_activity_results once; signals is separate from the active events socket. Filters never grant API privileges: the API checks current member/owner and workspace authorization on every request.

use_computer is not read-only. When it is withheld, pass computer_id explicitly, or bind a computer with MANDALA_COMPUTER_ID at stdio startup. HTTP callers must supply their own computer selection. These variables apply to both transports and the plugin forwards them. Embedders can pass readOnly: true and tags: ['input', 'guest'] in ServerConfig; the server does not read the environment itself.

Lifecycle operations

Every accepted create, clone, start, stop, suspend, restart, snapshot restore, resize, host move and delete records a lifecycle operation, and the tool that made the call says its operation_id — in its sentence, as (operation op_…), and in the JSON. move_computer carries it onto the outcome it reports. It is absent where the platform could not record one; the call happened either way.

get_operation reads one, and list_operations pages through them newest first (computer_id, limit, cursor, idempotency_key; computer_id is not defaulted to the selected computer, and idempotency_key finds the operation a call sent with that key recorded, even when the call's answer was lost). wait_for_operation polls one until it is final: it answers on succeeded, and a failed one is an error carrying the platform's error.code (start_failed, build_failed, computer_gone, move_failed, resize_not_applied, lost, and more may be added) and its sentence.

succeeded means the platform finished its step, not that the desktop has booted: wait_for_computer is still the wait for a desktop that answers. Most operations are already succeeded when their tool answers; a clone is running until its disk is copied, and a move until it lands. All three are reads, and all three stay registered when the lifecycle tools are withheld, since start, stop, suspend and restart record operations too.

The eleven lifecycle tools (create_computer, clone_computer, start_computer, stop_computer, suspend_computer, restart_computer, update_computer, move_computer, delete_computer, restore_snapshot, clone_snapshot) take an optional idempotency_key. Leave it out and a fresh one is sent for you; a failure whose outcome is unknown names the key it went with. What to do next depends on who answered:

• The answer was lost (the connection dropped after the request went out, or a proxy in front of the platform gave up), or the platform answers idempotency_in_progress: call the same tool again with the same idempotency_key. The platform answers with the first call's result, or says it is still running; the step is not done twice.
• The platform answered a 5xx that names no operation_id: nothing may have been done. A refusal given before the call was sent anywhere — another launch already in progress, no host to place it on, the template catalogue incomplete — names none, and the platform releases its key. Call the same tool again with the same idempotency_key: the platform carries it out if the key was released, or answers idempotency_outcome_unknown if not, and only then does the case below apply.
• The platform answered a 5xx that names an operation_id, or idempotency_outcome_unknown: the key is spent, and resending with it only answers idempotency_outcome_unknown. The platform then keeps the call's operation pending for up to an hour, whatever happened, while its host may be carrying it out. What to read depends on the tool:
  • start_computer, stop_computer, suspend_computer, delete_computer, move_computer and update_computer: read get_computer (and get_operation with the operation_id the error named: succeeded means the step happened, running means wait on it, and pending alone is no reason to wait). If the step did not take effect, send the call again with a new key, or none.
  • restart_computer, which get_computer cannot show: a computer reads running before and after a reset. Read the operation instead (get_operation with the operation_id the error named, or list_operations with the key): succeeded means the restart happened, running means wait on it. Anything else leaves the restart possibly done, so ask the user before sending it again with a new key, or none; a second restart resets the guest again.
  • create_computer, clone_computer, clone_snapshot and restore_snapshot, where a read of the computer made straight away can show no effect for a build or restore that then lands, and a second one is a second computer or a second overwrite: read the operation first — get_operation with the operation_id the error named, or list_operations with the key — and while it is pending or running, wait (wait_for_operation) and do not resend with any key. Only once it is final as failed (error.code lost, for one) or no longer found, AND get_computer (list_computers after a create or a clone) shows the step did not happen, send the call again with a new key, or none. If no operation is found at all, read the computer again after a few minutes before resending.
• idempotency_key_reused: a different call already used that key, and nothing was done. Send this one with a new key, or none.

Keys last 24 hours.

Who you are, and API keys

whoami takes no arguments and reads GET /api/v1/whoami: the person this server's API key was issued to, the account and role it acts with (the role as it is now), the workspace it is confined to, and the key itself — including manage_keys, whether it may manage API keys. It needs no permission and any role, and a suspended account can call it; the answer says so when the account is suspended. Behind the hosted server's OAuth sign-in, the key is the Connected app's (prefix oauth).

list_api_keys lists the key holder's API keys on the account, newest first — never a raw key. Each carries minted_by_key_id: the key that minted it over the API, or null for one minted from the dashboard. Revoking a key does not revoke the keys it minted. It needs this server's key to be allowed to manage keys, which only a person can turn on in the dashboard (Settings → Credentials → API keys → the key's menu → Allow managing keys); without it the tool answers the platform's 403, whose sentence says exactly that. A Connected app's key never has the permission.

Minting and revoking keys are not offered here, on purpose. A mint answers the new key in full, once, and a model's context — the conversation, the client's logs, whatever the transcript is shared with — is the worst place for a long-lived credential and one nobody can take it back from. A revoke is irreversible, and on a model's reading of a list it can cut off the person's CI, another agent, or this very session. People do both from the dashboard or with the mandala api-keys CLI.

list_workspaces, get_workspace and list_workspace_members read the account's workspaces, which partition its computers. A key confined to a workspace lists only that one, and is refused (403) the member list, which is the whole account's: workspaces do not divide people. All three are reads.

create_workspace, rename_workspace and delete_workspace change them, and need an owner's key that is not confined to a workspace: any key confined to one is refused (403), so none of them can touch the workspace this server's own key stands in. delete_workspace needs confirm: true and REVOKES every API key confined to the workspace, whoever holds it — a person's CI or another agent may be using one — and answers how many. Only an empty workspace can be deleted: one that still holds computers or active secrets is refused (409), saying how many, and nothing is deleted or revoked. Move its computers out with move_computer_to_workspace (or delete them) and delete its secrets with delete_secret, then send it again. None of the three is offered to a read-only session (MANDALA_READ_ONLY).

An account-wide key can work in one workspace without a key confined to it: create_computer(workspace_id: ...) creates the computer there, and names its secrets and an egress proxy's credentials_secret_id from that workspace's scope (its own secrets and the account-wide ones), and list_computers(workspace_id: ...) lists that workspace's computers, or with 'unassigned' the ones in none. A workspace the key cannot reach is not found, and so is 'unassigned' from a confined key. Left out, these two tools apply the saved profile's default workspace, as the secret tools do (see above); with no default, the computer goes in the key's own workspace or in none, and the listing is every computer the key can see.

Current account quota

get_account takes no arguments and reads GET /api/v1/account once with the caller's account credential. Viewer or stronger access is required. It reports account-wide aggregates, including for a workspace-scoped key, without resource identities. It needs no selected computer or model key, opens no event stream, and remains available with read-only and no-lifecycle filters. Select the account tag to expose it on its own.

The report includes the effective plan, pool ceilings, per-computer maxima, Windows capability, current consumption and remaining quota. Configured CPU and disk include kept computers regardless of power state; running or reserved RAM includes current reservations. CPU, MB, GB and snapshot bytes retain the API's units. get_usage separately reports historical metered consumption over time.

Quota is advisory: observed_at is an observation time, not a reservation or consistency token. Later create, resize, start or snapshot requests can still be refused, and existing 402 messages remain unchanged. Snapshot headroom is against indexed stored bytes; it does not include in-flight capture reservations and does not establish that a new capture will fit.

complete.computers and complete.snapshots are independent. An incomplete group has null for all its consumption and remaining figures, meaning unknown, while the other group can retain numeric values. Complete zero usage and zero remaining quota remain numeric zeros. Verified plan ceilings remain available in a partial report, including a no-plan account that still has retained usage. Malformed reports and HTTP failures remain errors, never an empty account. The tool prints unknown and advisory guidance before the projected public fields.

Things worth knowing

Every computer is a Linux desktop today. Windows guests are not offered on any plan; where this README mentions Windows it is describing behaviour the client already supports for when they are.

A running computer costs money, and a forgotten one keeps costing it. create_computer says so in its own description, and so does everything that starts a machine by a side door — restore_snapshot boots a stopped computer, and write_clipboard, read_file and cursor_position each resume a suspended one, because each has to reach the guest agent to do its job. All of them are charged like any other start, and on a plan at its limit can come back 402 rather than a result. The two reads are the surprising half of that list, and are why neither is annotated readOnlyHint: a host that auto-approves read-only tools would otherwise wake and bill a machine with nobody asked. screenshot, read_clipboard and list_windows genuinely do not start anything and keep the hint. When a stretch of work is over, suspend_computer (a pause: start_computer brings the same session back in about a second) or stop_computer (a shutdown: the disk is kept, the session is not). Idle suspend catches the ones a model forgets, but only after 30 minutes untouched. get_usage is what says what any of it cost.

Some defaults here differ from the SDKs', on purpose. screenshot's fresh defaults to true (the SDKs default to false), because after a click a cached frame can predate the action and a model reading it clicks again. Its width outside 64-3840 is refused rather than clamped as the API would. wait_for_computer polls every 2 seconds (an SDK launch polls every 3), and refuses a suspended computer and names start_computer, where the SDKs' waitForGuest resumes it.

A screenshot need not be the whole screen. screenshot takes region (a crop, in full-size capture pixels before any scaling), scale (0 to 1), format (png or jpeg) and quality (JPEG, 1-100). A cropped or scaled picture is in its own pixel space, so the tool says alongside it how to turn a position in it back into screen coordinates.

Live screenshots also return a capture name and the capture's original size, before cropping or scaling. Use that size instead of the computer record's resolution, which can differ. To crop a frame you already measured, send its capture with region; omit fresh or set it to false. Without capture, screenshots remain fresh by default. fresh: true cannot be combined with capture. If the capture is no longer held (stale_capture), take a new screenshot without capture or region, then remeasure the region and use the new capture name. Saved frames of suspended computers cannot be pinned this way.

Ten clicks need not be ten screenshots. Driving the desktop one tool at a time — screenshot, click, screenshot — puts an image in the calling model's context for every step. run_agent hands a task in plain language to the platform's own loop instead, which screenshots, decides and clicks inside the platform and answers with a sentence and the list of what it did. It is registered when a model key is present (see Configuration), and bills that key for the run, or else when the account's AI credits can pay (see Agents on AI credits). max_steps bounds the WORK rather than the bill: a step is one action on the desktop, one model reply can ask for several and spends a step on each, a paused turn costs tokens and no step, and not every step takes a screenshot. It defaults to 20 and is capped at 100 here.

A screenshot is how you find out what the screen looks like. A click that landed and a click that did nothing produce the same tool result, so a model that acts without looking is the single most common way one of these sessions goes wrong. The tools say so in their own descriptions and the server's instructions say it once more.

But a screenshot is no longer how you find out whether anything happened. The platform's computers report what they do — a window opening, closing or taking focus, the clipboard changing hands, a background command exiting, the desktop coming up, the machine going idle, every power transition — and wait_for_event blocks until one of those arrives instead of screenshotting in a loop to discover that nothing has.

The part worth understanding is where the socket lives. A model takes turns; it is not sitting in a loop reading a stream, and between two of its turns there is nobody here to read one. So this server holds the connection, one per computer, opened the first time a tool asks about it and kept across turns. What arrives while the model is doing something else is buffered, and the next wait_for_event or poll_events is handed it in order. The model holds a cursor and never learns that a socket exists.

Three consequences, and they are the whole of the design:

• A wait that times out has missed nothing. The stream stayed open while the tool call was not running. That is why the timeout is capped at 55 seconds rather than the fifteen minutes wait_for_computer allows — a short wait costs nothing, because calling again picks up exactly where it left off. A timeout is a normal answer here, not an error.
• An event that already happened still ends a wait. computer.ready fires once per desktop session, so a machine that has been up for an hour will never send it again; attach to one and you are handed a computer.ready marked synthesized rather than waiting forever for an event that cannot arrive.
• A hole in the history is answered, not forwarded. When the platform cannot replay from where this server had got to, the events that survived come back with a count of what did not and with the state the missing ones would have reported — the window listing and the computer's own record. The gap frame itself never reaches the model, because a model handed one would invent a recovery procedure.

Documentation truncated — see the full README on GitHub.