Dispatch Mode

Let your agent control your computer — run shell commands, take screenshots, click buttons, type text, and navigate any app on your Mac or PC.

How It Works

Your agent runs on a dedicated server 24/7. Dispatch Mode creates a secure tunnel between your agent and your personal computer. When you run the relay, your agent can see your screen, run commands, and interact with any application — just like a remote assistant sitting at your desk.

Your agent uses two methods depending on the task: direct shell execution for file operations, commands, and automation (fastest — no GUI needed), and visual control (screenshots + clicks) for interacting with app interfaces.

Quick Start

1. Go to instaclaw.io/settings and click "Connect Your Computer". The command is copied to your clipboard automatically.

2. Open Terminal and paste the command:

npx @instaclaw/dispatch@latest

3. Press Enter. Your agent connects automatically via encrypted WebSocket. You'll see a confirmation in Terminal:

  ✓ Connected to your agent!
  Mode: Supervised
  Your agent can now control this computer.

4. When you give your agent a task, press 'a' in Terminal to switch to autonomous mode — this lets your agent work through multi-step tasks without pausing for approval on each action.

The relay auto-reconnects if the connection drops. Press Ctrl+C to disconnect.

Requirements

  • Node.js 18 or later
  • macOS: Grant Accessibility and Screen Recording permissions to your terminal app (Terminal, iTerm, Warp, etc.)
  • Windows: No special permissions needed
  • Linux: X11 display server (Wayland not yet supported)

What Your Agent Can Do

Direct Shell Execution (primary method)

Your agent runs shell commands directly on your computer and gets the output back — no Terminal window needed. This is the fastest way to complete most tasks.

  • Run any shell command and get stdout/stderr back
  • Create, move, rename, and organize files and folders
  • Open applications
  • Install packages and run scripts
  • Get system information

Visual Control (for GUI tasks)

For tasks that need visual interaction — clicking buttons, filling forms, navigating apps — your agent sees your screen and controls your mouse and keyboard.

  • Take screenshots of your screen
  • Click at any position
  • Type text via keyboard
  • Press key combos (Cmd+C, Ctrl+V, etc.)
  • Drag and drop between positions
  • Scroll in any direction
  • List and switch between open windows
  • Batch actions — execute multiple visual actions in one round-trip for 2-3x faster task completion

Example: Organize Desktop Screenshots

You: "organize all the screenshots on my desktop into a Screenshots folder"

Your agent runs one command:

mkdir -p ~/Desktop/Screenshots && find ~/Desktop -maxdepth 1 -name 'Screenshot*' -type f -exec mv {} ~/Desktop/Screenshots/ \; && ls ~/Desktop/Screenshots/ | wc -l

Done in under 10 seconds. No clicking through Finder, no dragging files. The agent runs the command directly on your Mac and reports the result. Make sure you're in autonomous mode (press 'a') so the agent can execute without waiting for approval.

Modes

Supervised (default)

Every action the agent wants to take is shown in your terminal. Press Enter to approve or 'n' to deny. Screenshots and shell commands are shown before executing.

Autonomous (recommended for tasks)

Auto-approves most actions so your agent can work without interruption. Dangerous actions (passwords, deletions, purchases, sudo) still require your confirmation.

When you give your agent a task like "organize my desktop" or "fill out this form," it will suggest enabling autonomous mode so it can work through the steps without pausing for approval on each one. In supervised mode, multi-step tasks require you to press Enter for every single action — which gets tedious fast.

Three ways to enable autonomous mode:

  • Press 'a' in the terminal window running the relay — toggles between supervised and autonomous at any time
  • Start with the flag:
npx @instaclaw/dispatch@latest --autonomous
  • Your agent will ask: Before multi-step tasks, your agent will say "Press a in your terminal to switch to autonomous mode" — just follow its suggestion

Performance

  • Direct shell execution: Commands run instantly on your machine and return output — no screenshots or GUI interaction needed for file operations
  • Action batching: For visual tasks, your agent plans multiple steps (click, type, press Enter) and executes them in a single round-trip — 2-3x faster than one at a time
  • WebP screenshots: 50-60% smaller than JPEG, reducing transfer time without losing visual quality
  • Smart verification: Screenshots only at decision points, not after every action — cutting vision costs by up to 70%
  • TCP keepalive: Connection stays alive through NAT firewalls and home routers — no silent disconnects

macOS Permissions

macOS requires two permissions for visual computer control (shell execution works without these):

  1. Accessibility — allows mouse clicks and keyboard input.
    System Settings → Privacy & Security → Accessibility → enable your terminal app
  2. Screen Recording — allows screenshots.
    System Settings → Privacy & Security → Screen Recording → enable your terminal app

Note: Enabling Accessibility may close your Terminal app — this is normal macOS behavior. Just reopen Terminal and run the same command again. Your pairing code will still work.

macOS Sequoia may re-prompt for Screen Recording permission monthly. This is an Apple security feature, not an InstaClaw issue.

Security

  • All communication encrypted via TLS (WebSocket Secure) with self-signed certificates
  • HMAC-SHA256 authentication with timestamp + nonce — prevents replay attacks
  • Supervised mode lets you approve every action before it executes
  • Dangerous actions (passwords, deletions, purchases, sudo, rm -rf) always require confirmation, even in autonomous mode
  • Shell commands are filtered for dangerous patterns (curl|bash, chmod 777, etc.)
  • Command output is capped at 4KB to prevent data exfiltration
  • Press Ctrl+C at any time to immediately disconnect and stop all agent control
  • Connection auto-reconnects on network blips but requires re-authentication each time

FAQ

Does my agent have access to my computer when the relay isn't running?

No. Your agent can only control your computer while the relay is actively running in your terminal. Close it and the connection is severed immediately.

What data does the relay send to my agent?

Screenshots (as compressed WebP images), shell command output (stdout/stderr), and action results (success/failure). No files, passwords, or browsing history are ever sent. Command output is capped at 4KB.

Can I use this on multiple computers?

The relay connects one computer at a time. Run it on whichever machine you want your agent to control.

Why does my agent need to control my computer?

For tasks that require your installed apps, logged-in sessions, or local files — things your agent can't access from its server. Examples: organizing files on your desktop, filling out a form in your browser, editing a Figma file, running local scripts.

What if the connection drops?

The relay automatically reconnects with exponential backoff (1 second to 30 seconds). TCP keepalive prevents silent disconnects from NAT firewalls. If the relay can't reconnect, it shows a clear error message in your terminal.

Is the shell execution safe?

In supervised mode, every command is shown to you before executing. In autonomous mode, dangerous patterns (sudo, rm -rf, curl|bash) still require your approval. Output is capped at 4KB per command to prevent large data transfers.

How do I switch between supervised and autonomous mode?

Press 'a' in the terminal window running the relay to toggle modes at any time. Or start with --autonomous flag.