mirroir-mcp

We built an MCP server to control iPhones through macOS iPhone Mirroring — then realized the same tools work on any macOS window. Screenshot, tap, swipe, type, scroll_to, measure — from any MCP client. Same security model. Same scenarios. Same AI. No source code required.

When automation breaks — a button moves, a label changes, timing drifts — Agent Diagnosis tells you why and how to fix it. Self-diagnosing automation, not just self-running.

Input flows through Karabiner DriverKit virtual HID devices because iPhone Mirroring blocks standard CGEvent injection. The installer uses the standalone DriverKit package (no keyboard grabber, no modifier corruption). Existing Karabiner-Elements installs are detected and reused automatically.

Requirements

• macOS 15+
• iPhone connected via iPhone Mirroring

Install

/bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"

or via npx:

npx -y mirroir-mcp install

or via Homebrew:

brew tap jfarcand/tap && brew install mirroir-mcp

After install, approve the DriverKit system extension if prompted: System Settings > General > Login Items & Extensions. If you have Karabiner-Elements, enable all its toggles. If you installed the standalone DriverKit package, enable the Karabiner-DriverKit-VirtualHIDDevice toggle. The first time you take a screenshot, macOS will prompt for Screen Recording and Accessibility permissions. Grant both.

claude mcp add --transport stdio mirroir -- npx -y mirroir-mcp

{
  "servers": {
    "mirroir": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "mirroir-mcp"]
    }
  }
}

{
  "mcpServers": {
    "mirroir": {
      "command": "npx",
      "args": ["-y", "mirroir-mcp"]
    }
  }
}

codex mcp add mirroir -- npx -y mirroir-mcp

[mcp_servers.mirroir]
command = "npx"
args = ["-y", "mirroir-mcp"]

npx mirroir-mcp setup

git clone https://github.com/jfarcand/mirroir-mcp.git
cd mirroir-mcp
./mirroir.sh

Examples

Paste any of these into Claude Code, Claude Desktop, ChatGPT, Cursor, or any MCP client:

Send an iMessage:

Open Messages, find my conversation with Alice, and send "running 10 min late".
Take a screenshot so I can confirm it was sent.

Add a calendar event:

Open Calendar, create a new event called "Dentist" next Tuesday at 2pm,
and screenshot the week view so I can see it.

Test a login flow:

Open my Expo Go app, tap on the "LoginDemo" project, and test the login
screen. Use test@example.com / password123. Take a screenshot after each step
so I can see what happened.

Record a bug repro video:

Start recording, open Settings, scroll down to General > About, then stop
recording. I need a video of the scroll lag I'm seeing.

Tip: describe_screen supports skip_ocr: true to return only the grid-overlaid screenshot without running Vision OCR, letting the MCP client use its own vision model instead (costs more tokens but can identify icons, images, and non-text UI elements).

Agent Diagnosis

UI automation is brittle. A button moves, a label changes, timing drifts, and your test silently fails. You stare at screenshots trying to figure out what went wrong. --agent fixes this: when a compiled scenario fails, it diagnoses why the step failed and tells you exactly how to fix it.

Diagnosis runs in two tiers. First, deterministic OCR analysis compares the compiled coordinates against what's actually on screen — fast, free, no API key needed. If you pass a model name, it sends the diagnostic context (expected vs. actual OCR, failure screenshots, step metadata) to an AI for richer analysis: root cause, suggested YAML edits, and whether recompilation will fix it.

mirroir test --agent scenario.yaml                    # deterministic OCR diagnosis
mirroir test --agent claude-sonnet-4-6 scenario.yaml  # deterministic + AI via Anthropic
mirroir test --agent gpt-4o scenario.yaml             # deterministic + AI via OpenAI
mirroir test --agent ollama:llama3 scenario.yaml      # deterministic + AI via local Ollama
mirroir test --agent copilot scenario.yaml            # deterministic + AI via Copilot CLI

Built-in models: claude-sonnet-4-6, claude-haiku-4-5, gpt-4o. Set the corresponding API key env var (ANTHROPIC_API_KEY, OPENAI_API_KEY).

Custom agents: Place a YAML profile in ~/.mirroir-mcp/agents/ (global) or <cwd>/.mirroir-mcp/agents/ (project-local, takes priority). Two modes:

# API mode — call a cloud provider
name: my-agent
mode: api
provider: anthropic
model: claude-sonnet-4-6-20250514
api_key_env: MY_KEY

# Command mode — run a local CLI
name: my-agent
mode: command
command: copilot
args: ["-p", "Analyze: ${PAYLOAD}"]

Command mode supports ${PAYLOAD} substitution in args for CLIs that take prompts as arguments (like claude --print -p or copilot -p). Without ${PAYLOAD}, the diagnostic JSON is piped to stdin.

The system prompt is loaded from ~/.mirroir-mcp/prompts/diagnosis.md (or <cwd>/.mirroir-mcp/prompts/diagnosis.md, project-local takes priority) — edit it to customize AI behavior. The default prompt is installed from the repo-level prompts/ directory.

All AI errors are non-fatal: deterministic diagnosis always runs regardless.

Scenarios

Scenarios are YAML files that describe multi-step automation flows as intents, not scripts. Steps like tap: "Email" don't specify coordinates — the AI finds the element by fuzzy OCR matching and adapts to unexpected dialogs, screen layout changes, and timing.

Cross-app workflow — get your commute ETA from Waze, then text it to someone via iMessage:

name: Commute ETA Notification
app: Waze, Messages
description: Get commute ETA from Waze, then send it via iMessage.

steps:
  - launch: "Waze"
  - wait_for: "Où va-t-on ?"
  - tap: "Où va-t-on ?"
  - wait_for: "${DESTINATION:-Travail}"
  - tap: "${DESTINATION:-Travail}"
  - wait_for: "Y aller"
  - tap: "Y aller"
  - wait_for: "min"
  - remember: "Read the commute time and ETA from the navigation screen."
  - press_home: true
  - launch: "Messages"
  - wait_for: "Messages"
  - tap: "New Message"
  - wait_for: "À :"
  - tap: "À :"
  - type: "${RECIPIENT}"
  - wait_for: "${RECIPIENT}"
  - tap: "${RECIPIENT}"
  - wait_for: "iMessage"
  - tap: "iMessage"
  - type: "${MESSAGE_PREFIX:-On my way!} {commute_time} to the office (ETA {eta})"
  - press_key: "return"
  - wait_for: "Distribué"
  - screenshot: "message_sent"

${VAR} placeholders are resolved from environment variables. Use ${VAR:-default} for fallback values. Place scenarios in ~/.mirroir-mcp/scenarios/ (global) or <cwd>/.mirroir-mcp/scenarios/ (project-local). Both directories are scanned recursively.

Scenario Marketplace

Ready-to-use scenarios that automate anything a human can do on an iPhone — tap, type, navigate, chain apps together. If you can do it manually, you can script it. Install from jfarcand/mirroir-scenarios:

Claude Code

claude plugin marketplace add jfarcand/mirroir-scenarios
claude plugin install scenarios@mirroir-scenarios

GitHub Copilot CLI

copilot plugin marketplace add jfarcand/mirroir-scenarios
copilot plugin install scenarios@mirroir-scenarios

Manual (all other clients)

git clone https://github.com/jfarcand/mirroir-scenarios ~/.mirroir-mcp/scenarios

Once installed, scenarios are available through the list_scenarios and get_scenario tools. Claude Code and Copilot CLI load the SKILL.md automatically, which teaches the AI how to interpret and execute each step type. For other clients, ask the AI to call list_scenarios and then execute the steps.

See Tools Reference for the full step type reference and directory layout.

Test Runner

Run scenarios deterministically from the command line — no AI in the loop. Steps execute sequentially: OCR finds elements, taps land on coordinates, assertions pass or fail. Designed for CI, regression testing, and scripted automation.

mirroir test [options] <scenario>...

Run a scenario:

mirroir test apps/settings/check-about

Run all scenarios with JUnit output:

mirroir test --junit results.xml --verbose

Validate scenarios without executing (dry run):

mirroir test --dry-run apps/settings/*.yaml

The test runner uses the same OCR and input subsystems as the MCP server. Steps like tap: "General" find the element via Vision OCR and tap at the detected coordinates. wait_for polls OCR until the label appears or times out. AI-only steps (remember, condition, repeat) are skipped with a warning.

Scenario resolution searches <cwd>/.mirroir-mcp/scenarios/ and ~/.mirroir-mcp/scenarios/ — same directories as list_scenarios. Pass a .yaml path to run a specific file, or a name to search the scenario directories.

Exit code is 0 when all scenarios pass, 1 when any step fails.

Compiled Scenarios

Compile a scenario once against a real device to capture coordinates, timing, and scroll counts. Replay with zero OCR — pure input injection plus timing. Like JIT compilation for UI automation.

Compile (learning run):

mirroir compile apps/settings/check-about

Run compiled (zero OCR):

mirroir test apps/settings/check-about       # auto-detects .compiled.json
mirroir test --no-compiled check-about        # force full OCR

Each OCR-dependent step (~500ms per call) becomes a direct tap at cached coordinates, a timed sleep, or a replayed scroll sequence. A 10-step scenario that spent 5+ seconds on OCR runs in under a second.

Compiled files are invalidated automatically when the source YAML changes (SHA-256 hash), the window dimensions change, or the format version bumps. See Compiled Scenarios for the file format, architecture, and design rationale.

When a compiled step fails, use --agent for AI-powered failure diagnosis. See Agent Diagnosis.

Recorder

Record user interactions with iPhone Mirroring as a scenario YAML file. Click, swipe, and type on the mirrored iPhone — the recorder captures everything via a passive CGEvent tap, labels taps with OCR, and outputs a ready-to-edit scenario.

mirroir record [options]

Record a login flow:

mirroir record -o login-flow.yaml -n "Login Flow" --app "MyApp"

Fast recording without OCR:

mirroir record --no-ocr -o quick-capture.yaml

The recorder installs a passive CGEvent tap that observes mouse and keyboard events on the mirroring window. Taps are labeled with the nearest OCR text element. Swipe gestures are detected from mouse drag distance and direction. Keyboard input is grouped into type: and press_key: steps. Press Ctrl+C to stop and save.

The output is a starting point — review the YAML, replace any FIXME coordinate-only taps with text labels, and add wait_for steps where needed.

Doctor

Run 10 prerequisite checks to verify your setup is working:

mirroir doctor

Checks DriverKit extension, helper daemon, iPhone Mirroring connection, screen recording permissions, and more. Each failed check includes an actionable fix hint.

mirroir doctor --json    # machine-readable output
mirroir doctor --no-color # plain text (no ANSI colors)

Updating

# curl installer (re-run — pulls latest and rebuilds)
/bin/bash -c "$(curl -fsSL https://mirroir.dev/get-mirroir.sh)"

# npx (always fetches latest)
npx -y mirroir-mcp install

# Homebrew
brew upgrade mirroir-mcp
sudo brew services restart mirroir-mcp

# From source
git pull
sudo ./scripts/reinstall-helper.sh

Uninstall

# Homebrew
sudo brew services stop mirroir-mcp
brew uninstall mirroir-mcp

# From source — removes helper daemon, Karabiner config changes,
# and optionally standalone DriverKit or Karabiner-Elements
./uninstall-mirroir.sh

Configuration

All timing delays and numeric constants have sensible defaults compiled into the binary. To override any value, create a settings.json file:

# Project-local (takes priority)
mkdir -p .mirroir-mcp
cat > .mirroir-mcp/settings.json << 'EOF'
{
  "keystrokeDelayUs": 20000,
  "clickHoldUs": 100000
}
EOF

# Global
cat > ~/.mirroir-mcp/settings.json << 'EOF'
{
  "keystrokeDelayUs": 20000
}
EOF

Only include the values you want to change — everything else uses the compiled default. Resolution order for each key (first found wins):

1. Project-local <cwd>/.mirroir-mcp/settings.json
2. Global ~/.mirroir-mcp/settings.json
3. MIRROIR_<SCREAMING_SNAKE_CASE> environment variable
4. Compiled default (TimingConstants.swift)

Environment variables use screaming snake case with an MIRROIR_ prefix. For example, keystrokeDelayUs becomes MIRROIR_KEYSTROKE_DELAY_US.

Documentation

Contributing

Contributions are welcome! By submitting a pull request or patch, you agree to the Contributor License Agreement. Your Git commit metadata (name and email) serves as your electronic signature — no separate form to sign.

The CLA ensures the project can be maintained long-term under a consistent license. You retain full ownership of your contributions — the CLA simply grants the maintainer the right to distribute them as part of the project. Key provisions:

Why "mirroir"? — It's the old French spelling of miroir (mirror). A nod to the author's roots, not a typo.