desk-mcp
0 Trust – 100% Control | 0 Magic – 100% Transparency
Desktop automation MCP for AI agents — screenshot, mouse, keyboard, window inspection. Lets any MCP-capable agent (Claude, Gemini, Codex, Copilot, Vibe) see and interact with the KpihX-Ubuntu desktop.
The Problem
AI agents are blind to the desktop by default. They can read files and call APIs, but
they cannot see what is on screen, click buttons, type in forms, or react to GUI state.
desk-mcp bridges that gap — it exposes the desktop as a set of simple MCP tools that
any agent can call over the standard stdio transport.
Agent CLI ──stdio──► desk-mcp ──XDG Portal──► Screenshot
└───────xdotool───────► Click / Type / Key / Scroll
└───────xdotool───────► Window list & geometry
Architecture
desk-mcp serve (FastMCP, stdio transport)
│
├── screenshot() XDG Desktop Portal → /usr/bin/python3 + dbus + GLib
│ Full screen → optional crop (window name or {x,y,w,h})
│
├── get_windows() xdotool search + getwindowgeometry
├── get_screen() xdotool getdisplaygeometry + env vars
│
├── click() xdotool mousemove + click
├── double_click() xdotool click --repeat 2
├── right_click() xdotool click 3
├── move_mouse() xdotool mousemove
├── type_text() xdotool type --delay <ms>
├── key() xdotool key --clearmodifiers <combo>
└── scroll() xdotool click (button 4/5/6/7)
Screenshot backend: XDG Desktop Portal via dbus-python + GLib. Works natively
on GNOME Wayland without any dialog or user interaction. Does NOT depend on
gnome-screenshot or grim.
Input backend: xdotool via XWayland. Covers all X11 and XWayland apps.
For pure Wayland-native apps (e.g. WaveTerm in native mode), mouse coordinates are
correct but window auto-detection via get_windows() may not see them.
Install
Prerequisites
sudo apt install xdotool python3-gi python3-dbus
From PyPI
uv tool install desk-mcp
Editable (development)
git clone git@github.com:KpihX/desk-mcp.git ~/Work/AI/MCPs/desk_mcp
cd ~/Work/AI/MCPs/desk_mcp
uv tool install --editable .
Usage
Claude Code (~/.claude.json)
"desk-mcp": {
"command": "/home/kpihx/.local/bin/desk-mcp",
"args": ["serve"],
"env": {
"DISPLAY": ":0",
"DBUS_SESSION_BUS_ADDRESS": "unix:path=/run/user/1000/bus",
"XDG_RUNTIME_DIR": "/run/user/1000",
"WAYLAND_DISPLAY": "wayland-0"
}
}
Note: The display env vars must be injected explicitly because Claude Code does not inherit the user's graphical session environment.
Gemini (~/.gemini/extensions/desk-mcp/gemini-extension.json)
{
"name": "desk-mcp",
"version": "0.2.0",
"description": "Desktop automation MCP for Gemini — screenshot, mouse, keyboard via xdotool.",
"mcpServers": {
"desk-mcp": {
"command": "/home/kpihx/.local/bin/desk-mcp",
"args": ["serve"],
"env": {}
}
}
}
CLI test
desk-mcp status # check environment and tool availability
desk-mcp screenshot # take a test screenshot and print the path
desk-mcp serve # start the MCP server (stdio)
Tools
| Tool | Description |
|---|---|
screenshot | Full screen or cropped to window name / region {x,y,w,h} |
get_windows | List all XWayland-visible windows with IDs and geometry |
get_screen | Screen resolution, session type, display vars |
click | Left / right / middle click at (x, y) |
double_click | Double-click at (x, y) |
right_click | Right-click at (x, y) — opens context menu |
move_mouse | Move mouse without clicking |
type_text | Type text at current keyboard focus |
key | Press key combo: "ctrl+c", "Return", "super", "alt+F4" |
scroll | Scroll up/down/left/right at (x, y) |