Qanat Goose MCP Extension

🔄 Looking for the standalone prototype? See Qanat.

Qanat Goose MCP Extension is an agent‑based Goose MCP‑UI extension that wraps the Square MCP Server, integrating sandbox catalog/orders with voice and gesture agents for an interactive seller dashboard.

🏗️ Architecture

Qanat integrates with Goose Desktop as a custom MCP-UI extension:

Goose Desktop (MCP-UI surface)
    ↓
Qanat MCP-UI Extension ← Voice/Gesture Agents  
    ↓
Square MCP Server
    ↓  
Square API

🚀 Quick Start

Setup Environment

# Copy environment template
cp env.example .env

# Edit .env with your API keys
# SQUARE_API_KEY=your_key_here
# ELEVENLABS_API_KEY=your_key_here

Install Dependencies
```
pip install -r requirements.txt
```

Run the MCP Server

npm run dev
# or
python -m backend.mcp_servers.qanat_server

Connect to Goose Desktop
- Add Qanat as MCP-UI extension in Goose settings
- Server will run on localhost:3001

🔄 Running Both Qanat Demos

There are two versions of Qanat available: Qanat (Baseline Prototype)

Standalone dashboard using Web Speech API and seeded demo data.
Runs entirely in the browser, with fallback APIs when Square is unreachable.
To run:
Clone the qanat repo.
Install dependencies (npm install).
Start the backend mock server (npm run dev).
Start the frontend (cd frontend && npm run dev).
Visit http://localhost:5173 in Chrome and allow microphone access. Qanat Goose MCP Extension (Advanced)
Goose MCP‑UI extension wrapping Square MCP Server.
Integrates Square sandbox catalog/orders with voice and gesture agents.
To run:
Clone the qanat-goose-mcp repo.
Copy env.template → .env and add your Square sandbox OAuth credentials.
Start the MCP server (npm run dev or python -m backend.mcp_servers.qanat_server).
In Goose Desktop, add Qanat as an MCP‑UI extension with endpoint http://localhost:3001.
Interact via voice, gesture, or UI buttons.

📁 Project Structure

qanat_v2/
├── backend/                    # Core backend services
│   ├── agents/                # Goose subagents (voice, gesture)
│   ├── services/              # Square API integration
│   └── mcp_servers/           # MCP protocol servers
├── config/                    # Configuration files
│   ├── extensions/            # Extension configs (Square, voice, gesture)
│   └── environments/          # Environment loading utilities
├── docs/                      # Project documentation
│   ├── PROJECT_BOARD.md       # Development task breakdown
│   ├── PLAN.md               # MVP development plan  
│   └── Qanat_Architecture_Overview.md
├── requirements.txt           # Python dependencies
├── package.json              # Project metadata & scripts
└── env.example               # Environment template

🎯 MVP Features

Catalog Dashboard: View and manage Square catalog items
Orders Dashboard: Monitor recent orders and status updates
Voice Control: "Refresh catalog" command via ElevenLabs
Gesture Control: Toggle item status with thumb-up gesture via MediaPipe
Interactive UI: Clickable tables and action buttons in Goose Desktop

🛠️ Development

See docs/PROJECT_BOARD.md for detailed development tasks and timeline.

Key Components

MCP-UI Server: backend/mcp_servers/qanat_server.py
Square Integration: backend/services/
Voice Agent: Uses ElevenLabs for speech-to-text
Gesture Agent: Uses MediaPipe for hand gesture recognition

Configuration Files

config/extensions/square_mcp.json - Square API integration config
config/extensions/qanat_mcp_ui.json - UI component definitions
config/extensions/elevenlabs_voice.json - Voice command mappings
config/extensions/mediapipe_gesture.json - Gesture recognition setup

🔧 Scripts

npm run dev          # Start development server
npm run start        # Start production server  
npm run test         # Run tests
npm run setup        # Install dependencies
npm run config       # Validate configuration

📋 Environment Variables

Required variables (copy from env.example):

SQUARE_API_KEY - Your Square API key
SQUARE_ENVIRONMENT - sandbox/production
ELEVENLABS_API_KEY - ElevenLabs API key for voice features
MEDIAPIPE_CONFIDENCE_THRESHOLD - Gesture detection sensitivity

🎮 Usage

Voice Commands

"Refresh catalog" - Reload catalog items
"Show orders" - Display recent orders
"Help" - Show available commands

Gesture Controls

👍 Thumb up - Toggle item active/inactive status
👉 Point - Select table row
✋ Open palm - Refresh current view
✌️ Peace sign - Switch catalog/orders view

🧪 Testing

# Run all tests
npm run test

# Test Square integration
python -m pytest tests/test_square_services.py

# Test voice commands  
python -m pytest tests/test_voice_agent.py

# Test gesture recognition
python -m pytest tests/test_gesture_agent.py

📝 License

MIT License - see LICENSE file for details.

Built with ❤️ for Square sellers using Goose Desktop #Block #Goose

Contact

Reach me here or on Discord: @professordnyc

qanat-goose-mcp