Playwright AI Integration

Submit natural language prompts to an AI chatbot, which will use a browser through an MCP (Model Context Protocol) server to perform actions and generate Playwright tests automatically.

Features

🤖 AI-Powered Test Generation: Use natural language to describe what you want to test
🎭 Playwright Integration: Automatically generates valid Playwright test code
🌐 Browser Automation: Real-time browser control through MCP server
📡 WebSocket Updates: Live feedback as actions are executed
🎨 Beautiful UI: Clean, modern web interface
🔧 Configurable: Support for headless/headed browser modes

Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Web Client    │───▶│   Express       │───▶│   OpenAI API    │
│   (Frontend)    │    │   Server        │    │   (GPT-4)       │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                │
                                ▼
                        ┌─────────────────┐
                        │   MCP Server    │
                        │   (Playwright)  │
                        └─────────────────┘
                                │
                                ▼
                        ┌─────────────────┐
                        │   Browser       │
                        │   (Chromium)    │
                        └─────────────────┘

Prerequisites

Node.js (v18 or higher)
OpenAI API key
Modern web browser

Installation

Navigate to the project directory:
```
cd playwright-ai-integration
```
Install dependencies:
```
npm install
```
Install Playwright browsers:
```
npm run install-browsers
```
Set up environment variables:
```
cp .env.example .env
```
Edit .env and add your OpenAI API key:
```
OPENAI_API_KEY=your_openai_api_key_here
```

Usage

Starting the Server

Start the main server:
```
npm start
```
Open your browser and navigate to:
```
http://localhost:3000
```

Using the Interface

Start a Session:
- Enter your OpenAI API key (if not set in environment)
- Click "Start Session" to initialize the MCP server
Submit a Prompt:
- Enter a natural language description of what you want to test
- Examples:
  - "Go to google.com and search for playwright testing"
  - "Navigate to github.com, click on the search box, and search for microsoft/playwright"
  - "Open example.com and take a screenshot of the page"
Watch the Magic:
- See real-time updates as actions are executed
- View the generated Playwright test code
- Copy the test to use in your project

Example Prompts

Basic Navigation:

Go to example.com and take a screenshot

Form Interaction:

Navigate to httpbin.org/forms/post, fill in the customer name field with "John Doe", and submit the form

Search Functionality:

Go to google.com, search for "playwright testing", and click on the first result

Complex Workflows:

Open github.com, click on sign in, wait for the login form to appear, and take a screenshot

Available MCP Tools

The MCP server provides the following tools for browser automation:

launch_browser - Launch a new browser instance
navigate_to - Navigate to a specific URL
click_element - Click on elements using CSS selectors
fill_input - Fill input fields with text
wait_for_element - Wait for elements to appear
take_screenshot - Capture screenshots
get_page_content - Get page information
generate_test - Generate Playwright test code
close_browser - Close the browser instance

Development

Project Structure

playwright-ai-integration/
├── src/
│   ├── server.js           # Express server with WebSocket support
│   ├── ai-integration.js   # AI service for processing prompts
│   └── mcp-server.js      # MCP server for browser automation
├── public/
│   └── index.html         # Web interface
├── tests/
│   └── example.spec.js    # Example generated test
├── package.json
├── playwright.config.js
└── README.md

Running in Development Mode

npm run dev

This starts the server with file watching enabled.

Running the MCP Server Standalone

npm run mcp-server

Running Generated Tests

npm test

Or with UI mode:

npm run test:ui

API Endpoints

POST /api/start-session - Initialize AI integration session
POST /api/process-prompt - Process a natural language prompt
GET /api/session-status - Get current session status
POST /api/stop-session - Stop the current session

Configuration

Environment Variables

OPENAI_API_KEY - Your OpenAI API key (required)
OPENAI_MODEL - OpenAI model to use (default: gpt-4o-mini)
- Options: gpt-4o-mini, gpt-4o, gpt-4-turbo, gpt-3.5-turbo
- Note: gpt-4o-mini is recommended for better availability and lower costs
PORT - Server port (default: 3000)
HEADLESS - Run browser in headless mode (default: false)

Playwright Configuration

The project includes a playwright.config.js file with optimized settings for:

Multiple browser support (Chrome, Firefox, Safari)
Screenshot and video recording on failures
HTML reporting
Trace collection

Troubleshooting

Common Issues

"MCP server not connected" error:
- Ensure Node.js is properly installed
- Check that no other process is using the required ports
- Restart the session
"Browser not launched" error:
- Run npm run install-browsers to install Playwright browsers
- Check system permissions for browser execution
429 Rate Limit Error:

This error indicates you've hit OpenAI's rate limits. Solutions:
- Use a different model: Add OPENAI_MODEL=gpt-4o-mini to your .env file
  - gpt-4o-mini has higher rate limits and is more cost-effective
- Wait and retry: The system now includes automatic retry logic with exponential backoff
- Check your tier: New OpenAI accounts have lower rate limits. Upgrade at https://platform.openai.com/account/billing
- Monitor usage: Check your current limits at https://platform.openai.com/account/rate-limits
- Generate new API key: Sometimes old keys have cached rate limit states
OpenAI API errors:
- Verify your API key is correct and has sufficient credits
- Check your OpenAI API usage limits
- Ensure you're using a supported model (see OPENAI_MODEL in Environment Variables)
Element not found errors:
- The AI might generate selectors that don't exist on the target page
- Try more specific prompts or different selector strategies

Debug Mode

Set environment variable for verbose logging:

DEBUG=* npm start

Contributing

Fork the repository
Create a feature branch
Make your changes
Add tests if applicable
Submit a pull request

License

This project is licensed under the MIT License.

Acknowledgments

Playwright for browser automation
OpenAI for AI capabilities
Model Context Protocol for the integration framework

playwright-ai-integration