@matware/e2e-runner

JSON-driven E2E test runner. Define browser tests as simple JSON action arrays, run them in parallel against a Chrome pool. No JavaScript test files, no complex setup.

[
  {
    "name": "login-flow",
    "actions": [
      { "type": "goto", "value": "/login" },
      { "type": "type", "selector": "#email", "value": "user@test.com" },
      { "type": "type", "selector": "#password", "value": "secret" },
      { "type": "click", "text": "Sign In" },
      { "type": "assert_text", "text": "Welcome back" },
      { "type": "screenshot", "value": "logged-in.png" }
    ]
  }
]

Why

• No code -- Tests are JSON files. QA, product, and devs can all write them.
• Parallel -- Run N tests simultaneously against a shared Chrome pool.
• Portable -- Chrome runs in Docker, tests run anywhere.
• CI-ready -- JUnit XML output, exit code 1 on failure, error screenshots.
• AI-native -- Built-in MCP server for Claude Code integration.

Quick Start

# Install
npm install @matware/e2e-runner

# Scaffold project structure
npx e2e-runner init

# Start Chrome pool (requires Docker)
npx e2e-runner pool start

# Run all tests
npx e2e-runner run --all

The init command creates:

e2e/
  tests/
    01-sample.json      # Sample test suite
  screenshots/          # Reports and error screenshots
e2e.config.js           # Configuration file

Test Format

Each .json file in e2e/tests/ contains an array of tests. Each test has a name and sequential actions:

[
  {
    "name": "homepage-loads",
    "actions": [
      { "type": "goto", "value": "/" },
      { "type": "wait", "selector": ".hero" },
      { "type": "assert_text", "text": "Welcome" },
      { "type": "assert_url", "value": "/" },
      { "type": "screenshot", "value": "homepage.png" }
    ]
  }
]

Suite files can have numeric prefixes for ordering (01-auth.json, 02-dashboard.json). The --suite flag matches with or without the prefix, so --suite auth finds 01-auth.json.

Available Actions

Click by Text

When click uses text instead of selector, it searches across interactive elements:

button, a, [role="button"], [role="tab"], [role="menuitem"], div[class*="cursor"], span

{ "type": "click", "text": "Sign In" }

CLI

# Run tests
npx e2e-runner run --all                  # All suites
npx e2e-runner run --suite auth           # Single suite
npx e2e-runner run --tests path/to.json   # Specific file
npx e2e-runner run --inline '<json>'      # Inline JSON

# Pool management
npx e2e-runner pool start                 # Start Chrome container
npx e2e-runner pool stop                  # Stop Chrome container
npx e2e-runner pool status                # Check pool health

# Other
npx e2e-runner list                       # List available suites
npx e2e-runner init                       # Scaffold project

CLI Options

Configuration

Create e2e.config.js (or e2e.config.json) in your project root:

export default {
  baseUrl: 'http://host.docker.internal:3000',
  concurrency: 4,
  retries: 2,
  testTimeout: 30000,
  outputFormat: 'both',

  hooks: {
    beforeEach: [{ type: 'goto', value: '/' }],
    afterEach: [{ type: 'screenshot', value: 'after-test.png' }],
  },

  environments: {
    staging: { baseUrl: 'https://staging.example.com' },
    production: { baseUrl: 'https://example.com', concurrency: 5 },
  },
};

Config Priority (highest wins)

1. CLI flags (--base-url, --concurrency, ...)
2. Environment variables (BASE_URL, CONCURRENCY, ...)
3. Config file (e2e.config.js or e2e.config.json)
4. Defaults

When --env <name> is set, the matching profile from environments overrides everything.

Environment Variables

Hooks

Hooks run actions at lifecycle points. Define them globally in config or per-suite in the JSON file:

{
  "hooks": {
    "beforeAll": [{ "type": "goto", "value": "/login" }],
    "beforeEach": [{ "type": "goto", "value": "/" }],
    "afterEach": [],
    "afterAll": []
  },
  "tests": [
    { "name": "test-1", "actions": [...] }
  ]
}

Suite-level hooks override global hooks per key (non-empty array wins). The plain array format ([{ name, actions }]) is still supported.

Retries and Timeouts

Override globally or per-test:

{
  "name": "flaky-test",
  "retries": 3,
  "timeout": 15000,
  "actions": [...]
}

• Retries: Each attempt gets its own fresh timeout. Tests that pass after retry are flagged as "flaky" in the report.
• Timeout: Applied via Promise.race(). Defaults to 60s.

CI/CD

JUnit XML

npx e2e-runner run --all --output junit
# or: --output both (JSON + XML)

Output saved to e2e/screenshots/junit.xml.

GitHub Actions

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx e2e-runner pool start
      - run: npx e2e-runner run --all --output junit
      - uses: mikepenz/action-junit-report@v4
        if: always()
        with:
          report_paths: e2e/screenshots/junit.xml

Exit Codes

Programmatic API

import { createRunner } from '@matware/e2e-runner';

const runner = await createRunner({ baseUrl: 'http://localhost:3000' });

// Run all suites
const report = await runner.runAll();

// Run a specific suite
const report = await runner.runSuite('auth');

// Run a specific file
const report = await runner.runFile('e2e/tests/login.json');

// Run inline test objects
const report = await runner.runTests([
  {
    name: 'quick-check',
    actions: [
      { type: 'goto', value: '/' },
      { type: 'assert_text', text: 'Hello' },
    ],
  },
]);

Lower-Level Exports

import {
  loadConfig,
  waitForPool, connectToPool, getPoolStatus, startPool, stopPool,
  runTest, runTestsParallel, loadTestFile, loadTestSuite, loadAllSuites, listSuites,
  generateReport, generateJUnitXML, saveReport, printReport,
  executeAction,
} from '@matware/e2e-runner';

Claude Code Integration (MCP)

The package includes a built-in MCP server that gives Claude Code native access to the test runner. Install once and it's available in every project.

Via npm (requires Node.js):

claude mcp add --transport stdio --scope user e2e-runner \
  -- npx -y -p @matware/e2e-runner e2e-runner-mcp

Via Docker (no Node.js required):

claude mcp add --transport stdio --scope user e2e-runner \
  -- docker run -i --rm fastslack/e2e-runner-mcp

MCP Tools

Once installed, Claude Code can run tests, analyze failures, create new test files, and manage the Chrome pool as part of its normal workflow. Just ask:

"Run all E2E tests" "Create a test that verifies the checkout flow" "What's the status of the Chrome pool?"

Verify Installation

claude mcp list
# e2e-runner: ... - Connected

Architecture

bin/cli.js            CLI entry point (manual argv parsing)
bin/mcp-server.js     MCP server entry point (stdio transport)
src/config.js         Config cascade: defaults -> file -> env -> CLI -> profile
src/pool.js           Chrome pool: Docker Compose lifecycle + WebSocket
src/runner.js         Parallel test executor with retries and timeouts
src/actions.js        Action engine: maps JSON actions to Puppeteer calls
src/reporter.js       JSON reports, JUnit XML, console output
src/mcp-server.js     MCP server: exposes tools for Claude Code
src/logger.js         ANSI colored logger
src/index.js          Programmatic API (createRunner)
templates/            Scaffolding templates for init command

How It Works

1. Pool: A Docker container running browserless/chrome provides shared Chrome instances via WebSocket.
2. Runner: Spawns N parallel workers. Each worker connects to the pool, opens a new page, and executes actions sequentially.
3. Actions: Each JSON action maps to a Puppeteer call (page.goto, page.click, page.type, etc.).
4. Reports: Results are collected, aggregated into a report, and saved as JSON and/or JUnit XML.

The baseUrl defaults to http://host.docker.internal:3000 because Chrome runs inside Docker and needs to reach the host machine.

Requirements

• Node.js >= 20
• Docker (for the Chrome pool)

License

Copyright 2025 Matias Aguirre (fastslack)

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.