Lastest2

Free, open-source visual regression testing with AI-generated tests

Record it. Test it. Ship it. — $0 forever.

Features • Quick Start • How It Works • Why Lastest2 • Comparison • Commands • Config

The Problem

Visual regression testing is either expensive, flaky, or painful to maintain.

• Percy: $199-5,000+/mo depending on screenshots. Cloud-only
• Applitools: $699+/mo. Enterprise pricing, steep learning curve
• Chromatic: $179+/mo. Locked to Storybook
• Meticulous: Custom pricing. Session-based, no self-hosting
• Argos: $100+/mo. OSS core but cloud execution
• Playwright native: Free, but no dashboard, no collaboration, no AI
• BackstopJS: Free, but maintenance mode and no UI

Meanwhile, you just need to know: "Did my last commit break the UI?"

The Solution

Lastest2 is a free, self-hosted visual regression testing platform that records your tests, writes them with AI, runs them anywhere, and fixes them when they break — all in one tool.

1. Point it at your app
2. Record your user flows (point-and-click, no code)
3. AI generates resilient test code with multi-selector fallback
4. Run locally, on remote runners, or in an embedded browser container
5. Screenshots compared with 3 diff engines (pixelmatch, SSIM, Butteraugli)
6. Review and approve visual changes — or let AI auto-classify them

Your data stays on your server. Your screenshots never leave your infra. It costs $0 forever.

Three Ways to Work

Lastest2 adapts to how you want to build tests — from fully manual to fully autonomous.

1. AI-Free (Manual Recording)

Open the recorder, click through your app, hit stop. Lastest2 captures every interaction and generates deterministic Playwright code — no AI involved, no API keys needed. You own the test code and can edit it by hand.

Best for: Teams that don't want AI, air-gapped environments, simple flows.

2. AI-Assisted (Human-in-the-Loop)

AI generates, fixes, or enhances tests — but you review and approve before anything is saved. Feed it a URL and get a test back. Import OpenAPI specs or user stories and AI extracts test cases. When a test breaks, AI proposes a fix and you decide whether to accept it.

Best for: Day-to-day development, iterating on tests, fixing breakages fast.

3. Full Autonomous (Play Agent)

One click kicks off a 9-step pipeline: scan your repo for routes, classify your app type, generate tests, run them, fix failures (up to 3 attempts per test), re-run, and report results. The agent pauses and asks for help only when it hits something it can't resolve on its own (missing settings, server offline). You resume and it picks up where it left off.

Best for: Onboarding a new project, generating full coverage from scratch, CI bootstrapping.

Three Ways to Run

Once your tests exist, you have three execution modes:

All three modes support both running and recording tests. Builds can be triggered manually (click Run), by webhook (PR opened/updated), or on push to monitored branches via CI/CD (GitHub Action or CLI runner). Smart Run analyzes git diffs to run only affected tests.

Build Once, Run Forever — $0

Tests are recorded or generated once, then stored as code. Every subsequent run re-executes the same code, captures new screenshots, and diffs them against approved baselines.

• First run: screenshot becomes the baseline
• Every run after: new screenshot is SHA256-hashed — if it matches the baseline, instant pass (no pixel comparison needed). If it differs, the diff engine runs and you review the change.
• AI costs are one-time: AI is only used during test creation and fixing. Running tests uses zero AI — it's pure Playwright execution.
• No per-screenshot pricing: unlike Percy ($5K/mo for 100K shots) or Chromatic ($179+/mo), every run is free regardless of volume.

Create tests (one-time)          Run tests (forever, $0)
┌──────────────────────┐         ┌──────────────────────┐
│ Manual recording     │         │ Execute Playwright    │
│   — or —             │  ────▶  │ Capture screenshots   │
│ AI generation        │  save   │ Diff against baseline │
│   — or —             │         │ Review changes        │
│ Play Agent autonomy  │         │ Approve/reject        │
└──────────────────────┘         └──────────────────────┘
  AI may be used here              No AI needed here

Features

Core

• Record Browser Interactions — Point-and-click test recording via Playwright with multiple engines (custom recorder or Playwright Inspector). No code required.
• AI Test Generation — Claude generates robust test code with multi-selector fallback (data-testid → id → role → aria-label → text → css → OCR).
• Multi-Engine Visual Diffing — Three comparison engines: pixelmatch (pixel-perfect), SSIM (structural similarity), and Butteraugli (human-perception-aligned). Choose the best trade-off between speed and accuracy.
• Multi-Step Screenshots — Capture multiple labeled screenshots per test run for multi-page flow testing.
• Approval Workflow — Review visual diffs before they become baselines. Catch regressions, approve intentional changes.
• Git-Aware Builds — Run tests per branch/commit. Compare across PRs. Track coverage.
• Branch Comparison — Dedicated compare view for side-by-side branch-to-branch test result diffing.
• Test Suites — Organize tests into ordered suites for structured execution.
• Test Versioning — Full version history with change reasons (manual edit, AI fix, AI enhance, restored).
• Test Composition — Cherry-pick tests and pin specific test versions per build via the Compose page. Override latest with any historical version.
• Functional Area Hierarchy — Organize tests into nested parent/child functional areas with drag-and-drop reordering.
• Debug Mode — Step-by-step test execution with live feedback for diagnosing failures.
• Testing Templates — One-click preset configurations for common app types: SaaS/Dashboard, Marketing Website, Canvas/WebGL, E-commerce, Documentation, Mobile-First, SPA, and CMS.
• Auto-Detect Capabilities — Recording automatically detects required browser capabilities (file upload, clipboard, downloads, network interception) and enables corresponding Playwright settings.
• Early Adopter Mode — Team-level toggle to access experimental features before general release.

AI-Powered

• Multiple AI Providers — Claude CLI, OpenRouter, Claude Agent SDK, direct Anthropic API, or Ollama (local models).
• Separate AI Diff Provider — Use a different AI provider for diff analysis than test generation.
• AI Diff Analysis — AI-powered visual diff classification (insignificant/meaningful/noise) with confidence scores and change categories.
• AI Test Fixing — Automatically fix failing tests or enhance existing ones.
• Spec-Driven Testing — Import OpenAPI specs, user stories, or markdown files. AI extracts stories and generates tests automatically.
• Route Discovery — AI scans your source code to discover routes and suggest tests.
• MCP Selector Validation — Real-time selector validation on live pages via Claude MCP.

Stabilization & Flaky Test Prevention

• Text-Region-Aware Diffing — OCR-based two-pass comparison with separate thresholds for text vs non-text regions. Reduces false positives from dynamic text and cross-OS font rendering.
• Timestamp Freezing — Replace Date.now() and new Date() with fixed values for deterministic screenshots.
• Random Value Seeding — Seed Math.random() for consistent outputs.
• Cross-OS Consistency — Bundled fonts + Chromium flags for identical screenshots across operating systems.
• Burst Capture — Multi-frame instability detection: take N screenshots and compare for stability before saving.
• Auto-Mask Dynamic Content — Automatically detect and mask timestamps, UUIDs, and relative times before comparison.
• Network Idle Waiting — Wait for network activity to settle before capture.
• DOM Stability Detection — Wait for DOM mutations to stop before screenshot.
• Third-Party Blocking — Block third-party domains with configurable allowlist, mock external images.
• Font Loading Wait — Wait for webfonts to load, or disable them entirely.
• Loading Indicator Hiding — Auto-hide spinners and loading states with custom selectors.
• Page Shift Detection — Detect vertical content shifts (inserted/deleted rows) with fuzzy row matching.

Integrations

• GitHub — OAuth login, repo sync, PR comments, webhook-triggered builds, reusable GitHub Action.
• GitLab — OAuth login (self-hosted supported), MR comments, webhook triggers.
• Google OAuth — Sign in with Google.
• Google Sheets — Use spreadsheet data as test data sources with per-team OAuth, multi-tab support, custom header rows, fixed ranges, and caching.
• Notifications — Slack, Discord, custom webhooks, and GitHub/GitLab PR comments for build results.
• Email — Team invitation emails via Resend.

Infrastructure

• Smart Run — Analyzes git diffs to run only tests affected by your changes.
• Remote Runners (v2) — Distributed test execution with concurrent multi-task support, SHA256 code integrity verification, remote recording, heartbeat polling with command queuing, and per-test abort support.
• Parallel Test Execution — Configurable max parallel tests for local and remote runners.
• Docker Deployment — Production-ready multi-stage Docker setup based on official Playwright image with persistent volumes.
• VSCode Extension API — REST + SSE API (/api/v1/) for IDE integration.
• Accessibility Audits — Automated axe-core checks on every screenshot capture.
• Network & Console Tracking — Capture network requests and browser console errors during test runs.

Advanced

• Ignore Regions — Mask dynamic areas (timestamps, ads, counters) from diff comparison with configurable mask styles (solid-color or placeholder-text).
• Planned Screenshots — Compare against design files (Figma exports, etc.) with separate planned vs actual diff tracking.
• Branch Baseline Management — Fork baselines per branch, merge back on PR merge, promote test versions across branches. SHA256-based carry-forward matching.
• Setup & Teardown Orchestration — Repository-default multi-step setup and teardown sequences, build-level execution, and per-test overrides with skip/add extra steps. Supports Playwright (browser), API (HTTP seeding), and test-as-setup/teardown script types. Teardown errors are non-blocking.
• App State Inspection — Access internal app state during tests (window.__APP_STATE__, Redux stores, etc.) for complex assertions.
• Selector Stats — Track selector success/failure rates and response times for automatic optimization recommendations.
• Diff Sensitivity — Configurable pixel/percentage thresholds for unchanged/flaky/changed classification.
• AI Prompt Logs — Full audit trail of all AI requests and responses.
• Background Jobs — Queue tracking for long-running operations (AI scans, builds).
• Diff Engine Benchmarks — Built-in benchmark framework comparing all three diff engines across synthetic test scenarios with timing and accuracy metrics.

Team & Auth

• Multi-Tenant Teams — Slug-based team workspaces with invitations.
• Role-Based Access — Owner, admin, member, viewer roles.
• Multiple Auth Methods — Email/password (Argon2 hashing), GitHub OAuth, GitLab OAuth, Google OAuth via better-auth.
• Email Invitations — Send team invitations via Resend with verification and password reset tokens.

Quick Start

Option 1: Docker (recommended)

git clone https://github.com/dexilion-team/lastest2.git
cd lastest2
docker-compose up -d

Open http://localhost:3000 — that's it.

Option 2: From source

git clone https://github.com/dexilion-team/lastest2.git
cd lastest2
pnpm install
pnpm dev

Open http://localhost:3000

First steps

1. Create an account (local, no external auth required)
2. Add a repository and set its local path or connect GitHub/GitLab
3. Click Record — interact with your app, Lastest2 captures everything
4. AI generates test code automatically
5. Run the test — screenshots are captured and diffed against baselines
6. Review visual changes and approve or reject

Requirements

• Docker: Docker 20+ and Docker Compose
• From source: Node.js 18+, pnpm

How It Works

┌──────────────────┐     ┌─────────────┐     ┌─────────────┐
│   Create Tests   │ ──▶ │   Run       │ ──▶ │   Review    │
│                  │     │             │     │             │
│ Record manually  │     │ Local or    │     │ Approve/    │
│ AI-assisted      │     │ remote or   │     │ Reject      │
│ Play Agent auto  │     │ CI/CD       │     │ changes     │
└──────────────────┘     └─────────────┘     └─────────────┘
   One-time cost           $0 per run         New baseline
   (AI optional)           (no AI needed)     saved

Core Flow

1. Create: Build tests your way — record manually in the browser, let AI generate from a URL or spec, or let the Play Agent autonomously scan your entire app.

2. Run: Execute tests locally, on remote runners, or in CI/CD. Screenshots are captured at key steps. No AI needed — pure Playwright execution at zero cost.

3. Compare: New screenshots are diffed against baselines using your chosen engine (pixelmatch, SSIM, or Butteraugli). Text-region-aware comparison available. Accessibility audits run automatically.

4. Review: Visual diffs are classified (unchanged/flaky/changed). AI can optionally auto-classify with confidence scores. Approve intentional changes — they become the new baseline.

5. Fix: When tests break, AI can propose fixes (human-in-the-loop) or the Play Agent can fix and re-run autonomously. Or edit the code by hand — your choice.

Why Lastest2

Comparison

What makes Lastest2 different

• Record + AI generate + run + diff + approve in one self-hosted tool — no competitor does all five
• Three execution modes: local, remote runners (@lastest/runner on npm), or embedded browser container with live streaming — no local Playwright install needed
• Autonomous Play Agent: one-click 9-step pipeline scans routes, generates tests, runs them, fixes failures, and reports results
• AI auto-fix: tests break as your UI evolves, Lastest2 fixes them automatically
• $0 with unlimited screenshots — Percy charges ~$5K/mo for 100K shots
• Your data never leaves your server — screenshots stay local, no cloud dependency
• 5 AI providers including Ollama — run AI completely locally with zero API costs
• Spec-driven testing — feed it OpenAPI specs, user stories, or markdown files and get tests back
• 3 diff engines — pixelmatch, SSIM, and Butteraugli with OCR-based text-region-aware comparison
• 12 stabilization features — timestamp freezing, random seeding, burst capture, auto-masking, DOM/network stability, and more
• Auto-capability detection — recordings auto-detect clipboard, upload, download, and network needs

Commands

pnpm dev          # Start development server on localhost:3000
pnpm build        # Production build
pnpm start        # Start production server
pnpm lint         # Run ESLint
pnpm test         # Run unit tests (Vitest)
pnpm test:watch   # Run unit tests in watch mode
pnpm test:coverage # Run tests with coverage report
pnpm test:ui      # Run tests with Vitest UI
pnpm db:studio    # Open Drizzle Studio for database inspection
pnpm db:push      # Push schema changes to database
pnpm db:generate  # Generate Drizzle migrations
pnpm db:reset     # Reset database (removes SQLite DB + screenshots/baselines)
pnpm db:seed      # Seed test data
pnpm test:visual  # Run visual tests via CLI (see below)

CLI Test Runner (CI/CD)

Run visual regression tests from the command line for GitHub Actions or other CI pipelines:

pnpm test:visual --repo-id <id> [options]

Options

GitHub Actions Example

- name: Run Visual Tests
  run: pnpm test:visual --repo-id ${{ env.REPO_ID }} --base-url http://localhost:3000
  env:
    REPO_ID: your-repo-id

The runner automatically captures GITHUB_HEAD_REF, GITHUB_REF_NAME, and GITHUB_SHA for git tracking.

GitHub Action

Use the reusable GitHub Action for zero-config CI/CD integration. No local Playwright install needed — tests run on your Lastest2 server via a remote runner.

- name: Run visual regression tests
  uses: dexilion-team/lastest2/action@main
  with:
    server-url: ${{ secrets.LASTEST_SERVER_URL }}
    runner-token: ${{ secrets.LASTEST_RUNNER_TOKEN }}
    timeout: '300'
    fail-on-changes: 'false'

Inputs

Outputs

Results are automatically posted to the GitHub Actions step summary.

Smart Run

Run only tests affected by your code changes:

1. Select a feature branch (not main/master)
2. Lastest2 compares against the default branch via GitHub/GitLab API
3. Tests are matched to changed files by URL patterns and code references
4. Only affected tests run, skipping unchanged areas

This dramatically reduces test time for large suites while maintaining coverage for changed code.

Docker Deployment

Deploy Lastest2 on your home server or any Docker host:

# Quick start
docker-compose up -d

# View logs
docker-compose logs -f

# Stop
docker-compose down

Uses the official Playwright base image (mcr.microsoft.com/playwright) with Node.js 20, multi-stage build, and health checks via GET /api/health. Runs as non-root user.

Volumes

Environment Variables for Docker

GITHUB_CLIENT_ID=your-github-app-id
GITHUB_CLIENT_SECRET=your-github-app-secret

A development compose file (docker-compose.dev.yml) is also available.

Remote Runners

Run tests on remote machines by deploying runners that connect back to your Lastest2 server via WebSocket.

Setup

1. Register a runner in Settings → Runners
2. Copy the token (shown only once)
3. Install and run on your target machine

# Install from npm
npm install -g @lastest/runner

# Start as daemon
lastest2-runner start -t YOUR_TOKEN -s https://your-lastest2-server

# Or run in foreground
lastest2-runner run -t YOUR_TOKEN -s https://your-lastest2-server

Runner CLI

lastest2-runner start -t <token> -s <server-url>  # Start as background daemon
lastest2-runner stop                               # Stop the daemon
lastest2-runner status                             # Show runner status
lastest2-runner log [-f] [-n <lines>]              # View logs (-f to follow)
lastest2-runner run -t <token> -s <server-url>     # Run in foreground

Runner Capabilities

• Run: Execute tests remotely
• Record: Record new tests on remote machines
• Parallel: Configurable max parallel tests per runner
• System Info: Automatic OS, architecture, memory, and CPU reporting

Config stored in ~/.lastest2/ (runner.pid, runner.log, runner.config.json).

Google Sheets Integration

Use spreadsheet data as test data sources:

1. Connect your Google account in Settings → Google Sheets
2. Select spreadsheets and configure data sources with aliases (e.g., "users", "products")
3. Reference data in test code via the cached headers and rows

Supports per-team OAuth, automatic token refresh, multi-tab spreadsheets, custom header row selection, and fixed data ranges.

Custom Webhooks

Send build results to any HTTP endpoint. Configure in Settings → Notifications. Supports custom HTTP methods and headers.

Payload Format

{
  "event": "build.completed",
  "buildId": "abc123",
  "status": "safe_to_merge | review_required | blocked",
  "totalTests": 10,
  "passedCount": 8,
  "failedCount": 1,
  "changesDetected": 1,
  "flakyCount": 0,
  "gitBranch": "main",
  "gitCommit": "abc123",
  "buildUrl": "https://your-instance/builds/abc123",
  "timestamp": "2024-01-01T00:00:00.000Z"
}

VSCode Extension API

A REST + SSE API is available at /api/v1/ for IDE integration:

• Repos — list, get repositories
• Functional Areas — list, create, manage test areas
• Tests — CRUD operations, run individual tests
• Builds — trigger and monitor builds
• Runs — view test run results
• Events — SSE stream at /api/v1/events for real-time test updates

Settings

All configuration lives under a unified Settings page:

Tech Stack

• Framework: Next.js 16 (App Router)
• UI: React 19, Radix UI, Tailwind CSS 4
• Browser Automation: Playwright
• Visual Diffing: pixelmatch, SSIM, Butteraugli
• Accessibility: axe-core
• Database: SQLite + Drizzle ORM (WAL mode)
• Auth: better-auth (email/password with Argon2, GitHub, GitLab, Google OAuth)
• AI: Claude (Agent SDK, CLI, OpenRouter, direct Anthropic API), Ollama
• OCR Fallback: Tesseract.js
• Test Data: Google Sheets integration
• Email: Resend
• Testing: Vitest (unit), Playwright (visual)
• State: TanStack React Query

Environment Variables

# GitHub OAuth (for repository sync + login)
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
GITHUB_REDIRECT_URI=              # Optional
GITHUB_WEBHOOK_SECRET=            # Optional: verify webhook signatures

# GitLab OAuth (supports self-hosted instances)
GITLAB_CLIENT_ID=
GITLAB_CLIENT_SECRET=
GITLAB_REDIRECT_URI=              # Optional
GITLAB_INSTANCE_URL=              # Default: https://gitlab.com
GITLAB_WEBHOOK_SECRET=            # Optional: verify webhook signatures

# Google OAuth (for login)
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
GOOGLE_REDIRECT_URI=              # Optional

# Google Sheets OAuth (separate scope from login)
GOOGLE_SHEETS_REDIRECT_URI=       # Separate redirect for Sheets OAuth

# Email (optional, for invitations)
RESEND_API_KEY=
EMAIL_FROM=

# Advanced
DATABASE_PATH=                    # Default: ./lastest2.db
MONITORED_BRANCHES=               # Default: main,master,develop
NEXT_PUBLIC_APP_URL=              # Your app's public URL
NEXT_PUBLIC_BASE_URL=             # Base URL for API calls

Roadmap

• GitHub Actions integration (CLI runner)
• Slack/Discord notifications
• Team collaboration features
• GitLab integration (OAuth, MR comments, webhooks)
• Docker deployment
• Smart run (git-diff based test selection)
• Custom webhook notifications
• Google OAuth
• Google Sheets test data integration
• AI diff analysis with confidence scoring
• Spec-driven test generation
• Accessibility audits (axe-core)
• VSCode Extension API
• Remote runners with WebSocket
• Test versioning and history
• Planned screenshots (design comparison)
• Ignore regions for dynamic content
• Setup script orchestration
• Ollama support (local AI models)
• Cross-OS screenshot consistency
• Flaky test prevention (timestamp/random freezing, burst capture)
• Auto-mask dynamic content
• Page shift detection
• Multi-step screenshots
• Debug mode
• Branch comparison view
• App state inspection
• Network & console error tracking
• GitHub Action (reusable composite action)
• Test composition (cherry-pick tests + version overrides)
• Testing templates (8 preset configurations)
• Teardown orchestration (default + per-test overrides)
• Branch baseline management (fork/merge/promote)
• Functional area hierarchy (parent/child organization)
• Multi-engine diffing (SSIM, Butteraugli alongside pixelmatch)
• Text-region-aware diffing (OCR-based)
• Diff engine benchmark framework
• Auto-detect capabilities from recording
• Early adopter mode (experimental feature gating)
• Runner v2 (concurrent execution, code integrity, remote recording)
• better-auth migration (replaced Clerk)
• Remote runner NPM package (@lastest/runner on npm)
• Embedded browser container (live streaming, no local Playwright needed)
• Runner management UI (register, monitor, configure from dashboard)

Contributing

PRs welcome! See CONTRIBUTING.md for guidelines.

License

FSL-1.1-ALv2 — see LICENSE

_{Built for solo founders who ship fast and break things (then fix them before users notice).}