redflag-mcp

MCP server exposing AML red flag knowledge as queryable tools. Compliance officers ask natural-language questions; the server returns relevant, sourced red flags from a local vector database.

Overview

Three distinct workflows:

1. Extraction — pull AML red flags out of PDFs or web pages using an LLM and save them as YAML
2. Ingestion — embed the YAML files and load them into the local vector database
3. Query — MCP server answers semantic search queries against that database

Extraction Pipeline

scripts/extract.py takes a regulatory document (PDF file or URL), sends its text to an OpenAI model, and writes a structured YAML file into data/source/. Each extracted entry includes a source_url linking back to the original document.

Prerequisites

uv sync
export OPENAI_API_KEY=sk-...

Adding PDFs in bulk (recommended workflow)

Step-by-step:

1. Add the source URL to red_flag_sources/pdflinks.txt — one URL per line, in serial order. Line 1 → key 001, line 2 → 002, etc.
2. Download the PDF and save it to red_flag_sources/pdf/ named NNN_short_descriptive_name.pdf, where NNN matches its line position in pdflinks.txt.
3. Regenerate the registry: uv run python scripts/build_sources_registry.py
4. Run extraction: uv run python scripts/extract.py --parallel

Key constraint: the NNN_ prefix in the filename must match the line number in pdflinks.txt. Line 1 = 001_*.pdf, line 2 = 002_*.pdf, etc. This is how the extractor links each PDF to its public source URL.

PDFs are stored in red_flag_sources/pdf/ and must be named with a zero-padded serial prefix:

red_flag_sources/pdf/
  001_fincen_alert_russian_sanctions_evasion.pdf
  002_ffiec_bsa_aml_examination_manual.pdf
  003_fatf_guidance_virtual_assets.pdf

Each serial number maps to a public URL for the source document. Maintain this mapping in red_flag_sources/pdflinks.txt — one URL per line, in serial order:

# FinCEN Russian Sanctions Evasion Alert
https://fincen.gov/sites/default/files/2022-06/Alert%20FIN-2022-Alert001_508C.pdf

# FFIEC BSA/AML Examination Manual
https://bsaaml.ffiec.gov/manual

# FATF Guidance on Virtual Assets
https://www.fatf-gafi.org/...

Blank lines and lines starting with # are ignored. After editing pdflinks.txt, regenerate sources.yaml:

uv run python scripts/build_sources_registry.py

Then run batch extraction:

uv run python scripts/extract.py --parallel

Only new (unprocessed) PDFs are extracted — previously processed sources are skipped automatically.

Batch extraction commands

# Sequential batch
uv run python scripts/extract.py

# Parallel batch (4 workers by default)
uv run python scripts/extract.py --parallel

# Parallel batch with custom worker count
uv run python scripts/extract.py --parallel 8

# Force re-extract everything
uv run python scripts/extract.py --force --parallel

# Process only PDFs in a serial range (e.g. 001 through 005)
uv run python scripts/extract.py --range 001-005

# Range + parallel
uv run python scripts/extract.py --range 001-005 --parallel

# Force re-extract a range
uv run python scripts/extract.py --force --range 001-005 --parallel

Note: --range applies only to numbered PDFs. Web URLs in Weblinks.md are excluded when a range is active.

Single source (ad hoc)

# Extract from a local PDF
uv run python scripts/extract.py red_flag_sources/pdf/001_fincen_alert.pdf

# Extract from a URL
uv run python scripts/extract.py https://example.com/regulatory-guidance

# Re-extract a source that was already processed
uv run python scripts/extract.py --force red_flag_sources/pdf/001_fincen_alert.pdf

For single-source PDFs, add the URL to pdflinks.txt and run build_sources_registry.py first so the extractor can populate source_url in the output.

What it does

1. Fetches the document — downloads the web page (strips nav/footer/scripts) or reads text from the PDF via pdfplumber
2. Sends to OpenAI — prompts gpt-4o-mini (override with OPENAI_EXTRACTION_MODEL) to extract every distinct AML red flag indicator as structured JSON
3. Validates — each returned flag is checked against the RedFlagSource schema; invalid entries are skipped with a warning
4. Writes YAML — saves to data/source/<slug>.yaml, one entry per red flag
5. Updates the manifest — records the source in data/source/.extracted_sources.yaml to prevent re-processing

Output schema

Each entry in the YAML file has the following fields:

Deduplication

data/source/.extracted_sources.yaml tracks every processed source by its canonical path or URL. Sources already in the manifest are skipped in both batch and single-source mode. Use --force to re-extract a source regardless.

Ingestion

After extraction, embed the YAML files and load them into the vector database:

uv run python scripts/ingest.py

This reads all YAML files in data/source/, generates embeddings with nomic-embed-text-v1.5, and upserts records into LanceDB at data/vectors/. Run this before connecting the MCP server to Claude Desktop — the ~275 MB embedding model downloads on first use and connection timeouts will occur if it happens at server startup.

MCP Server

# Start server (stdio mode, for Claude Desktop / Claude Code)
uv run python -m redflag_mcp

# Start in MCP inspector
uv run mcp dev src/redflag_mcp/server.py

# Start as HTTP server (for OpenAI agents or other HTTP clients)
MCP_TRANSPORT=http MCP_HOST=0.0.0.0 MCP_PORT=8000 uv run python -m redflag_mcp

The server exposes three tools: search_red_flags, get_red_flag, and list_filters. It is fully offline after ingestion — no API keys required at query time.

Development

uv sync                          # Install dependencies
uv run pytest tests/             # Run tests
uv run ruff check src/           # Lint
uv run mypy src/                 # Type check