smart-search

A local-first MCP server that makes PDF and DOCX files searchable from Claude Code via semantic search. Documents are extracted with Docling, chunked using structure-aware hierarchical chunking, embedded with nomic-embed-text-v1.5 (ONNX), and stored in LanceDB. Everything runs on CPU with no cloud dependencies and no GPU required.

Version: 0.1.0 (Foundation)

MCP Tools

The server exposes two tools to Claude Code.

knowledge_search

Search the knowledge base for document chunks matching a natural language query.

Returns a formatted block with source file name, page number, section heading path, chunk text, and relevance score for each result.

Note (v0.1): All three mode values currently execute semantic search. Keyword and hybrid modes using SQLite FTS5 + Reciprocal Rank Fusion are planned for v0.3.

knowledge_stats

Returns counts and metadata about the indexed knowledge base: document count, chunk count, index size in MB, last indexed timestamp, and file formats present.

No parameters.

Prerequisites

• Python 3.11 or later
• uv (recommended) or pip

The embedding model (nomic-ai/nomic-embed-text-v1.5) is downloaded from Hugging Face on first run and cached locally. No GPU or internet connection is required after that initial download.

Installation

Clone the repository and install in editable mode with development dependencies:

git clone <repository-url>
cd smart-search
uv pip install -e ".[dev]"

To install without development dependencies:

uv pip install -e .

Indexing Documents

Documents must be indexed before they can be searched. Use DocumentIndexer directly from Python, or integrate it into a CLI or script.

Index a single file:

from smart_search.config import get_config
from smart_search.chunker import DocumentChunker
from smart_search.embedder import Embedder
from smart_search.store import ChunkStore
from smart_search.indexer import DocumentIndexer

config = get_config()
store = ChunkStore(config)
store.initialize()

indexer = DocumentIndexer(
    config=config,
    chunker=DocumentChunker(config),
    embedder=Embedder(config),
    store=store,
)

result = indexer.index_file("/path/to/document.pdf")
print(result.status, result.chunk_count)

Index a folder:

result = indexer.index_folder("/path/to/documents", recursive=True)
print(f"Indexed: {result.indexed}, Skipped: {result.skipped}, Failed: {result.failed}")

Files already indexed at the same content hash are skipped automatically. Pass force=True to re-index regardless.

MCP Server Setup

Configure Claude Code

Copy .mcp.json.example to .mcp.json and update the paths to match your environment:

{
  "mcpServers": {
    "smart-search": {
      "command": "/path/to/your/venv/bin/python",
      "args": ["-m", "smart_search.server"],
      "cwd": "/path/to/smart-search"
    }
  }
}

On Windows, use the full path to python.exe inside your virtual environment's Scripts directory.

Run the server manually

python -m smart_search.server

The server communicates over stdio (MCP standard transport).

Configuration

All settings can be overridden with environment variables prefixed SMART_SEARCH_. The defaults are suitable for local use out of the box.

Paths are resolved to absolute paths at startup, so relative values are interpreted relative to the working directory where the server process starts.

Running Tests

The test suite uses pytest. Tests are split into fast unit tests and slow integration tests that load ML models or process real files.

Run fast tests only (default):

pytest

Run all tests including slow ones:

pytest --override-ini="addopts="

Run with coverage:

pytest --cov=smart_search --cov-report=term-missing

Slow tests are marked with @pytest.mark.slow. The default pytest invocation excludes them so the suite completes quickly without loading ML models.

Project Structure

src/smart_search/
  server.py    - FastMCP entry point; registers knowledge_search and knowledge_stats
  indexer.py   - Document ingestion pipeline (chunk, embed, store, dedup)
  chunker.py   - Docling DocumentConverter and HierarchicalChunker wrapper
  embedder.py  - nomic-embed-text-v1.5 ONNX embedding generation
  store.py     - LanceDB vector store and SQLite metadata store
  search.py    - Semantic search with Smart Context result formatting
  models.py    - Pydantic models: Chunk, SearchResult, IndexStats
  config.py    - Settings with SMART_SEARCH_ environment variable overrides

tests/
  test_models.py   - Chunk, SearchResult, IndexStats validation
  test_config.py   - Environment variable override and path resolution
  test_chunker.py  - DocumentChunker (slow: requires Docling)
  test_embedder.py - Embedder (slow: loads ONNX model)
  test_store.py    - ChunkStore LanceDB and SQLite operations
  test_indexer.py  - DocumentIndexer pipeline integration
  test_search.py   - SearchEngine formatting and filtering
  test_server.py   - FastMCP tool registration and dispatch

Tech Stack