deepghs-mcp

A Python MCP server for the DeepGHS anime AI ecosystem. Connect it to any MCP-compatible client (Claude Desktop, Cursor, etc.) to browse datasets, discover pre-built character training sets, look up tags across 18 platforms, and generate complete data pipeline scripts — all directly from your AI assistant.

✨ Features

📦 Dataset & Model Discovery

• Browse all DeepGHS datasets — Danbooru2024 (8M+ images), Sankaku, Gelbooru, Zerochan, BangumiBase, and more
• Full file trees — see exactly which tar/parquet files a dataset contains and how large they are before downloading anything
• Model catalog — find the right model for your task: CCIP, WD Tagger Enhanced, aesthetic scorer, face/head detector, anime classifier, and more
• Live demos — browse DeepGHS Spaces (interactive web apps) for testing models without code

🏷️ Cross-Platform Tag Intelligence

• site_tags lookup — 2.5M+ tags unified across 18 platforms in one query
• Tag format translation — Danbooru uses hatsune_miku, Zerochan uses Hatsune Miku, Pixiv uses 初音ミク — this tool maps them all together
• Ready-to-use Parquet queries — get copy-paste code to filter the tag database programmatically

🎯 Character Dataset Finder

• Pre-built LoRA datasets — search both deepghs and CyberHarem namespaces for existing character image collections
• Ready-to-run download commands — get the exact cheesechaser command to pull what you need
• Smart fallback — if no pre-built dataset exists, the tool hands off directly to the waifuc script generator

🤖 Training Pipeline Code Generation

• waifuc scripts — generate complete, annotated Python data collection pipelines for any character from any source (Danbooru, Pixiv, Gelbooru, Zerochan, Sankaku, or Auto)
• cheesechaser scripts — generate targeted download scripts to pull specific post IDs from indexed multi-TB datasets without downloading the whole archive
• Format-aware — crop sizes, bucket ranges, and export formats automatically adjusted for SD 1.5, SDXL, or Flux

📦 Installation

Prerequisites

• Python 3.10+
• git

Quick Start

1. Clone the repository:

git clone https://github.com/citronlegacy/deepghs-mcp.git
cd deepghs-mcp

2. Run the installer:

chmod +x install.sh && ./install.sh
# or without chmod:
bash install.sh

3. Or install manually:

pip install -r requirements.txt

🔑 Authentication

HF_TOKEN is optional for public datasets but strongly recommended — it raises HuggingFace's API rate limit and is required for any gated or private repositories.

Get your token at huggingface.co/settings/tokens (read access is sufficient).

Without it, the server still works for all public DeepGHS datasets.

▶️ Running the Server

python deepghs_mcp.py
# or via the venv created by install.sh:
.venv/bin/python deepghs_mcp.py

⚙️ Configuration

Claude Desktop

Add the following to your claude_desktop_config.json:

{
  "mcpServers": {
    "deepghs": {
      "command": "/absolute/path/to/.venv/bin/python",
      "args": ["/absolute/path/to/deepghs_mcp.py"],
      "env": {
        "HF_TOKEN": "hf_your_token_here"
      }
    }
  }
}

Other MCP Clients

• Command: /absolute/path/to/.venv/bin/python
• Args: /absolute/path/to/deepghs_mcp.py
• Transport: stdio

💡 Usage Examples

Browse available datasets

"What anime datasets does DeepGHS have on HuggingFace?"

The assistant calls deepghs_list_datasets and returns all datasets sorted by download count — Danbooru2024, Sankaku, Gelbooru WebP, BangumiBase, site_tags, and more — with links and update dates.

Check dataset contents before downloading

"What files are in deepghs/danbooru2024? How big is it?"

The assistant calls deepghs_get_repo_info and returns the full file tree — every .tar and .parquet file with individual and total sizes — so you know exactly what you're committing to before you download.

Find a pre-built character dataset

"Is there already a dataset for Rem from Re:Zero I can use for LoRA training?"

The assistant calls deepghs_find_character_dataset, searches both deepghs and CyberHarem namespaces, and returns any matches with download counts and a one-liner download command.

Example response:

## Character Dataset Search: Rem

Found 2 dataset(s):

### CyberHarem/rem_rezero
Downloads: 4.2K | Likes: 31 | Updated: 2024-08-12

Download with cheesechaser:
  from cheesechaser.datapool import SimpleDataPool
  pool = SimpleDataPool('CyberHarem/rem_rezero')
  pool.batch_download_to_directory('./dataset', max_workers=4)

Look up a tag across all platforms

"What is the correct tag for 'Hatsune Miku' on Danbooru, Zerochan, and Pixiv?"

The assistant calls deepghs_search_tags and returns the format for every platform, plus a pre-filtered link to the site_tags dataset viewer and Parquet query code.

Example response:

Tag Lookup: Hatsune Miku

  Danbooru:  hatsune_miku
  Gelbooru:  hatsune_miku
  Sankaku:   character:hatsune_miku
  Zerochan:  Hatsune Miku
  Pixiv:     初音ミク (Japanese preferred)
  Yande.re:  hatsune_miku
  Wallhaven: hatsune miku

This directly solves the MultiBoru tag normalization problem.

Generate a full data collection pipeline

"Generate a waifuc script to build a LoRA dataset for Surtr from Arknights, from Danbooru and Pixiv, for SDXL, safe-only."

The assistant calls deepghs_generate_waifuc_script and returns a complete annotated Python script. Here is what that script does when run:

1.  Crawls Danbooru (surtr_(arknights)) + Pixiv (スルト アークナイツ)
2.  Converts all images to RGB with white background
3.  Drops monochrome, sketch, manga panels, and 3D renders
4.  Filters to safe rating only
5.  Deduplicates near-identical images (差分 / variants)
6.  Drops images with no detected face
7.  Splits group images — each character becomes its own crop
8.  CCIP identity filter — AI verifies every image actually shows Surtr
9.  WD14 auto-tagging — writes .txt caption files automatically
10. Resizes and pads to 1024×1024 (SDXL standard)
11. Exports in kohya_ss-compatible folder structure

No manual curation. No manual tagging. Drop the output folder straight into your trainer.

Generate a targeted download script

"Give me a cheesechaser script to download post IDs 1234, 5678, 9012 from deepghs/danbooru2024."

The assistant calls deepghs_generate_cheesechaser_script and returns a complete Python script — with options for downloading by post ID list, downloading everything, or filtering from the Parquet index by tag.

Find the right model for a task

"Does DeepGHS have a model for scoring image aesthetics?"

The assistant calls deepghs_list_models with search: "aesthetic" and returns matching models with pipeline task, download counts, and direct HuggingFace links.

Try a model in the browser

"Is there a live demo for the DeepGHS face detection model?"

The assistant calls deepghs_list_spaces with search: "detection" and returns matching Spaces with direct links to the interactive demos.

🛠️ Available Tools

📖 Tools Reference

deepghs_list_datasets

Lists all public datasets from DeepGHS, sortable and filterable by keyword.

Parameters

deepghs_list_models

Lists all public models — CCIP, WD Tagger Enhanced, aesthetic scorer, face/head/person detectors, anime classifier, furry detector, NSFW censor, style era classifier, and more.

Parameters

deepghs_list_spaces

Lists all public Spaces — live demos for face detection, head detection, CCIP character similarity, WD tagger, aesthetic scorer, reverse image search, Danbooru character lookup, and more.

Parameters

deepghs_get_repo_info

Get full metadata for any dataset, model, or space — including the complete file tree with individual file sizes. Essential before downloading a multi-TB dataset.

Parameters

Tip: Datasets containing .tar files automatically get a ready-to-copy cheesechaser snippet appended.

deepghs_search_tags

Look up any tag across 18 platforms using the deepghs/site_tags dataset. Returns per-platform format guidance, a pre-filtered dataset viewer link, and Parquet query code.

Parameters

LLM Tip: Call this before deepghs_generate_waifuc_script to confirm the correct Danbooru/Pixiv tag format for a character.

deepghs_find_character_dataset

Searches deepghs and CyberHarem on HuggingFace for pre-built character datasets. CyberHarem datasets are built with the full automated pipeline: crawl → CCIP filter → WD14 tag → upload.

Parameters

deepghs_generate_waifuc_script

Generates a complete, annotated Python pipeline script using waifuc.

Pipeline actions included (in order):

Parameters

Source tag formats:

deepghs_generate_cheesechaser_script

Generates a Python download script using cheesechaser to pull specific images from indexed tar datasets without downloading entire archives.

Parameters

🗂️ Key DeepGHS Datasets

🔄 Recommended Workflows

Workflow A: Use a pre-built dataset

1. deepghs_find_character_dataset   → check if dataset exists
2. deepghs_get_repo_info            → inspect file sizes
3. deepghs_generate_cheesechaser_script → get download command
4. Run the script → train

Workflow B: Build a new dataset from scratch

1. deepghs_search_tags              → find the correct Danbooru tag
2. deepghs_generate_waifuc_script   → generate full pipeline script
3. Run the script                   → crawl, filter, tag, crop
4. Review output                    → remove any remaining noise
5. Train with kohya_ss

Workflow C: Mine specific images from a large dataset

1. deepghs_get_repo_info            → inspect dataset Parquet structure
2. deepghs_search_tags              → confirm tag names
3. deepghs_generate_cheesechaser_script → generate Parquet-filter script
4. Run the script                   → downloads only matching images

🤖 Notes for LLMs

• Check pre-built first: Always call deepghs_find_character_dataset before generating a waifuc script. CyberHarem has hundreds of ready-to-use LoRA datasets.
• Danbooru tag format: character_(series) with underscores — rem_(re:zero), not Rem (Re:Zero). Use deepghs_search_tags to confirm.
• File sizes: Datasets like danbooru2024 are multi-TB. Always check deepghs_get_repo_info before recommending a full download. Use cheesechaser with post IDs for targeted access.
• CCIP is essential: It's the most important pipeline step — without it, 20–40% of a character dataset will be wrong-character noise. Always include it in waifuc scripts.
• Pixiv source: Requires a pixiv_token. If the user hasn't set one up, suggest Danbooru + Gelbooru instead.
• Model format crop sizes: SD1.5 = 512×512, SDXL/Flux = 1024×1024. This controls AlignMinSizeAction and PaddingAlignAction in generated scripts.

⚠️ Known Limitations

• HuggingFace rate limits: Without HF_TOKEN, the Hub API may throttle requests on heavy usage.
• Gated datasets: Some datasets require explicit approval on HuggingFace before downloading. The server returns a clear error with guidance.
• CyberHarem search: Niche characters may need manual browsing at huggingface.co/CyberHarem.
• waifuc runtime: The generated scripts require waifuc installed separately (not in this server's deps). First run downloads ~500MB of CCIP + WD14 model weights.

🐛 Troubleshooting

Server won't start:

• Ensure Python 3.10+: python --version
• Re-run the installer: bash install.sh

Rate limit / 429 errors:

• Set HF_TOKEN in your MCP env config
• Get a free token: huggingface.co/settings/tokens

403 / Forbidden on a dataset:

• The dataset is gated — visit the dataset page on HuggingFace and click "Request Access"
• Ensure the HF_TOKEN is from an account that has been granted access

Character dataset not found:

• Try alternate spellings: "Rem", "rem_(re:zero)", "rem re:zero"
• Browse manually: huggingface.co/CyberHarem
• Generate from scratch with deepghs_generate_waifuc_script

waifuc script fails:

• Install waifuc: pip install git+https://github.com/deepghs/waifuc.git
• GPU support: pip install "waifuc[gpu]" (much faster CCIP + tagging)
• First run downloads ~500MB of model weights — this is expected

🤝 Contributing

Pull requests are welcome! If a tool is returning incorrect data, a script template is outdated, or a new DeepGHS dataset or model should be highlighted, please open an issue or PR.

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

📄 License

MIT License — see LICENSE for details.

🔗 Links

• 🤗 DeepGHS on HuggingFace
• 🧰 waifuc — data pipeline
• 🧀 cheesechaser — targeted HF downloader
• 🖼️ imgutils — image processing
• 🤖 cyberharem — automated LoRA pipeline
• 🔧 MCP documentation
• 🐛 Bug Reports
• 💡 Feature Requests

Related MCP Servers

• gelbooru-mcp — Search Gelbooru, generate SD prompts from character tag data
• zerochan-mcp — Browse Zerochan's high-quality anime image board