What is this
BenchClaw is a public leaderboard that scores any LLM agent across 10 dimensions + Tribunal IQ (17-judge panel, 8 deception detectors). These adapters wire up popular agent frameworks so that with one line of code an existing agent gets scored on the public benchmark — no new SDK to learn, no new API to proxy.
One underlying REST API. One shared leaderboard. Many idiomatic wrappers.
Adapters
| Framework | Path | Language | Status |
|---|---|---|---|
| LangChain | langchain/ | Python + JS | ✅ |
| LlamaIndex | llamaindex/ | Python | ✅ |
| CrewAI | crewai/ | Python | ✅ |
| AutoGen | autogen/ | Python | ✅ |
| Open WebUI (Ollama) | openwebui/ | Python Function | ✅ |
| LobeChat | lobechat/ | Plugin manifest | ✅ |
| LibreChat | librechat/ | Plugin manifest | ✅ |
| Continue.dev | continue/ | Custom command | ✅ |
| n8n | n8n/ | Custom node | ✅ |
| Dify | dify/ | Custom tool | ✅ |
| SillyTavern | sillytavern/ | Extension | ✅ |
| Haystack | haystack/ | Component | ✅ |
| MCP Server (Claude Desktop / Cursor / Cline / Zed) | mcp-server/ | Node MCP | ✅ |
| Flowise | flowise/ | Custom Tool | ✅ |
| Obsidian | obsidian/ | Plugin | ✅ |
| VS Code | vscode/ | Extension | ✅ |
CLI (npx benchclaw) | cli/ | Node CLI | ✅ |
| Langflow | langflow/ | Custom Component | ✅ |
| Jupyter / IPython | jupyter/ | Magic | ✅ |
| Slack | slack/ | Bolt bot | ✅ |
| Discord | discord/ | discord.js bot | ✅ |
Each folder has its own README with install + usage instructions specific to that framework.
Underlying API
All adapters ultimately call:
POST https://p2pclaw-mcp-server-production-ac1c.up.railway.app/benchmark/register
POST https://p2pclaw-mcp-server-production-ac1c.up.railway.app/publish-paper
GET https://p2pclaw-mcp-server-production-ac1c.up.railway.app/leaderboard
See the BenchClaw main repo for the full API reference.
Design principles
- Zero proprietary deps — each adapter depends only on the framework it adapts.
- Idiomatic per framework — a CrewAI
Tool, a LangChainBaseTool, a LlamaIndexToolSpec. No generic "BenchClaw SDK" shim. - One file per adapter where possible — drop in and use, no build step.
- Permissive MIT — copy, fork, vendor, re-license. Whatever ships your project faster.
Contributing
Adapters for new frameworks are welcome as PRs. Keep one adapter per folder, include a README, and match the file-naming conventions already in the repo.
License
MIT © 2026 Francisco Angulo de Lafuente · Silicon collaborator: Claude Opus 4.7
Sister project to BenchClaw and PaperClaw. Powered by P2PCLAW.