promptup-plugin
AI coding skill evaluator for Claude Code — 11-dimension scoring, decision intelligence, PR reports.
Zero infrastructure. Just SQLite at ~/.promptup/.
Install
npx promptup-plugin
Installs MCP server, commands, hooks, statusline, and default config. Restart Claude Code to activate.
How it works
Session tracking
Every Claude Code session is passively tracked via hooks. On session creation, the current git branch is recorded. This links sessions to PRs automatically.
Evaluation
/pup:eval spawns an independent claude -p to analyze your session transcript across 11 dimensions. It extracts developer decisions (steers, rejects, validates, scopes) and generates coaching recommendations with before/after examples from your actual prompts. Falls back to heuristic scoring if Claude is unavailable.
PR reports
/pup:pr-report matches sessions to the current branch, auto-evaluates unscored sessions, gathers all decisions, and computes a Decision Quality Score (DQS). The session-to-PR link uses sessions.branch — set when the session was created. Optionally posts as a GitHub PR comment.
DQS formula
DQS = autonomy * 25% + discipline * 25% + validation * 30% + diversity * 20%
- Autonomy: How often you steer, reject, or modify AI output vs accepting
- Discipline: Ratio of modifications to blind accepts
- Validation: How often you verify/test output
- Diversity: How many decision types you use (steer, reject, validate, modify, scope, accept)
Commands
| Command | What |
|---|---|
/pup:eval | Evaluate session across 11 skill dimensions |
/pup:pr-report | DQS report for current branch (add --post to comment on PR) |
/pup:status | Sessions tracked, evaluations, decision counts |
/pup:config | View/modify settings (/pup:config evaluation.auto_trigger=prompt_count) |
/pup:update | Check for and install updates |
MCP Tools
| Tool | What |
|---|---|
evaluate_session | Score session, extract decisions, coaching recommendations |
generate_pr_report | Match sessions to branch, compute DQS, optionally post to GitHub |
get_status | Tracking status and recent activity |
configure | Show/modify config (dot-path keys) |
Statusline
pupmeter shows your latest composite score in the Claude Code status bar with a recommendation tip.
Hooks
| Hook | What | Async |
|---|---|---|
PostToolUse | Logs tool events to ~/.promptup/tool-events.jsonl | Yes |
Stop | Captures session end to ~/.promptup/session-end.json | Yes |
SessionStart | Background update check against npm | Yes |
UserPromptSubmit | Auto-eval every N prompts (off by default) | Yes |
Configuration
/pup:config # show all settings
/pup:config evaluation.auto_trigger=prompt_count # enable auto-eval
/pup:config evaluation.interval=5 # eval every 5 prompts
/pup:config evaluation.weight_profile=bugfix # scoring profile
/pup:config decisions.signal_filter=all # show all decisions
Config file: ~/.promptup/config.json
11 Dimensions
Base (interaction quality): task decomposition, prompt specificity, output validation, iteration quality, strategic tool usage, context management
Domain (depth of understanding): architectural awareness, error anticipation, technical vocabulary, dependency reasoning, tradeoff articulation
Requirements
- Node.js >= 20
- Claude Code
Manual setup
{
"mcpServers": {
"promptup": {
"command": "node",
"args": ["~/.promptup/plugin/dist/index.js"]
}
}
}
Uninstall
npx promptup-plugin --uninstall
License
MIT