Corpus freshness
Answers come only from a local index of the Claude Code docs, never from model memory. The index re-syncs from code.claude.com daily, picking up anything that changed.
last sync 2w ago · 205 pages · 5,635 chunks · 135 updated
Eval scores
0.85
retrieval hit@5
0.6958
MRR
0.85
answer pass rate
± 0.05, measured
5/5
refusals
correctly declined
Answerable
| question | grounded | cited | complete | no fab |
|---|---|---|---|---|
| What actually happens under the hood when Claude Code works on a task? | ✓ | ✓ | ✓ | ✓ |
| I just got Claude Code installed — how do I kick off my first session? | ✓ | ✓ | ✓ | ✓ |
| How do I give Claude a set of project instructions it picks up automatically every session? | ✓ | ✓ | ✓ | ✓ |
| What happens to my conversation when it gets really long and runs out of room? | ✓ | ✓ | ✓ | ✓ |
| How can I make my code formatter run automatically every time Claude edits a file? | ✓ | ✓ | ✓ | ✓ |
| How do I set up my own custom subagent for a specific kind of task? | ✓ | ✓ | ✓ | ✓ |
| I keep repeating the same workflow with Claude — how do I package it up as a skill? | ✓ | ✓ | ✓ | ✓ |
| How do I hook Claude Code up to a remote MCP server? | ✗ | ✗ | ✗ | ✗ |
| When two settings files set the same thing to different values, which one wins? | ✓ | ✓ | ✓ | ✗ |
| How do I make Claude lay out a plan before it starts changing my code? | ✓ | ✓ | ✓ | ✓ |
| How do I change which model Claude Code runs on? | ✓ | ✓ | ✓ | ✗ |
| How do I get started building an agent with the Claude Agent SDK? | ✓ | ✓ | ✓ | ✓ |
| How do I get the Claude Code extension installed in VS Code? | ✓ | ✓ | ✓ | ✓ |
| How do I wire up Claude Code to run inside a GitHub Actions workflow? | ✓ | ✓ | ✓ | ✓ |
| How do I turn on the sandbox for the Bash tool? | ✓ | ✓ | ✓ | ✓ |
| How do I run Claude Code from a shell script without the interactive interface? | ✓ | ✓ | ✓ | ✓ |
| What can I do to cut down how many tokens Claude Code burns through? | ✓ | ✓ | ✓ | ✓ |
| Claude made a bunch of changes I don't like — how do I roll them back? | ✓ | ✓ | ✓ | ✓ |
| How do I run Claude Code against Amazon Bedrock? | ✓ | ✓ | ✓ | ✓ |
| What does it take to write my own plugin for Claude Code? | ✓ | ✓ | ✓ | ✓ |
Refusals
- r-01correctly declined · sentinel
- r-02correctly declined · sentinel
- r-03correctly declined · sentinel
- r-04correctly declined · sentinel
- r-05correctly declined · sentinel
Boundary
- Can I make Claude Code read its answers out loud to me?expected refuse · ok
- What's the largest file Claude Code can read in a single operation?expected refuse · ok
- Can I schedule Claude Code to automatically review every new pull request?expected answer · ok
Config
k 5 · threshold 0.35 (calibrated 2026-07-23) · claude-haiku-4-5 · judge claude-sonnet-4-6 · text-embedding-3-small
vs baseline: retrieval hit@5 +0.0000, MRR +0.0000, answers -0.0500