Server data from the Official MCP Registry
Context-window optimization: caching, compression & smart file tools that cut tokens 60-90%.
About
Context-window optimization: caching, compression & smart file tools that cut tokens 60-90%.
Security Report
The visible portions (README, package.json, and the start of src/server/index.ts) show no data exfiltration, hardcoded credentials, or obfuscated code, and the server's tools match its developer-tooling purpose. The audit is incomplete because the provided source is truncated, so the implementation of system-operation tools (smart_process, smart_service, smart_cron, smart_user), the postinstall script, and the telemetry and network code could not be verified. These unverified areas are the main reason the score is not higher. Supply chain analysis found 7 known vulnerabilities in dependencies (0 critical, 3 high severity). Package verification found 1 issue.
4 files analyzed · 12 issues found
Security scores are indicators to help you make informed decisions, not guarantees. Always review permissions before connecting any MCP server.
Permissions Required
This plugin requests these system permissions. Most are normal for its category.
What You'll Need
Set these up before or after installing:
Environment variable: TOKEN_OPTIMIZER_CACHE_DIR
How to Install
Add this to your MCP configuration file:
{
"mcpServers": {
"io-github-ooples-token-optimizer-mcp": {
"env": {
"TOKEN_OPTIMIZER_CACHE_DIR": "your-token-optimizer-cache-dir-here"
},
"args": [
"-y",
"@ooples/token-optimizer-mcp"
],
"command": "npx"
}
}
}Documentation
View on GitHubFrom the project's GitHub README.
How it compares to HeadRoom
Providers cache the prompt prefix: cached tokens re-read at 0.1x, rewritten ones bill at 1.25x. Most compressors optimise bytes removed and ignore that multiplier. This one optimises the bill.
Measured against their real implementation, on their own fixtures. Not a reimplementation and not our fixtures: their harness compresses the payload, dumps it, their resolver redeems their own markers, and ours is handed the identical bytes. Their column is the best of the nine configurations that capture holds, row by row.
Two of our arms are in the table, and they answer different questions. ours
is the block engine with nowhere to move a block, which is how the MCP tools and
the hooks run: everything it removes, it removes by compressing, and nothing
leaves the request. ours, proxy is the proxy a Claude Code session runs
through, which also moves a block it could not compress out to a file and
leaves a marker naming the path -- the same kind of trade their content-cache
reference makes. ours, preset is the block engine with that move switched on
at spillWholeBlockBelow: 0.9.
| workload | payload | theirs | ours | ours, proxy | ours, preset | zero-turn ids |
|---|---|---|---|---|---|---|
| agentic-conversation | 244,908 | 37.3% / 33.4% | 42.9% / 30.8% | 52.4% / 49.0% | 100.0% / 100.0% | 2099 / 2099 / 0 |
| agent-loop | 170,938 | 42.2% / 54.3% | 69.3% / 59.1% | 92.0% / 94.1% | 93.6% / 94.3% | 185 / 185 / 52 |
| agent-loop-logs | 327,500 | 51.5% / 53.0% | 61.9% / 58.1% | 88.7% / 91.5% | 96.2% / 97.0% | 2034 / 2034 / 81 |
| api-responses | 49,127 | 88.2% / 89.3% | 66.6% / 59.1% | 96.4% / 96.8% | 96.6% / 96.8% | 699 / 98 / 23 |
| browser-session | 781,774 | 21.7% / 13.1% | 93.7% / 26.5% | 55.5% / 90.2% | 100.0% / 99.9% | 334 / 334 / 0 |
| code-search | 131,264 | 47.4% / 59.7% | 81.8% / 70.8% | 97.7% / 97.9% | 95.7% / 96.1% | 201 / 201 / 43 |
| database-rows | 45,929 | 88.6% / 89.2% | 65.6% / 56.5% | 92.4% / 93.1% | 92.6% / 93.4% | 590 / 78 / 49 |
| codebase-exploration | 137,961 | 99.7% / 99.7% | 16.0% / 12.7% | 99.6% / 99.6% | 100.0% / 99.9% | 540 / 3 / 0 |
| grep-output | 75,219 | 99.6% / 99.6% | 56.1% / 51.9% | 99.5% / 99.6% | 99.9% / 99.9% | 1005 / 3 / 0 |
| log-entries | 91,385 | 80.8% / 83.2% | 63.5% / 52.0% | 95.4% / 95.7% | 95.5% / 96.0% | 1180 / 237 / 46 |
| human-authored-json | 37,425 | 30.6% / 30.0% | 66.6% / 66.0% | 81.6% / 83.7% | 96.7% / 96.9% | 154 / 154 / 11 |
| issue-triage | 109,354 | 53.1% / 55.2% | 88.1% / 85.7% | 97.2% / 97.6% | 95.1% / 95.7% | 59 / 59 / 21 |
| rag-conversation | 171,913 | 0.0% / 0.1% | 84.6% / 83.2% | 99.1% / 99.4% | 100.0% / 100.0% | 42 / 42 / 0 |
| raw-build-log | 158,057 | 99.8% / 99.8% | 70.6% / 48.2% | 99.7% / 99.8% | 100.0% / 100.0% | 430 / 3 / 0 |
| relevance-probe | 49,157 | 64.1% / 71.0% | 82.4% / 77.2% | 97.1% / 97.8% | 95.8% / 96.6% | 225 / 225 / 13 |
| search-results | 96,647 | 95.0% / 95.4% | 74.3% / 59.4% | 96.5% / 96.5% | 96.6% / 96.6% | 505 / 62 / 34 |
| repeated-reads | 158,285 | 37.4% / 30.4% | 71.7% / 68.5% | 99.5% / 99.6% | 100.0% / 100.0% | 394 / 388 / 0 |
| sre-debugging | 312,276 | 88.4% / 89.9% | 67.4% / 62.4% | 98.5% / 98.7% | 97.1% / 97.5% | 3116 / 506 / 82 |
Recorded 2026-10-17 at b1231a8c, by the command in the
record's regenerate field, over HeadRoom 0.37.0's own fixtures and harness. That arm runs
HeadRoom itself, so CI does not re-derive it -- it checks these figures and
this provenance against the record instead.
Over the corpus the proxy removes 82.5% of the characters and 91.5% of the tokens; their best configuration removes 51.5% and 61.4%. It is cheaper than their best arm on 18 of 18 rows when nothing is ever fetched back, which is the end of the range their markers are built for, and on 17 of 18 when every moved block is fetched back. The block engine alone removes 72.0% and 55.9%: more of the characters than theirs, fewer of the tokens.
Where they still win, and why. On codebase-exploration, grep-output,
raw-build-log, sre-debugging and the four raw-data rows their best arm takes
83.2-99.8% off the tokens and the block engine 12.7-62.4%. Their number there is
a content-cache reference: the block is not made smaller, it is taken out of
the request, put in a store, and replaced by a <<ccr:hash,kind,size>> marker.
An engine with nowhere to put a block cannot match that, so on those rows the
ours column loses on reduction, and on cost when nothing is fetched back. The
proxy makes the same move: it is ahead of them on five of those eight rows, level
on grep-output and raw-build-log, and a tenth of a point behind on
codebase-exploration. The one row where the proxy costs more than they do is
agentic-conversation, and only when every block it moved is fetched back:
they move none of that conversation out, so they cost the same at either end,
while the proxy moves part of it and pays a retrieval for each part it wants
back.
The other column is a turn. A marker is not the content: to read what it stands for, the agent spends a request. Of the 13,792 retention units planted in the corpus, the block engine keeps 5,223 in the text and 8,569 more rebuildable from the text alone, so every one of them is there with no extra turn; theirs keeps 6,711 in the text and 7,075 behind its store, one retrieval call away. 0 are unrecoverable on our side and 6 on theirs. Their best arms force 17 round trips across the corpus to get their content back; the block engine forces 0. The proxy, like theirs, trades a turn for every block it moves, and that turn is what the fetch-rate end of the cost model prices.
What their marker costs is priced, not just its size. Their marker is only
redeemable because their proxy also sends a headroom_retrieve tool with the
request -- on by default (ccr_inject_tool), and on every later request once a
session has used it -- so every row where their best arm leaves a marker is
charged for that tool's definition as well, recorded from their own package by
bench/compression/headroom/ccr-tool.py. The turn that first adds the tool also
rewrites the whole cached prefix behind it; that is not charged, which errs in
their favour. Ours needs no tool: our marker names a path the agent opens with
the Read it already has, and the path is priced at the length the proxy really
writes (<home>/.tok/<pid>/<15 digits>, src/compress/spill-path.ts) rather than
a shorter invented one.
browser-session is mostly the fixture. This payload holds four images, two
of them distinct, and the base64 in them is generated rather than photographic:
one distinct image alone folds from 160,032 characters to 5,572, which no real
PNG would do. What carries over to a real session is the duplication -- half the
image bytes are a second copy of an image already in the request, which is
44.3% of the whole payload. The run the marker names is still in the output
above it, so the reader rebuilds it without asking for anything, which is why
the characters fall much further than the tokens: the image tokens are counted
from pixels on both arms either way.
Reproduce the whole table:
python bench/compression/headroom/run-theirs.py <headroom-clone> <out-dir>
python bench/compression/headroom/resolve-theirs.py <headroom-clone> <out-dir>
node bench/compression/head-to-head.mjs <out-dir>
Against their published design
A second arm reimplements their published design from their own benchmark
generator's definitions — opaque hash markers, history compressed, a retrieval
tool and system message injected — so the four workloads with a published
comparator can be checked without their clone. node bench/compression/proof.mjs:
| workload | payload | theirs | ours | verdict |
|---|---|---|---|---|
| code-search | 17765 | 92.1% | 97.8% | ours |
| sre-debugging | 65694 | 92.2% | 98.6% | ours |
| issue-triage | 54174 | 72.8% | 97.5% | ours |
| codebase-exploration | 78502 | 47.4% | 54.7% | ours |
Four workloads, because those are the four this arm has a published comparator for. The head-to-head harness scores eighteen; the other fourteen are not in this arm, and are scored against their real implementation in the table above.
codebase-exploration was theirs and is now ours, by 7.3 points. An earlier
version of this table claimed 61.3% for us on that row, then parity, then a
published loss at 46.0% against their 47.4%. The first two were figures from a
branch rather than from here; the third was true on this tree when it was
written; four log engine commits then moved it to 54.2%, masking an interior
hashbang so a concatenated block still parses moved it to 64.9%, and
authenticating every marker with a six-character stamp took 0.2 of a point
back, and naming a spilled file by a digest of its contents, the way the proxy
sink really names it, instead of by its position in a counter, took another
half point, to reach 64.2%. Then the currency stopped being an estimate: every
figure in the table above is now the count Anthropic's own tokenizer returns
for the text, recorded into a committed fixture so this stays offline, and
under it the row reads 53.7%, and pricing the path the proxy's sink really
writes -- content-addressed and decimal, where this arm had invented a 32-digit
hex name -- moved it to 54.7%. That is the largest move in this history and
the only one that is not about the engine at all. Dividing characters by four
had been charging the control's hex markers too much and our own stamp far too
little -- the stamp was six characters of a vowelless base-31 alphabet then,
and a random string over a large alphabet is close to the worst case a BPE
tokenizer has, so it cost a measured 5.52 tokens where chars/4 billed 2. Those
three are the only figures here that moved the wrong way, and none of them is a
regression: each is the harness charging us for something it had been leaving
out. Being able to see that cost is what paid for the next move: the stamp is
now nine decimal digits, which carry slightly more entropy than those six
characters did and cost 4 tokens on every draw rather than a mean of 5.52 that
ranged from four to seven. Measured against the old stamp on this row -- the
same tree, the same fixture, only the stamp pinned back to its base-31 form --
that bought a tenth of a point, and a tenth is the honest size of it: a
marker got better than a quarter cheaper, but markers are a small part of what
the engine emits, so the headline moves far less than the per-marker saving
suggests. A
number in prose is a fact about the tree it was measured on, which is why
bench/compression/readme-table.check.mjs re-derives every figure in the block
above from the harness rather than trusting it.
Against the four comparators in this reimplemented arm: ours on all four. Against their real implementation the answer is the table at the top of this section, and it is a split rather than a sweep: the proxy is cheaper than their best arm on every row when nothing is fetched back and takes more of the tokens overall, while the block engine alone keeps every identifier without a turn and loses on reduction wherever their best arm moves a block out of the request.
The two columns come from different arms of the same engine, and that is the
point. v3-history compresses history too and matches them byte for byte;
v1-frontier leaves the cached prefix alone and wins the invoice by 11–42%.
Maximising bytes removed is available and is not the default, because on a
cached prefix it costs money.
The competitor arm is this repository's reimplementation of their published design — opaque hash markers, history compressed, a retrieval tool and system message injected — not their binary, and it is held to the same signed-content guard we hold ourselves to.
On whole sessions
node bench/compression/session-replay.mjs replays real recorded conversations
turn by turn and prices each the way a provider does. Four independent sessions,
whole request including system prompt and tool schema, against no proxy:
| conversation length | ours | cheaper on |
|---|---|---|
| 10 turns | 0.953x | 4 of 4 |
| 20 turns | 0.915x | 4 of 4 |
This is the region nothing else touches: conversation history is ~65% of a live request and the only part that grows every turn. The saving grows with session length, because model reasoning accumulates.
On end-to-end agent tasks
THOL, 16 real tasks against a no-proxy control:
- cheaper on 11 of 16 tasks
- median cost 0.926x, median turns 0.671x
- zero quality cost: mean score 0.994 against control's 0.994, lower on 0 of 16 tasks
One run per task, so the aggregate interval still spans 1.0; the per-task tally and the score parity are the solid parts. Every figure here is regenerated from the committed benchmarks, which ship in this repository.
The 30-second version
Your agent burns most of its context on work it already did: re-reading files that have not changed, dumping a whole file to see three lines, running unbounded searches, and re-deriving conclusions it reached last session and then forgot.
Token Optimizer attacks that on four fronts.
1. It makes the expensive call impossible. Install the plugin and a built-in
Read of a 200 KB file is denied, with the refusal naming the cached,
diffed replacement. Same for Grep, Glob, Edit, Write, and cat /
head / grep -r through the shell. Re-reading a file you already read this
session returns only a diff — usually the single biggest win, and one that
size-based rules structurally cannot catch. There is no setting to turn on.
2. It remembers what your agent worked out. A per-project knowledge graph accumulates findings, decisions and dead ends as a side effect of working, then feeds them back the moment the agent touches the relevant file. A finding costs ~150 tokens to carry. Re-deriving it costs 5k–50k.
3. It measures itself, in public, and tells you when it is losing. A materialized before/actual-return measurement for MCP progressive disclosure, with later expansions debited from the same net. Modeled graph substitutions and the randomized control arm for downstream graph effects remain separate. Every number is measured, visibly collecting, excluded, or absent.
4. It attributes the traffic. Returned context, optional cost equivalents, and net transport avoided are grouped by operation and MCP handshake identity. Codex, Claude Code, Gemini, and any other connected client get separate rows. Old records without identity remain explicitly unattributed instead of being assigned to whichever agent happens to be open now.
No account and no hosted service. MIT, so it is usable at work.
Nothing is measured about you unless you ask for it. Anonymous usage data is
off by default and takes two separate opt-ins: TOKEN_OPTIMIZER_TELEMETRY=1
aggregates counts into a file on your own disk, and TOKEN_OPTIMIZER_BEACON=1
is what uploads any of it. Setting the first opens no socket. With both set,
the events already on disk are sent once per session at start-up -- in a batch,
never on the hot path -- and the file is cleared only after the receiver has
accepted them. install_doctor prints which of the two is on, how many events
are waiting and where they would go; it reports without sending. A package
built without the upload key cannot transmit at all, and the doctor says so
rather than letting an opt-in look like it is working.
DO_NOT_TRACK=1 overrides both, whatever else is set. The version check
npm run doctor and install_doctor make is a plain GET to the registry npm
installs from, carries nothing about you, and runs only when you ask for a
diagnosis -- never during a session. DO_NOT_TRACK=1 or
TOKEN_OPTIMIZER_UPDATE_CHECK=0 suppresses it before a socket is opened. An event may only
contain numbers and flags -- a string is dropped before it is written, so a
path, a prompt or an error message cannot travel even by mistake -- and the
machine identifier is a salted hash, never your hostname. What the counts are
about is the hook ledger your own project already keeps: how often the graph
answered instead of a tool, what each arm of the holdout was delivered and
withheld, and whether the saving exceeded the cost -- reduced to integers once
every six hours, never the ledger itself, which names your files. Usage is
counted as a rollup rather than a row per request: one event per 200 requests,
holding the request count, bytes in and out, how often compression paid, how
often knowledge was injected, and how many blocks were elided, plus one final
event when the proxy stops. That keeps the totals exact -- a row per request
would overflow the batch on a busy session and leave you reporting its first
five hundred -- at the cost of the per-request distribution, which is not
something anyone downstream needs. npm run doctor
prints exactly which of these is on and what has been recorded.
What the dashboard proves on a real machine
The screenshots in this README come from the shipped server reading persisted local data, not a design mockup. In the capture above it reports:
- 43,491 net verified MCP transport tokens avoided in the current live proof: 54,037 gross reduction minus a deliberate 10,546-token expansion;
- 486,074,740 historical/tool-reported tokens quarantined, dominated by repository scan volume that never entered model context;
- a live Codex / Claude Code / Gemini stdio smoke against AiDotNet, with each client attributed independently;
- 2,648 graph nodes, 6,527 edges, and 58 findings across 11 local projects;
- more than 1,000 hook runs with zero failures and zero timeouts across six active CLI clients in the selected rolling 24-hour window;
- 6,332 tokens of modeled graph-substitution potential, excluded from the
verified headline while the causal graph-reuse study remains
Collecting.
The dashboard now reads native CLI usage receipts and prices uncached input, cache reads, cache writes, and output with the exact captured provider, model, route, request-time tier, and versioned official source. Ambiguous model ids stay Not priced instead of receiving a blended guess. API/list-price equivalents are kept separate from provider-reported charges and are never labeled as a subscription invoice. See the token accounting contract.
Quick start
Claude Code — install the plugin, not the bare MCP server. The plugin is what enforces; adding the server alone just gives the model tools it can ignore.
/plugin marketplace add ooples/token-optimizer-mcp
/plugin install token-optimizer@token-optimizer
/reload-plugins
That is the entire installation. All sixteen clients →
Then, whenever you want to know what to do next:
token_audit
One ranked queue: what is costing the most per session, with an optional monthly cost equivalent only after you configure your own effective rate. Each line names how to fix it. Not a dashboard, not six reports — a queue.
Subagents and workflows
Agents are where token spend concentrates. A long-running subagent re-sends its whole context on every turn, so every byte a tool adds is paid for again on each later turn. In one measured session, 42 workflow subagents made 4,483 model calls. The six longest made 218–348 calls each and peaked at 336K–609K tokens of context. Together they spent 4.0M output tokens and 1.0B cache-read tokens. The bill is the re-reading, not the writing.
What works with no setup
The plugin's hooks run for subagents exactly as they do for the main session. Nothing needs to be configured per agent.
- Whole-file reads are outlined, through
Readand through the shell. Agents mostly read files withcat,typeorGet-Content, rarely withRead. In the measured session that was 2,071 shell reads against 9Readcalls. On the first whole read of a large source file, either kind returns a line-numbered outline of its symbols instead of the file. A second whole read of the same file returns the file itself. A line range is never outlined:sed -n 'A,Bp',head,Get-Content -TotalCountand a pagedReadall pass through untouched. - Each subagent is briefed once. A subagent never sees the SessionStart guidance, so its first tool call carries a four-line briefing instead: load the optimizer tools if they are listed, read ranges rather than whole files, and keep the final report short. The tools are named only conditionally, so a hooks-only install is never told to call something it does not have.
- Advisories are said once. A routing hint such as "use smart_grep" appears at most once per agent for each kind of call, not on every matching command.
- Past findings arrive only when they name what you are running. A finding
with no explicit trigger is delivered with a command only when its claim names
that command's program and subcommand, such as
npx jestordotnet build, not when the two merely share a word. - Repeated build and test output is bounded, as it is in the main session.
- Through the proxy, a range you asked for comes back whole. A paged
Reador a shell slice is never folded, and a fold inside a whole-file read points at the file's own line numbers.
What to add when you brief an agent
Claude Code gives subagents MCP tools as deferred tools: the schemas are not loaded until the agent asks for them. An agent that never loads them falls back to shell commands. Put this in the agent's prompt:
Before your first file read or search, load the token-optimizer tools:
ToolSearch("smart_read smart_grep smart_glob").
Use smart_read for files (it returns a diff on re-reads), smart_grep for
searching, and smart_glob for finding files. Do not print whole files or search
the tree through the shell. When you need part of a file, read a line range.
The server's own instructions now say the same thing to every client, deferred or not.
Keep agents short
The plugin can only shrink what enters an agent's context. The number of turns and the size of the context it carries are set by how the work is split up:
- One bounded task per agent. Several short agents cost less than one agent that runs for hundreds of turns, because each starts from a small context.
- Give the agent the facts it needs (paths, line numbers, the exact question) instead of asking it to rediscover them.
- Ask for a structured result so the agent stops when it has the answer.
Measuring an agent
Each subagent writes its own transcript next to the session's, under
~/.claude/projects/<project>/<session>/subagents/. The usage block on every
assistant row records input_tokens, output_tokens,
cache_read_input_tokens and cache_creation_input_tokens.
Count each response once. One model response is written as several
assistant rows that share a message.id, and each row repeats the usage
recorded so far. Keep the LAST row per message.id, which holds the final
usage, and sum those. Summing every row overstates the cost: on the session
above it reported 1.8B cache-read tokens instead of 1.0B. Keeping the first row
instead understates output, at 0.5M instead of 4.0M.
The number of distinct message.ids is the number of model calls the agent
made. Read against its peak context, it shows whether the agent should have
been split.
The knowledge graph — the part nothing else has
Every agent session ends the same way: the reasoning evaporates. The next session re-derives it, at full price, forever.
This builds a living per-project graph — nodes for files, symbols, tasks and
findings; edges for derived_from, contains, supersedes, contradicts,
related — and it fills itself in from real work. No ingestion job, no
embedding model, no rebuild step, no query to formulate.
you touch src/auth.ts
│
├─ verify() compares exp against the LOCAL clock (finding, 0.9)
├─ per-host retry budgets; global was rejected — deadlock (decision)
├─ ! the skew fix was reverted once already (dead end)
└─ [git] 47 changes in 90d, last three: "fix token expiry",
"revert skew fix", "fix token expiry again"
None of that is in your repository. It exists only because an agent once burned tokens finding it out — and every other tool throws it away at the end of the session.
What a default install actually produces. The structural graph — files,
symbols, tasks, and the edges between them — is captured from ordinary tool
traffic with no configuration at all. Findings are produced two ways. At session
end, derive reads evidence already on disk (command outcomes and exit codes,
red-to-green transitions, corrections, re-read churn) and writes findings from
it: no model call, no credential, nothing sent anywhere. And the active model
records durable conclusions itself through wiki_write.
The model-based semantic harvest is the third path, and the only one that
needs something you do not already have. It is not opt-in —
TOKEN_OPTIMIZER_HARVEST=0 turns it off — but its real gate is a credential:
with none it reports off:no-key, which is the state on CI, corporate machines,
and subscription-only logins. Point TOKEN_OPTIMIZER_HARVEST_ENDPOINT at a
local model and it runs free and private, with nothing leaving the machine.
npx token-optimizer-doctor states which of these is live.
Why this is not RAG
| Classic RAG | This |
|---|---|
| Retrieves evidence; the model re-derives meaning each time | Retrieves verdicts — the reasoning already happened |
| Index built by a batch ingestion job | Accretes from real agent traffic — coverage follows attention |
| Similarity search | Traversal — this symbol and its callers |
| Model must formulate a query | Fires when the model reaches for a file |
| Staleness invisible; serves rotted chunks confidently | Staleness computed from content hashes, served with the invalidating diff |
| Returns only what is in the documents | Returns dead ends, which exist nowhere in your source tree |
Traversal plus lexical search: deterministic, instant, explainable, and it works offline.
The zero-turn refusal
A plain deny costs a full turn: the model calls Read, is refused, re-plans,
calls smart_read. But at refusal time we already hold the file and the
snapshot the graph stored — so the refusal carries the answer inside it.
Nothing to re-plan, no second call. Turn cost drops from one to zero.
And when the graph already holds the verdict a tool output would support, the output never enters context at all. Not compressed. Absent.
The dashboard
npm install
npm run build
npm run dashboard # http://localhost:3100
The overview answers the questions a token optimizer should answer first:
- How much MCP context did it avoid? The headline is gross materialized payload reduction minus every later linked expansion. Graph estimates are intentionally separate.
- How much context still reached the agents? Every successful current MCP result records its actual returned text, even when no valid before-state exists. That row is context-accounted but savings-unmeasured.
- Which agent and action spent it? The client ledger and action table show
operations, returned context, optional cost equivalent, and net tokens
avoided. Lifecycle-only clients say
Not measured; no zero is invented. - Did remembering cost more than it saved? Delivery and semantic-harvest tokens are charged to the graph. A causal benefit is added only after the treated/holdout evidence gate passes.
Walkthrough: get useful data, not an empty dashboard
- Install the MCP server and the native adapter for your CLI. The MCP handshake provides per-client accounting; native lifecycle hooks provide automatic routing, capture, health, and delivery where the client protocol permits it.
- Use
smart_read,smart_grep,smart_glob,smart_edit, or any other MCP operation normally. Every successful result records returned context; tools with a comparable materialized before-state also record a gross reduction; laterexpandcalls debit that reduction. - Let the active model record durable conclusions with
wiki_write. Before a new agent re-derives work, callwiki_readfor the project or the files it is about to touch. Native clients can also deliver matching knowledge automatically. Usewiki_queryto read the graph directly — one finding by key, a ranked BM25 search over claims, a node with its neighbours, or the graph's own audit — which is how a subagent that never sees the SessionStart briefing reaches what previous sessions established. - Open
http://localhost:3100. Use Overview for combined accounting and What it knows for capture health, graph exploration, audits, and causal evidence. - To register existing local repositories without reading their source, run
npm run projects:discover -- /absolute/path/to/repos. This makes coverage gaps explicit; it does not fabricate findings.
For maintainers, this live smoke exercises the shipped stdio transport and creates separately attributed rows without seeding the analytics database:
npm run dashboard:attribution-smoke -- /absolute/path/to/project /absolute/path/to/large-file
npm run dashboard:verify-live -- http://localhost:3100
The health panel is deliberately operational rather than a raw text dump. Each client has activity, runtime failures/timeouts, policy outcomes, and observed surface coverage. Diagnostics keep no prompts, commands, paths, or tool output.
Modeled substitution potential and causal graph effects are different claims.
The first is a full-file counterfactual and is never promoted to the verified
MCP headline. The second asks whether delivered knowledge prevented later reads;
it uses a control arm and remains Collecting until there are at least 20
treated file touches and 5 holdouts with valid downstream joins.
Drag to orbit, scroll to zoom, click a node for provenance, or switch to the bounded one-hop focus view. The default All known projects scope pools local graphs through opaque project IDs; filesystem paths never reach the browser.
Audit tab. Contradictions, stale findings, and low-confidence claims remain reviewable instead of silently becoming model truth.
Evidence console. Client/model/task cohorts, matched effects with 95% intervals, live outcome joins, harm feedback, and capability tiers for all 16 clients. Release and superiority claims fail closed while evidence is missing.
One-click Markdown export. The accumulated graph becomes documentation you can inspect, edit, and commit.
Server-side by design: the browser asks for a neighbourhood, a search result or a page. A mature graph holds thousands of nodes, and shipping it wholesale would make every page load a multi-megabyte download for a view that shows twenty things.
The default All known projects scope combines captured graphs through an opaque machine-local project registry; filesystem paths never reach the browser. Lifecycle hooks register repositories as they are used. To backfill existing local checkouts without reading their source files, run the bounded discovery command against one or more explicit roots:
npm run projects:discover -- /absolute/path/to/repos /absolute/path/to/worktrees
Coverage distinguishes repositories with graph data from known repositories
whose capture has not started. The balance cards are backed by persisted events:
Memory deliveries counts graph context actually supplied to an agent,
Kept back for comparison counts randomized control touches, and Cost of
remembering combines delivered-context tokens with measured semantic-write
payload cost. Reading avoided stays Collecting or Not measured until at
least 20 treated file touches and 5 holdouts exist with a downstream join; the
dashboard does not manufacture a savings estimate from missing data.
See the cross-client capability contract and the live evaluation suite.
Cross-client lifecycle diagnostics
Every native hook writes the same bounded JSONL lifecycle record, including Claude Code's custom router and compaction paths. Records carry the client and plugin versions, event, hashed session/turn correlation, latency, outcome, input/output byte counts, and response key shape. They deliberately retain no prompt, command, tool output, file content, or raw working-directory path. The fields include OpenTelemetry log severity and resource semantics so the local files can be collected without inventing a second schema.
npm run diagnostics # last 24 hours, summary-first JSON
npm run diagnostics -- --hours 72 --output hook-summary.json
npm run diagnostics -- --include-events --limit 100 --output hook-report.json
Raw event rows are opt-in and capped at 1,000. The default report contains aggregate health and at most twenty recent failures/timeouts, keeping routine troubleshooting output small enough for CLI and model context windows.
The dashboard's Capture health panel shows runs, failures, timeouts and
p50/p95 latency by client. Logs rotate at 5 MiB, retain at most 40 files for 14
days, and live under .token-optimizer/logs when a state directory is set (or
~/.token-optimizer/logs otherwise). TOKEN_OPTIMIZER_LOG_DIR,
TOKEN_OPTIMIZER_LOG_MAX_BYTES, TOKEN_OPTIMIZER_LOG_MAX_FILES, and
TOKEN_OPTIMIZER_LOG_RETENTION_DAYS override those operational defaults.
What it does that other optimizers do not
Compression that does not break the cache, or lose the needle
An optional local proxy compresses tool results, search output, logs and
conversation history on the way to the model. A hook cannot do this: the
PostToolUse output schema is {hookEventName, additionalContext?, classifierContext?} and updatedOutput occurs nowhere in the client, so a hook
can add context but never replace a result. The proxy never tries to -- it
rewrites the outbound request, where those results already sit as history.
Two things make it different from simply compressing harder.
It never rewrites cached content. A cached prefix bills at 0.1x and a cache write at 1.25x, so compressing history can cut tokens while multiplying the bill. Measured on our own benchmark: compressing behind the cache breakpoint removes more raw tokens on every workload and costs more money on every workload. Compression stops at the frontier, and cache-weighted tokens are reported beside raw ones so the trap is visible rather than inferred.
It keeps the rows that matter. Eliding a long array after the first few rows scored 95.7% on a search payload here and destroyed both the UUID record and the error record planted in it -- the only two rows anyone would have searched for. The engine now keeps every row that departs from the shape, wherever it sits: 92.4% with the needles intact. A gate in the benchmark fails the build if a planted needle disappears, because a size metric alone cannot tell compression from truncation.
Reduction over the content each strategy is permitted to rewrite, on fixtures matching the four workloads HeadRoom publishes, is the guarded table earlier in this section -- 97.5%, 97.8%, 98.6% and 54.7% against their 72.8%, 92.1%, 92.2% and 47.4%. It is stated once and checked there rather than restated here, which is how this copy came to claim 98.9%, 98.2%, 92.8% and 61.3% long after the tree had moved.
Documentation truncated — see the full README on GitHub.
Reviews
No reviews yet
Be the first to review this server!
More Developer Tools MCP Servers
Rea
Freeby Morluto · Developer Tools
Reverse engineer anything from your terminal or agent with one CLI and MCP server.
Paperclip
Freeby Paperclipai · Developer Tools
Trending hip-hop artist momentum scores across four cultural dimensions.
Git
Freeby Modelcontextprotocol · Developer Tools
Read, search, and manipulate Git repositories programmatically
Fetch
Freeby Modelcontextprotocol · Developer Tools
Web content fetching and conversion for efficient LLM usage
Toleno
Freeby Toleno · Developer Tools
Toleno Network MCP Server — Manage your Toleno mining account with Claude AI using natural language.
MCP Marketplace
Freeby mcp-marketplace · Developer Tools
Search and install MCP servers from inside your AI client.
