docs/cache-harness.md

Cache Harness & Prompt-Cache Probe

Source Specs: v2-cache-harness-prd.md, docs/18-cache-harness.md, lean-context-prompt.md, docs/cache-harness-threat-model.md

Picasso treats prompt caching and context reduction as a unified harness problem across packages/cache-harness, packages/picasso-cache-probe, and packages/core/src/dsh/project.ts.

---

1. Architecture & Packages

Component	Path	Responsibility
Cache Harness	`packages/cache-harness`	Owns canonical prompt section ordering, stable-prefix vs. dynamic-suffix hashing, token-budgeted context packing, repo-map slicing, privacy/secret redaction filters, provider cache policy translation (Anthropic `cache_control` breakpoints, OpenAI `prompt_cache_key`, Gemini cached content handles, vLLM workspace-scoped `cache_salt`), and cache-miss attribution.
Runtime Cache Probe	`packages/picasso-cache-probe`	Cordis plugin mounted in the engine that intercepts the `llm/stream` waterfall and `tools/result` / `tool/result` hooks to log request metadata, stream usage chunks (`cached_tokens`, cache read/write tokens), and tool calls to `$DSH_HOME/picasso-probe/probe.jsonl`, plus registers the `picasso_cache_probe` inspection tool.
Event Projector	`packages/core/src/dsh/project.ts`	Projects the engine's token-meter usage chunks into normalized `cache-usage` `LoopEvent` records so `picasso run --cache observe --cache-report`, `picasso cache status	stats	inspect`, and the Mac Thread Inspector report cached vs. uncached input tokens.
Scenario Evaluation Suite	`scripts/cache/*`	Offline H0–H3 benchmark scenarios that validate stable-prefix hashes, token budgets, trajectory safety, and before/after context savings.

---

2. Prompt-Cache Invariants

Canonical Prefix Ordering (`tools -> system -> Tier 1 index`):

Static, repeatable sections always precede dynamic turn history.
Tool schemas, system instructions, and the Tier 1 routing index (AGENTS.md + compact skill index) are deterministically sorted and serialized with zero timestamps or random IDs.

Dynamic Suffix Isolation:

User prompts, Tier 2 on-demand loads (deferred tool results, MCP tool outputs, SKILL.md bodies, recalled memory bodies, mode-specific prompt sections), and per-turn reminders enter strictly as appended history messages via agent/pre-step or tool results.

Zero Prefix Drops Across On-Demand Loads & Modes:

Invoking tool_call, <server>_call, recall, or tool-skill on turn $N$ must never alter the tools array or system prompt on turn $N+1$.
Switching between default, plan, and auto permission modes preserves the exact Tier 1 byte prefix.

Privacy & Isolation Guardrails:

Sponsored content (ads) and telemetry metadata never enter model prompt sections.
Shared or self-hosted endpoints (e.g., vLLM) scope cache_salt per workspace/account and hash salts in exported reports so KV blocks never leak across workspaces.

---

3. Measuring & Verifying Lean Context

When modifying instruction loaders, skills, tool gateways, MCP proxies, or memory recall:

Capture first-turn request payloads on both TUI and Mac relay for hello and a coding prompt using packages/picasso-cache-probe ($DSH_HOME/picasso-probe/probe.jsonl) or packages/cache-harness.
Compare byte/token breakdowns (Tool schemas, System prompt, Instruction files, Skills, Memory, MCP) against reports/lean-context/baseline.md and record updated figures in reports/lean-context/after.md.
Verify that turn 2+ cached_tokens do not drop after loading a Tier 2 tool, skill, or memory body.