diff --git a/.agents/skills/cavecrew/README.md b/.agents/skills/cavecrew/README.md new file mode 100644 index 0000000..0801ba0 --- /dev/null +++ b/.agents/skills/cavecrew/README.md @@ -0,0 +1,67 @@ +# cavecrew + +Decision guide. When to delegate to caveman subagents instead of doing the work inline. + +## What it does + +Tells main thread when to spawn a caveman-style subagent. Compact return +contracts can reduce repeated prose when results return to main context, but +effect depends on task, agent, and delegation count. This skill publishes no +universal reduction rate. + +Three subagents: + +| Subagent | Job | Use when | +|----------|-----|----------| +| `cavecrew-investigator` | Locate code (read-only) | "Where is X defined / what calls Y / list uses of Z" | +| `cavecrew-builder` | Surgical edit, 1-2 files | Scope is obvious, ≤2 files. Refuses 3+ file scope. | +| `cavecrew-reviewer` | Diff/file review | One-line findings with severity emoji | + +Use vanilla `Explore` or `Code Reviewer` when you want prose, architecture commentary, or rationale. Use main thread directly for one-line answers and 3+ file refactors. + +This skill is a decision guide, not a slash command. It activates when the conversation mentions delegation. + +## How to invoke + +Triggers on phrases like "delegate to subagent", "use cavecrew", "spawn investigator", "save context", "compressed agent output". + +## Example chaining + +Locate → fix → verify (most common): + +1. `cavecrew-investigator` returns site list (`path:line`, symbol, note) +2. Main thread picks 1-2 sites, hands paths to `cavecrew-builder` +3. `cavecrew-reviewer` audits the resulting diff + +Parallel scout: spawn 2-3 `cavecrew-investigator` calls in one message with different angles (defs, callers, tests). Aggregate in main. + +## Model overrides + +By default, `cavecrew-reviewer` and `cavecrew-investigator` pin `model: haiku` in their frontmatter; `cavecrew-builder` has no `model:` line (uses the API session default). Set env vars in your shell before launching Claude Code to override per-agent: + +| Env var | Agent | +|---|---| +| `CAVECREW_REVIEWER_MODEL` | `cavecrew-reviewer` | +| `CAVECREW_BUILDER_MODEL` | `cavecrew-builder` | +| `CAVECREW_INVESTIGATOR_MODEL` | `cavecrew-investigator` | + +Example: run reviewer on sonnet and keep others on default. + +```sh +export CAVECREW_REVIEWER_MODEL=sonnet +``` + +Use the same model name strings you'd use in any Claude Code agent frontmatter (e.g. `haiku`, `sonnet`, `opus`). + +Overrides patch only `model:` line in installed agent frontmatter; prompt body +stays untouched and continues receiving upstream updates. Only plugin installs +have local agent files to patch. Empty variables do nothing. Patch persists until +plugin update or reinstall. + +## See also + +- [`SKILL.md`](./SKILL.md): full decision matrix and output contracts +- [`agents/cavecrew-investigator.md`](../../agents/cavecrew-investigator.md) +- [`agents/cavecrew-builder.md`](../../agents/cavecrew-builder.md) +- [`agents/cavecrew-reviewer.md`](../../agents/cavecrew-reviewer.md) +- [Caveman README](../../README.md): repo overview diff --git a/.agents/skills/cavecrew/SKILL.md b/.agents/skills/cavecrew/SKILL.md new file mode 100644 index 0000000..efa413f --- /dev/null +++ b/.agents/skills/cavecrew/SKILL.md @@ -0,0 +1,82 @@ +--- +name: cavecrew +description: > + Decision guide for delegating to caveman-style subagents. Tells the main + thread WHEN to spawn `cavecrew-investigator` (locate code), `cavecrew-builder` + (1-2 file edit), or `cavecrew-reviewer` (diff review) instead of doing the + work inline or using vanilla `Explore`. Subagent output is caveman-compressed + so the tool-result injected back into main context is ~60% smaller — main + context lasts longer across long sessions. + Trigger: "delegate to subagent", "use cavecrew", "spawn investigator/builder/reviewer", + "save context", "compressed agent output". +--- + +Cavecrew = three subagent presets that emit caveman output. Same job as Anthropic defaults (`Explore`, edit-style agents, reviewer); difference is the tool-result they return is compressed, so main context shrinks per delegation. + +## When to use cavecrew vs alternatives + +| Task | Use | +|---|---| +| "Where is X defined / what calls Y / list uses of Z" | `cavecrew-investigator` | +| Same but you also want suggestions/architecture commentary | `Explore` (vanilla) | +| Surgical edit, ≤2 files, scope obvious | `cavecrew-builder` | +| New feature / 3+ files / cross-cutting refactor | Main thread or `feature-dev:code-architect` | +| Review diff, branch, or file for bugs | `cavecrew-reviewer` | +| Deep code review with rationale + alternatives | `Code Reviewer` (vanilla) | +| One-line answer you already know | Main thread, no subagent | + +Rule of thumb: **if you'd want the subagent's output in 1/3 the tokens, pick cavecrew. If you'd want prose, pick vanilla.** + +## Why this exists (the real win) + +Subagent tool results get injected into main context verbatim. A vanilla `Explore` that returns 2k tokens of prose costs 2k tokens of main-context budget every time. The same finding from `cavecrew-investigator` returns ~700 tokens. Across 20 delegations in one session that's the difference between context exhaustion and finishing the task. + +## Output contracts + +What main thread can rely on per agent: + +**`cavecrew-investigator`** +``` +
: +- path:line — `symbol` — short note +totals: . +``` +Or `No match.` Always file-path-first, line-number-attached, backticked symbols. Safe to grep with `path:\d+`. + +**`cavecrew-builder`** +``` + — . +verified: . +``` +Or one of: `too-big.` / `needs-confirm.` / `ambiguous.` / `regressed.` (terminal first token). + +**`cavecrew-reviewer`** +``` +path:line: : . . +totals: N🔴 N🟡 N🔵 N❓ +``` +Or `No issues.` Findings sorted file → line ascending. + +## Chaining patterns + +**Locate → fix → verify** (most common): +1. `cavecrew-investigator` returns site list. +2. Main thread picks 1-2 sites, hands paths to `cavecrew-builder`. +3. `cavecrew-reviewer` audits the diff. + +**Parallel scout** (when investigation is broad): +Spawn 2-3 `cavecrew-investigator` calls in one message (different angles: defs vs callers vs tests). Aggregate in main thread. + +**Single-shot edit** (when site is already known): +Skip investigator. Hand exact path:line to `cavecrew-builder` directly. + +## What NOT to do + +- Don't use `cavecrew-builder` when you don't already know the file. Spawn investigator first or main thread will eat tokens passing context. +- Don't chain `cavecrew-investigator → cavecrew-builder` for a 5-file refactor. Builder will return `too-big.` and you'll have wasted a turn. +- Don't ask `cavecrew-reviewer` for "general feedback" — it returns findings only, no architecture opinions. Use `Code Reviewer` for that. +- Don't expect prose. Cavecrew output is structured, sometimes terse to the point of cryptic. If a human will read it directly, paraphrase. + +## Auto-clarity (inherited) + +Subagents drop caveman → normal English for security warnings, irreversible-action confirmations, and any output where fragment ambiguity could be misread. Resume caveman after. diff --git a/.agents/skills/caveman-commit/README.md b/.agents/skills/caveman-commit/README.md new file mode 100644 index 0000000..d5aee01 --- /dev/null +++ b/.agents/skills/caveman-commit/README.md @@ -0,0 +1,44 @@ +# caveman-commit + +Terse Conventional Commits. Why over what. + +## What it does + +Generates commit messages in Conventional Commits format. Subject ≤50 chars, hard cap 72. Imperative mood. Body only when the *why* is non-obvious or there are breaking changes. No AI attribution, no "this commit does X", no emoji unless the project uses them. Body always required for breaking changes, security fixes, data migrations, and reverts — future debuggers need the context. + +Outputs only the message. Does not stage, commit, or amend. + +## How to invoke + +``` +/caveman-commit +``` + +Also triggers on phrases like "write a commit", "commit message", "generate commit". + +## Example output + +Diff: new endpoint for user profile. + +``` +feat(api): add GET /users/:id/profile + +Mobile client needs profile data without the full user payload +to reduce LTE bandwidth on cold-launch screens. + +Closes #128 +``` + +Diff: breaking API rename. + +``` +feat(api)!: rename /v1/orders to /v1/checkout + +BREAKING CHANGE: clients on /v1/orders must migrate to /v1/checkout +before 2026-06-01. Old route returns 410 after that date. +``` + +## See also + +- [`SKILL.md`](./SKILL.md) — full LLM-facing instructions +- [Caveman README](../../README.md) — repo overview diff --git a/.agents/skills/caveman-commit/SKILL.md b/.agents/skills/caveman-commit/SKILL.md new file mode 100644 index 0000000..b9999e3 --- /dev/null +++ b/.agents/skills/caveman-commit/SKILL.md @@ -0,0 +1,65 @@ +--- +name: caveman-commit +description: > + Ultra-compressed commit message generator. Cuts noise from commit messages while preserving + intent and reasoning. Conventional Commits format. Subject ≤50 chars, body only when "why" + isn't obvious. Use when user says "write a commit", "commit message", "generate commit", + "/commit", or invokes /caveman-commit. Auto-triggers when staging changes. +--- + +Write commit messages terse and exact. Conventional Commits format. No fluff. Why over what. + +## Rules + +**Subject line:** +- `(): ` — `` optional +- Types: `feat`, `fix`, `refactor`, `perf`, `docs`, `test`, `chore`, `build`, `ci`, `style`, `revert` +- Imperative mood: "add", "fix", "remove" — not "added", "adds", "adding" +- ≤50 chars when possible, hard cap 72 +- No trailing period +- Match project convention for capitalization after the colon + +**Body (only if needed):** +- Skip entirely when subject is self-explanatory +- Add body only for: non-obvious *why*, breaking changes, migration notes, linked issues +- Wrap at 72 chars +- Bullets `-` not `*` +- Reference issues/PRs at end: `Closes #42`, `Refs #17` + +**What NEVER goes in:** +- "This commit does X", "I", "we", "now", "currently" — the diff says what +- "As requested by..." — use Co-authored-by trailer +- "Generated with Claude Code" or any AI attribution — unless the user's own rule requires an `Assisted-by`/AI-attribution trailer, then add it as a trailer +- Emoji (unless project convention requires) +- Restating the file name when scope already says it + +## Examples + +Diff: new endpoint for user profile with body explaining the why +- ❌ "feat: add a new endpoint to get user profile information from the database" +- ✅ + ``` + feat(api): add GET /users/:id/profile + + Mobile client needs profile data without the full user payload + to reduce LTE bandwidth on cold-launch screens. + + Closes #128 + ``` + +Diff: breaking API change +- ✅ + ``` + feat(api)!: rename /v1/orders to /v1/checkout + + BREAKING CHANGE: clients on /v1/orders must migrate to /v1/checkout + before 2026-06-01. Old route returns 410 after that date. + ``` + +## Auto-Clarity + +Always include body for: breaking changes, security fixes, data migrations, anything reverting a prior commit. Never compress these into subject-only — future debuggers need the context. + +## Boundaries + +Only generates the commit message. Does not run `git commit`, does not stage files, does not amend. Output the message as a code block ready to paste. "stop caveman-commit" or "normal mode": revert to verbose commit style. diff --git a/.agents/skills/caveman-compress/README.md b/.agents/skills/caveman-compress/README.md new file mode 100644 index 0000000..a784452 --- /dev/null +++ b/.agents/skills/caveman-compress/README.md @@ -0,0 +1,176 @@ +

+ +

+ +

caveman-compress

+ +

+ shrink memory file. save token every session. +

+ +--- + +A Claude Code skill that compresses project memory files (`CLAUDE.md`, todos, +preferences) into caveman format, reducing repeated input size. + +Claude loads `CLAUDE.md` on every session start, so large files add repeated +input tokens. Caveman shortens supported natural-language files. + +## What It Do + +``` +/caveman-compress CLAUDE.md +``` + +``` +CLAUDE.md ← compressed (Claude reads smaller file each session) +CLAUDE.original.md ← human-readable backup (you edit this) +``` + +Original remains in data directory rather than next to live file, so skill +auto-loaders do not read it twice. Path is +`$XDG_DATA_HOME/caveman-compress/backups//` on macOS and Linux, +or `%LOCALAPPDATA%\caveman-compress\backups\\` on Windows. Edit +`.original.md` there, then run skill again to re-compress. + +## Benchmarks + +Real results on real project files: + +| File | Original | Compressed | Saved | +|------|----------:|----------:|------:| +| `claude-md-preferences.md` | 706 | 285 | 59.6% | +| `project-notes.md` | 1145 | 535 | 53.3% | +| `claude-md-project.md` | 1122 | 636 | 43.3% | +| `todo-list.md` | 627 | 388 | 38.1% | +| `mixed-with-code.md` | 888 | 560 | 36.9% | +| Average | 898 | 481 | 46% | + +All fixture validations passed: headings, code blocks, URLs, and file paths were +preserved exactly. + +## Before / After + + + + + + +
+ +### Original (706 tokens) + +> "I strongly prefer TypeScript with strict mode enabled for all new code. Please don't use `any` type unless there's genuinely no way around it, and if you do, leave a comment explaining the reasoning. I find that taking the time to properly type things catches a lot of bugs before they ever make it to runtime." + + + +### rock Caveman (285 tokens) + +> "Prefer TypeScript strict mode always. No `any` unless unavoidable; comment why if used. Proper types catch bugs early." + +
+ +This fixture produced 59.6% fewer counted tokens. Structural validation passed; +result does not prove semantic equivalence on other files or models. + +## Security + +`caveman-compress` is flagged as Snyk High Risk due to subprocess and file I/O +patterns detected by static analysis. See [SECURITY.md](./SECURITY.md) for why +these operations exist and how paths are constrained. + +## Install + +Compress is built in with the `caveman` plugin. Install `caveman` once, then use `/caveman-compress`. + +If you need local files, the compress skill lives at: + +```bash +skills/caveman-compress/ +``` + +Requires Python 3.10 or newer. + +## Usage + +``` +/caveman-compress +``` + +Examples: +``` +/caveman-compress CLAUDE.md +/caveman-compress docs/preferences.md +/caveman-compress todos.md +``` + +### What files work + +| Type | Compress? | +|------|-----------| +| `.md`, `.txt`, `.rst`, `.typ`, `.typst`, `.tex` | Yes | +| Extensionless natural language | Yes | +| `.py`, `.js`, `.ts`, `.json`, `.yaml` | ❌ Skip (code/config) | +| `*.original.md` | ❌ Skip (backup files) | + +## How It Work + +``` +/caveman-compress CLAUDE.md + ↓ +detect file type (no tokens) + ↓ +Claude compresses (tokens: one call) + ↓ +validate output (no tokens) + checks: headings, code blocks, URLs, file paths, bullets + ↓ +if errors: Claude fixes cherry-picked issues only (tokens: targeted fix) + does NOT recompress; only patches broken parts + ↓ +retry up to 2 times + ↓ +write compressed → CLAUDE.md +write original → CLAUDE.original.md +``` + +Only two things use tokens: initial compression + targeted fix if validation fails. Everything else is local Python. + +## What Is Preserved + +Caveman compress natural language. It never touch: + +- Code blocks (` ``` ` fenced or indented) +- Inline code (`` `backtick content` ``) +- URLs and links +- File paths (`/src/components/...`) +- Commands (`npm install`, `git commit`) +- Technical terms, library names, API names +- Headings (exact text preserved) +- Tables (structure preserved, cell text compressed) +- Dates, version numbers, numeric values + +## Why This Matter + +`CLAUDE.md` loads on every session start. A 1,000-token project memory file adds +1,000 input tokens each time project opens, or 100,000 across 100 sessions. + +Caveman reduced counted tokens by about 46% on five listed fixtures. Validators +confirmed headings, code blocks, URLs, and file paths. They did not establish +general semantic or task-quality equivalence. + +``` +┌────────────────────────────────────────────┐ +│ TOKEN SAVINGS PER FILE █████ 46% │ +│ FIXTURES IN TABLE 5 │ +│ STRUCTURAL VALIDATION passed on all │ +│ SETUP TIME █ 1x │ +└────────────────────────────────────────────┘ +``` + +## Part of Caveman + +This skill is part of the [caveman](https://github.com/JuliusBrussee/caveman) toolkit. + +- `caveman`: ask Claude to answer in shorter prose +- `caveman-compress`: shorten supported project-memory files with backups and validation diff --git a/.agents/skills/caveman-compress/SECURITY.md b/.agents/skills/caveman-compress/SECURITY.md new file mode 100644 index 0000000..0efa9fe --- /dev/null +++ b/.agents/skills/caveman-compress/SECURITY.md @@ -0,0 +1,31 @@ +# Security + +## Snyk High Risk Rating + +`caveman-compress` receives a Snyk High Risk rating due to static analysis heuristics. This document explains what the skill does and does not do. + +### What triggers the rating + +1. **subprocess usage**: The skill calls the `claude` CLI via `subprocess.run()` as a fallback when `ANTHROPIC_API_KEY` is not set. The subprocess call uses a fixed argument list — no shell interpolation occurs. User file content is passed via stdin, not as a shell argument. + +2. **File read/write**: The skill reads the file the user explicitly points it at, compresses it, and writes the result back to the same path. A `.original.md` backup is saved to an out-of-tree data dir (`$XDG_DATA_HOME/caveman-compress/backups//`, or `%LOCALAPPDATA%\caveman-compress\backups\\` on Windows). Beyond the target file and that backup location, no files are read or written. + +### What the skill does NOT do + +- Does not execute user file content as code +- Does not make network requests except to Anthropic's API (via SDK or CLI) +- Does not access files outside the path the user provides +- Does not use shell=True or string interpolation in subprocess calls +- Does not collect or transmit any data beyond the file being compressed + +### Auth behavior + +If `ANTHROPIC_API_KEY` is set, the skill uses the Anthropic Python SDK directly (no subprocess). If not set, it falls back to the `claude` CLI, which uses the user's existing Claude desktop authentication. + +### File size limit + +Files larger than 500KB are rejected before any API call is made. + +### Reporting a vulnerability + +If you believe you've found a genuine security issue, please open a GitHub issue with the label `security`. diff --git a/.agents/skills/caveman-compress/SKILL.md b/.agents/skills/caveman-compress/SKILL.md new file mode 100644 index 0000000..0b95aab --- /dev/null +++ b/.agents/skills/caveman-compress/SKILL.md @@ -0,0 +1,111 @@ +--- +name: caveman-compress +description: > + Compress natural language memory files (CLAUDE.md, todos, preferences) into caveman format + to save input tokens. Preserves all technical substance, code, URLs, and structure. + Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md. + Trigger: /caveman-compress FILEPATH or "compress memory file" +--- + +# Caveman Compress + +## Purpose + +Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `.original.md`, but NOT beside the source file — it lives in an out-of-tree data dir (`$XDG_DATA_HOME/caveman-compress/backups//`, or `%LOCALAPPDATA%\caveman-compress\backups\\` on Windows) so skill auto-loaders don't re-ingest it as a live file. + +## Trigger + +`/caveman-compress ` or when user asks to compress a memory file. + +## Process + +1. The compression scripts live in `scripts/` (adjacent to this SKILL.md). If the path is not immediately available, search for `scripts/__main__.py` next to this SKILL.md. + +2. From the directory containing this SKILL.md, run: + +python3 -m scripts + +3. The CLI will: +- detect file type (no tokens) +- call Claude to compress +- validate output (no tokens) +- if errors: cherry-pick fix with Claude (targeted fixes only, no recompression) +- retry up to 2 times +- if still failing after 2 retries: report error to user, leave original file untouched + +4. Return result to user + +## Compression Rules + +### Remove +- Articles: a, an, the +- Filler: just, really, basically, actually, simply, essentially, generally +- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend" +- Hedging: "it might be worth", "you could consider", "it would be good to" +- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because" +- Connective fluff: "however", "furthermore", "additionally", "in addition" + +### Preserve EXACTLY (never modify) +- Code blocks (fenced ``` and indented) +- Inline code (`backtick content`) +- URLs and links (full URLs, markdown links) +- File paths (`/src/components/...`, `./config.yaml`) +- Commands (`npm install`, `git commit`, `docker build`) +- Technical terms (library names, API names, protocols, algorithms) +- Proper nouns (project names, people, companies) +- Dates, version numbers, numeric values +- Environment variables (`$HOME`, `NODE_ENV`) + +### Preserve Structure +- All markdown headings (keep exact heading text, compress body below) +- Bullet point hierarchy (keep nesting level) +- Numbered lists (keep numbering) +- Tables (compress cell text, keep structure) +- Frontmatter/YAML headers in markdown files + +### Compress +- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize" +- Fragments OK: "Run tests before commit" not "You should always run tests before committing" +- Drop "you should", "make sure to", "remember to" — just state the action +- Merge redundant bullets that say the same thing differently +- Keep one example where multiple examples show the same pattern + +CRITICAL RULE: +Anything inside ``` ... ``` must be copied EXACTLY. +Do not: +- remove comments +- remove spacing +- reorder lines +- shorten commands +- simplify anything + +Inline code (`...`) must be preserved EXACTLY. +Do not modify anything inside backticks. + +If file contains code blocks: +- Treat code blocks as read-only regions +- Only compress text outside them +- Do not merge sections around code + +## Pattern + +Original: +> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production. + +Compressed: +> Run tests before push to main. Catch bugs early, prevent broken prod deploys. + +Original: +> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens. + +Compressed: +> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens. + +## Boundaries + +- ONLY compress natural language files (.md, .txt, .typ, .typst, .tex, extensionless) +- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh +- If file has mixed content (prose + code), compress ONLY the prose sections +- If unsure whether something is code or prose, leave it unchanged +- Original file is backed up as FILE.original.md before overwriting — in the out-of-tree backup data dir (see Purpose), not beside the source file +- Never compress FILE.original.md (skip it) diff --git a/.agents/skills/caveman-compress/scripts/__init__.py b/.agents/skills/caveman-compress/scripts/__init__.py new file mode 100644 index 0000000..16b8c53 --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/__init__.py @@ -0,0 +1,9 @@ +"""Caveman compress scripts. + +This package provides tools to compress natural language markdown files +into caveman format to save input tokens. +""" + +__all__ = ["cli", "compress", "detect", "validate"] + +__version__ = "1.0.0" diff --git a/.agents/skills/caveman-compress/scripts/__main__.py b/.agents/skills/caveman-compress/scripts/__main__.py new file mode 100644 index 0000000..4e28416 --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/__main__.py @@ -0,0 +1,3 @@ +from .cli import main + +main() diff --git a/.agents/skills/caveman-compress/scripts/benchmark.py b/.agents/skills/caveman-compress/scripts/benchmark.py new file mode 100644 index 0000000..97d081b --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/benchmark.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 +from pathlib import Path +import sys + +# Support both direct execution and module import +try: + from .validate import validate +except ImportError: + sys.path.insert(0, str(Path(__file__).parent)) + from validate import validate + +try: + import tiktoken + _enc = tiktoken.get_encoding("o200k_base") +except ImportError: + _enc = None + + +def count_tokens(text): + if _enc is None: + return len(text.split()) # fallback: word count + return len(_enc.encode(text)) + + +def benchmark_pair(orig_path: Path, comp_path: Path): + orig_text = orig_path.read_text(encoding="utf-8", errors="ignore") + comp_text = comp_path.read_text(encoding="utf-8", errors="ignore") + + orig_tokens = count_tokens(orig_text) + comp_tokens = count_tokens(comp_text) + saved = 100 * (orig_tokens - comp_tokens) / orig_tokens if orig_tokens > 0 else 0.0 + result = validate(orig_path, comp_path) + + return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid) + + +def print_table(rows): + print("\n| File | Original | Compressed | Saved % | Valid |") + print("|------|----------|------------|---------|-------|") + for r in rows: + print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'✅' if r[4] else '❌'} |") + + +def main(): + # Direct file pair: python3 benchmark.py original.md compressed.md + if len(sys.argv) == 3: + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + if not orig.exists(): + print(f"❌ Not found: {orig}") + sys.exit(1) + if not comp.exists(): + print(f"❌ Not found: {comp}") + sys.exit(1) + print_table([benchmark_pair(orig, comp)]) + return + + # Glob mode: repo_root/tests/caveman-compress/ + # __file__ lives at /skills/caveman-compress/scripts/benchmark.py + # Walk up four dirs: scripts → caveman-compress → skills → repo_root. + tests_dir = Path(__file__).resolve().parents[3] / "tests" / "caveman-compress" + if not tests_dir.exists(): + print(f"❌ Tests dir not found: {tests_dir}") + sys.exit(1) + + rows = [] + for orig in sorted(tests_dir.glob("*.original.md")): + comp = orig.with_name(orig.stem.removesuffix(".original") + ".md") + if comp.exists(): + rows.append(benchmark_pair(orig, comp)) + + if not rows: + print("No compressed file pairs found.") + return + + print_table(rows) + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/caveman-compress/scripts/cli.py b/.agents/skills/caveman-compress/scripts/cli.py new file mode 100644 index 0000000..75ea8a6 --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/cli.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +""" +Caveman Compress CLI + +Usage: + caveman +""" + +import sys + +# Force UTF-8 on stdout/stderr before any code can print. Windows consoles +# default to cp1252 and crash on the ❌ glyphs in error/validation branches, +# masking the real error and leaving the user with a half-compressed file. +for _stream in (sys.stdout, sys.stderr): + reconfigure = getattr(_stream, "reconfigure", None) + if callable(reconfigure): + try: + reconfigure(encoding="utf-8", errors="replace") + except Exception: + pass + +from pathlib import Path + +from .compress import backup_dir_for, compress_file +from .detect import detect_file_type, should_compress + + +def print_usage(): + print("Usage: caveman ") + + +def main(): + if len(sys.argv) != 2: + print_usage() + sys.exit(1) + + filepath = Path(sys.argv[1]) + + # Check file exists + if not filepath.exists(): + print(f"❌ File not found: {filepath}") + sys.exit(1) + + if not filepath.is_file(): + print(f"❌ Not a file: {filepath}") + sys.exit(1) + + filepath = filepath.resolve() + + # Detect file type + file_type = detect_file_type(filepath) + + print(f"Detected: {file_type}") + + # Check if compressible + if not should_compress(filepath): + print("Skipping: file is not natural language (code/config)") + sys.exit(0) + + print("Starting caveman compression...\n") + + try: + success = compress_file(filepath) + + if success: + print("\nCompression completed successfully") + backup_path = backup_dir_for(filepath) / (filepath.stem + ".original.md") + print(f"Compressed: {filepath}") + print(f"Original: {backup_path}") + sys.exit(0) + else: + print("\n❌ Compression failed after retries") + sys.exit(2) + + except KeyboardInterrupt: + print("\nInterrupted by user") + sys.exit(130) + + except Exception as e: + print(f"\n❌ Error: {e}") + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/caveman-compress/scripts/compress.py b/.agents/skills/caveman-compress/scripts/compress.py new file mode 100644 index 0000000..3e75e4b --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/compress.py @@ -0,0 +1,505 @@ +#!/usr/bin/env python3 +""" +Caveman Memory Compression Orchestrator + +Usage: + python scripts/compress.py +""" + +import os +import re +import shutil +import stat +import subprocess +import sys +import tempfile +from pathlib import Path +from typing import List + +# Windows consoles default to cp1252, which cannot encode the emoji glyphs in +# our status lines; replace unencodable characters instead of crashing. +for _stream in (sys.stdout, sys.stderr): + try: + _stream.reconfigure(errors="replace") + except Exception: + pass + +# A fence marker at the start of a line, at CommonMark's 0-3 space indent. +FENCE_LINE_REGEX = re.compile(r"^\s{0,3}(`{3,}|~{3,})") + +# YAML frontmatter: starts at file start with --- on its own line, ends with --- on its own line. +# Captures the entire block (including delimiters and trailing newline) and the body after. +FRONTMATTER_REGEX = re.compile( + r"\A(---\r?\n.*?\r?\n---\r?\n)(.*)", re.DOTALL +) + + +def split_frontmatter(text: str): + """Split YAML frontmatter from body. Returns (frontmatter, body). + + Memory files (and many other markdown docs) start with a YAML frontmatter + block delimited by `---` lines. The compression LLM has a habit of stripping + or rewriting these despite preserve-structure rules in the prompt — so we + surgically remove the frontmatter before compression and prepend it back + verbatim to the output. Files without frontmatter pass through unchanged. + """ + m = FRONTMATTER_REGEX.match(text) + if m: + return m.group(1), m.group(2) + return "", text + +# Filenames and paths that almost certainly hold secrets or PII. Compressing +# them ships raw bytes to the Anthropic API — a third-party data boundary that +# developers on sensitive codebases cannot cross. detect.py already skips .env +# by extension, but credentials.md / secrets.txt / ~/.aws/credentials would +# slip through the natural-language filter. This is a hard refuse before read. +SENSITIVE_BASENAME_REGEX = re.compile( + r"(?ix)^(" + r"\.env(\..+)?" + r"|\.netrc" + r"|credentials(\..+)?" + r"|secrets?(\..+)?" + r"|passwords?(\..+)?" + r"|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?" + r"|authorized_keys" + r"|known_hosts" + r"|.*\.(pem|key|p12|pfx|crt|cer|jks|keystore|asc|gpg)" + r")$" +) + +SENSITIVE_PATH_COMPONENTS = frozenset({".ssh", ".aws", ".gnupg", ".kube", ".docker"}) + +SENSITIVE_NAME_TOKENS = ( + "secret", "credential", "password", "passwd", + "apikey", "accesskey", "token", "privatekey", +) + + +def backup_dir_for(filepath: Path) -> Path: + """Resolve the out-of-tree backup directory for a given source file. + + Backups must live OUTSIDE the source directory so skill auto-loaders + (Claude Code rules/, opencode instructions/, etc.) stop re-ingesting the + `.original.md` copies as live files. Base dir is platform-aware: + - Windows: %LOCALAPPDATA%\\caveman-compress\\backups + - else: $XDG_DATA_HOME/caveman-compress/backups if set, + else ~/.local/share/caveman-compress/backups + + The source file's parent-dir name is mirrored under the base to reduce + cross-project collisions (e.g. two `task.md` files in different repos). + """ + if os.name == "nt" or sys.platform == "win32": + local_appdata = os.environ.get("LOCALAPPDATA") + base = Path(local_appdata) if local_appdata else Path.home() / "AppData" / "Local" + base = base / "caveman-compress" / "backups" + else: + xdg = os.environ.get("XDG_DATA_HOME") + base = Path(xdg) if xdg else Path.home() / ".local" / "share" + base = base / "caveman-compress" / "backups" + return base / filepath.parent.name + + +def is_sensitive_path(filepath: Path) -> bool: + """Heuristic denylist for files that must never be shipped to a third-party API.""" + name = filepath.name + if SENSITIVE_BASENAME_REGEX.match(name): + return True + lowered_parts = {p.lower() for p in filepath.parts} + if lowered_parts & SENSITIVE_PATH_COMPONENTS: + return True + # Normalize separators so "api-key" and "api_key" both match "apikey". + lower = re.sub(r"[_\-\s.]", "", name.lower()) + return any(tok in lower for tok in SENSITIVE_NAME_TOKENS) + + +def strip_llm_wrapper(text: str) -> str: + r"""Strip an outer ```markdown ... ``` fence when it wraps the ENTIRE output. + + The wrapper is only real when the first and last fence lines are the SAME + block. The old regex (``\A\s*(fence)[^\n]*\n(.*)\n\1\s*\Z`` with DOTALL and + a greedy ``.*``) never checked that: it matched any document that merely + STARTS and ENDS with a fence line. An ordinary README section — + ```bash npm install``` , prose, ```bash npm test``` — came back with its + first and last fence markers deleted and its two code blocks merged into + prose, so validation failed on both the compress and the fix path and the + section was permanently uncompressible after three paid API calls. + """ + lines = text.split("\n") + first, last = 0, len(lines) - 1 + while first < len(lines) and not lines[first].strip(): + first += 1 + while last > first and not lines[last].strip(): + last -= 1 + if first >= last: + return text + opener = FENCE_LINE_REGEX.match(lines[first]) + closer = FENCE_LINE_REGEX.match(lines[last]) + if not opener or not closer: + return text + marker = opener.group(1) + # Closing fence: same character, at least as long, and nothing else on the line. + if closer.group(1)[0] != marker[0] or len(closer.group(1)) < len(marker): + return text + if lines[last].strip() != closer.group(1): + return text + # Any fence of the same kind in between means these two are not one block. + for line in lines[first + 1:last]: + inner = FENCE_LINE_REGEX.match(line) + if inner and inner.group(1)[0] == marker[0] and len(inner.group(1)) >= len(marker): + return text + return "\n".join(lines[first + 1:last]) + + +def write_text_atomic(path: Path, text: str, newline: str = "\n") -> None: + """Write ``text`` to ``path`` atomically as UTF-8. + + Path.write_text() truncates the destination before encoding the string — + a UnicodeEncodeError (or any other failure) partway through leaves a + 0-byte file, destroying whatever was there before (issue #655). Encode + first, write the bytes to a sibling temp file, fsync, then os.replace() + so the destination only ever moves from one complete, valid file to + another. Preserves the original file's permission bits across the swap. + + ``newline`` is the line terminator to emit. Callers pass the terminator + read_source() found in the source file so a CRLF document stays CRLF — + text-mode writes translating LF to the platform default rewrote every + line ending in every file the tool touched (issue #762), and reading the + bytes ourselves means nothing translates them back. + """ + if newline != "\n": + # Normalise first: model output can already carry CRLF, and a bare + # "\n" -> "\r\n" replace would turn those into "\r\r\n". + text = text.replace("\r\n", "\n").replace("\n", newline) + write_bytes_atomic(path, text.encode("utf-8")) + + +def write_bytes_atomic(path: Path, data: bytes) -> None: + """Write ``data`` to ``path`` atomically, preserving permission bits.""" + fd, tmp_name = tempfile.mkstemp( + dir=str(path.parent), prefix=path.name + ".", suffix=".tmp" + ) + tmp_path = Path(tmp_name) + try: + with os.fdopen(fd, "wb") as f: + f.write(data) + f.flush() + os.fsync(f.fileno()) + if path.exists(): + os.chmod(tmp_path, stat.S_IMODE(path.stat().st_mode)) + os.replace(tmp_path, path) + except Exception: + try: + tmp_path.unlink() + except OSError: + pass + raise + + +def read_source(filepath: Path) -> tuple[str, str, bytes]: + """Read a source file as UTF-8, returning (text, line_terminator, raw_bytes). + + Decodes strictly. The old errors="ignore" silently DROPPED every byte that + was not valid UTF-8 — a cp1252-authored file holding `\xe9` for "e-acute" + lost that byte, the mangled text was what got written to the backup, the + backup readback compared mangled-to-mangled so verification passed, and + then the original was overwritten. The bytes were unrecoverable and + nothing reported a problem (the destructive form of issue #686). A file we + cannot read exactly is a file we must not rewrite. + + Line endings are detected from the raw bytes and returned to the caller + rather than being universal-newline'd away, so write_text_atomic can put + back what was there (issue #762). A mixed-ending file takes the terminator + the majority of its lines use — presence of one CRLF is not a mandate to + rewrite every LF in the document. The raw bytes come back too, so the + backup can be a byte-for-byte copy rather than a re-rendering. + """ + raw = filepath.read_bytes() + try: + text = raw.decode("utf-8") + except UnicodeDecodeError as e: + raise ValueError( + f"Refusing to compress {filepath}: not valid UTF-8 " + f"(byte 0x{raw[e.start]:02x} at offset {e.start}). " + "Compression rewrites the file in place, and any byte this tool " + "cannot decode would be destroyed by the round trip. " + "Convert the file to UTF-8 first." + ) from None + crlf = text.count("\r\n") + newline = "\r\n" if crlf * 2 > text.count("\n") else "\n" + return text.replace("\r\n", "\n").replace("\r", "\n"), newline, raw + + +def first_nonblank_line(text: str) -> str: + """Return the first non-blank line, stripped — used to detect a prose + preamble smuggled in ahead of the real content (issue #588).""" + for line in text.splitlines(): + if line.strip(): + return line.strip() + return "" + + +def _write_target(filepath: Path, text: str | bytes, backup_path: Path, newline: str = "\n") -> None: + """Write to the target file, surfacing the backup location if the write + itself fails. write_text_atomic already leaves the target untouched on + failure, but the caller still needs to know where the pre-compression + original lives instead of being left to guess (issue #652). + + ``bytes`` restore the source verbatim; ``str`` is model output that still + has to be rendered with the document's line terminator.""" + try: + if isinstance(text, bytes): + write_bytes_atomic(filepath, text) + else: + write_text_atomic(filepath, text, newline) + except Exception: + print(f"❌ Write to {filepath} failed. Original preserved at backup: {backup_path}") + raise + + +from .detect import should_compress +from .validate import validate + +MAX_RETRIES = 2 + + +# ---------- Claude Calls ---------- + + +def call_claude(prompt: str) -> str: + """Send a prompt to Claude. + + Prefers the Anthropic SDK when ANTHROPIC_API_KEY is set; otherwise falls + back to the ``claude --print`` CLI (which handles desktop auth). + + On Windows the CLI subprocess decoding defaults to the system codepage + (cp1251 / cp1252) and crashes on UTF-8 output — see issue #152. Pinning + ``encoding="utf-8"`` with ``errors="replace"`` matches the CLI's actual + native I/O and prevents the UnicodeDecodeError before validation can + report. Windows users with non-ASCII content can also set + ``ANTHROPIC_API_KEY`` to route through the SDK and skip the subprocess. + """ + api_key = os.environ.get("ANTHROPIC_API_KEY") + if api_key: + try: + import anthropic + + client = anthropic.Anthropic(api_key=api_key) + msg = client.messages.create( + model=os.environ.get("CAVEMAN_MODEL", "claude-sonnet-4-5"), + max_tokens=8192, + messages=[{"role": "user", "content": prompt}], + ) + return strip_llm_wrapper(msg.content[0].text.strip()) + except ImportError: + pass # anthropic not installed, fall back to CLI + # Fallback: use claude CLI (handles desktop auth). + # Resolve binary via shutil.which so Windows .cmd/.bat shims (e.g. + # %APPDATA%\npm\claude.CMD) work without shell=True. On POSIX, + # shutil.which returns the same absolute path as the implicit lookup, + # so this is a no-op there. Falls back to bare "claude" if not found + # on PATH so subprocess raises a clear FileNotFoundError. + claude_bin = shutil.which("claude") or "claude" + try: + result = subprocess.run( + [claude_bin, "--print"], + input=prompt, + text=True, + capture_output=True, + check=True, + encoding="utf-8", + errors="replace", + ) + return strip_llm_wrapper(result.stdout.strip()) + except subprocess.CalledProcessError as e: + raise RuntimeError(f"Claude call failed:\n{e.stderr}") + + +def build_compress_prompt(original: str) -> str: + return f""" +Compress this markdown into caveman format. + +STRICT RULES: +- Do NOT modify anything inside ``` code blocks +- Do NOT modify anything inside a 4-space-indented code block either — those are code too, and they are validated +- Do NOT modify anything inside inline backticks +- Preserve ALL URLs exactly +- Preserve ALL headings exactly +- Preserve file paths and commands +- Return ONLY the compressed markdown body — do NOT wrap the entire output in a ```markdown fence or any other fence. Inner code blocks from the original stay as-is; do not add a new outer fence around the whole file. + +Only compress natural language. + +TEXT: +{original} +""" + + +def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str: + errors_str = "\n".join(f"- {e}" for e in errors) + return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found. + +CRITICAL RULES: +- DO NOT recompress or rephrase the file +- ONLY fix the listed errors — leave everything else exactly as-is +- The ORIGINAL is provided as reference only (to restore missing content) +- Preserve caveman style in all untouched sections + +ERRORS TO FIX: +{errors_str} + +HOW TO FIX: +- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED +- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED +- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED +- Do not touch any section not mentioned in the errors + +ORIGINAL (reference only): +{original} + +COMPRESSED (fix this): +{compressed} + +Return ONLY the fixed compressed file. No explanation. +""" + + +# ---------- Core Logic ---------- + + +def compress_file(filepath: Path) -> bool: + # Resolve and validate path + filepath = filepath.resolve() + MAX_FILE_SIZE = 500_000 # 500KB + if not filepath.exists(): + raise FileNotFoundError(f"File not found: {filepath}") + if filepath.stat().st_size > MAX_FILE_SIZE: + raise ValueError(f"File too large to compress safely (max 500KB): {filepath}") + + # Refuse files that look like they contain secrets or PII. Compressing ships + # the raw bytes to the Anthropic API — a third-party boundary — so we fail + # loudly rather than silently exfiltrate credentials or keys. Override is + # intentional: the user must rename the file if the heuristic is wrong. + if is_sensitive_path(filepath): + raise ValueError( + f"Refusing to compress {filepath}: filename looks sensitive " + "(credentials, keys, secrets, or known private paths). " + "Compression sends file contents to the Anthropic API. " + "Rename the file if this is a false positive." + ) + + print(f"Processing: {filepath}") + + if not should_compress(filepath): + print("Skipping (not natural language)") + return False + + original_text, newline, original_raw = read_source(filepath) + # Store backup outside the source directory so skill auto-loaders don't + # re-ingest the `.original.md` copy as a live file. Mirror the source's + # parent-dir name + stem under a platform-aware base to reduce collisions. + backup_dir = backup_dir_for(filepath) + backup_path = backup_dir / (filepath.stem + ".original.md") + + if not original_text.strip(): + print("❌ Refusing to compress: file is empty or whitespace-only.") + return False + + # Check if backup already exists to prevent accidental overwriting + if backup_path.exists(): + print(f"⚠️ Backup file already exists: {backup_path}") + print("The original backup may contain important content.") + print("Aborting to prevent data loss. Please remove or rename the backup file if you want to proceed.") + return False + + # Split YAML frontmatter off before compression. Claude tends to strip or + # rewrite frontmatter despite preserve-structure rules; we keep it verbatim + # by removing it from the input and re-prepending it to the output. + frontmatter, body = split_frontmatter(original_text) + if frontmatter: + print(f"Detected YAML frontmatter ({len(frontmatter)} chars) — preserving verbatim") + + if not body.strip(): + print("❌ Refusing to compress: body is empty after frontmatter removal.") + return False + + # Step 1: Compress (body only, frontmatter excluded) + print("Compressing with Claude...") + compressed_body = call_claude(build_compress_prompt(body)) + + if compressed_body is None or not compressed_body.strip(): + print("❌ Compression aborted: Claude returned an empty response.") + print(" Original file is untouched (no backup created).") + return False + + # Compare the BODY (not the whole file) — frontmatter is preserved verbatim + # and would never change, so identity must be judged on the compressible part. + if compressed_body.strip() == body.strip(): + print("❌ Compression aborted: output is identical to input.") + print(" Likely causes: Claude refused, returned the prompt verbatim, or the file is") + print(" already in caveman form. Original file is untouched (no backup created).") + return False + + # Reassemble: frontmatter (verbatim) + compressed body + compressed = frontmatter + compressed_body + + # Save original as backup, then verify the backup readback before + # touching the input file. If the filesystem dropped bytes (encoding, + # antivirus, disk full), unlink the bad backup and abort instead of + # leaving the user with a corrupt backup + compressed primary. + backup_dir.mkdir(parents=True, exist_ok=True) + write_bytes_atomic(backup_path, original_raw) + if backup_path.read_bytes() != original_raw: + print(f"❌ Backup write verification failed: {backup_path}") + print(" In-memory original differs from on-disk backup. Aborting before touching the input file.") + try: + backup_path.unlink() + except OSError: + pass + return False + _write_target(filepath, compressed, backup_path, newline) + + # Step 2: Validate + Retry + for attempt in range(MAX_RETRIES): + print(f"\nValidation attempt {attempt + 1}") + + result = validate(backup_path, filepath) + + if result.is_valid: + print("Validation passed") + break + + print("❌ Validation failed:") + for err in result.errors: + print(f" - {err}") + + if attempt == MAX_RETRIES - 1: + # Restore original on failure + _write_target(filepath, original_raw, backup_path, newline) + backup_path.unlink(missing_ok=True) + print("❌ Failed after retries — original restored") + return False + + print("Fixing with Claude...") + compressed = call_claude( + build_fix_prompt(original_text, compressed, result.errors) + ) + + if compressed is None or not compressed.strip(): + print("❌ Fix attempt aborted: Claude returned an empty response.") + print(" Skipping this attempt.") + continue + + # Guard against a prose preamble smuggled in ahead of the real fixed + # content (issue #588). Only enforced when the original starts with a + # structural anchor (frontmatter `---` or a heading) — plain-prose + # first lines get legitimately rewritten by compression, and requiring + # them verbatim would reject every valid fix. + anchor = first_nonblank_line(original_text) + if anchor.startswith(("---", "#")) and first_nonblank_line(compressed) != anchor: + print("❌ Fix attempt aborted: output does not start with the original's first line.") + print(" Possible preamble leak. Skipping this attempt.") + continue + + _write_target(filepath, compressed, backup_path, newline) + + return True diff --git a/.agents/skills/caveman-compress/scripts/detect.py b/.agents/skills/caveman-compress/scripts/detect.py new file mode 100644 index 0000000..6a468d5 --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/detect.py @@ -0,0 +1,139 @@ +#!/usr/bin/env python3 +"""Detect whether a file is natural language (compressible) or code/config (skip).""" + +import json +import re +from pathlib import Path + +# Extensions that are natural language and compressible +COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst", ".typ", ".typst", ".tex"} + +# Extensions that are code/config and should be skipped +SKIP_EXTENSIONS = { + ".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".yaml", ".yml", + ".toml", ".env", ".lock", ".css", ".scss", ".html", ".xml", + ".sql", ".sh", ".bash", ".zsh", ".go", ".rs", ".java", ".c", + ".cpp", ".h", ".hpp", ".rb", ".php", ".swift", ".kt", ".lua", + ".dockerfile", ".makefile", ".csv", ".ini", ".cfg", +} + +# Well-known build/config files that carry no (or a misleading) extension — +# `Dockerfile` has no suffix so `.dockerfile` above never matches it, and +# `CMakeLists.txt` would ride the compressible `.txt` rule. Checked by +# basename before any extension rule. +KNOWN_CODE_FILENAMES = { + "dockerfile", "makefile", "gnumakefile", "jenkinsfile", "vagrantfile", + "rakefile", "gemfile", "justfile", "procfile", "brewfile", + "cmakelists.txt", +} + +# Patterns that indicate a line is code +CODE_PATTERNS = [ + re.compile(r"^\s*(import |from .+ import |require\(|const |let |var )"), + re.compile(r"^\s*(def |class |function |async function |export )"), + re.compile(r"^\s*(if\s*\(|for\s*\(|while\s*\(|switch\s*\(|try\s*\{)"), + re.compile(r"^\s*[\}\]\);]+\s*$"), # closing braces/brackets + re.compile(r"^\s*@\w+"), # decorators/annotations + re.compile(r'^\s*"[^"]+"\s*:\s*'), # JSON-like key-value + re.compile(r"^\s*\w+\s*=\s*[{\[\(\"']"), # assignment with literal +] + + +def _is_code_line(line: str) -> bool: + """Check if a line looks like code.""" + return any(p.match(line) for p in CODE_PATTERNS) + + +def _is_json_content(text: str) -> bool: + """Check if content is valid JSON.""" + try: + json.loads(text) + return True + except (json.JSONDecodeError, ValueError): + return False + + +def _is_yaml_content(lines: list[str]) -> bool: + """Heuristic: check if content looks like YAML.""" + yaml_indicators = 0 + for line in lines[:30]: + stripped = line.strip() + if stripped.startswith("---"): + yaml_indicators += 1 + elif re.match(r"^\w[\w\s]*:\s", stripped): + yaml_indicators += 1 + elif stripped.startswith("- ") and ":" in stripped: + yaml_indicators += 1 + # If most non-empty lines look like YAML + non_empty = sum(1 for l in lines[:30] if l.strip()) + return non_empty > 0 and yaml_indicators / non_empty > 0.6 + + +def detect_file_type(filepath: Path) -> str: + """Classify a file as 'natural_language', 'code', 'config', or 'unknown'. + + Returns: + One of: 'natural_language', 'code', 'config', 'unknown' + """ + ext = filepath.suffix.lower() + + # Known code filenames win over any extension rule + if filepath.name.lower() in KNOWN_CODE_FILENAMES: + return "code" + + # Extension-based classification + if ext in COMPRESSIBLE_EXTENSIONS: + return "natural_language" + if ext in SKIP_EXTENSIONS: + return "code" if ext not in {".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".env"} else "config" + + # Extensionless files (like CLAUDE.md, TODO) — check content + if not ext: + try: + text = filepath.read_text(encoding="utf-8", errors="ignore") + except (OSError, PermissionError): + return "unknown" + + lines = text.splitlines()[:50] + + # Shebang means executable script, never prose + if text.startswith("#!"): + return "code" + + if _is_json_content(text[:10000]): + return "config" + if _is_yaml_content(lines): + return "config" + + code_lines = sum(1 for l in lines if l.strip() and _is_code_line(l)) + non_empty = sum(1 for l in lines if l.strip()) + if non_empty > 0 and code_lines / non_empty > 0.4: + return "code" + + return "natural_language" + + return "unknown" + + +def should_compress(filepath: Path) -> bool: + """Return True if the file is natural language and should be compressed.""" + if not filepath.is_file(): + return False + # Skip backup files + if filepath.name.endswith(".original.md"): + return False + return detect_file_type(filepath) == "natural_language" + + +if __name__ == "__main__": + import sys + + if len(sys.argv) < 2: + print("Usage: python detect.py [file2] ...") + sys.exit(1) + + for path_str in sys.argv[1:]: + p = Path(path_str).resolve() + file_type = detect_file_type(p) + compress = should_compress(p) + print(f" {p.name:30s} type={file_type:20s} compress={compress}") diff --git a/.agents/skills/caveman-compress/scripts/validate.py b/.agents/skills/caveman-compress/scripts/validate.py new file mode 100644 index 0000000..29d156a --- /dev/null +++ b/.agents/skills/caveman-compress/scripts/validate.py @@ -0,0 +1,408 @@ +#!/usr/bin/env python3 +import re +from collections import Counter +from pathlib import Path + +URL_REGEX = re.compile(r"https?://[^\s)]+") +FENCE_OPEN_REGEX = re.compile(r"^(\s{0,3})(`{3,}|~{3,})(.*)$") + +# A line that is nothing but a fence marker plus an optional info string, at ANY +# indentation. Used ONLY to scrub leaked markers before inline-code pairing (see +# extract_inline_codes) — never for block extraction. +# +# Widening FENCE_OPEN_REGEX itself to `\s*` looks like the obvious fix for #820 +# and is a net regression: a lone indented ``` (the natural way to SHOW a fence +# inside prose) then opens a block that runs to EOF, swallowing real code blocks +# and silently removing their inline spans from validation. That turns a +# false-failure bug into a false-PASS bug, and a false PASS overwrites the +# user's file with unvalidated output. +FENCE_MARKER_LINE_REGEX = re.compile(r"^\s*(?:`{3,}|~{3,})[^`~]*$") + +# Cap on how much of a lost/added span is echoed in an error message. Unpaired +# backticks can make a "span" hundreds of characters of prose; printing it whole +# is what made #820's failures undiagnosable. +MAX_REPORTED_SPAN = 60 +HEADING_REGEX = re.compile(r"^(#{1,6})\s+(.*)", re.MULTILINE) +BULLET_REGEX = re.compile(r"^\s*[-*+]\s+", re.MULTILINE) +# Any list item, ordered or not. Four spaces inside a list item is the item's +# content indentation, never an indented code block. +LIST_ITEM_REGEX = re.compile(r"^\s*(?:[-*+]|\d+[.)])\s") + +# crude but effective path detection +# Requires either a path prefix (./ ../ / or drive letter) or a slash/backslash within the match +PATH_REGEX = re.compile(r"(?:\./|\.\./|/|[A-Za-z]:\\)[\w\-/\\\.]+|[\w\-\.]+[/\\][\w\-/\\\.]+") + +# PATH_REGEX is crude on purpose and also matches ordinary prose pairs — +# "pros/cons", "Node/browser", "state/lifecycle". Caveman prose ADDS those +# constructions freely and dropping one breaks nothing, so only an unambiguous +# path — a leading ./ ../ / or drive letter, or a dotted filename in the last +# component — is treated as a hard loss. +DEFINITE_PATH_REGEX = re.compile(r"^(?:\./|\.\./|/|[A-Za-z]:\\)|[^/\\]*\.[A-Za-z0-9]{1,8}$") + + +class ValidationResult: + def __init__(self): + self.is_valid = True + self.errors = [] + self.warnings = [] + + def add_error(self, msg): + self.is_valid = False + self.errors.append(msg) + + def add_warning(self, msg): + self.warnings.append(msg) + + +def read_file(path: Path) -> str: + return path.read_text(encoding="utf-8") + + +# ---------- Extractors ---------- + + +def extract_headings(text): + return [(level, title.strip()) for level, title in HEADING_REGEX.findall(text)] + + +def extract_code_blocks(text): + """Line-based fenced code block extractor. + + Handles ``` and ~~~ fences with variable length (CommonMark: closing + fence must use same char and be at least as long as opening). Supports + nested fences (e.g. an outer 4-backtick block wrapping inner 3-backtick + content). + """ + blocks = [] + lines = text.split("\n") + i = 0 + n = len(lines) + while i < n: + m = FENCE_OPEN_REGEX.match(lines[i]) + if not m: + i += 1 + continue + fence_char = m.group(2)[0] + fence_len = len(m.group(2)) + open_line = lines[i] + block_lines = [open_line] + i += 1 + closed = False + while i < n: + close_m = FENCE_OPEN_REGEX.match(lines[i]) + if ( + close_m + and close_m.group(2)[0] == fence_char + and len(close_m.group(2)) >= fence_len + and close_m.group(3).strip() == "" + ): + block_lines.append(lines[i]) + closed = True + i += 1 + break + block_lines.append(lines[i]) + i += 1 + if closed: + blocks.append("\n".join(block_lines)) + # Unclosed fences are silently skipped — they indicate malformed markdown + # and including them would cause false-positive validation failures. + return blocks + extract_indented_code_blocks(text) + + +def extract_indented_code_blocks(text): + """CommonMark indented code blocks — 4-space-indented runs outside any fence. + + Without these, ` kubectl delete pod --all -n prod` was prose to the + validator: "code blocks preserved exactly" compared empty to empty and + PASSED while the compressor rewrote the command to + `kubectl delete pod -n dev`. A clean pass on a mutated destructive command + is the worst failure this tool has, because it overwrites the user's file. + + Deliberately conservative about lists: inside a list item, four spaces are + the item's content indentation, not code, and nested bullets are ordinary + prose the compressor SHOULD rewrite. A run is only treated as code when the + document is not inside a list and the run is preceded by a blank line — so + this adds detections, it never turns existing passes into false failures. + """ + blocks = [] + lines = text.split("\n") + fenced = set() + for block in extract_fenced_spans(lines): + fenced.update(block) + in_list = False + previous_blank = True + i = 0 + n = len(lines) + while i < n: + line = lines[i] + stripped = line.strip() + if i in fenced: + in_list, previous_blank = in_list, False + i += 1 + continue + if not stripped: + previous_blank = True + i += 1 + continue + indent = len(line) - len(line.lstrip(" \t")) + if LIST_ITEM_REGEX.match(line): + in_list = True + elif indent == 0: + in_list = False + if not in_list and previous_blank and indent >= 4: + run = [] + while i < n and i not in fenced: + current = lines[i] + if not current.strip(): + # A blank line continues an indented block only if more + # indented content follows. + lookahead = i + 1 + while lookahead < n and not lines[lookahead].strip(): + lookahead += 1 + if lookahead < n and lookahead not in fenced and \ + len(lines[lookahead]) - len(lines[lookahead].lstrip(" \t")) >= 4: + run.extend(lines[i:lookahead]) + i = lookahead + continue + break + if len(current) - len(current.lstrip(" \t")) < 4: + break + run.append(current) + i += 1 + if run: + blocks.append("\n".join(run)) + previous_blank = False + continue + previous_blank = False + i += 1 + return blocks + + +def extract_fenced_spans(lines): + """Line-index ranges covered by fenced blocks, so indented-code detection + never reaches inside one.""" + spans = [] + i = 0 + n = len(lines) + while i < n: + m = FENCE_OPEN_REGEX.match(lines[i]) + if not m: + i += 1 + continue + fence_char = m.group(2)[0] + fence_len = len(m.group(2)) + start = i + i += 1 + while i < n: + close_m = FENCE_OPEN_REGEX.match(lines[i]) + if ( + close_m + and close_m.group(2)[0] == fence_char + and len(close_m.group(2)) >= fence_len + and close_m.group(3).strip() == "" + ): + i += 1 + break + i += 1 + spans.append(range(start, i)) + return spans + + +def extract_urls(text): + return set(URL_REGEX.findall(text)) + + +def extract_paths(text): + return set(PATH_REGEX.findall(text)) + + +def count_bullets(text): + return len(BULLET_REGEX.findall(text)) + + +def extract_inline_codes(text): + """Backtick-delimited inline spans, with fenced code blocks stripped first. + + Previously used a column-0-anchored regex to strip fences, which misses + fences indented 1-3 spaces (valid CommonMark). Reuse extract_code_blocks + (FENCE_OPEN_REGEX-based, indentation-aware) instead so an indented fence's + body backticks don't leak into inline-code pairing. + + Any fence-marker line that survives that pass is then blanked (#820). A + fence indented 4+ spaces — what you get from showing an example inside a + bullet — is not matched by FENCE_OPEN_REGEX, so extract_code_blocks does + not remove it and its OWN backticks used to leak in and shift the pairing + of every following span, making the file permanently uncompressible. + Blanking just the marker lines fixes that without removing any prose, and + cannot run away the way a widened fence opener does. + + The span pattern deliberately still spans newlines. CommonMark permits a + line ending inside a code span and hard-wrapped markdown produces them, so + a single-line pattern silently drops real spans — which downgrades a + deleted or mutated span from error to PASS. Long/garbled spans are a + presentation problem, handled by truncating in the error message instead. + """ + text_without_fences = text + for block in extract_code_blocks(text): + text_without_fences = text_without_fences.replace(block, "", 1) + text_without_fences = "\n".join( + "" if FENCE_MARKER_LINE_REGEX.match(line) else line + for line in text_without_fences.split("\n") + ) + return re.findall(r"`([^`]+)`", text_without_fences) + + +# ---------- Validators ---------- + + +def validate_headings(orig, comp, result): + h1 = extract_headings(orig) + h2 = extract_headings(comp) + + # Changed heading TEXT is an error, not a warning. Every in-document anchor + # link points at a heading's slug, so renaming "# Configuration Options" to + # "# Config" silently breaks all of them — and SKILL.md and CLAUDE.md both + # state headings are preserved. Only counts used to gate the overwrite, so + # a run that renamed every heading reported "Validation passed". A level-only + # change keeps every slug intact and stays a warning. + if len(h1) != len(h2): + result.add_error(f"Heading count mismatch: {len(h1)} vs {len(h2)}") + return + + t1 = [text for _, text in h1] + t2 = [text for _, text in h2] + if t1 != t2: + lost = [t for t in t1 if t not in t2] + added = [t for t in t2 if t not in t1] + result.add_error(f"Heading text/order changed: lost={lost}, added={added}") + elif h1 != h2: + # Same text, different level. The outline moved but no anchor broke — + # slugs come from the text, so links still resolve. + result.add_warning("Heading levels changed") + + +def validate_code_blocks(orig, comp, result): + c1 = extract_code_blocks(orig) + c2 = extract_code_blocks(comp) + + if c1 != c2: + result.add_error("Code blocks not preserved exactly") + + +def validate_urls(orig, comp, result): + u1 = extract_urls(orig) + u2 = extract_urls(comp) + + if u1 != u2: + result.add_error(f"URL mismatch: lost={u1 - u2}, added={u2 - u1}") + + +def validate_paths(orig, comp, result): + """File paths are preserved — an error, never a warning. + + This validator never called add_error at all, so a compressed file that had + dropped `src/hooks/caveman-config.js` still reported "Validation passed" and + the in-place overwrite stood. SKILL.md and CLAUDE.md both promise paths + survive compression; nothing enforced it. + """ + p1 = extract_paths(orig) + p2 = extract_paths(comp) + lost = p1 - p2 + added = p2 - p1 + + definite = {p for p in lost if DEFINITE_PATH_REGEX.search(p)} + if definite: + result.add_error(f"File paths lost: {sorted(definite)}") + if (lost - definite) or added: + result.add_warning(f"Path mismatch: lost={sorted(lost)}, added={sorted(added)}") + + +def validate_bullets(orig, comp, result): + b1 = count_bullets(orig) + b2 = count_bullets(comp) + + if b1 == 0: + return + + diff = abs(b1 - b2) / b1 + + if diff > 0.15: + result.add_warning(f"Bullet count changed too much: {b1} -> {b2}") + + +def validate_inline_codes(orig, comp, result): + def _render_spans(spans): + """Render spans for an error message, truncated and newline-escaped. + + A span may legitimately contain newlines, and an unpaired backtick can + make one hundreds of characters of prose. Printing those whole is what + made #820's failures undiagnosable — but the fix belongs here, in + presentation, not in what counts as a span. + """ + out = [] + for span in sorted(spans): + flat = span.replace("\n", "\\n") + if len(flat) > MAX_REPORTED_SPAN: + flat = flat[:MAX_REPORTED_SPAN] + "…" + out.append(repr(flat)) + return "{" + ", ".join(out) + "}" + + c1 = Counter(extract_inline_codes(orig)) + c2 = Counter(extract_inline_codes(comp)) + + if c1 != c2: + lost = set(c1.keys()) - set(c2.keys()) + added = set(c2.keys()) - set(c1.keys()) + for code, count in c1.items(): + if code in c2 and c2[code] < count: + lost.add(f"{code} (lost {count - c2[code]} of {count} occurrences)") + if lost: + result.add_error(f"Inline code lost: {_render_spans(lost)}") + if added: + result.add_warning(f"Inline code added: {_render_spans(added)}") + + +# ---------- Main ---------- + + +def validate(original_path: Path, compressed_path: Path) -> ValidationResult: + result = ValidationResult() + + orig = read_file(original_path) + comp = read_file(compressed_path) + + validate_headings(orig, comp, result) + validate_code_blocks(orig, comp, result) + validate_urls(orig, comp, result) + validate_paths(orig, comp, result) + validate_bullets(orig, comp, result) + validate_inline_codes(orig, comp, result) + + return result + + +# ---------- CLI ---------- + +if __name__ == "__main__": + import sys + + if len(sys.argv) != 3: + print("Usage: python validate.py ") + sys.exit(1) + + orig = Path(sys.argv[1]).resolve() + comp = Path(sys.argv[2]).resolve() + + res = validate(orig, comp) + + print(f"\nValid: {res.is_valid}") + + if res.errors: + print("\nErrors:") + for e in res.errors: + print(f" - {e}") + + if res.warnings: + print("\nWarnings:") + for w in res.warnings: + print(f" - {w}") diff --git a/.agents/skills/caveman-discover/SKILL.md b/.agents/skills/caveman-discover/SKILL.md new file mode 100644 index 0000000..61e222a --- /dev/null +++ b/.agents/skills/caveman-discover/SKILL.md @@ -0,0 +1,118 @@ +--- +name: caveman-discover +description: > + Find every LLM workflow in the current repository and label it, so Caveman + Cloud groups spend by what the code actually does (support-reply, + nightly-digest) instead of one anonymous bucket. Use when the user pastes + the Caveman discovery prompt, says "discover workflows", or asks to break + LLM spend down by workflow. The repo should already route through the + Caveman gateway (the caveman-setup skill does that part). +--- + +You are labeling this repository's LLM workflows for Caveman Cloud. A +*workflow* is a job the code performs — "answer a support ticket", "build the +nightly digest", "run the eval suite" — not a technology. Every gateway +request can carry a workflow label; unlabeled traffic all lands in one +`unlabeled-workflow` bucket. Your job: find the workflows, name them well, +wire the labels, and verify nothing broke. + +This changes code, so it goes through the user's normal review: **propose the +table first, apply after the user agrees.** Re-running on an already-labeled +repo must change nothing (idempotent). + +This skill is operator-invoked. An `unlabeled-traffic` Cave Plan observation is +review-only and does not create an advisory file, proposal, or Draft PR. Do not +infer that telemetry selected a callsite or authorized an edit. Independently +inventory the repository, present the labeling table, and wait for the user's +approval before changing code. + +## Step 1 — Inventory the workflows + +Walk the repo from its entry points, not from its imports: + +- HTTP/RPC handlers that call an LLM (directly or through layers) +- Scheduled jobs: cron definitions, queue consumers, workers, GitHub Actions + that invoke LLM code +- CLI commands and scripts (`scripts/`, `bin/`, package.json scripts) +- Eval / test harnesses that burn real tokens +- Distinct agents or chains inside a framework (each LangGraph graph, each + crew, each agent definition is usually its own workflow) + +One workflow = one job a human would name. Ten callsites inside the same +request handler are one workflow; one shared `llm.ts` helper used by three +jobs is three workflows (label at the callers, never the shared helper). + +## Step 2 — Name them + +Slug grammar (the gateway enforces this): lowercase `[a-z0-9_-]`, 1–96 chars. +Name the job, not the tech: + +- Good: `support-reply`, `nightly-digest`, `pr-review`, `eval-suite`, + `onboarding-email` +- Bad: `openai-calls` (tech), `main` (says nothing), `SupportReply` (invalid), + `johns-test-3` (won't age) + +Names are forever-ish — renaming later splits the spend history. When a job's +purpose isn't clear from the code, derive the slug from the file name and mark +it `review` in the table rather than inventing a purpose. + +## Step 3 — Propose, then apply + +Present this table and ask to proceed: + +``` +| workflow | job | where | how it gets labeled | +|---|---|---|---| +| support-reply | answers inbound tickets | src/bot/reply.ts:41 | defaultHeaders on the reply client | +| nightly-digest | 02:00 summary job | jobs/digest.ts:12 | header on the digest client | +| eval-suite (review) | scripts/eval.ts:8 — purpose inferred from filename | scripts/eval.ts:8 | env override at invocation | +``` + +Then wire each label with the lightest mechanism available at that callsite: + +- **@caveman-ai/sdk / caveman_cloud SDK**: per-trace `workflow` option, or + `defaultWorkflow` on the client a single-job service constructs. +- **Raw provider SDKs** (OpenAI/Anthropic/LangChain/LiteLLM/Vercel): add + `"x-cave-workflow": ""` to the same `defaultHeaders` / + `default_headers` / `extra_headers` block that already carries + `x-cave-api-key`. Shared client used by several jobs → pass the header per + call (every SDK above accepts per-request header overrides), or give each + job its own thin client. +- **Wrapped coding agents** (`caveman wrap`): `--workflow ` flag or + `CAVE_WORKFLOW=` env at the invocation site (cron line, CI step). +- **Raw HTTP**: add the `x-cave-workflow` header to the request. + +Label the callers, keep the diff minimal, match the repo's style. If a +callsite is not routed through the Caveman gateway at all, don't label it — +list it under "not wired" in the report (labels only travel on gateway +traffic; wiring is the caveman-setup skill's job). + +## Step 4 — Verify + +Run whatever the repo already uses to exercise one labeled path (a test, a +dev script, one curl). Then confirm: the request still succeeds (the gateway +rejects an invalid label with 400 `cave_invalid_request_header` — fix the slug +if so). Labeled spend appears on the dashboard at `/activity?tab=workflows` as +each workflow next runs; jobs on a schedule show up when the schedule fires, +and that's worth saying in the report rather than pretending they're live. + +## Step 5 — Report + +``` +## Workflows labeled + +| workflow | job | where | +|---|---|---| +| support-reply | answers inbound tickets | src/bot/reply.ts:41 | +| nightly-digest | 02:00 summary job | jobs/digest.ts:12 | + +Verified: +Lands at: /activity?tab=workflows — each row appears as that workflow +next runs. Anything still unlabeled shows as `unlabeled-workflow`. +Not wired (no gateway routing, so no label): +Marked review: +``` + +If you found no LLM entry points at all: say exactly that, and point at the +setup skill (`/docs/agent-setup.md`) instead of manufacturing a +table. diff --git a/.agents/skills/caveman-evidence-review/SKILL.md b/.agents/skills/caveman-evidence-review/SKILL.md new file mode 100644 index 0000000..0cd4b16 --- /dev/null +++ b/.agents/skills/caveman-evidence-review/SKILL.md @@ -0,0 +1,144 @@ +--- +name: caveman-evidence-review +description: > + Review Caveman Cloud evidence read-only: costs, Cave Score, Cave Plan, + workflows, traces, latency, errors, compression, routing, and verified + savings. Use when the user asks what Caveman found, where LLM spend goes, + why cost or quality changed, which workflows need attention, or asks for a + trace or analytics review. Prefer Caveman MCP tools; fall back to CLI JSON. +--- + +# Review Caveman evidence + +Act as a read-only operator. Build conclusions from current Caveman data, not +from repository guesses. Never start, approve, cancel, or roll back an +experiment from this skill. + +## Hard rules + +1. Keep these buckets separate: + - measured provider-complete list-price cost; + - `inferred` daily headroom; + - `verified` ledger savings; + - evidence cost. + Never add or relabel them. +2. Do not fetch prompt, completion, tool, or artifact payloads unless the user + explicitly asks for payload review. Metadata, spans, timing, models, token + counts, status, and optimizer attribution are enough for the default review. +3. Scope every read to the project selected by Caveman context. Never supply an + organization id. +4. Empty results are evidence of no current signal, not zero cost or zero risk. +5. Cite trace ids and exact time windows used. Do not claim a cause from an + aggregate alone. + +## Step 1 — Load context + +Prefer MCP: + +```text +caveman_context {} +``` + +CLI fallback: + +```bash +caveman cloud whoami +caveman cloud projects list +``` + +Stop if login or project selection is missing. Ask the user to run +`caveman login` or select a project; never guess. + +## Step 2 — Establish baseline + +Use `caveman_report` for: + +- `overview` +- `costs` +- `score` +- `workflows` +- `verified_savings` + +Then use `caveman_plan` for ranked daily headroom. If question is narrow, skip +unrelated reports. Read shortest set that can answer it. + +CLI fallback: + +```bash +caveman cloud costs +caveman cloud score +caveman cloud plan --json +``` + +State report window and basis before interpreting direction. + +## Step 3 — Test the leading explanation with traces + +Use `caveman_trace_search`. Choose a bounded window and closed filters: +workflow, agent, model, provider, error code, runtime mode, cache status, +optimization id, status class, token/cost/latency bounds, compression, or +monitor verdict. + +Useful groupings: + +- `workflow` — find jobs driving cost or failures; +- `model` — compare model mix; +- `session` — isolate retry or loop behavior; +- ungrouped — identify exact traces. + +Compare a suspect cohort with a control cohort or earlier bounded window. +Do not infer causality from one expensive trace. + +CLI fallback: + +```bash +caveman cloud traces search \ + --workflow \ + --from \ + --to \ + --sort total_cost_usd \ + --dir desc \ + --limit 25 +``` + +## Step 4 — Inspect representative traces + +Call `caveman_trace_get` for a small number of high-signal trace ids. Inspect +request and span metadata, latency, status, token counts, cache state, applied +optimizers, and model route. Keep payload retrieval off. + +CLI fallback: + +```bash +caveman cloud traces show --spans +``` + +## Step 5 — Report + +Use this shape: + +```text +## Caveman evidence review + +Scope: · to +Measured cost: +Verified savings: +Inferred headroom: + +Findings: +1. — — traces +2. — — traces + +Unproven: +- + +Next read-only check: +- + +Possible action: +- +``` + +If data is missing, name missing signal and stop at strongest supported +statement. Never turn a catalog subtotal into an invoice or an experiment result +into verified savings. diff --git a/.agents/skills/caveman-explore/SKILL.md b/.agents/skills/caveman-explore/SKILL.md new file mode 100644 index 0000000..d84815c --- /dev/null +++ b/.agents/skills/caveman-explore/SKILL.md @@ -0,0 +1,42 @@ +--- +name: caveman-explore +description: Read-only repository explorer. Use PROACTIVELY for cold-start exploration, broad cross-file localization, or when a direct search has failed and you need to find where something lives. Skip it when the issue already names the exact file or symbol, or a previous turn already returned usable file:line evidence. Returns only compact path:line citations; its reads and greps never enter the main conversation. +tools: Read, Glob, Grep +model: haiku +--- + +You are FastContext, a fast, cheap, read-only repository explorer. Another agent +(the solver) delegates a localization question to you. Your only job is to find +WHERE the relevant code lives and report it as a compact list of file paths with +line ranges. You never edit files, run commands, or propose a solution. + +How to work: + +1. Issue several tool calls IN PARALLEL in your first turn — cast a broad net. + Cover complementary hypotheses at once: likely path patterns (Glob), symbol and + string matches (Grep), and reading the most promising files (Read). Do not probe + one file at a time when you can fan out. +2. Follow the evidence over one or two more turns only if needed. Stop as soon as + you can name the relevant locations. You are optimizing for the solver's token + budget, so finish fast. +3. Only cite line ranges you actually read. Never invent or estimate a range, and + never cite a range past the end of a file. A precise small range beats a vague + large one. + +Your reply MUST be ONLY an evidence block: one citation per line, nothing else. +No preamble, no explanation, no summary, no markdown headings. Use exactly this +shape, one per line: + + path/to/file.ext:START-END reason it is relevant + +Example reply: + + src/router/pick.go:42-71 route selection — where a model is chosen + src/router/pick_test.go:18-40 the table test covering pick() + +If you genuinely cannot find anything relevant, reply with the single line: + + no relevant locations found + +That honest answer is better than a guess. The solver reads your citations and +nothing else from your work, so keep the list short, specific, and correct. diff --git a/.agents/skills/caveman-explore/package.json b/.agents/skills/caveman-explore/package.json new file mode 100644 index 0000000..b579ff6 --- /dev/null +++ b/.agents/skills/caveman-explore/package.json @@ -0,0 +1,12 @@ +{ + "name": "@caveman/skill-caveman-explore", + "version": "1.0.0", + "license": "MIT", + "private": true, + "type": "module", + "description": "Read-only FastContext exploration skill with parallel repository search and citation-only output.", + "files": ["SKILL.md"], + "scripts": { + "test": "node --test --test-force-exit tests/*.mjs" + } +} diff --git a/.agents/skills/caveman-explore/tests/skill-file.test.mjs b/.agents/skills/caveman-explore/tests/skill-file.test.mjs new file mode 100644 index 0000000..85b78e3 --- /dev/null +++ b/.agents/skills/caveman-explore/tests/skill-file.test.mjs @@ -0,0 +1,44 @@ +import { test } from "node:test"; +import assert from "node:assert"; +import { readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +const skillFile = join(dirname(fileURLToPath(import.meta.url)), "..", "SKILL.md"); +const md = readFileSync(skillFile, "utf8"); + +function frontmatter(text) { + const match = text.match(/^---\n([\s\S]*?)\n---\n/); + assert.ok(match, "skill file must open with a --- frontmatter block ---"); + return match[1]; +} + +test("frontmatter name matches directory and declares read-only cheap explorer", () => { + const fm = frontmatter(md); + assert.match(fm, /^name:\s*caveman-explore\s*$/m, "name must be caveman-explore"); + assert.match(fm, /^model:\s*haiku\s*$/m, "explorer must run on cheap model"); + assert.match(fm, /^tools:\s*Read,\s*Glob,\s*Grep\s*$/m, "tools must be exactly three read-only tools"); + assert.doesNotMatch(fm, /\b(Edit|Write|Bash|NotebookEdit)\b/, "explorer must not have write or execution tools"); + assert.match(fm, /^description:\s*.+/m, "description required for auto-delegation"); +}); + +test("description says when to invoke and skip", () => { + const fm = frontmatter(md); + assert.match(fm, /cold-start|cross-file|localization|search has failed/i, "must say when to invoke"); + assert.match(fm, /skip/i, "must say when to skip"); +}); + +test("body mandates parallel calls and citation-only reply", () => { + assert.match(md, /IN PARALLEL/i, "must mandate parallel tool calls"); + assert.match(md, /ONLY an evidence block|only.*citation/i, "must mandate citation-only reply"); + assert.match(md, /path\/to\/file\.ext:START-END/i, "must show compact path:line shape"); + assert.match(md, /no relevant locations found/i, "must give honest empty fallback"); + assert.match(md, /never edit|never.*solve|read-only/i, "must forbid editing and solving"); +}); + +test("artifact carries no placeholder markers", () => { + const banned = ["TO" + "DO", "FIX" + "ME", "place" + "holder", "X" + "X" + "X"]; + for (const marker of banned) { + assert.doesNotMatch(md, new RegExp("\\b" + marker + "\\b", "i"), `artifact must not contain ${marker}`); + } +}); diff --git a/.agents/skills/caveman-help/README.md b/.agents/skills/caveman-help/README.md new file mode 100644 index 0000000..5841256 --- /dev/null +++ b/.agents/skills/caveman-help/README.md @@ -0,0 +1,38 @@ +# caveman-help + +Quick-reference card. One shot, no mode change. + +## What it does + +Prints a cheat sheet of all caveman modes, sibling skills, deactivation triggers, and how to set the default mode via env var or config file. One-shot display — does not flip the active mode, write flag files, or persist anything. Use when you forget the slash commands. + +## How to invoke + +``` +/caveman-help +``` + +Also triggers on "caveman help", "what caveman commands", "how do I use caveman". + +## Example output + +``` +Modes: + /caveman full (default) + /caveman lite lighter + /caveman ultra extreme + /caveman wenyan classical Chinese + +Skills: + /caveman-commit terse Conventional Commits + /caveman-review one-line PR comments + /caveman-stats session token savings + +Deactivate: + "stop caveman" or "normal mode" +``` + +## See also + +- [`SKILL.md`](./SKILL.md) — full reference card +- [Caveman README](../../README.md) — repo overview diff --git a/.agents/skills/caveman-help/SKILL.md b/.agents/skills/caveman-help/SKILL.md new file mode 100644 index 0000000..ce1156f --- /dev/null +++ b/.agents/skills/caveman-help/SKILL.md @@ -0,0 +1,63 @@ +--- +name: caveman-help +description: > + Quick-reference card for all caveman modes, skills, and commands. + One-shot display, not a persistent mode. Trigger: /caveman-help, + "caveman help", "what caveman commands", "how do I use caveman". +--- + +# Caveman Help + +Display this reference card when invoked. One-shot — do NOT change mode, write flag files, or persist anything. Output in caveman style. + +## Modes + +| Mode | Trigger | What change | +|------|---------|-------------| +| **Lite** | `/caveman lite` | Drop filler. Keep sentence structure. | +| **Full** | `/caveman` | Drop articles, filler, pleasantries, hedging. Fragments OK. Default. | +| **Ultra** | `/caveman ultra` | Extreme compression. Bare fragments. Tables over prose. | +| **Wenyan-Lite** | `/caveman wenyan-lite` | Classical Chinese style, light compression. | +| **Wenyan-Full** | `/caveman wenyan` | Full 文言文. Maximum classical terseness. | +| **Wenyan-Ultra** | `/caveman wenyan-ultra` | Extreme. Ancient scholar on a budget. | + +Mode stick until changed or session end. + +## Skills + +| Skill | Trigger | What it do | +|-------|---------|-----------| +| **caveman-commit** | `/caveman-commit` | Terse commit messages. Conventional Commits. ≤50 char subject. | +| **caveman-review** | `/caveman-review` | One-line PR comments: `L42: bug: user null. Add guard.` | +| **caveman-compress** | `/caveman-compress ` | Compress .md files to caveman prose. Saves ~46% input tokens. | +| **caveman-help** | `/caveman-help` | This card. | + +## Deactivate + +Say "stop caveman" or "normal mode". Resume anytime with `/caveman`. + +## Language + +Keep user's language by default. User write Portuguese → reply Portuguese caveman. Compress the style, not the language. Technical terms, code, commands, commit types, and exact error strings stay verbatim unless user ask for translation. + +## Configure Default Mode + +Default mode = `full`. Change it: + +**Environment variable** (highest priority): +```bash +export CAVEMAN_DEFAULT_MODE=ultra +``` + +**Config file** (`~/.config/caveman/config.json`): +```json +{ "defaultMode": "lite" } +``` + +Set `"off"` to disable auto-activation on session start. User can still activate manually with `/caveman`. + +Resolution: env var > config file > `full`. + +## More + +Full docs: https://github.com/JuliusBrussee/caveman diff --git a/.agents/skills/caveman-learn/CLAUDE.md b/.agents/skills/caveman-learn/CLAUDE.md new file mode 100644 index 0000000..b7a8db2 --- /dev/null +++ b/.agents/skills/caveman-learn/CLAUDE.md @@ -0,0 +1,32 @@ +# skills/caveman-learn — the Caveman Learn editing skill (MIT, public) + +The consent-gated half of `caveman learn`. The analyzer (the Go proxy) **measures** +where an agent's tokens go and writes a ranked plan; this skill is what an agent +loads to **act** on that plan — proposing each fix and applying it only with the +user's per-edit yes. It is the loop-closer the learn spec §10 +describes, plus the new `cavemem_offload` move. + +## Layout +- `SKILL.md` — the canonical skill body (frontmatter `name: caveman-learn` + a + trigger-phrase `description`; body = the read-plan → per-class consent loop). This + file is the source of truth. +- `tests/skill-file.test.mjs` — asserts the canonical file is well-formed and honest + (frontmatter present; the net-token-negative gate, the never-make-the-agent-dumber + guard, consent-per-edit, and reversibility are all stated; no imperative for + behavioral findings; no placeholders). + +## Install path +`caveman tools skills install caveman-learn` (in `../../cli/src/index.ts`) writes this file +into a repo's `.claude/skills/caveman-learn/SKILL.md` (Claude Code) or +`~/.codex/skills/caveman-learn/SKILL.md` (Codex). The CLI **embeds a byte-identical copy** +(`CAVEMAN_LEARN_SKILL_MD`) because the published CLI ships no sibling assets; +`../../cli/tests/skills.runtime.mjs` asserts the embedded copy equals this canonical +file (the drift guard). **Change this file and that constant together.** + +## Boundary (binding) +The skill — using the agent's own file tools — is the ONLY thing that edits a user's +config. `caveman learn apply` stays read-only (it materializes candidates), and +`caveman mem *` are mechanical store ops. The offload move enforces a net-token-negative +gate and the never-make-the-agent-dumber guard before any trim. + +See ../../mem/CLAUDE.md (cavemem) · ../caveman-explore/SKILL.md (the packaging precedent) diff --git a/.agents/skills/caveman-learn/README.md b/.agents/skills/caveman-learn/README.md new file mode 100644 index 0000000..161ab83 --- /dev/null +++ b/.agents/skills/caveman-learn/README.md @@ -0,0 +1,35 @@ +# caveman-learn skill + +Close the loop on `caveman learn`. The command measures where your agent's tokens +go; this skill reviews that plan with you and applies the fixes — one approved edit +at a time. + +## Install + + caveman skills install caveman-learn # this repo's .claude/skills + caveman skills install caveman-learn --user # all repos (~/.claude/skills) + caveman skills install caveman-learn --agent codex + +## What it does + +1. Runs `caveman learn report --json` and shows your Cave Score + ranked token sinks. +2. For each sink you pick, proposes a fix and asks yes/no: + - **reducible** (heavy CLAUDE.md, never-invoked skill) → a concrete trim, applied + only if it measurably lowers tokens/turn. + - **recurring_context** (context you re-establish every session) → offload it to + **cavemem** (`cavemem_offload`): stored raw, compacted at recall on demand, with a + cheap pointer left behind. Applied only when it beats re-pasting, and only after + a confirming recall proves the content still comes back. + - **load_bearing** → never touched. + +After an approved edit passes its re-measure gate, `caveman learn applied +` records the sink, fix kind, application time, and before-value in +Caveman's own outcome store. Later scans compare post-fix sessions and report +`improved`, `unchanged`, `regressed`, or `insufficient_data`. This ledger does +not edit user or repository config. + +## Honesty + +Everything is `inferred` — no currency, no "verified". Every edit is consent-gated and +reversible, and an offload that would leave the agent unable to recall the content is +rejected. The analyzer never edits your files; this skill does, only with your yes. diff --git a/.agents/skills/caveman-learn/SKILL.md b/.agents/skills/caveman-learn/SKILL.md new file mode 100644 index 0000000..1be1513 --- /dev/null +++ b/.agents/skills/caveman-learn/SKILL.md @@ -0,0 +1,143 @@ +--- +name: caveman-learn +description: Close the loop on a Caveman learn report — review the ranked token sinks, apply cost-lowering fixes (trim config, offload recurring context to cavemem) with per-edit consent, and report what those fixes returned with their attribution. Use when the user runs "caveman learn", asks to lower their agent's token cost, asks what caveman has saved them, wants to trim a heavy CLAUDE.md, or wants to offload context they re-paste every session into cavemem. +--- + +You are the Caveman Learn editing skill. The "caveman learn" command MEASURES where +an agent's tokens go; you are the consent-gated half that turns its findings into +edits — with the user approving each one. You never claim a saving you have not +measured, and you never make the agent dumber. + +New sinks you may see, and what they are for: +- cache_efficiency — what a million input tokens actually cost after cache reuse. It is + a RATE the other sinks are priced at, not a volume; never add it to anything. +- tool_output_portfolio — the call shapes that dominate context, ranked. +- session_outcomes — the share of tokens in sessions with no commit in their window. + Correlational. Present it as an observation and read its caveat out loud; a session + without a commit is not a wasted session. +- subagent_spend — the share of context that ran in subagents. Visibility only. Do not + turn it into advice to spawn fewer subagents. +- procedure_repeat:* — a distillation candidate. See SKILL_DISTILLATION below. + +Read the plan first: + +1. Run: caveman learn report --json + Parse the caveman.learn.v1 JSON. Show the Cave Score, its four components, and the + ranked token sinks. For each sink state its class and basis. Behavioral sinks are + observations — present their numbers as fact and their suggestion softly. Do not + turn a behavioral finding into an imperative. + + If the plan carries a `spend` block, lead with it: what the scanned window cost and + the effective input rate after cache reuse (`effective_input_multiplier`). Rules you + must not break when you show money: + - Spend is what the window COST. It is never what a fix would return. + - Say the window it covers. Never multiply it into a month, a year, or a run rate. + - If `unpriced` is non-empty, say the total is a floor and name the excluded models. + - Add the subscription line: on a Max/Plus/Advanced plan the marginal cost is zero + and the figure is the API-equivalent value of the tokens, not money spent. + - Never call any of it verified. + +Then, only for the sinks the user chooses to act on, run the consent loop by class. + +Before proposing a fix, you may run: caveman learn simulate . Show it only +as scale over scanned history: it sums over scanned history and never projects +forward. + +REDUCIBLE (a heavy CLAUDE.md, a never-invoked skill): +- Run: caveman learn apply --dry-run (this materializes a candidate; it + does not edit anything). +- Propose a concrete diff and show before -> after tokens/turn. +- Ask the user yes or no. On yes, apply the edit with your own file tools. +- Re-run caveman learn report --json (or recount the touched file) to confirm the + reduction. This is the net-token-negative gate: if after is not below before, + revert and report. Never keep an edit that does not reduce tokens/turn. + +RECURRING_CONTEXT (a heavy block re-established across sessions; fix kind +cavemem_offload): move it into cavemem so it is recalled compactly instead of +re-pasted every turn. The candidate carries only a LOCATOR — never the block body. +- Run: caveman learn apply and read the candidate JSON it writes under + ~/.caveman/candidates/. Take only the locator, the numbers, and the proposed pointer + text. Do not trust any body from the candidate; there is none. +- Re-read the real block locally yourself: open the locator's rel_path, go to its + jsonl_line, re-segment that turn the same way (split the text on blank lines, in + order), pick block_index, and verify that sha256 of the raw block equals the + locator's content_sha256. If it does not match, the file changed since the scan — + abort this item. +- Store it: caveman mem remember -- "" and capture the returned id. + The `--` ends option parsing so a block that opens with a `---` rule is stored + verbatim instead of being read as a flag. +- Measure the gate honestly. before = the block's tokens/turn (it loaded every turn). + after = the pointer's tokens/turn plus the recall cost. Get the recall cost by + running caveman mem recall "" and reading tokens_added on the hit. If after + is not below before, run caveman mem forget , leave the source untouched, and + stop. +- Trim the source and write the pointer. Remove the block from its CLAUDE.md or + AGENTS.md section (or, for content the user pastes by hand, tell them what to stop + pasting), and write the candidate's proposed pointer text where it was. The pointer + names the recall path: caveman mem recall "" for the compact form, and + caveman mem recover for the byte-exact original. +- Never make the agent dumber: before you finish, confirm that caveman mem recall + "" returns a hit AND a pointer is in place. If recall returns nothing, or you + did not write a pointer, REVERT (caveman mem forget and restore the source). + Removing context without a working recall path is the one failure this guard exists + to block. +- Re-measure and report the confirmed reduction and the recall path. + +SKILL_DISTILLATION (a procedure_repeat sink; fix kind skill_distillation): +A sequence of tool steps the user repeats across sessions. Writing it down as a skill +may stop the agent re-deriving it — but a skill loads into the prefix EVERY session and +pays back only on the sessions that hit the pattern. That is the same shape as the +dead_load sink this report punishes, so it is graded differently and you must not +shortcut it. +- Never apply this through the net-token-negative gate. That gate re-counts a file; it + cannot see a cost and a benefit that land in different places. +- Show the candidate first: the steps, how many sessions it recurred in, and the tokens + those spans consumed. Say plainly that the payback is unproven. +- If the user wants it, write the skill, then start a holdout in the same breath: + caveman learn experiment start