Iteratively improve docs by testing them with coding agents
A CLI runs coding agents on tasks that need your docs, analyzes transcripts for failures, and generates doc variants to fix them, repeating until the problems are resolved. ·
Done withClaude CodeCodex
- What they did
- You save a task with success criteria, and Gauge starts fresh Docker sessions of Claude Code, Codex, Pi or OpenCode. The sessions get your working-tree inputs, such as Skills or files. Your own coding agent reads the transcripts, publishes findings, edits the guidance, and reruns the same eval. Gauge then compares the before and after batches.
- How it went
- The README gives no measured results, only an example release-notes Skill eval. Docs as an input are listed under both local and hosted modes, but the hosted mode adds response rewriting, so local testing of docs changes is less clear.
- Worth knowing
- Needs Node 22.13+, Docker, and agent credentials. Agent usage is billed through your provider or subscription. Hosted mode uses workspace credits.
Try it yourself with Claude Code
Run a docs improvement loop on my project in [path to repo or docs folder]. Come up with 5 realistic tasks that require using my docs, attempt each as a fresh agent that only has the docs, and review the transcripts for where it got stuck or guessed. Then edit the docs to fix those failures and re-run the tasks, stopping when all tasks succeed or after 3 rounds, and give me a summary of each change before you save it.
Want help getting this working? Join the Vibe Coding Academy ↗ Run by Alex Finn, who also runs this site.
Discussion on HN · Nov 4, 2026
Related use cases
Turn tickets into merged pull requests
Pulls tasks from GitHub Issues or Linear, then runs Claude Code or Codex in isolated Kubernetes pods to carry each one through to a PR.
Handle family errands and recipe sorting with Codex
A communications professional had Codex email her son's doctor for a refill and sort 240 recipes down to 62 family-fit ones into an index, spreadsheet and Drive folder.
Have Claude read GitHub code and post drafts
A commenter lets Claude read code from GitHub and post drafts for them, and uses Claude Code in the desktop app for file editing and running commands.
Build a C compiler with a team of parallel agents
Anthropic's engineering write-up on tasking Opus 4.6 agent teams with building a C compiler.
Build and ship a micro-SaaS in three hours
A developer used paid Replit to build and deploy a micro-SaaS to production in about 3 hours, with database, domain, hosting and monitoring handled.
Build small features and integrations with Cursor agent mode
A commenter uses Cursor agent mode for smaller features, third-party integrations, background jobs, notifications and activity tracking, and has it log key findings to a worklog file for the next run.
Get 5 like this every morning
The best things people got an AI agent to do, each with the prompt to try it.