Triage failed GitHub Actions runs and self-heal flaky tests

A QA engineer's OpenClaw agent reads failed CI logs, classifies failures, posts analysis to GitHub and Slack and opens issues; flakiness fell from 8% to 0.8%, though it once wrongly altered an assertion.

Done withOpenClaw

What they did

The author built an OpenClaw agent over a weekend that watches GitHub Actions for failed runs and reads the logs and test output. It classifies each failure as flaky, a real regression or an environment issue. It then posts its analysis to GitHub and Slack and opens issues. A separate self-healing locator agent tries alternative strategies when selectors break and suggests fixes.

How it went

Time-to-diagnosis fell from 45 to 6 minutes, and flakiness dropped from 8% to 0.8%. The self-healing agent once changed an assertion to match the output instead of fixing the locator. Code review caught it.

Worth knowing

Token costs neared $4,000 a month until the author fed the agent only the relevant 500-1000 tokens, which cut it to $400. A confidence threshold gates auto-merges.

Source: suneetmalhotra.com · Undated