Build and test an image editor with agent-driven headless GUI
A developer had Opus build an image editor and a Sonnet subagent test it via a headless sway setup with scripted mouse and keyboard input, catching most bugs; manual checks were still needed. ·
Done withClaude CodeClaude
- What they did
- The developer hand-wrote scripts so the agent could drive the app headlessly: a headless sway instance, a daemon using Wayland virtual keyboard and mouse protocols via a unix socket, and bash glue. Commands (start, screenshot, mouse move/click, type) were documented in claude.md. Opus built the editor; a Sonnet subagent did interactive testing. The developer specified how each tool and panel should work.
- How it went
- Testing was mostly automatic and Sonnet caught most bugs itself. The developer still did a cursory manual check, since headless sway differs from a real desktop environment with KDE's platform theme.
- Worth knowing
- Scripted input mocking was far more reliable than the accessibility API route (pyatspi2) tried first; a Sonnet subagent also cut cost and avoids inheriting Opus's long context.
Try it yourself with Claude Code
Build a small [type of GUI app, e.g. image editor] in this folder using [language/toolkit]. Then set up a headless display environment, such as a headless Wayland compositor, and have a separate subagent test the app by scripting mouse and keyboard input and taking screenshots. Fix every bug the tester finds, repeat until a full test pass finds no new problems, and give me a list of things you could not verify so I can check them by hand.
Want help getting this working? Join the Vibe Coding Academy ↗ Run by Alex Finn, who also runs this site.
Source: HN · Nov 17, 2026
Related use cases
Build a C compiler with a team of parallel agents
Anthropic's engineering write-up on tasking Opus 4.6 agent teams with building a C compiler.
Hand off coding sessions between agent accounts at usage limits
A developer built an app that tracks usage across multiple Codex and Claude Code accounts and auto-hands-off conversations when one hits its limit.
Build and deploy a blog aggregator website in a day
An author used Claude Code to turn a year-old requirements doc into a live Vue 3 site on Netlify that pulls their posts from several platforms.
Submitted by from search gap: 网站
Review Linux kernel patches
Google engineers launched Sashiko, an agentic AI code reviewer for the Linux kernel.
Custom agent
Monitor tasks and open pull requests with long-running agents
A commenter runs a persistent script that watches tasks and uses Cursor to open pull requests, building a documentation summarizer and a data pipeline.
Give an agent a rack-mounted phone to operate over ADB
The author set up a rack-mounted Fairphone 6 so their agent can use it through ADB.
Custom agent
Get 5 like this every morning
The best things people got an AI agent to do, each with the prompt to try it.