Play Game Boy games through an emulator
A developer built an agent where Claude watches game frames from PyBoy, decides actions and presses buttons, using memory tools and periodic summaries. ·
Done withClaude
- What they did
- You supply a ROM, an Anthropic API key and optionally a saved state. Claude 3.7 gets recent screenshots from PyBoy and answers with tool calls. It sends button sequences, sets a goal, and adds, updates or removes memory items. In the default turn-based mode, the emulator advances only when Claude sends inputs. Every 30 turns by default, it writes a progress summary.
- How it went
- The README gives no results or progress numbers. It describes a working agent with frame logging for debugging. Movement is purely button-based, with no coordinate helpers.
- Worth knowing
- Context is capped by settings: 30 messages and 5 screenshots by default. Thinking budget is 16000 tokens per turn, so API costs may add up.
Try it yourself with Claude
Build me a Python agent that plays [Game Boy game] through the PyBoy emulator using Claude's API. Each step it should send the current screen frame to Claude, get back a button press, and apply it, with a simple memory file and a summary written every [number] steps. It's done when I can run one command, watch it play for a few minutes, and see the summary log; tell me which ROM and API key setup I need to provide.
Want help getting this working? Join the Vibe Coding Academy ↗ Run by Alex Finn, who also runs this site.
Source: github.com · Oct 29, 2026 · Submitted by from search gap: 游戏
Related use cases
Build a C compiler with a team of parallel agents
Anthropic's engineering write-up on tasking Opus 4.6 agent teams with building a C compiler.
Sort tax documents and fill IRS forms
Author had Claude sort wage forms, 1099s and brokerage summary, extract numbers, and fill blank IRS forms, but did not let it file via browser automation.
Build a report-writing program for real estate appraisals
A real estate appraiser used Claude to build a report-writing program that removes tedious work from the appraisal process.
Build a racing game from a single prompt
Codex made a racing game with multiple racers, eight maps and items from one prompt, acting as designer, developer and QA tester by playing it.
Build game architecture and animation merging with Codex
A commenter had Codex do fundamental architecture work for their game, reading engine docs and implementing better animation merging.
Get 5 like this every morning
The best things people got an AI agent to do, each with the prompt to try it.