Build an NES harness so an agent can play games
A commenter is building an NES harness where an agent reads emulator state; it struggled detecting dialogs and menus, so they plan a per-frame state recorder.
Done with a custom-built or unnamed agent
- What they did
- The commenter is building a harness that lets an agent play NES games by reading emulator and game state. The agent kept stalling on detecting dialogs, menu navigation and screen changes. After a break, the commenter asked whether state could simply be read every frame instead of using clever detection. The agent agreed.
- How it went
- Not finished. The commenter hasn't resumed work, and plans to have the agent build a small recorder for frame-accurate state changes. The agent also raised a performance worry, which the commenter dismissed as nonsense, noting it was a flash model.
- Worth knowing
- When the agent churns on a clever approach, step back and suggest a simpler one yourself; agents struggle to notice ambiguity or missing context.
Try it yourself with Claude Code
Help me build a harness so an AI agent can play [NES game name] in an emulator. Set up [emulator, e.g. FCEUX or a Python NES library] so the agent can read game state like RAM values and screen frames, send controller inputs, and record a per-frame state log. Include detection for dialogs and menus (for example by watching specific RAM addresses), and stop when the agent can get through [first goal, e.g. the opening town] and show me the log format.
Want help getting this working? Join the Vibe Coding Academy ↗ Run by Alex Finn, who also runs this site.
Source: HN · Oct 25, 2026
Related use cases
Play Pokémon Silver on an emulator via Computer Use
A user had ChatGPT with Computer Use control a Pokémon Silver emulator, picking a starter and starting a playthrough, though usage limits were consumed by setup and repetitive dialogue.
Build a 2D platformer game without coding
A content creator with no coding experience used Claude Code to build a black-and-white wave-combat platformer in HTML canvas and played it live.
Play a button-clicking game via a browser agent
A developer wired Gemini's Computer Use model to Playwright in Java and had it play a simple button-clicking game to verify click-coordinate scaling.
Vibe-code a Flappy Bird game for DeskHog
A PostHog tutorial has Cursor in Agent mode write a Flappy Bird-style game for the DeskHog device from its README and hardware details.
Build four games in parallel with agents
Four Grok 4.20 agents built a tower defense, mini Factorio, CHIP-8 emulator and roguelike shooter in parallel over a day.
Get 5 like this every morning
The best things people got an AI agent to do, each with the prompt to try it.