Clear daily game tasks in Honkai: Star Rail by computer use
NUS researchers had Claude Computer Use open menus, pick a stage, set run counts and start auto-battle in Honkai: Star Rail, and it also managed decks in Hearthstone. ·
Done withClaude computer use
- What they did
- The NUS team built an automated GUI framework on the Claude Computer Use API. Claude works from live screenshots and keeps the earlier ones as context. Told to finish the day's Simulated Universe, it opened the game menu, found the Golden Calyx, set 6 challenge runs, started auto-battle, waited for the fight to end and exited. Human reviewers monitored the runs.
- How it went
- It completed the Star Rail routine and also made and renamed Hearthstone decks. Across 20 tasks on 12 apps or sites, it failed on some fine-grained web steps and on Word and PowerPoint text-field selection.
- Worth knowing
- Tests ran on Windows at 1366x768 and macOS at 1344,756, with human review. Complex interfaces are where it failed.
Try it yourself with Claude computer use
Open [game name] and complete my daily tasks: [list tasks, e.g. spend stamina on stage X, set run count to Y, start auto-battle]. Describe each screen as you go, and stop when all tasks are done or if you hit anything unexpected. Pause and ask me before spending any in-game currency or making any purchase.
Want help getting this working? Join the Vibe Coding Academy ↗ Run by Alex Finn, who also runs this site.
Source: qbitai.com · Nov 12, 2026 · Submitted by from search gap: 游戏
Related use cases
Fill out a Form 1040 with computer use
A tester gave Claude Computer Use data stored on disk and checked whether it could find the fields and fill a simplified Form 1040 accurately.
Find vegetarian restaurants and save notes via computer use
The author ran Anthropic's computer-use demo in Docker and had Claude find 5 vegetarian-friendly restaurants near Bond Street and save details to a notepad file.
Click add-to-cart button on a website with computer use
Víctor Paytuvi had Claude computer use find and click a site's add-to-cart button on the first try after Gemini failed.
Build a racing game from a single prompt
Codex made a racing game with multiple racers, eight maps and items from one prompt, acting as designer, developer and QA tester by playing it.
Run a D&D dungeon crawl with an AI dungeon master
A browser D&D 5e dungeon crawl where an AI dungeon master runs the game for a three-character party without inventing rules.
Custom agent
Get 5 like this every morning
The best things people got an AI agent to do, each with the prompt to try it.