3D action & open-world missions
Navigate and complete missions in 3D worlds.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Doom · Minecraft · open-world titles
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R13.01.01.02-T01 Reach and press the exit switch of Doom E1M1
Original instruction
GZDoom runs Doom (harness-supplied IWAD) at the start of E1M1 on 'Hurt Me Plenty'. Navigate the level with keyboard and mouse, survive the zombiemen and press the exit switch.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Doom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlkey· Shortcutsnav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
GZDoom stat log written by the harness: level E1M1 exited via the normal exit with player alive. Console, cheats (iddqd/idclip) and savegames are disabled; demo recording (.lmp) replays the run to confirm.
Proposed partial credit
60% scaled by distance along the reference route; 40% exit switch pressed
R13.01.01.02-T02 Take the gold key and open the locked door in Wolfenstein 3D level 1
Original instruction
A browser port of Wolfenstein 3D (shareware episode, harness-hosted) is at the start of E1L1. Find the gold key and open the gold-locked door without dying.
- Platforms
web· Any OS (web)- Applications
- Wolfenstein 3D
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsnav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Game-state bridge (harness only) reports key=gold true and the specific door's state open, player health > 0. Debug keys removed from the build. DevTools are disabled by browser policy and game objects are closure-scoped, so state cannot be patched by script; only trusted (isTrusted) keys are accepted.
Proposed partial credit
50% gold key obtained; 50% locked door opened
R13.01.01.02-T03 Reach the first slipgate of Quake e1m1
Original instruction
Quake (QuakeSpasm, harness-supplied pak files) is loaded at the start of e1m1 on Normal skill. Fight through the base and enter the slipgate at the end of the level.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Quake
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlkey· Shortcutsnav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Engine log: level change from e1m1 via the exit trigger. The console is disabled (no god/noclip/map cheats); demo file replays the run.
Proposed partial credit
60% scaled by route progress; 40% slipgate entered
R13.01.01.02-T04 Survive 60 seconds in ViZDoom Defend the Center
Original instruction
A ViZDoom window shows the 'defend_the_center' scenario: you are fixed in the middle of a circular room, can only turn and shoot, and monsters approach from all sides. Survive for 60 seconds.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Doom · ViZDoom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
ViZDoom episode variables (harness side): episode time ≥ 60 s with health > 0. The agent controls the game through OS key events on the visible window; the Python API handle is not exposed.
Proposed partial credit
50% scaled by survival time up to 30 s; 50% scaled from 30 to 60 s
R13.01.01.02-T05 Collect medkits to survive in ViZDoom Health Gathering Supreme
Original instruction
ViZDoom runs 'health_gathering_supreme': the floor drains health and medkits are scattered among poison vials in a maze-like room. Move to collect medkits and avoid vials to survive the full 2,100-tic episode.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- ViZDoom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsnav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ViZDoom episode stats: survived tics (pass at 2,100) and medkits collected. The ViZDoom Python API handle is not exposed to the agent and the game console is disabled; only OS key events on the window are accepted.
Proposed partial credit
50% scaled by survived tics up to 1,050; 50% scaled from 1,050 to 2,100
R13.01.01.02-T06 Find two secrets and exit Doom E1M2
Original instruction
In GZDoom (Doom IWAD), start E1M2 'Nuclear Plant' on 'Hurt Me Plenty'. Find at least two of the level's secret areas and then exit the level. The automap is allowed; cheats are not.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Doom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsnav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
long· Long-horizonhidden· Hidden/stale state- Proposed difficulty
- 5 / 5
Proposed verifier
Intermission stats (harness reads the level-stat log): secrets found ≥ 2 and level exited normally. Console disabled; .lmp demo replay confirms.
Proposed partial credit
25% first secret found; 25% second secret found; 50% level exited
R13.01.01.02-T07 Walk to the nearest village in Minecraft within ten in-game minutes
Original instruction
A Minecraft Java 1.20.4 world (seed in seed.txt, survival, peaceful) is loaded at spawn. Find and walk to the nearest village within 10 in-game minutes (12,000 ticks). F3 coordinates are disabled ('reducedDebugInfo'); use landmarks and exploration.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Minecraft
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Server plugin logs player position each second: pass when within 32 blocks of the nearest village centre before tick 12,000. Commands and the debug screen are disabled; chat is muted.
Proposed partial credit
50% scaled by distance to the village closed over the first half of the way; 50% scaled over the second half, full credit on arrival in time
R13.01.01.02-T08 Kill a zombie at night in Minecraft without dying
Original instruction
Minecraft Java survival world (normal difficulty), night just started, player has a stone sword, leather armour and 5 bread. Kill at least one zombie and survive until sunrise without dying.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Minecraft
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Server statistics file: minecraft:killed/zombie ≥ 1 and deaths = 0 at world time 23,000 (sunrise). Commands disabled.
Proposed partial credit
50% zombie killed; 50% survived to sunrise
R13.01.01.02-T09 Locate and enter a stronghold with eyes of ender
Original instruction
A Minecraft Java survival world starts with the player holding 12 eyes of ender, iron gear, food and 64 cobblestone. Throw eyes to triangulate the stronghold, dig down, and reach the end-portal room.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Minecraft
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Server plugin: player bounding box inside the end-portal room structure; deaths = 0. F3 coordinates disabled; commands disabled.
Proposed partial credit
30% stronghold direction narrowed (distance to it < 300 blocks); 30% stronghold entered; 40% portal room reached
R13.01.01.02-T10 Defeat the first boss in Valheim with provided gear
Original instruction
A local Valheim world (fixed seed) starts beside the Eikthyr altar with a crude bow, 100 fire arrows, flint axe, leather armour and food. Summon Eikthyr with the provided trophies and defeat it.
- Platforms
win· Windowslin· Linux- Applications
- Valheim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
World save parsed after the run: global key 'defeated_eikthyr' set. Devcommands disabled (BepInEx not loaded for the agent); the server-side save is the source of truth.
Proposed partial credit
20% boss summoned; 50% scaled by boss health removed; 30% defeated
R13.01.01.02-T11 Climb to the rooftop marker in Goat Simulator 3
Original instruction
Goat Simulator 3 is loaded on the opening map with a waypoint marker on a tall building. Reach the rooftop marker by climbing ladders, ledges and objects.
- Platforms
win· Windows- Applications
- Goat Simulator 3
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
A harness mod logs player position: pass when inside the marker volume. Without the mod, a VLM judge checks the final frame for the rooftop view (stated fallback, audited by a human sample). The agent has no shell or mod-loader access, and the harness mod exposes no console or teleport, so position changes come only from gameplay input.
Proposed partial credit
50% scaled by height gained over the first half of the climb; 50% scaled over the second half, full credit inside the volume
R13.01.01.02-T12 Repair the scanner and take off in No Man's Sky
Original instruction
A No Man's Sky save starts at the crash site on the first planet. Gather the required resources, repair the multi-tool Scanner, then repair the ship's launch thruster and take off.
- Platforms
win· Windows- Applications
- No Man's Sky
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollclick· Point & click- Work product
WP11· Game/sim state- Conditions
long· Long-horizon- Proposed difficulty
- 3 / 5
Proposed verifier
Save JSON (decompressed by the harness): Scanner tech not damaged; launch thruster repaired; player state 'in ship' with altitude above ground after the run. The agent has no shell or file-manager access, and the harness hashes the save at launch and logs each write by process, so a save edited outside the game is detected.
Proposed partial credit
35% scanner repaired; 35% thruster repaired; 30% take-off
R13.01.01.02-T13 Retrieve the Dragonstone from Bleak Falls Barrow in Skyrim
Original instruction
Skyrim Special Edition save begins at Riverwood with quest 'Bleak Falls Barrow' active, level-6 character with gear. Travel to Bleak Falls Barrow, solve the claw door and pillar puzzles, defeat the draugr overlord and take the Dragonstone.
- Platforms
win· Windows- Applications
- Skyrim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Save file parsed (inventory and quest stage): Dragonstone in inventory and quest stage ≥ 30. Console commands disabled via an INI lock.
Proposed partial credit
20% barrow reached; 20% pillar puzzle; 20% claw door; 40% Dragonstone obtained
R13.01.01.02-T14 Complete the first four test chambers in Portal
Original instruction
Portal (Steam, offline mode) is at the start of test chamber 00. Complete chambers 00 through 03, placing portals and boxes on buttons as required.
- Platforms
win· Windowsmac· macOSlin· Linux- Applications
- Portal
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlgeom· Precise geometrynav· Pan/zoom/scroll- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Source engine console log (written via -condebug, console input disabled): map transition events through testchmb_a_02 reached and chamber 03 exit elevator triggered.
Proposed partial credit
25% chamber 00 completed; 25% chamber 01; 25% chamber 02; 25% chamber 03
R13.01.01.02-T15 Follow a waypoint route through a cave in a voxel game
Original instruction
A Luanti (Minetest) world has a numbered sign trail through a dark cave system; the player has torches. Reach waypoint sign 8 (the cave exit) within 5 real-time minutes without falling into the lava pool.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Minetest
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Server mod logs position and deaths: pass when within 3 nodes of sign 8 before 300 s with no deaths. Chat commands and fly/noclip privileges revoked.
Proposed partial credit
50% scaled by waypoints 1-4 reached; 50% scaled by waypoints 5-8 reached; 0 on death
R13.01.01.02-T16 Clear Doom II MAP01 on Ultra-Violence losing at most half health
Original instruction
In GZDoom with Doom II (harness-supplied IWAD), play MAP01 'Entryway' on Ultra-Violence from a pistol start. Exit the level with at least 50% health remaining and at least 80% kills.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Doom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Level-stat log: exit with health ≥ 50 and kill percentage ≥ 80. Cheats and console disabled; demo replay confirms.
Proposed partial credit
40% exit; 30% scaled by kill percentage up to the 80 target; 30% health ≥ 50
R13.01.01.02-T17 Find the exit of the ViZDoom My Way Home maze
Original instruction
ViZDoom runs 'my_way_home': a maze of rooms with the green vest goal in a fixed room and a random start. Reach the vest within the 2,100-tic limit.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- ViZDoom
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Episode reward and position from ViZDoom (harness side): goal picked up before timeout. Run across 3 random starts; pass rate is the score. The ViZDoom Python API handle is not exposed to the agent and the console is disabled; only OS key events on the window are accepted.
Proposed partial credit
34% goal reached from start 1; 33% from start 2; 33% from start 3
R13.01.01.02-T18 Follow the companion to camp in a Red Dead Redemption 2 mission
Original instruction
Red Dead Redemption 2 (story mode, save at chapter 1 'Outlaws from the West' after the cabin) is loaded. Ride with the gang through the snow following Dutch, stay within the mission area, and reach the next checkpoint at the Colter camp.
- Platforms
win· Windows- Applications
- Red Dead Redemption 2
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
rt· Real-time controlnav· Pan/zoom/scrollkey· Shortcuts- Work product
WP11· Game/sim state- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Checkpoint reached detected by a save-file mission-state byte; fallback: perceptual hash of the checkpoint screen. Mission failures ('You left your gang') are counted. No trainers or script hooks are installed, and the agent has no shell access to the save folder, so progress comes only from play.
Proposed partial credit
60% scaled by route progress (VLM-free hash checkpoints); 40% camp checkpoint reached
R13.01.01.02-T19 Defend a Minecraft village through a full raid
Original instruction
Minecraft Java survival (normal), player with iron armour, bow, 128 arrows, shield, iron sword and food, has the Bad Omen effect next to a village. Trigger the raid and defeat all waves so the 'Hero of the Village' effect is granted, with at least 5 villagers alive.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Minecraft
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Server plugin: raid status 'victory' event, player has hero_of_the_village effect, villager count in the village ≥ 5. Commands disabled.
Proposed partial credit
60% scaled by waves defeated; 25% victory; 15% ≥ 5 villagers alive
R13.01.01.02-T20 Win a bot deathmatch in OpenArena
Original instruction
OpenArena runs a free-for-all deathmatch on map oa_dm1 against 3 bots at skill 2 ('Hurt me plenty'), frag limit 10. Win the match (first to 10 frags).
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- OpenArena
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
rt· Real-time controlkey· Shortcutsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Server log parsed: match end with the player's frag count = 10 and highest. The console is disabled for the agent and no rcon password is issued, so frags cannot be set by command; demo recorded.
Proposed partial credit
50% scaled by player frags / winner frags up to 0.5; 50% scaled from 0.5 to 1.0 (full credit for winning)
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.