Simulation scene setup
Build physics scenes with robots, objects, sensors and lighting.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Isaac Sim · Gazebo · MuJoCo
- Class GUI-need label
G2· GUI-essential
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R07.09.01.03-T01 Set up a Franka tabletop scene with three rigid cubes in Isaac Sim
Original instruction
In Isaac Sim, create a stage with a ground plane, a dome light (intensity 1000), a Franka at the origin and three 5 cm cubes with rigid bodies and colliders at (0.4,0,0.025), (0.4,0.1,0.025), (0.5,-0.1,0.025) m. Press Play for 2 s, stop, and save output/franka_cubes.usd.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader opens the USD with usd-core: prims, RigidBodyAPI and CollisionAPI present, cube poses within 2 mm, light intensity exact; a headless rerun keeps cubes resting (<2 mm drift). Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
25% ground/light/robot; 45% cubes with physics at right poses; 30% stable after play and saved.
R07.09.01.03-T02 Insert models into a Gazebo world and save the SDF
Original instruction
In Gazebo Harmonic GUI, open fixtures/empty_lab.sdf, insert three boxes (0.5 m cube) at (2,0), (2,1.5), (-1,2) on the ground, add the 'TurtleBot 4' Fuel model at the origin facing +Y, and save the world as output/lab_obstacles.sdf.
- Platforms
lin· Linuxmac· macOS- Applications
- Gazebo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses the SDF: 3 box models with size 0.5 and poses within 2 cm, z resting on ground; TurtleBot model included with yaw 90+/-3 deg; world loads headless without errors. Agent has no terminal; gz service/topic calls from outside the GUI are logged and fail.
Proposed partial credit
45% boxes correct; 35% robot pose; 20% world saved and loads.
R07.09.01.03-T03 Read a joint angle after setting an actuator slider in MuJoCo
Original instruction
Open fixtures/humanoid.xml in MuJoCo 'simulate'. Reset, set the actuator 'right_knee' control slider to 0.5 (others 0), run for exactly 2.0 s of simulation time, pause, and write the right_knee joint angle in degrees (1 decimal) to output/answer.txt.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- MuJoCo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
drag· Drag & dropclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader replays the same controls in MuJoCo headless and compares the angle (+/-1.0 deg); simulate's saved key/history file must show the control slider set via UI. The agent VM has no terminal and no MuJoCo Python bindings, so the angle can only be read in simulate.
Proposed partial credit
30% correct actuator and value; 30% stopped at 2.0 s; 40% angle within tolerance.
R07.09.01.03-T04 Mount a wrist RGB-D camera with set intrinsics and a ROS 2 bridge
Original instruction
In fixtures/ur10_cell.usd (Isaac Sim), attach an RGB-D camera to the UR10 tool0 frame 5 cm along Z, set resolution 1280x720 and focal/aperture to match input/camera_intrinsics.yaml, and use the ROS 2 Camera Helper action graph to publish RGB, depth and camera_info at 30 Hz. Save the stage.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim · ROS 2
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader loads the USD: camera prim under tool0 with offset within 1 mm; derived fx, fy, cx, cy within 0.5 px of the YAML; headless run publishes the 3 topics at 30+/-2 Hz with matching camera_info. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
25% camera mounted correctly; 30% intrinsics; 30% ROS 2 graph publishing; 15% rate.
R07.09.01.03-T05 Lay out a warehouse aisle in Gazebo from a floor sketch
Original instruction
In Gazebo, build the warehouse aisle in input/aisle_layout.png: 8 shelf units (Fuel model 'Warehouse Shelf'), two rows facing each other with a 2.4 m aisle, a pallet jack at the aisle end and 4 pallets on the floor positions shown. Save output/aisle.sdf.
- Platforms
lin· Linuxmac· macOS- Applications
- Gazebo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
drag· Drag & dropgeom· Precise geometryvis· Visual judgment- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses SDF: model counts exact; shelf poses within 5 cm/3 deg of reference layout; aisle width 2.4+/-0.05 m; no interpenetrating models (bounding-box check). Agent has no terminal; gz service/topic calls from outside the GUI are logged and fail.
Proposed partial credit
20% counts; 45% shelf layout; 20% pallets and jack; 15% no overlaps.
R07.09.01.03-T06 Tune ramp friction in Isaac Sim so a box stops in a target zone
Original instruction
In fixtures/ramp.usd, a 20 deg ramp leads to a floor with a marked 30 cm target zone. Assign physics materials (static/dynamic friction, restitution 0) to the ramp and floor so the released 2 kg box slides down and comes to rest fully inside the zone. Save the stage.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader reruns physics headless 5 times: box rests inside the zone in all runs, final velocity <1 mm/s; materials bound to the ramp and floor prims only. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
20% materials assigned; 20% box slides; 60% rests in zone (pro-rata over runs).
R07.09.01.03-T07 Build a settled bin-picking scene with 20 parts in Isaac Sim
Original instruction
In Isaac Sim, create a bin (fixtures/bin.usd) on a table, drop 20 instances of input/bracket.usd at random orientations above it, simulate until they settle, freeze them as static poses, add three rect lights and an overhead camera framing the bin, and save output/bin_scene.usd.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropgeom· Precise geometryvis· Visual judgment- Work product
WP04· 3D/spatial model- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Grader: 20 part prims all inside bin volume, no pairwise penetration >1 mm, positions not on a grid (orientation entropy check), camera frustum contains the whole bin (projection test), 3 lights present. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
35% 20 settled parts inside bin; 20% no penetration; 25% camera framing; 20% lights.
R07.09.01.03-T08 Create an e-puck arena world in Webots
Original instruction
In Webots, create a new world with a 1x1 m rectangle arena, four 0.1 m high walls, a point light at (0,0,1), an e-puck at (-0.3,-0.3) facing +X running the 'braitenberg' controller, and 5 small cylinders at the positions in input/obstacles.csv. Save output/arena.wbt.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Webots
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses the .wbt: arena size, wall height, light, e-puck pose (2 cm/3 deg) and controller name, 5 cylinders within 1 cm; world runs 10 s headless without errors. Agent has no terminal or text editor; the .wbt must be saved by the Webots GUI (editor header/version), and writes to it by any other process are detected via the file audit log.
Proposed partial credit
20% arena and walls; 30% robot and controller; 35% obstacles; 15% runs.
R07.09.01.03-T09 Write the Franka cube scene in MJCF (control pair of T01)
Original instruction
Control pair of T01: with no GUI, write output/franka_cubes.xml in MJCF that includes fixtures/franka_panda.xml, a ground plane, a directional light and the three 5 cm free-joint cubes at the T01 positions. Simulate 2 s headless and save the final qpos to output/qpos.npy.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- MuJoCo · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Control pair of T01: grader loads the MJCF: bodies, sizes and positions within 2 mm; cubes rest (<2 mm drift after 2 s); qpos.npy matches grader rerun.
Proposed partial credit
40% cubes correct; 30% stable; 30% qpos matches.
R07.09.01.03-T10 Import a URDF arm with tuned joint drives in Isaac Sim
Original instruction
Import input/arm7.urdf with the Isaac Sim URDF importer as a fixed-base articulation, set each joint's drive stiffness and damping from input/drive_gains.csv, set the initial joint targets to the 'home' pose, play 5 s and save output/arm7.usd. The arm must hold home within 1 deg under gravity.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader opens USD: articulation root on base, fixed joint to world, drive gains per joint exact, targets equal home; headless 5 s run shows every joint within 1 deg of home. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
20% import settings (fixed base); 40% drive gains; 20% targets; 20% holds pose.
R07.09.01.03-T11 Light an Isaac Sim scene to a target exposure
Original instruction
In fixtures/product_table.usd, set up a dome light with input/studio.hdr and two rect lights so the render from camera '/World/Cam' has mean luminance between 0.40 and 0.50 and fewer than 1% clipped pixels, with the product's shadow falling to the left. Save the stage and render output/render.png.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader re-renders from the saved stage: mean luminance in range, clipped <1%, shadow centroid left of product centroid; render.png SSIM >=0.9 to grader render. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
40% luminance range; 20% clipping; 20% shadow direction; 20% consistent saved render.
R07.09.01.03-T12 Assemble a three-robot Gazebo world with namespaced bridges and an actor
Original instruction
Create output/multi_robot.sdf in Gazebo with three differential-drive robots (namespaces r1-r3, each with a 2D lidar at 10 Hz), a walking actor following the waypoints in input/actor_path.csv in a loop, and a ros_gz_bridge config output/bridge.yaml exposing each robot's /scan, /odom and /cmd_vel.
- Platforms
lin· Linux- Applications
- Gazebo · ROS 2
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropcode· Code & commands- Work product
WP04· 3D/spatial model- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Grader launches the world and bridge headless: 9 namespaced topics exist with correct types, lidar rates 10+/-1 Hz, actor positions at sampled times within 10 cm of the waypoint path, robots spawn without collision.
Proposed partial credit
30% robots and sensors; 30% bridge topics; 25% actor path; 15% clean spawn.
R07.09.01.03-T13 Build a conveyor sorting scene in CoppeliaSim
Original instruction
In CoppeliaSim, build a scene with a conveyor (0.2 m/s), a proximity sensor 1.0 m along the belt, a UR5 with suction pad beside it and a box spawning every 4 s at the belt start. Save output/sorting.ttt. Logic may stay unimplemented; geometry and object hierarchy must be correct.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CoppeliaSim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
drag· Drag & dropclick· Point & clickgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader loads the .ttt via the CoppeliaSim remote API in a clean instance: conveyor velocity, sensor pose (2 cm), UR5 with suction pad attached to the tip, spawner script present; 10 s run spawns 2-3 boxes. In the agent's CoppeliaSim instance the ZMQ remote API add-on and the Lua/Python commander are disabled, so objects can only be built in the GUI.
Proposed partial credit
30% conveyor and sensor; 30% robot with pad; 20% spawner; 20% scene runs.
R07.09.01.03-T14 Generate a 2D occupancy map of a warehouse scene in Isaac Sim
Original instruction
In fixtures/small_warehouse.usd, place 5 pallets at the positions in input/obstacles.csv, then use the Occupancy Map extension to generate a map at 0.05 m resolution with origin (-10,-10) covering 20x20 m at 0.1-0.6 m height, and export output/map.png and output/map.yaml (ROS map_server format).
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader: pallet poses within 2 cm; map.yaml resolution/origin exact; map.png IoU of occupied cells >=0.95 vs grader-generated map. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
30% pallets placed; 20% map parameters; 50% map IoU.
R07.09.01.03-T15 Set Mars gravity and physics step in a Gazebo world
Original instruction
Open fixtures/rover_flat.sdf in Gazebo, set gravity to (0,0,-3.71), max step size 0.001 s and real-time factor 1.0 using the GUI component inspector, and save as output/rover_mars.sdf.
- Platforms
lin· Linuxmac· macOS- Applications
- Gazebo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader parses SDF physics and gravity values exactly; all other elements unchanged (canonical XML diff). Agent has no terminal; gz service/topic calls from outside the GUI are logged and fail.
Proposed partial credit
60% physics values exact; 40% no collateral changes.
R07.09.01.03-T16 Pose a checkerboard for camera calibration renders in Isaac Sim
Original instruction
In fixtures/calib_cell.usd, place the 9x6 checkerboard (input/checker.usd) at 5 distinct poses in front of '/World/Cam' (tilts of at least 20 deg in two axes across poses, board filling 20-60% of the frame) and render one image per pose to output/calib_##.png.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
drag· Drag & dropgeom· Precise geometryvis· Visual judgment- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader runs OpenCV findChessboardCorners on each image (must succeed on all 5), computes board coverage fraction and pose diversity from the saved stage variants; calibrateCamera reprojection error <0.5 px. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
40% boards detected; 20% coverage range; 20% pose diversity; 20% calibration quality.
R07.09.01.03-T17 Configure physics options in MuJoCo simulate and save the model
Original instruction
Load fixtures/block_stack.xml in MuJoCo simulate. Using the Physics panel, set timestep 0.002, integrator implicitfast, friction cone elliptic and noslip iterations 5, then use 'Save xml' to write output/block_stack_tuned.xml. Run 3 s and confirm the stack does not topple.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- MuJoCo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses the saved XML option values exactly; headless 3 s run keeps all blocks within 5 mm of start; the agent VM has no terminal or MuJoCo Python bindings, so the file must come from simulate's Save XML (generated header checked; hand edits of the input fail).
Proposed partial credit
60% options exact; 20% saved via simulate; 20% stack stable.
R07.09.01.03-T18 Build a measured digital twin of a lab cell in Isaac Sim
Original instruction
Using input/cell_survey.xlsx (measured positions) and photos input/cell_cam*.jpg, build the cell in Isaac Sim: table, UR5e on its pedestal, two fixtures, a parts tray and the ceiling camera. Place each within 1 cm of survey values and tune the camera pose so its render overlays the reference photo edges.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim · Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometryvis· Visual judgmentmulti· Cross-app- Work product
WP04· 3D/spatial model- Conditions
long· Long-horizonmultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
Grader: object poses within 1 cm/2 deg of survey; render from the saved camera vs reference photo edge map IoU >=0.6 (partial from 0.4). Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
40% object poses; 20% hierarchy and scale; 40% camera overlay IoU.
R07.09.01.03-T19 Create a heightmap terrain world and spawn a rover on it in Gazebo
Original instruction
In Gazebo, create a world with the heightmap input/dem.png scaled to 100x100x10 m, a sun with shadows, and the Fuel rover 'X2 UGV' placed at (10,20) resting on the terrain with no penetration. Save output/terrain.sdf.
- Platforms
lin· Linuxmac· macOS- Applications
- Gazebo
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses SDF: heightmap size exact, sun cast_shadows true; headless 3 s settle shows rover wheels in contact and base height change <5 cm. Agent has no terminal; gz service/topic calls from outside the GUI are logged and fail.
Proposed partial credit
35% heightmap configured; 15% sun; 50% rover placed and stable.
R07.09.01.03-T20 Set up a conveyor that delivers boxes in a timed window in Isaac Sim
Original instruction
In fixtures/line.usd, use the Conveyor extension to create a 1.8 m belt at 0.3 m/s along +X, add a box emitter at the belt start and a stop plate at the end. Each box must reach the stop plate 6.0+/-0.3 s after spawning. Save output/line_conveyor.usd.
- Platforms
lin· Linuxwin· Windows- Applications
- Isaac Sim
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsgeom· Precise geometry- Work product
WP11· Game/sim state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader runs 20 s headless: belt speed 0.3 m/s, at least 3 boxes spawned, each arrival time 6.0+/-0.3 s, boxes stay on belt. Script Editor and Kit Python console are disabled; omni.kit command log must show only UI-origin commands.
Proposed partial credit
30% belt geometry and speed; 20% emitter; 50% arrival timing.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.