遥操作装配与灵巧操作
Teleoperated assembly & dexterous manipulation
Teleoperate arms and dexterous hands in simulation to assemble, insert and move objects.
20 条本项目编写的提案 · 分类 v1.1 · 新增基准运行 0 次。下方验证方式与部分得分规则是设计，不是执行结果。
- 典型应用
- MuJoCo · Isaac Sim · LeRobot · teleop UIs
- 小类 GUI 需求标签
G3· 仅 GUI
任务提案 · 20
展开查看完整原文与建议检查方式；两种界面均保留提案英文原文。
R07.09.02.01-T01 Push a cube into a target square with MuJoCo mouse perturbation
原始指令
In MuJoCo simulate, load fixtures/push_table.xml (a 5 cm cube on a table with a 10 cm target square). Using only Ctrl+right-drag force perturbation on the cube while the sim runs in real time, push it so its center rests inside the square with yaw error <15 deg.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 2 / 5
建议验证方式
Grader reads the final state from simulate's recorded trajectory: cube center inside target, yaw error <15 deg, cube never left the table. Only perturbation events in the log may move the cube; qpos edits via the UI panel are detected and fail.
建议部分得分
50% cube center inside square (half of that if within 5 cm); 30% yaw; 20% stayed on table.
R07.09.02.01-T02 Pick a block and place it on a coaster with keyboard teleop
原始指令
In the browser teleop UI for the simulated Franka (fixtures/teleop_pick), use the keyboard end-effector jog controls (translate/rotate, gripper open/close) to pick the red 4 cm block and place it upright on the coaster, centered within 1 cm.
- 平台
web· 任意系统（网页）- 应用
- teleop UIs · MuJoCo
- 交互表面
brow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
key· 快捷键rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 2 / 5
建议验证方式
Grader reads the episode log: block on coaster with center error <1 cm and tilt <5 deg after release, no block drop >5 cm. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
30% grasp achieved; 50% placement within 1 cm (half of that within 3 cm); 20% upright and no drops.
R07.09.02.01-T03 Insert a peg into a 0.5 mm-clearance hole under a force limit
原始指令
In Isaac Sim with the 6-DoF space-mouse emulator panel controlling the UR5e (fixtures/peg_hole.usd), insert the 20 mm peg into the hole with 0.5 mm clearance to a depth of at least 20 mm while keeping wrist force below 30 N throughout.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
AP· API 足够- 交互原语
drag· 拖放rt· 实时控制geom· 精确几何vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader reads the logged contact/FT stream: final insertion depth >=20 mm, max |F| <30 N, no penetration beyond 1 mm into the hole wall (physics state). Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% peg aligned over hole; 50% depth reached (pro-rata); 30% force limit respected.
R07.09.02.01-T04 Reorient a cube in-hand with a dexterous hand to show a target face
原始指令
In MuJoCo, teleoperate the Shadow Hand (fixtures/shadow_cube.xml) through the provided joint-synergy sliders so the lettered cube shows face 'C' upward with orientation error below 15 deg, without dropping the cube or touching the table.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 4 / 5
建议验证方式
Grader computes the cube quaternion at end: face C normal within 15 deg of +Z, cube never contacted the table and remained in the hand (palm-distance check). Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
50% orientation (pro-rata to 45 deg); 30% no drop; 20% no table contact.
R07.09.02.01-T05 Hand a cube from one ALOHA arm to the other
原始指令
In LeRobot's gym_aloha transfer-cube task, use the browser leader-arm teleop UI to pick the cube with the right arm, hand it over mid-air to the left arm and hold it above the table for 2 s. Record the episode to output/episode_0.
- 平台
lin· Linuxmac· macOS- 应用
- LeRobot · MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUIbrow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 4 / 5
建议验证方式
Grader replays the recorded episode in the env: task success flag true, handover happens with cube height >5 cm, hold >=2 s, episode saved in LeRobot dataset format. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
25% right-arm grasp; 35% handover; 25% hold 2 s; 15% episode saved.
R07.09.02.01-T06 Assemble three gears onto shafts on a simulated task board
原始指令
In fixtures/nist_taskboard.usd (Isaac Sim), teleoperate the Franka with the space-mouse panel to place the small, medium and large gears onto their shafts in that order, each fully seated and meshed, within 15 minutes of sim time.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制geom· 精确几何vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
long· 长流程- 提案难度
- 5 / 5
建议验证方式
Grader reads final gear poses: each concentric with its shaft within 0.5 mm and seated within 1 mm of the stop; order from event log; teeth mesh check (relative angle) for adjacent gears. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
75% gears seated (25 pts each); 15% correct order; 10% meshing.
R07.09.02.01-T07 Stack three blocks into a tower that stays up for five seconds
原始指令
In the MuJoCo teleop UI (fixtures/stack3), use the end-effector jog controls to stack the blue, green and red blocks (bottom to top) on the marked spot so the tower stands for 5 s after release with top-block offset <1 cm.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
CL· CLI 足够- 交互原语
key· 快捷键drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader checks final state 5 s after last release: order blue-green-red, each block's xy offset <1 cm from the one below, all velocities ~0. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
30% two-block stack; 40% three-block stack in order; 30% stable 5 s with offsets in tolerance.
R07.09.02.01-T08 Route a deformable cable through three clips
原始指令
In fixtures/cable_route.xml (MuJoCo, flexible cable), teleoperate the two grippers to route the cable from its anchor through clips C1, C2 and C3 in order and leave it seated in all three, with the free end past C3.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制path· 连续路径vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
long· 长流程- 提案难度
- 5 / 5
建议验证方式
Grader checks cable body segments: for each clip at least one segment inside the clip slot volume, route order along cable arc length C1<C2<C3, cable not snagged on other geometry. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
75% clips correctly seated (25 pts each); 15% correct order; 10% free end past C3.
R07.09.02.01-T09 Open a drawer, place a mug inside and close it
原始指令
In the kitchen scene fixtures/drawer_mug.usd (Isaac Sim), teleoperate the Franka to open the top drawer at least 20 cm, place the mug upright inside it, then close the drawer to within 1 cm of fully closed.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader reads final joint of the drawer and mug pose: mug inside drawer volume and upright (<10 deg), drawer joint <1 cm; event log shows drawer opened >=20 cm. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
30% drawer opened; 40% mug placed upright inside; 30% drawer closed.
R07.09.02.01-T10 Pour beads from a cup into a bowl with minimal spill
原始指令
In fixtures/pour.usd (Isaac Sim particles), teleoperate the arm holding a cup of 500 beads so that at least 80% of beads end up in the bowl and fewer than 5% fall on the table, without the cup touching the bowl.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 4 / 5
建议验证方式
Grader counts particle positions at the end: fraction in bowl volume, fraction on table; contact log for cup-bowl contact. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
60% beads-in-bowl fraction (pro-rata from 0.4 to 0.8); 25% spill under 5 pct; 15% no cup-bowl contact.
R07.09.02.01-T11 Place a block on target under 300 ms teleop latency
原始指令
The teleop UI for fixtures/latency_place adds 300 ms command and video latency. Pick the cylinder and place it on the target ring within 1 cm, without knocking over the two nearby distractor cylinders.
- 平台
web· 任意系统（网页）- 应用
- teleop UIs · MuJoCo
- 交互表面
brow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
key· 快捷键rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader: final cylinder center within 1 cm of target and upright; distractors upright (tilt <10 deg); latency setting verified unchanged in the session config. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
50% placement; 30% distractors untouched; 20% completed under 4 min sim time.
R07.09.02.01-T12 Record five successful push-T teleop demonstrations in LeRobot
原始指令
Run LeRobot's gym_pusht teleop recorder (mouse-controlled pusher) and record 5 episodes into output/pusht_demos in LeRobot dataset format. At least 4 must reach coverage >=0.95 of the T target within 300 steps.
- 平台
lin· Linuxmac· macOS- 应用
- LeRobot · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP05· 时序媒体- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader loads the dataset with LeRobot: 5 episodes, valid schema and video frames; replays actions in the env and counts episodes with coverage >=0.95 within 300 steps. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% dataset valid; 80% successful episodes (20 pts each, up to 4).
R07.09.02.01-T13 Thread a nut onto a bolt with wrist rotation
原始指令
In fixtures/nut_bolt.xml (MuJoCo), teleoperate the 7-DoF arm and gripper to pick the M12 nut and thread it onto the vertical bolt at least 3 full turns using regrasp-and-rotate cycles.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制geom· 精确几何- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 4 / 5
建议验证方式
Grader reads the nut's screw-joint angle: >=1080 deg accumulated, axial position consistent with thread pitch; no cross-thread (nut tilt <3 deg). Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% nut started on thread; 60% turns (pro-rata to 3); 20% no cross-thread.
R07.09.02.01-T14 Drive a mobile manipulator to a shelf and pick a box
原始指令
In fixtures/shelf_store.usd (Isaac Sim), use the teleop UI (WASD base, arm jog panel) to drive the mobile manipulator through the aisle without collisions, pick the box labeled 'SKU-417' from shelf level 2 and bring it to the drop zone.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
key· 快捷键drag· 拖放rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader: correct box (prim id) inside drop zone at end; base-shelf collisions = 0 in contact log; other boxes on the shelf displaced <2 cm. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% reached shelf without collisions; 40% correct box picked; 30% delivered; 10% other boxes undisturbed.
R07.09.02.01-T15 Press four panel buttons in a given sequence
原始指令
In the teleop UI for fixtures/button_panel, press the buttons in the order Green, Blue, Blue, Red using the robot fingertip; each press must exceed 5 mm travel and no other button may be pressed.
- 平台
web· 任意系统（网页）- 应用
- teleop UIs · MuJoCo
- 交互表面
brow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
key· 快捷键rt· 实时控制- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 2 / 5
建议验证方式
Grader reads the button joint log: exact press sequence with travel >5 mm each, no extra presses. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
25% first press correct; 25% second; 25% third; 25% fourth (a press scores 0 if an extra press intervenes).
R07.09.02.01-T16 Fold a cloth in half with aligned corners
原始指令
In fixtures/cloth_fold.xml (MuJoCo flex cloth, 40x40 cm), teleoperate the two grippers to fold the cloth in half along its vertical midline so that the two left corners lie on the two right corners within 3 cm, and leave it flat.
- 平台
mac· macOSwin· Windowslin· Linux- 应用
- MuJoCo · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制path· 连续路径vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Grader reads final vertex positions: corner pair distances <3 cm, folded area 45-55% of original footprint, max height <4 cm. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
40% corner alignment; 40% folded area; 20% flatness.
R07.09.02.01-T17 Insert a USB-C plug with 0.2 mm tolerance
原始指令
In fixtures/usbc_insert.usd (Isaac Sim), teleoperate the gripper holding a USB-C plug (cable attached) into the laptop port: align orientation, insert to full depth with clearance 0.2 mm, peak force <15 N, without bending the cable over 90 deg.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · teleop UIs
- 交互表面
desk· 桌面 GUI- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制geom· 精确几何vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 5 / 5
建议验证方式
Grader: plug depth within 0.5 mm of fully seated, axis misalignment <1 deg at end, max contact force <15 N, cable bend <90 deg (segment angles). Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% aligned at port; 50% fully inserted; 20% force limit; 10% cable bend.
R07.09.02.01-T18 Script an impedance-control peg insertion (control pair of T03)
原始指令
Control pair of T03: without the teleop UI, write output/insert_peg.py using the Isaac Sim Python API with a Cartesian impedance or spiral-search strategy to insert the same 20 mm peg (0.5 mm clearance) to >=20 mm depth with force <30 N.
- 平台
lin· Linuxwin· Windows- 应用
- Isaac Sim · Python
- 交互表面
api· API / SDKterm· 终端- GUI 需求 · 任务设定
CL· CLI 足够- GUI 需求 · 普通权限
AP· API 足够- 交互原语
code· 代码与命令- 工作产物
WP06· 代码 / 工程- 条件
- 未声明
- 提案难度
- 3 / 5
建议验证方式
Control pair of T03: grader runs the script on 5 seeds with +/-2 mm initial offsets: success rate, depth and force thresholds as in T03.
建议部分得分
20% script runs; 60% success rate over 5 seeds; 20% force limit in all runs.
R07.09.02.01-T19 Hand over a tool between two arms in a WebXR teleop session
原始指令
In the Isaac Sim WebXR teleop session (WebXR emulator in the browser, hand-tracking poses driven by the emulator controls), grasp the screwdriver with the left robot hand, hand it to the right hand and place it in the tool tray slot.
- 平台
web· 任意系统（网页）- 应用
- Isaac Sim · teleop UIs
- 交互表面
xr· XR / 空间界面brow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
drag· 拖放rt· 实时控制geom· 精确几何vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
- 未声明
- 提案难度
- 4 / 5
建议验证方式
Grader: event log shows left grasp, simultaneous two-hand contact, right-hand-only hold; final screwdriver pose in slot within 1 cm/10 deg; no drop. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
25% left grasp; 35% handover; 30% slotted within tolerance; 10% no drop.
R07.09.02.01-T20 Recover after a scripted disturbance knocks the part loose
原始指令
In the teleop UI for fixtures/kit_assembly, pick the bracket and place it in fixture slot 2. Partway through, a scripted disturbance knocks the bracket out of the gripper; recover, re-grasp and complete the placement within the 6-minute session.
- 平台
web· 任意系统（网页）- 应用
- teleop UIs · MuJoCo
- 交互表面
brow· 浏览器- GUI 需求 · 任务设定
G3· 仅 GUI- GUI 需求 · 普通权限
G2· GUI 必要- 交互原语
key· 快捷键rt· 实时控制vis· 视觉判断- 工作产物
WP11· 游戏 / 仿真状态- 条件
interrupt· 中断- 提案难度
- 3 / 5
建议验证方式
Grader: disturbance event fired; final bracket pose in slot 2 within 5 mm/5 deg; completion time <6 min; other kit parts unmoved. Sim state/control APIs are not exposed to the agent; grader checks that every command in the control log came from the teleop UI process.
建议部分得分
20% initial grasp; 30% re-grasp after disturbance; 40% final placement; 10% other parts untouched.
已有研究映射
以下沿用研究目录中的基准与小类映射，不是逐题标注、执行记录或产品背书。