Computer-Use Agent Taxonomy中文
R01.01.02.01 · TASK CLASS

Window layouts, desktops & displays

Tile and move app windows and system panels (Spotlight, Control Center), assign apps to Spaces/virtual desktops and arrange multi-monitor geometry to a spec.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Mission Control · Snap layouts · GNOME workspaces
Class GUI-need label
G3 · GUI-only

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R01.01.02.01-T01 Snap Word and Edge side by side

Original instruction

Word (fixtures\brief.docx) and Edge are open and overlapping on the single display. Snap Word to the left half and Edge to the right half of the screen.

Platforms
win · Windows
Applications
Snap layouts · Word · Edge
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
licensed · Licensed pro app
Proposed difficulty
2 / 5

Proposed verifier

Win32 GetWindowRect via UIA: Word rect within ±2% of left work-area half, Edge within ±2% of right half; both not minimized. PowerShell blocked by AppLocker and no AutoHotkey installed; Sysmon detecting SetWindowPos scripts -> 0.

Proposed partial credit

50% Word left half; 50% Edge right half

R01.01.02.01-T02 Tile Safari and Notes with macOS window tiling

Original instruction

Using macOS window tiling (drag to screen edge or Window > Move & Resize), tile Safari on the left half and Notes on the right half of the main display, with no gaps setting changed.

Platforms
mac · macOS
Applications
Mission Control · Safari · Notes
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

CGWindowList bounds: Safari within ±2% of left half of visible frame, Notes within ±2% of right half; com.apple.WindowManager tiling margin setting unchanged. Terminal and Script Editor withheld; eslogger detects osascript 'set bounds' -> 0.

Proposed partial credit

50% Safari left; 50% Notes right

R01.01.02.01-T03 Three-column snap layout and a snap group

Original instruction

On the 3440×1440 virtual display, use a three-column Snap layout to place Teams (left), Outlook (center) and Edge (right). Then minimize all three and restore them together via the snap group in the taskbar.

Platforms
win · Windows
Applications
Snap layouts · Teams · Outlook · Edge
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
hover · Hover & context click · Point & click geom · Precise geometry
Work product
WP08 · App/system state
Conditions
licensed · Licensed pro app
Proposed difficulty
3 / 5

Proposed verifier

Final rects match the 3-column template ±2%; snap-group restore observed via UIA event log (all three restored within 1 s from one taskbar action). PowerShell blocked by AppLocker; Sysmon detects SetWindowPos scripts -> 0.

Proposed partial credit

60% three windows in correct columns; 40% restored together via snap group

R01.01.02.01-T04 Split work across named virtual desktops

Original instruction

Create two virtual desktops named 'Research' and 'Comms'. Move Chrome and Zotero to Research, Teams and Outlook to Comms, and make Notepad show on all desktops. Desktop 1 must keep only File Explorer.

Platforms
win · Windows
Applications
Task View · Chrome · Zotero · Teams
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop hover · Hover & context type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

HKCU VirtualDesktops names; IVirtualDesktopManager.GetWindowDesktopId for each window maps to expected desktop; Notepad pinned (IsPinnedWindow). PowerShell blocked by AppLocker; Sysmon detects the VirtualDesktop module or MoveToDesktop tools -> 0 (control pair is T20).

Proposed partial credit

20% desktops named; 50% four windows on correct desktops; 15% Notepad on all; 15% Desktop 1 clean

R01.01.02.01-T05 Assign apps to Spaces from the Dock

Original instruction

With three Spaces on the main display, assign Mail to Desktop 2, Xcode to Desktop 3 and Calendar to All Desktops using each app's Dock menu Options > Assign To. Leave other apps unassigned.

Platforms
mac · macOS
Applications
Mission Control · Dock
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
hover · Hover & context click · Point & click
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

com.apple.spaces app-bindings: com.apple.mail→UUID of Space 2, com.apple.dt.Xcode→Space 3 UUID, com.apple.iCal→AllSpaces; no other bindings. Space UUID order from SpacesDisplayConfiguration. Terminal withheld; eslogger detects defaults write com.apple.spaces -> 0.

Proposed partial credit

30% Mail; 30% Xcode; 25% Calendar all; 15% no extra bindings

R01.01.02.01-T06 Stage Manager with a grouped Safari+Notes stage

Original instruction

Turn on Stage Manager with recent apps hidden and 'Show windows from an application' set to One at a Time. Create one stage that contains both Safari and Notes side by side, and a separate stage with Finder only.

Platforms
mac · macOS
Applications
Stage Manager · Safari · Notes · Finder
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop vis · Visual judgment
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

com.apple.WindowManager GloballyEnabled=1, AutoHide=1, AppWindowGroupingBehavior=0; grader activates Safari and checks via CGWindowList that Notes is on-screen and Finder is not, then activates Finder and checks the reverse. No API composes stages; Terminal withheld and eslogger detects defaults/osascript -> 0.

Proposed partial credit

30% Stage Manager settings; 50% Safari+Notes grouped; 20% Finder stage separate

R01.01.02.01-T07 Four fixed GNOME workspaces with apps placed

Original instruction

Switch GNOME to a fixed number of 4 workspaces, then put Firefox on workspace 2, Terminal on workspace 3 and Files on workspace 4 by dragging them in the Activities overview. Workspace 1 should be empty.

Platforms
lin · Linux
Applications
GNOME workspaces · Firefox · Files
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

gsettings org.gnome.mutter dynamic-workspaces=false, org.gnome.desktop.wm.preferences num-workspaces=4; wmctrl -l desktop index: Firefox 1, Terminal 2, Files 3 (0-based); none on 0. wmctrl/xdotool absent from the VM; auditd detects gsettings or window-move commands run in the placed Terminal -> 0.

Proposed partial credit

30% fixed 4 workspaces; 60% three windows placed (equal share each); 10% workspace 1 empty

R01.01.02.01-T08 Exact window positions and sizes without snapping

Original instruction

Place Calculator so its top-left corner is at (100,100) on monitor 1 and resize Notepad to 800×600 centered on monitor 2, by dragging title bars and edges only. Snapping must not be engaged.

Platforms
win · Windows
Applications
Calculator · Notepad
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

GetWindowRect: Calculator origin within ±8 px of (100,100); Notepad size 800×600 ±8 px and center within ±8 px of monitor 2 center; neither window in a snapped state (IsZoomed/snap flags false). PowerShell blocked by AppLocker; Sysmon detects SetWindowPos scripts -> 0.

Proposed partial credit

40% Calculator position; 30% Notepad size; 30% Notepad centered

R01.01.02.01-T09 Precise TextEdit and Finder bounds across two displays

Original instruction

Make the TextEdit window 900×700 pt with its top-left at the top-left of the right (external) display's visible area, and make a Finder window in Column view fill the bottom half of the left display's visible area.

Platforms
mac · macOS
Applications
TextEdit · Finder
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

CGWindowList bounds: TextEdit size ±6 pt, origin within ±6 pt of external visibleFrame top-left; Finder bounds within ±1% of left display bottom half, current view = column (Finder AX). Terminal and Script Editor withheld; eslogger detects osascript bounds setters -> 0.

Proposed partial credit

35% TextEdit size; 25% TextEdit position; 25% Finder half-display bounds; 15% column view

R01.01.02.01-T10 Two-monitor desk layout that survives a disconnect

Original instruction

Maximize Teams on monitor 2, snap Outlook left and VS Code right on monitor 1. Turn on 'Remember window locations based on monitor connection' and turn off 'Minimize windows when a monitor is disconnected'. The grader will unplug and replug monitor 2.

Platforms
win · Windows
Applications
Windows Settings · Teams · Outlook · VS Code
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop geom · Precise geometry click · Point & click
Work product
WP08 · App/system state
Conditions
licensed · Licensed pro app interrupt · Interruptions
Proposed difficulty
5 / 5

Proposed verifier

Before unplug: rects per spec ±2%; MonitorDisconnect settings registry values; after virtual detach/reattach Teams returns to monitor 2 maximized and the others keep their halves. PowerShell/regedit blocked by AppLocker; Sysmon detects SetWindowPos/reg.exe scripts -> 0.

Proposed partial credit

40% initial layout; 20% two settings; 40% layout restored after reconnect

R01.01.02.01-T11 Mission Control behavior settings

Original instruction

In Desktop & Dock: turn off 'Automatically rearrange Spaces based on most recent use', turn on 'Group windows by application', turn off 'When switching to an application, switch to a Space with open windows', keep 'Displays have separate Spaces' on.

Platforms
mac · macOS
Applications
Mission Control · System Settings
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

com.apple.dock mru-spaces=0, expose-group-apps=1; -g AppleSpacesSwitchOnActivate=0; com.apple.spaces spans-displays=0. Terminal withheld; eslogger exec audit detects defaults -> 0.

Proposed partial credit

25% Spaces auto-rearrange off; 25% group windows by app on; 25% switch-to-Space off; 25% separate Spaces kept on

R01.01.02.01-T12 Full-screen Split View at a 60/40 ratio

Original instruction

Put Safari and Pages (fixtures/memo.pages) into one full-screen Split View with Safari on the left, then drag the divider so Safari takes 60% of the width.

Platforms
mac · macOS
Applications
Mission Control · Safari · Pages
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop geom · Precise geometry hover · Hover & context
Work product
WP08 · App/system state
Conditions
licensed · Licensed pro app
Proposed difficulty
4 / 5

Proposed verifier

Both windows full-screen on the same Space (CGSCopySpacesForWindows); Safari left; width ratio Safari/(Safari+Pages) = 0.60 ±0.03. No scripting route composes Split View; Terminal withheld and eslogger detects osascript -> 0.

Proposed partial credit

40% both in one Split View; 20% Safari left; 40% ratio within tolerance

R01.01.02.01-T13 Restore a morning workspace from a layout spec

Original instruction

input/layout.json lists 6 windows (Outlook, Teams, Edge with 2 windows, Excel q3.xlsx, Notepad todo.txt) with target virtual desktop, monitor and rectangle. Open what is missing and arrange everything to the spec across 2 desktops and 2 monitors.

Platforms
win · Windows
Applications
Task View · Snap layouts · Excel · Edge
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop geom · Precise geometry multi · Cross-app
Work product
WP08 · App/system state
Conditions
long · Long-horizon licensed · Licensed pro app
Proposed difficulty
5 / 5

Proposed verifier

Per window: correct process+document, desktop id, monitor, rect IoU ≥0.95 vs spec. Score is mean over 6 windows. PowerShell blocked by AppLocker; Sysmon detects SetWindowPos/VirtualDesktop scripts -> 0.

Proposed partial credit

20% all windows open; 30% desktop assignment; 20% monitor; 30% rect IoU ≥0.95

R01.01.02.01-T14 Recreate a teammate's three-Space setup from a screenshot

Original instruction

input/workspace.png is a Mission Control screenshot of three Spaces with 7 windows (Safari, Notes, Terminal, Xcode, Preview, Messages, Calendar). Recreate it: same windows per Space and approximately the same tiling within each Space.

Platforms
mac · macOS
Applications
Mission Control · Safari · Xcode · Preview
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
vis · Visual judgment drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

Grader maps windows to Spaces via CGSCopySpacesForWindows: app-set per Space must match; per-window normalized rect IoU ≥0.8 vs annotated targets. No yabai installed and Script Editor withheld; eslogger detects osascript or other window-moving exec -> 0.

Proposed partial credit

20% three Spaces exist; 40% app-to-Space assignment; 40% per-window placement IoU

R01.01.02.01-T15 GNOME edge tiling and an always-on-top window

Original instruction

Enable edge tiling and 'center new windows' in GNOME, tile Text Editor to the left and Terminal to the right of workspace 1, and set the Settings window to Always on Top via its title-bar context menu.

Platforms
lin · Linux
Applications
GNOME workspaces · GNOME Tweaks · Text Editor
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop hover · Hover & context
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

gsettings org.gnome.mutter edge-tiling=true, center-new-windows=true; wmctrl -lG geometry halves ±2%; xprop _NET_WM_STATE contains _NET_WM_STATE_ABOVE for Settings. wmctrl/xdotool absent from the VM; auditd detects gsettings/window-move commands run in the tiled Terminal -> 0.

Proposed partial credit

25% two settings; 50% tiled halves; 25% always on top

R01.01.02.01-T16 Keep snapping but quiet Snap Assist

Original instruction

Keep window snapping on, but turn off 'When I snap a window, suggest what I can snap next to it', keep 'Show snap layouts when I drag a window to the top', set Alt+Tab to show open windows only, and show taskbar windows from all desktops.

Platforms
win · Windows
Applications
Windows Settings
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Explorer\Advanced: WindowArrangementActive=1, SnapAssist=0, EnableSnapBar=1, MultiTaskingAltTabFilter=3, VirtualDesktopTaskbarFilter=0. regedit/PowerShell blocked by AppLocker; Sysmon detects reg.exe writes -> 0.

Proposed partial credit

30% snapping and drag-top layouts kept on; 30% Snap Assist off; 20% Alt+Tab; 20% taskbar all desktops

R01.01.02.01-T17 Duplicate two displays and extend to a third

Original instruction

Three virtual displays are attached. Make displays 1 and 2 mirror each other (for the projector) while display 3 extends the desktop to the right of them, top-aligned, and keep display 1 as the main display.

Platforms
win · Windows
Applications
Windows Settings
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
AP · API-sufficient
Interaction primitives
click · Point & click drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

QueryDisplayConfig: displays 1 and 2 share one source (clone), display 3 separate source with x = clone width, y = 0 ±4; display 1 primary. cmd/PowerShell blocked by AppLocker; Sysmon detects DisplaySwitch/SetDisplayConfig tools -> 0.

Proposed partial credit

40% 1+2 duplicated; 40% display 3 extended and placed; 20% primary

R01.01.02.01-T18 Merge document windows into tabs and keep them on relaunch

Original instruction

Turn off 'Close windows when quitting an application', set 'Prefer tabs when opening documents' to Always, then merge the three open TextEdit windows (a.txt, b.txt, c.txt) into one tabbed window ordered a, b, c, and quit/relaunch TextEdit.

Platforms
mac · macOS
Applications
TextEdit · System Settings
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
drag · Drag & drop click · Point & click
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

-g NSQuitAlwaysKeepsWindows=1, AppleWindowTabbingMode=always; after grader relaunch, TextEdit AX shows one window with 3 tabs in order a,b,c. Terminal and Script Editor withheld; eslogger detects defaults/osascript -> 0.

Proposed partial credit

30% two settings; 40% merged in order; 30% restored after relaunch

R01.01.02.01-T19 Reorder, rename and clean up virtual desktops

Original instruction

There are four desktops: 'Desktop 1', 'Comms', 'Desktop 3' (empty) and 'Build'. Drag 'Comms' to first position in Task View, rename 'Build' to 'CI', close the empty desktop, and keep every window on the desktop it was on.

Platforms
win · Windows
Applications
Task View
Interfaces
desk · Desktop GUI
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
drag · Drag & drop type · Text & forms hover · Hover & context
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

VirtualDesktopIDs order: Comms first; names list = [Comms, Desktop 1, CI]; per-window desktop id unchanged vs seed (no window moved by closing). PowerShell blocked by AppLocker; Sysmon detects VirtualDesktop module use -> 0.

Proposed partial credit

35% order; 25% rename; 20% empty desktop closed; 20% windows unchanged

R01.01.02.01-T20 Virtual desktop split via PowerShell (control)

Original instruction

Control pair of T04: using the VirtualDesktop PowerShell module, create 'Research' and 'Comms', move Chrome and Zotero to Research, Teams and Outlook to Comms, and pin Notepad to all desktops.

Platforms
win · Windows
Applications
PowerShell · VirtualDesktop module
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Same desktop-name and window-to-desktop checks as T04; measures the GUI-vs-CLI gap on an identical end state.

Proposed partial credit

20% desktops named; 50% windows placed; 15% Notepad pinned; 15% Desktop 1 clean

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

In this workflow family