Computer-Use Agent Taxonomy中文
R02.05.01.02 · TASK CLASS

Desktop app & UI framework work

Implement or fix desktop UI in SwiftUI, WPF, Qt or Electron, checked on screen.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Xcode · Visual Studio · Qt Creator
Class GUI-need label
HY · Hybrid

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R02.05.01.02-T01 Build a macOS sidebar-detail browser with toolbar search in SwiftUI

Original instruction

In fixtures/Library.xcodeproj (macOS 14 target), replace the single list with a NavigationSplitView: sidebar of 4 collections, content list of books, detail pane. Add a toolbar search field filtering titles case-insensitively, and remember the selected collection across launches. Run it and check the layout at 900×600 and full screen.

Platforms
mac · macOS
Applications
Xcode · SwiftUI
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden macOS XCUITest: 3 columns exist; selecting 'Sci-Fi' lists 17 books; typing 'dune' filters to 2; relaunch restores 'Sci-Fi'. Screenshot at 900×600 has no clipped sidebar labels (OCR of all 4 names). Build has zero warnings.

Proposed partial credit

30% split view structure; 30% search filter; 20% selection persistence; 20% layout at both sizes

R02.05.01.02-T02 Fix a WPF DataGrid that does not refresh after edits

Original instruction

fixtures/Inventory.sln (WPF, .NET 8) shows stale totals: editing Quantity in the DataGrid does not update the row Total or the footer sum until restart. Fix the view model bindings in Visual Studio and verify in the running app.

Platforms
win · Windows
Applications
Visual Studio · WPF
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click type · Text & forms
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Hidden FlaUI test edits 3 quantities and asserts row Total cells and footer 'Sum' text update immediately to expected values. dotnet build has zero warnings; no code-behind event handlers added (Roslyn diff check).

Proposed partial credit

60% UI updates on edit (rows 30, footer 30); 20% MVVM kept (no code-behind); 20% clean build

R02.05.01.02-T03 Make a Qt settings dialog resize and stretch correctly

Original instruction

The Preferences dialog in fixtures/qtpad (Qt 6 Widgets, .ui file) clips controls when shrunk and leaves dead space when enlarged. Fix it in Qt Creator's Designer with proper layouts, stretch factors and minimum sizes so it matches input/prefs_spec.png at 480×360, 800×600 and 1200×900.

Platforms
mac · macOS win · Windows lin · Linux
Applications
Qt Creator · Qt Widgets
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click geom · Precise geometry vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden QTest resizes the dialog to the 3 sizes and asserts: no child widget extends outside the dialog, path QLineEdit grows with width, buttons stay bottom-right within 12 px margins. Dialog minimumSize ≥ declared spec. .ui must use layouts (no absolute geometry).

Proposed partial credit

30% no clipping; 30% stretch behavior; 20% button anchoring; 20% layouts instead of fixed geometry

R02.05.01.02-T04 Add native menus and platform shortcuts to an Electron app

Original instruction

fixtures/markdown-studio (Electron 31) has no application menu. Add File/Edit/View/Help menus with New, Open, Save, Export PDF, Toggle Preview; use Cmd on macOS and Ctrl on Windows, include the macOS app menu, and make the window follow the system dark/light theme.

Platforms
mac · macOS win · Windows
Applications
Electron · VS Code
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands key · Shortcuts click · Point & click
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Grader runs Playwright _electron on mac and win: Menu.getApplicationMenu() structure and accelerators match spec per platform; triggering accelerator Save writes the file; nativeTheme toggle changes body background ≥ΔE 30. macOS app menu present only on darwin.

Proposed partial credit

35% menu structure; 30% accelerators per OS; 15% actions work; 20% theme following

R02.05.01.02-T05 Make a WPF app crisp at 150% and 200% display scaling

Original instruction

fixtures/Plotter (WPF) is blurry and mis-sized on high-DPI monitors. Make it per-monitor DPI aware (v2), replace raster toolbar icons with vector or multi-scale assets, and fix hard-coded pixel offsets. Check it on a Windows VM at 100%, 150% and 200% scaling.

Platforms
win · Windows
Applications
Visual Studio · WPF
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Grader launches at each scaling (registry-set VM snapshots) and screenshots: toolbar icon edge sharpness (Laplacian variance) ≥ reference ×0.9; chart axis labels OCR-exact; window chrome to content offsets scale linearly ±1 px. App manifest declares PerMonitorV2.

Proposed partial credit

20% manifest DPI awareness; 40% icon sharpness at 1.5x and 2x scaling; 20% OCR labels; 20% layout offsets

R02.05.01.02-T06 Add a persisted theme switcher to a WinUI 3 settings page

Original instruction

In fixtures/Journal (WinUI 3, Windows App SDK 1.5), add a Settings page reachable from the NavigationView with a Light/Dark/Use system radio group that applies immediately and persists across restarts. Match the Fluent spacing in input/settings_mock.png.

Platforms
win · Windows
Applications
Visual Studio · WinUI 3
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands click · Point & click vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden WinAppDriver test: navigate to Settings, choose Dark, verify root RequestedTheme Dark and a sampled background pixel ΔE2000 ≤3 from #202020; restart and confirm Dark persists; choose system and confirm it follows OS. Screenshot SSIM ≥0.85 vs mock.

Proposed partial credit

20% navigation entry; 40% theme applies; 25% persistence; 15% layout match

R02.05.01.02-T07 Implement drag-to-reorder in a QML list

Original instruction

In fixtures/Tasks (Qt 6 Quick), let users reorder the task ListView by dragging a handle, with a drop indicator line and smooth displacement, and persist the new order to tasks.json. Dragging outside the list must cancel.

Platforms
mac · macOS win · Windows lin · Linux
Applications
Qt Creator · Qt Quick
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands drag · Drag & drop vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Hidden Qt Quick Test simulates press-move-release on the handle: dragging item 1 to slot 4 yields order [B,C,D,A,E] in the model and tasks.json; drag released outside restores order; drop indicator Item visible during drag. Frame rate during drag ≥50 fps measured via QQuickWindow frameSwapped.

Proposed partial credit

40% reorder correct; 20% persisted; 20% cancel outside; 20% indicator and smoothness

R02.05.01.02-T08 Restore window frame and split position across launches on macOS

Original instruction

fixtures/Notes (AppKit) always opens at a default size and the sidebar split resets. Implement state restoration: window frame, sidebar width, selected note, and full-screen state, working with multiple windows.

Platforms
mac · macOS
Applications
Xcode · AppKit
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden XCUITest opens 2 windows, sets frames and split widths to known values, quits and relaunches: both windows restored within ±2 pt, split widths ±2 pt, selections restored. Uses NSWindowRestoration or autosave names (static check).

Proposed partial credit

40% frames restored; 30% split widths; 20% selection; 10% multi-window

R02.05.01.02-T09 Harden an Electron app's renderer security without breaking features

Original instruction

fixtures/clipvault (Electron) runs with nodeIntegration on and no CSP. Enable contextIsolation and sandbox, disable nodeIntegration, add a preload with a minimal IPC allowlist, and a strict CSP. All existing features (clipboard history, search, export) must still work.

Platforms
mac · macOS win · Windows
Applications
Electron
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Electronegativity scan: zero HIGH findings. Grader's Playwright _electron e2e (8 specs) all pass. Runtime check: window.require undefined in renderer; ipcRenderer only exposes the 5 allowed channels; CSP header blocks inline script.

Proposed partial credit

30% security flags; 20% CSP; 20% IPC allowlist; 30% e2e suite passing

R02.05.01.02-T10 Package a Tauri app with installers and a file association

Original instruction

Package fixtures/mdview (Tauri 2) as a signed-with-test-cert .dmg for macOS and an .msi for Windows, with the icon set from input/icon.png and a .mdx file association that opens the file in the app. Install on both test VMs and open input/sample.mdx by double-click.

Platforms
mac · macOS win · Windows
Applications
Tauri · Xcode · Visual Studio
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click multi · Cross-app
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Grader inspects artifacts: Info.plist CFBundleDocumentTypes includes mdx; MSI registers ProgID for .mdx. On VMs, launching sample.mdx via the OS shell (open / start) opens the app with the file title shown (accessibility tree). Icon hash matches derived sizes.

Proposed partial credit

25% mac bundle; 25% msi; 30% association works on both; 20% icons

R02.05.01.02-T11 Add undo/redo to a WPF diagram canvas

Original instruction

fixtures/FlowSketch (WPF) lets users drag shapes and connect them on a Canvas but has no undo. Implement a command-based undo/redo stack (Ctrl+Z/Ctrl+Y and toolbar buttons) covering move, create, delete and connect; grouped moves count as one step.

Platforms
win · Windows
Applications
Visual Studio · WPF
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands drag · Drag & drop key · Shortcuts
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Hidden FlaUI test performs 12 scripted drag/create/connect/delete operations, then 12 undos and 6 redos; serialized canvas state (export JSON) must equal expected snapshots at each checkpoint. Toolbar buttons enable/disable correctly.

Proposed partial credit

40% undo correctness; 30% redo correctness; 15% grouped moves; 15% button states

R02.05.01.02-T12 Port a WinForms dialog to Avalonia for macOS and Windows

Original instruction

Port the Export dialog from fixtures/LegacyTool (WinForms, Designer.cs) to Avalonia 11 in the new fixtures/NewTool project so it runs on macOS and Windows: same controls, tab order, validation rules and default values, with layouts that scale instead of absolute positions.

Platforms
mac · macOS win · Windows
Applications
Visual Studio · Rider · Avalonia
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands vis · Visual judgment click · Point & click
Work product
WP06 · Code/engineering
Conditions
long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

Hidden Avalonia.Headless tests compare controls (names, types, defaults), tab order and validation error messages to a spec extracted from the WinForms dialog. Screenshots on mac and win at 1x and 2x: SSIM ≥0.80 vs reference renders, no clipped text (OCR all labels).

Proposed partial credit

25% control parity; 15% tab order; 25% validation; 35% rendering on both OS

R02.05.01.02-T13 Fix dark-mode contrast in a SwiftUI macOS app

Original instruction

fixtures/Budget renders unreadable gray-on-black text and invisible dividers in dark mode. Replace hard-coded colors with semantic colors or an asset catalog with light/dark variants so every text element meets 4.5:1 contrast in both appearances.

Platforms
mac · macOS
Applications
Xcode · SwiftUI
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Grader renders the 3 main views in light and dark via hidden snapshot tests and measures text/background contrast at OCR-located text boxes: all ≥4.5:1; dividers ≥1.5:1. Static check: no Color(red:green:blue:) literals remain in view files.

Proposed partial credit

60% contrast in dark (per view); 20% light unchanged; 20% no hard-coded colors

R02.05.01.02-T14 Set tab order and keyboard shortcuts in a Qt form

Original instruction

In fixtures/crm-lite (Qt Widgets), the New Contact form tabs through fields in a random order and has no mnemonics. Set the tab order to follow input/tab_order.txt, add &-mnemonics to all labels with buddies, and make Enter submit and Esc cancel.

Platforms
mac · macOS win · Windows lin · Linux
Applications
Qt Creator · Qt Widgets
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
click · Point & click key · Shortcuts
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Hidden QTest presses Tab from the first field and records focus sequence == spec (11 widgets); Alt+mnemonic focuses the right buddy (Ctrl+Option on macOS); Return triggers accepted(), Escape rejected().

Proposed partial credit

50% tab order; 30% mnemonics/buddies; 20% Enter/Esc

R02.05.01.02-T15 Fix anchoring and auto-scaling in a WinForms dialog

Original instruction

fixtures/Payroll's Employee dialog (WinForms, .NET 8) overlaps controls when the window is maximized or the system font is set to 125%. In the Visual Studio designer, fix Anchor/Dock settings and AutoScaleMode so the layout matches input/employee_spec.png at 640×480 and maximized, at 100% and 125% text size.

Platforms
win · Windows
Applications
Visual Studio · WinForms
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click geom · Precise geometry vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden FlaUI test captures bounding rectangles of 18 controls in 4 configurations: no overlaps, buttons anchored bottom-right, grid fills remaining space ±4 px; AutoScaleMode=Dpi. Edits must be in Designer.cs (generated region) and match designer serialization; a Procmon audit log flags any write to Designer.cs not made by devenv.exe.

Proposed partial credit

40% no overlaps; 30% anchoring per config; 20% AutoScaleMode; 10% designer-valid code

R02.05.01.02-T16 Stop an Electron renderer crash when opening large files

Original instruction

fixtures/logscope (Electron) crashes with an out-of-memory renderer when opening input/big.log (1.2 GB). Stream and virtualize the file view so it opens in under 3 s with scrolling, and search still works. Check memory in DevTools.

Platforms
mac · macOS win · Windows
Applications
Electron · Chrome DevTools
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands nav · Pan/zoom/scroll vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Grader opens big.log via Playwright _electron: first rows visible ≤3 s, renderer process memory ≤600 MB, scroll to 90% shows correct line numbers; search 'ERROR 0x7f' returns 3 hits with correct lines. No renderer crash events.

Proposed partial credit

30% opens without crash; 25% time and memory limits; 25% scroll correctness; 20% search

R02.05.01.02-T17 Add a menu bar extra and Settings scene to a SwiftUI app

Original instruction

Extend fixtures/FocusTimer (macOS SwiftUI) with a MenuBarExtra showing the remaining time and Start/Pause/Reset items, and a Settings scene (Cmd+,) with work/break durations stored in UserDefaults. The menu bar title must update every second while running.

Platforms
mac · macOS
Applications
Xcode · SwiftUI
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Hidden XCUITest clicks the status item, starts the timer, samples the title over 3 s (decreasing by 1 s steps), pauses, resets. Settings window opens via Cmd+, and changed durations persist in defaults read output.

Proposed partial credit

35% menu bar extra and actions; 25% live title updates; 25% settings persist; 15% shortcut

R02.05.01.02-T18 Remove UI-thread freezes from a WPF import screen

Original instruction

Importing input/orders_200k.csv in fixtures/OrderDesk (WPF) freezes the window for ~40 s. Use the Visual Studio profiler to find the blocking work, move it off the UI thread with progress reporting and cancellation, and keep the DataGrid virtualized.

Platforms
win · Windows
Applications
Visual Studio · WPF
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click vis · Visual judgment
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Grader runs a hidden FlaUI test that starts the import and pings the window every 100 ms via UIA: max response latency ≤150 ms during import; progress bar advances; Cancel stops within 1 s leaving 0 partial rows; full import gives 200,000 rows.

Proposed partial credit

40% responsiveness; 20% progress; 20% cancellation; 20% correct row count

R02.05.01.02-T19 Draw a custom chart widget to match reference renders

Original instruction

Implement SparkBarWidget in fixtures/qtdash (Qt 6 Widgets, QPainter) to match input/ref/*.png: rounded bars, baseline, min/max labels, hover tooltip with value, and HiDPI-correct rendering. Use the 5 datasets in input/data/ for the reference cases.

Platforms
mac · macOS win · Windows lin · Linux
Applications
Qt Creator · Qt Widgets
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment hover · Hover & context
Work product
WP06 · Code/engineering
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Grader renders the widget offscreen at 1x and 2x for 5 datasets: SSIM ≥0.92 vs references; hover at bar centers produces QToolTip text equal to value. No external chart library linked (CMake check).

Proposed partial credit

50% SSIM at 1x/2x; 20% labels; 20% hover tooltips; 10% no external chart lib

R02.05.01.02-T20 Ship an auto-update flow for an Electron app against a test server

Original instruction

Add electron-updater to fixtures/notely so a 1.0.0 build checks the local test update server (fixtures/update-server), downloads 1.0.1, verifies its signature, prompts 'Restart to update', and installs on restart, on both macOS and Windows test VMs. A tampered package must be rejected.

Platforms
mac · macOS win · Windows
Applications
Electron · electron-builder
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands click · Point & click multi · Cross-app
Work product
WP06 · Code/engineering
Conditions
long · Long-horizon risk · Irreversible/risky
Proposed difficulty
5 / 5

Proposed verifier

On each VM, grader installs 1.0.0, triggers check, accepts the dialog: app relaunches reporting version 1.0.1 (app.getVersion via IPC). With fixtures/tampered-1.0.1 served, update is refused and 1.0.0 remains. Uses only the local test server.

Proposed partial credit

20% updater wired; 30% macOS update; 30% Windows update; 20% tampered package rejected

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

In this workflow family