Computer-Use Agent Taxonomy中文
R01.11.03.01 · TASK CLASS

Watch, foldable & tablet interactions

Complete tasks on round watch faces, foldable and multi-pane tablet layouts.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Wear OS emulator · watchOS Simulator · iPad Simulator
Class GUI-need label
G3 · GUI-only

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R01.11.03.01-T01 Wear OS: start a 5-minute timer using rotary input

Original instruction

On the Wear OS 5 round emulator (Large Round), open the Timer app and set a custom 5:00 timer using the rotating crown (rotary input) rather than the preset chips, then start it.

Platforms
and · Android
Applications
Wear OS emulator · Timer
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
touch · Touch gestures rt · Real-time control
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Grader (adb to watch AVD) dumpsys of the timer service shows a running 300 s timer; input log contains rotary scroll events and no preset-chip taps.

Proposed partial credit

60% 5:00 timer running; 40% rotary route used.

R01.11.03.01-T02 Wear OS: add the Weather tile and move it to first position

Original instruction

On the Wear OS emulator, add the Weather tile to the tile carousel and reorder tiles so Weather is first, keeping the others in their existing order.

Platforms
and · Android
Applications
Wear OS emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Tile service registry (dumpsys / tiles provider) lists Weather at index 0; other tiles keep relative order. adb and root shell are withheld from the agent (screen input only), so the tile registry cannot be changed by command; the tile must be added on the watch.

Proposed partial credit

50% Weather tile added; 30% first position; 20% others' order kept.

R01.11.03.01-T03 Wear OS: reply 'On my way' to a message notification

Original instruction

An SMS from 'Sam' ('Where are you?') arrives on the paired phone and appears on the Wear OS emulator. From the watch notification, send the canned reply 'On my way'.

Platforms
and · Android
Applications
Wear OS emulator · Messages
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
xdevice · Cross-device risk · Irreversible/risky
Proposed difficulty
2 / 5

Proposed verifier

Paired phone AVD mmssms.db has one outgoing SMS to Sam's number with body 'On my way'; reply originated from the watch (RemoteInput log on watch).

Proposed partial credit

40% notification opened on watch; 60% exact reply sent once.

R01.11.03.01-T04 Wear OS: enable bedtime mode and theater mode from quick settings

Original instruction

On the Wear OS emulator, swipe down to Quick Settings and turn on Bedtime mode, then turn on Theater mode. Battery saver must stay off.

Platforms
and · Android
Applications
Wear OS emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Watch settings: bedtime mode on (zen/bedtime flag), theater_mode_on=1, low_power=0. adb and root shell are withheld from the agent (screen input only), so settings put is unavailable; modes must be toggled in the watch's quick settings.

Proposed partial credit

40% bedtime on; 40% theater on; 20% battery saver off.

R01.11.03.01-T05 watchOS Simulator: set a weekday 6:15 alarm with the Digital Crown

Original instruction

On the Apple Watch Series 10 (46 mm) Simulator paired with an iPhone Simulator, open Alarms, add an alarm at 6:15 AM using the Digital Crown to set hours and minutes, set repeat to weekdays, label 'Run', and save.

Platforms
ios · iOS/iPadOS
Applications
watchOS Simulator · Alarms
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures rt · Real-time control
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Grader XCUITest accessibility snapshot of the watch Alarms list shows 6:15 AM 'Run' weekdays enabled; paired phone Clock alarm store mirrors it. Host shell and xcrun simctl are withheld from the agent, so the alarm store cannot be edited; the alarm must be set with the Digital Crown.

Proposed partial credit

50% time correct; 25% repeat weekdays; 25% label.

R01.11.03.01-T06 watchOS: edit a Modular face to put Calendar in the top-left complication

Original instruction

On the watchOS Simulator, edit the active Modular watch face: set the top-left complication to Calendar, the large centre complication to Weather Conditions, and change the colour to Blue.

Platforms
ios · iOS/iPadOS
Applications
watchOS Simulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures rt · Real-time control
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Watch face configuration read from the simulator's face library (NanoTimeKit face JSON in the simulator data dir): slots and colour as specified. Host shell and xcrun simctl are withheld from the agent, so the face library JSON cannot be edited; the face must be edited on the watch.

Proposed partial credit

40% top-left Calendar; 40% centre Weather; 20% colour.

R01.11.03.01-T07 Foldable AVD: start a form folded, unfold, and finish without data loss

Original instruction

On the Pixel Fold AVD (folded), open FormDemo 'Shipping' and fill name and street. Unfold the device (emulator posture control), then complete city, postcode and country on the large screen and submit.

Platforms
and · Android
Applications
FormDemo · Pixel Fold emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

FormDemo backend receives one submission with all 5 fields exact; window log shows a posture change from folded to open mid-form. adb and root shell are withheld from the agent (screen input only) and FormDemo's backend accepts only the app session, so fields must be typed in the app; posture changes via the emulator toolbar.

Proposed partial credit

30% fields entered before unfold kept; 50% submission exact; 20% posture transition happened.

R01.11.03.01-T08 Foldable: tabletop posture with VLC controls on the lower half

Original instruction

On the Pixel Fold AVD, play Movies/demo.mp4 in VLC, set the device to half-opened tabletop posture, and ensure the video occupies the top half while controls are on the bottom half; pause at 00:30 (±2 s).

Platforms
and · Android
Applications
VLC · Pixel Fold emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Emulator posture = half-opened, hinge horizontal; VLC window layout info (WindowManager FoldingFeature log) shows video above the fold; media_session paused at 30±2 s. adb shell and the emulator console (adb emu/telnet posture commands) are not available to the agent, so posture must be set from the emulator's extended-controls panel.

Proposed partial credit

40% tabletop posture; 30% video layout above fold; 30% paused at time.

R01.11.03.01-T09 Unfolded foldable: drag a photo from Files into a Mail draft

Original instruction

On the unfolded Pixel Fold AVD, put Files and MailDemo side by side and drag Pictures/plan.png from Files into a new MailDemo draft to boss@example.test with subject 'Plan', then save the draft (do not send).

Platforms
and · Android
Applications
Files · MailDemo · Pixel Fold emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop multi · Cross-app
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app
Proposed difficulty
3 / 5

Proposed verifier

Mail server Drafts has one message with subject Plan and an attachment md5 equal to plan.png; Sent is empty; drag-and-drop event observed (ClipData via DragEvent log).

Proposed partial credit

30% split view set up; 50% attachment via drag; 20% draft saved not sent.

R01.11.03.01-T10 Tablet taskbar: build a split pair, swap sides, save as app pair

Original instruction

On the Pixel Tablet AVD (Android 15, landscape), drag Chrome and Calendar from the taskbar into split screen, swap their sides so Calendar is left, set the divider to about 1/3 for Calendar, and save as an app pair.

Platforms
and · Android
Applications
Android launcher · Pixel Tablet emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

dumpsys window: Calendar left with width ratio 0.28–0.38; favorites.db contains app pair item with both components. adb and root shell are withheld from the agent (screen input only), so am/wm windowing commands and favorites.db edits are unavailable; the pair must be built from the taskbar.

Proposed partial credit

30% split created from taskbar; 20% sides swapped; 25% ratio; 25% app pair saved.

R01.11.03.01-T11 iPad Stage Manager: three windows in one stage with Notes at 1/3 width

Original instruction

On the iPad Pro 13-inch Simulator with Stage Manager on, group Safari, Notes and Files in one stage, resize Notes to about one-third of the screen width (±5%), and leave Safari as the largest window.

Platforms
ios · iOS/iPadOS
Applications
iPadOS Stage Manager · Safari · Notes · Files
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

XCUITest snapshot of SpringBoard scene frames: all three windows in the active stage; Notes width / screen width in [0.28, 0.38]; Safari area largest. Host shell, Xcode and xcrun simctl are withheld from the agent, so windows can only be arranged with Stage Manager gestures.

Proposed partial credit

40% three apps in one stage; 35% Notes width; 25% Safari largest.

R01.11.03.01-T12 iPad Split View: drag a photo from Photos into a Notes note

Original instruction

On the iPad Air Simulator, open Photos and Notes in Split View and drag IMG_0045 from Photos into the note 'Site visit' below its second paragraph.

Platforms
ios · iOS/iPadOS
Applications
Photos · Notes · iPadOS multitasking
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop multi · Cross-app
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app
Proposed difficulty
3 / 5

Proposed verifier

NoteStore.sqlite: note 'Site visit' has an image attachment matching IMG_0045 (pHash) positioned after paragraph 2; Split View active in snapshot. Host shell and xcrun simctl are withheld from the agent, so NoteStore.sqlite cannot be edited; the photo must be dragged across Split View.

Proposed partial credit

30% split view; 50% image attached; 20% correct position.

R01.11.03.01-T13 Tablet two-pane mail: triage ten messages without leaving landscape

Original instruction

On the Pixel Tablet AVD in landscape, in Thunderbird (Android) two-pane view, move the 6 messages from 'ci@builds.test' to folder 'CI', flag the 2 from 'cfo@corp.test', and mark the remaining 2 as read.

Platforms
and · Android
Applications
Thunderbird for Android · Pixel Tablet emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

IMAP state: CI folder has the 6 messages; CFO messages flagged; remaining 2 seen; orientation never left landscape (window log). adb is withheld and the agent has no IMAP credentials (account pre-provisioned in Thunderbird), so triage must be done in the app.

Proposed partial credit

40% CI moves; 30% flags; 20% read state; 10% landscape kept.

R01.11.03.01-T14 Wear OS: change watch face and set its complications

Original instruction

On the Wear OS emulator, switch to the 'Analog' watch face from the face picker, then set its complications to Steps (top) and Battery (bottom).

Platforms
and · Android
Applications
Wear OS emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Watch face service state (dumpsys wallpaper/watchface): active face component Analog; complication slot providers = steps and battery. adb and root shell are withheld from the agent (screen input only), so watch-face and complication commands are unavailable; both must be set on the watch.

Proposed partial credit

40% face switched; 30% top complication; 30% bottom complication.

R01.11.03.01-T15 Wear OS: stopwatch with three laps about 10 seconds apart

Original instruction

On the Wear OS emulator, start the Stopwatch, record laps at approximately 10 s, 20 s and 30 s (each within ±0.5 s), then stop it at 35 s (±1 s).

Platforms
and · Android
Applications
Wear OS emulator · Stopwatch
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures rt · Real-time control
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Stopwatch state (app prefs/dumpsys): 3 laps with cumulative times within ±0.5 s of targets; stopped total 34–36 s. adb and root shell are withheld from the agent (screen input only), so run-as pref edits are unavailable; laps must be tapped on the watch.

Proposed partial credit

60% three laps within ±0.5 s (20 each); 20% stop time; 20% stopwatch state saved.

R01.11.03.01-T16 Foldable: take a self-timer photo in tabletop posture

Original instruction

On the Pixel Fold AVD, open Camera, set the device to tabletop posture, set the 3-second timer and take one photo of the virtual scene with the rear camera.

Platforms
and · Android
Applications
Camera · Pixel Fold emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

New JPEG in DCIM/Camera; camera prefs timer=3; posture log shows half-opened at capture time (photo timestamp falls within tabletop interval). adb and root shell are withheld from the agent (screen input only), so camera intents and file pushes are unavailable; the photo must be taken in the Camera app.

Proposed partial credit

40% photo taken; 30% timer 3 s; 30% tabletop posture at capture.

R01.11.03.01-T17 Wear OS: reply with free text using the round-screen keyboard

Original instruction

A message from 'Lee' arrives on the Wear OS emulator. Reply with the free text 'Running 10 min late' using the on-watch keyboard or handwriting input (voice input is disabled), not a canned reply.

Platforms
and · Android
Applications
Wear OS emulator · Messages
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures type · Text & forms path · Continuous path
Work product
WP08 · App/system state
Conditions
xdevice · Cross-device risk · Irreversible/risky
Proposed difficulty
4 / 5

Proposed verifier

Paired phone mmssms.db: outgoing SMS to Lee with body exactly 'Running 10 min late' (case-sensitive); one message only. adb and root shell are withheld from the agent (screen input only), so SMS cannot be injected via content/am commands; the reply must be typed on the watch keyboard.

Proposed partial credit

30% reply initiated; 70% exact text sent once (35 pts if ≤2 char edits off).

R01.11.03.01-T18 Wear OS + phone: change watch notification settings from the companion app

Original instruction

Using the Pixel Watch companion app on the paired phone AVD, stop MailDemo notifications from showing on the watch while keeping them on the phone, then confirm on the watch that a test MailDemo notification no longer appears.

Platforms
and · Android
Applications
Pixel Watch app · Wear OS emulator · MailDemo
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll multi · Cross-app
Work product
WP08 · App/system state
Conditions
xdevice · Cross-device multiapp · Multi-app
Proposed difficulty
5 / 5

Proposed verifier

Companion bridging config shows MailDemo muted for the watch; grader posts a test notification: present on phone (dumpsys notification), absent on watch. adb shell on both AVDs (cmd notification, settings put) is not available to the agent, so the watch setting must be changed in the Pixel Watch app.

Proposed partial credit

50% bridging setting changed; 30% phone still receives; 20% watch suppressed in test.

R01.11.03.01-T19 Foldable expense flow across fold changes with a receipt attachment

Original instruction

On the Pixel Fold AVD, start ExpenseDemo's 'New claim' folded and enter merchant and amount; unfold and in split screen with Gallery drag receipt_44.jpg into the claim; fold again, choose category 'Travel' and submit. State must survive each posture change.

Platforms
and · Android
Applications
ExpenseDemo · Gallery · Pixel Fold emulator
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop type · Text & forms multi · Cross-app
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

ExpenseDemo backend: one claim with merchant, amount, category exact and attachment md5 = receipt_44.jpg; posture log shows folded→open→folded during the claim. adb and root shell are withheld from the agent (screen input only), and ExpenseDemo's backend accepts claims only from the app session (no API token issued).

Proposed partial credit

25% fields kept across posture changes; 35% attachment via drag; 20% category; 20% single submission.

R01.11.03.01-T20 Wear OS: rotary scroll to the 37th contact and start a call

Original instruction

On the Wear OS emulator (LTE, contacts synced), open Contacts, scroll with the rotating crown to 'Marta Quintero' (37th of 80, no search or index), and start a call (sandbox telephony).

Platforms
and · Android
Applications
Wear OS emulator · Contacts
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
rt · Real-time control touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Emulator telephony log shows an outgoing call to Marta's number from the watch; rotary scroll events present; no search activity. adb and root shell are withheld from the agent (screen input only), so am start CALL intents are unavailable; the call must be placed from the watch Contacts list.

Proposed partial credit

40% reached correct contact; 60% call placed to correct number.

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.