Home screen, notifications & quick settings
Arrange icons, folders and widgets; act on and triage notifications; toggle radios and modes.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Android launcher · iOS Springboard · System UI
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R01.11.01.01-T01 Android: create 'Travel' folder of four apps at page 2, first cell
Original instruction
On the Pixel 8 AVD (Android 15, Launcher3), create a home-screen folder named 'Travel' containing exactly Maps, Calendar, Clock and Files, placed on home page 2 at row 1, column 1. Do not change other icons or the dock.
- Platforms
and· Android- Applications
- Android launcher
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader (rooted AVD) reads Launcher3 favorites.db: one folder titled Travel on screen 1 at cellX=0,cellY=0 with exactly those 4 components; all other rows unchanged vs snapshot. adb and root shell are withheld from the agent (screen input only), so favorites.db can only change through launcher gestures.
Proposed partial credit
30% folder created and named; 40% exact members; 20% position; 10% other items untouched.
R01.11.01.01-T02 Android: add and resize a 4x2 clock widget on page 2
Original instruction
On the Android 15 AVD, add the Clock app's digital clock widget to home page 2, resized to exactly 4 columns × 2 rows, with its top-left cell at row 2, column 1. Leave page 1 untouched.
- Platforms
and· Android- Applications
- Android launcher · Clock
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & dropgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
favorites.db: appwidget item with the DeskClock digital provider on screen 1, spanX=4, spanY=2, cellX=0, cellY=1; page 1 rows identical to snapshot. adb and root shell are withheld from the agent (screen input only), so favorites.db and widget bindings can only change through launcher gestures.
Proposed partial credit
40% widget added on page 2; 40% size 4x2; 20% exact position.
R01.11.01.01-T03 Android: swap Chrome into the dock and park Messages on page 1
Original instruction
On the Android 15 AVD, move Chrome into the dock slot currently occupied by Messages, and place Messages on home page 1 at row 4, column 4. No other dock icon may move.
- Platforms
and· Android- Applications
- Android launcher
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
favorites.db: hotseat rank of former Messages slot now holds Chrome; Messages on screen 0 cellX=3 cellY=3; other hotseat items unchanged. adb and root shell are withheld from the agent (screen input only), so favorites.db can only change through launcher gestures.
Proposed partial credit
50% Chrome in correct dock slot; 30% Messages position; 20% other dock items unchanged.
R01.11.01.01-T04 Android: inline-reply to the middle message of a grouped SMS notification
Original instruction
Three SMS from +1-555-0142 ('Lunch?', 'Bring the keys', 'At gate 4') arrive as a grouped notification. Without opening Messages, expand the group and use inline reply on the 'Bring the keys' notification to send 'Got them'. Leave the notifications otherwise as they are.
- Platforms
and· Android- Applications
- System UI · Messages
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
mmssms.db has one outgoing SMS to +15550142 body 'Got them'; dumpsys activity shows Messages was never resumed during the episode; emulator SMS sink receives exactly one message. adb and root shell are withheld from the agent (screen input only), so the reply cannot be injected via content/am commands and must be sent from the notification.
Proposed partial credit
30% notification expanded; 50% correct reply sent once; 20% app not opened.
R01.11.01.01-T05 Android: triage 12 notifications: dismiss promos, snooze invoice, keep ongoing
Original instruction
Twelve notifications from five test apps are posted (ShopDemo promos, MailDemo, BankDemo, Calendar reminder, ongoing Recorder). Dismiss every notification from ShopDemo's 'Promotions' channel, snooze MailDemo's 'Invoice due' for 1 hour, and leave everything else, including the ongoing recording, in the shade.
- Platforms
and· Android- Applications
- System UI
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
dumpsys notification --noredact: active keys equal the expected set; snoozed list contains the invoice key with ~60 min duration (±2 min); no channel settings changed. Notification snooze must be enabled beforehand by fixture. adb and root shell are withheld from the agent (screen input only), so cmd notification cannot dismiss or snooze; triage must happen in the shade.
Proposed partial credit
40% promos dismissed exactly; 30% invoice snoozed 1 h; 30% others untouched.
R01.11.01.01-T06 Android: from a notification, silence one channel and block another
Original instruction
A ShopDemo 'Promotions' notification is in the shade. Using its long-press controls and the app's notification settings, turn off only the 'Promotions' channel and set 'Order updates' to Silent, leaving 'Delivery alerts' at its current importance.
- Platforms
and· Android- Applications
- System UI · Android Settings
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesclick· Point & click- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
dumpsys notification: channel importance Promotions=NONE, Order updates=LOW, Delivery alerts unchanged; app-level notifications still enabled. adb and root shell are withheld from the agent (screen input only), so channel settings cannot be changed by shell commands; they must be changed from the notification's UI.
Proposed partial credit
40% promotions off; 40% order updates silent; 20% others unchanged.
R01.11.01.01-T07 Android: reorder Quick Settings tiles and remove one
Original instruction
Edit the Quick Settings panel so the first four tiles are, in order, Internet, Bluetooth, Flashlight, Do Not Disturb, and remove the Nearby Share tile. Keep all other tiles in their existing relative order.
- Platforms
and· Android- Applications
- System UI
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
settings get secure sysui_qs_tiles: first four entries internet,bt,flashlight,dnd; 'nearby' absent; remaining tiles keep relative order vs snapshot. adb shell (and so `settings put`) is not available to the agent, which acts only through the emulator screen, so the tile list can change only in the Quick Settings edit panel.
Proposed partial credit
50% first four tiles in order; 25% nearby removed; 25% relative order of rest kept.
R01.11.01.01-T08 Android: airplane mode on with only Bluetooth re-enabled
Original instruction
From the Quick Settings shade only, turn on airplane mode, then re-enable Bluetooth while airplane mode stays on and Wi-Fi stays off. Also enable Battery Saver.
- Platforms
and· Android- Applications
- System UI
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
touch· Touch gestures- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
settings global: airplane_mode_on=1, bluetooth_on=1, wifi_on=0, low_power=1. Settings app must not appear as resumed activity in the episode log. adb and root shell are withheld from the agent (screen input only), so settings put and cmd connectivity are unavailable; toggles must be set in Quick Settings.
Proposed partial credit
30% airplane on; 30% bluetooth on; 20% wifi off; 20% battery saver.
R01.11.01.01-T09 Android: weekday 'Sleep' Do Not Disturb schedule with exceptions
Original instruction
Create a Do Not Disturb schedule named 'Sleep' active 22:30–06:45 Monday–Friday that allows calls only from starred contacts, allows repeat callers, allows alarms, and blocks all app notifications except Clock. Leave other DND rules unchanged.
- Platforms
and· Android- Applications
- Android Settings · System UI
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
dumpsys notification zen config: automatic rule 'Sleep' with schedule days 2–6, start 22:30, end 06:45; policy calls=starred, repeatCallers=true, alarms=true; app bypass list={Clock}. Other rules identical to snapshot. adb and root shell are withheld from the agent (screen input only), so zen rules cannot be written by shell commands; the schedule must be built in Settings.
Proposed partial credit
30% rule with name and days; 20% times; 30% call/alarm policy; 20% app exceptions.
R01.11.01.01-T10 iOS Simulator: 'Planning' folder and Maps in the dock
Original instruction
On the iPhone 16 Simulator (iOS 18), create a folder 'Planning' containing Calendar, Reminders and Contacts in the bottom row of page 1, and move Maps into the dock replacing Music. Do not delete any app.
- Platforms
ios· iOS/iPadOS- Applications
- iOS Springboard
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader reads the simulator's SpringBoard IconState.plist: folder 'Planning' with exactly those bundle IDs on page 1 last row; dock contains Maps, not Music; Music still on a home page; app count unchanged. Host shell and xcrun simctl are withheld from the agent, so the simulator's IconState.plist can only change through SpringBoard gestures.
Proposed partial credit
30% folder with name; 30% exact members and row; 30% dock swap; 10% no apps removed.
R01.11.01.01-T11 iOS: medium Calendar widget plus a Smart Stack on page 1
Original instruction
On the iOS 18 Simulator, add a medium Calendar 'Up Next' widget at the top of home page 1 and, below it, a Smart Stack containing Reminders and Weather widgets (small size). Move displaced icons to page 2 without deleting any.
- Platforms
ios· iOS/iPadOS- Applications
- iOS Springboard
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
IconState.plist widget entries: Calendar medium at page 1 index 0; a stack with Reminders and Weather small widgets on page 1; total icon set across pages unchanged. Host shell and xcrun simctl are withheld from the agent, so the simulator's IconState.plist can only change through SpringBoard gestures.
Proposed partial credit
35% Calendar medium widget placed; 40% Smart Stack contents; 25% no icons lost.
R01.11.01.01-T12 iOS: 'Deep Work' Focus with allowed apps and a linked home page
Original instruction
Create a custom Focus 'Deep Work' on the iOS 18 Simulator that allows notifications only from Calendar and Reminders, allows calls from Favorites, hides notification badges, and shows only home page 2 while active. Turn it on.
- Platforms
ios· iOS/iPadOS- Applications
- iOS Settings · iOS Springboard
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader reads simulator DoNotDisturb/Focus store (ModeConfigurations.json) for a mode named Deep Work: allowed apps set, calls=favorites, badges hidden, home page filter = page 2, mode active. Host shell and xcrun simctl are withheld from the agent, so the simulator's Focus store can only change through Settings and SpringBoard.
Proposed partial credit
20% focus created; 30% app allow-list; 20% calls/badges; 30% linked home page and active.
R01.11.01.01-T13 iOS: customize Control Center: add two controls, remove Now Playing
Original instruction
On the iOS 18 Simulator, edit Control Center so it contains Low Power Mode and Screen Recording controls on the first page and no longer contains the Now Playing module. Do not change other controls.
- Platforms
ios· iOS/iPadOS- Applications
- iOS Control Center
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader reads the simulator Control Center configuration plist: low-power and screen-recording modules on page 0, now-playing absent, other module identifiers unchanged. The agent has no host shell or simctl access and cannot reach the simulator's data container, so the plist can change only through the Control Center editor.
Proposed partial credit
35% low power added; 35% screen recording added; 30% now playing removed and rest unchanged.
R01.11.01.01-T14 Android: Simple Calendar month widget 4x3 at top of page 1
Original instruction
Place the Simple Calendar month widget on home page 1 sized 4×3 with top-left at row 1, column 1, moving any displaced icons to page 2 in their original order. The widget must show October 2026.
- Platforms
and· Android- Applications
- Android launcher · Simple Calendar
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
favorites.db: Simple Calendar month provider on screen 0 at 0,0 span 4x3; displaced icons present on screen 1 preserving relative order; widget configured (app prefs show widget id bound). adb and root shell are withheld from the agent (screen input only), so favorites.db and the widget binding can only change through launcher gestures.
Proposed partial credit
40% widget placed; 30% span 4x3; 30% displaced icons preserved.
R01.11.01.01-T15 Android: set home-screen-only wallpaper from Photos
Original instruction
Set /sdcard/Pictures/harbor.jpg as the home screen wallpaper only (lock screen keeps the current image), using the Photos/Gallery 'Use as' or the Wallpaper & style picker.
- Platforms
and· Android- Applications
- Android launcher · Gallery
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gestures- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
dumpsys wallpaper: system wallpaper id changed and its bitmap pHash matches harbor.jpg (cropped allowed); lock wallpaper id unchanged. adb and root shell are withheld from the agent (screen input only), so the wallpaper cannot be set via shell calls or file pushes; it must be set from Photos.
Proposed partial credit
60% home wallpaper set to harbor.jpg; 40% lock wallpaper unchanged.
R01.11.01.01-T16 Android: remove six home icons without uninstalling apps
Original instruction
Remove these icons from home page 1: Camera, Contacts, Files, Phone shortcut duplicate, Markor and VLC, by dragging to 'Remove' (not 'Uninstall'). All six apps must remain installed and the dock unchanged.
- Platforms
and· Android- Applications
- Android launcher
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & drop- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
favorites.db: none of the six items on screen 0; pm list packages (grader) still includes all apps; hotseat unchanged. Any uninstall scores 0. adb and root shell are withheld from the agent (screen input only), so favorites.db can only change through launcher gestures.
Proposed partial credit
60% six icons removed; 40% apps still installed and dock unchanged.
R01.11.01.01-T17 Android: copy a one-time code from a notification into a mock app
Original instruction
MockBank (sandbox app) is on its 2-step screen. A notification from MockBankSMS shows a 6-digit test code. Use the notification's 'Copy' action or read it, enter it in MockBank, and then dismiss only that notification.
- Platforms
and· Android- Applications
- System UI · MockBank
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
MockBank backend log shows successful verification within 60 s with the exact code; dumpsys notification shows the code notification gone and others still present. adb and root shell are withheld from the agent (screen input only), so the code cannot be read via dumpsys and must be copied from the notification.
Proposed partial credit
60% code verified; 20% within time limit; 20% only that notification dismissed.
R01.11.01.01-T18 Android: recover an amount from notification history
Original instruction
Notification history is enabled. Earlier, a BankDemo notification about a card charge at 'Harbor Cafe' was dismissed. Find it in Settings › Notifications › Notification history and write the charged amount (e.g. 12.40) to the Markor note history_answer.md.
- Platforms
and· Android- Applications
- Android Settings · Markor
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrolltype· Text & forms- Work product
WP12· Answer/decision- Conditions
hidden· Hidden/stale statemultiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Markor file content equals the amount in the seeded notification (exact string after trim). The notification was posted with a random amount per seed; BankDemo has no in-app history. adb and root shell are withheld from the agent (screen input only), so dumpsys notification history is unavailable; the amount must be read in Settings.
Proposed partial credit
40% reached notification history; 60% correct amount written.
R01.11.01.01-T19 Android: rebuild a two-page home layout from a reference screenshot
Original instruction
fixtures/target_home.png (shown in Photos) shows a target layout: 2 home pages, 3 folders (names visible), 1 widget and a 5-icon dock. Rearrange the AVD's home screen to match it exactly, creating folders and moving icons as needed.
- Platforms
and· Android- Applications
- Android launcher · Photos
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
long· Long-horizonmultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
favorites.db vs target spec: per-item screen, cell and container, folder names and members, widget span and position, dock order. Score is the fraction of matched items; extra items penalised. adb and root shell are withheld from the agent (screen input only), so favorites.db can only change through launcher gestures.
Proposed partial credit
30% folders and names; 30% icon positions; 20% widget; 20% dock order.
R01.11.01.01-T20 Android: rule-based notification triage under a live stream
Original instruction
Over 3 minutes, 30 notifications arrive. Follow fixtures/rules.md: reply 'Call you at 6' inline to SMS from 'Mom'; tap 'Archive' on MailDemo newsletters; mute the 'Soccer' group conversation for 8 hours; dismiss nothing else. New notifications keep arriving while you work.
- Platforms
and· Android- Applications
- System UI · Messages · MailDemo
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & formsnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
interrupt· Interruptionslong· Long-horizonrisk· Irreversible/risky- Proposed difficulty
- 5 / 5
Proposed verifier
mmssms.db: one reply per Mom message; mail server: newsletters in Archive only; conversation channel for Soccer muted until ~+8 h; active-notification set equals expected remaining set. Extra replies or archives penalised. adb/root shell and mail-server credentials are withheld from the agent (screen input only), so replies, archives and mutes must go through the UI.
Proposed partial credit
30% Mom replies exact; 25% newsletters archived; 25% Soccer muted; 20% no collateral changes.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.