TV & 10-foot interfaces
Navigate TV launchers and streaming apps with D-pad focus.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Android TV emulator · tvOS simulator
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.07.01.02-T01 Turn on large captions with D-pad navigation only
Original instruction
On the Android TV emulator, using only D-pad keys (up/down/left/right/center/back), open Settings and turn on captions with text size 'Large' and caption style 'White on black'.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV Settings
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
adb settings secure: accessibility_captioning_enabled=1, accessibility_captioning_font_scale=1.5 (large), preset id for white-on-black. Input audit: only DPAD/ENTER/BACK keycodes. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
40% captions on; 30% size; 20% style; 10% D-pad-only compliance.
R12.07.01.02-T02 Add an app to the launcher favorites row
Original instruction
On the Google TV-style launcher (Android TV emulator), add the app 'NewsNow' to the Favorites row as the third item, using only remote keys (long-press center opens the move menu).
- Platforms
and· Android- Applications
- Android TV emulator · TV launcher
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Launcher DB/content provider: favorites row contains NewsNow at index 2; other favorites order unchanged. Key-event-only action space. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
60% added to favorites; 30% position; 10% others unchanged.
R12.07.01.02-T03 Set the screensaver to start after five minutes
Original instruction
On the Android TV emulator, using the remote only, set the screen saver (Ambient mode) to start after 5 minutes and set 'Put device to sleep' to 1 hour.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV Settings
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
adb settings: screensaver delay 300000 ms; sleep timeout 3600000 ms. Key-event-only input. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
50% screensaver delay; 50% sleep timeout.
R12.07.01.02-T04 Search a video app and play the second result
Original instruction
On the Android TV emulator, open the 'StreamBox' Leanback app clone, search 'cooking' with the on-screen keyboard, and play the 2nd result in the results row. Report the playing title.
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutstype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
dumpsys media_session: active session title equals 2nd result's title for query 'cooking' (seeded ordering); report matches; no voice-search intent used. adb shell (am start search intents, input text) is not available to the agent, so the query must be typed on the on-screen keyboard.
Proposed partial credit
40% search performed; 40% correct item playing; 20% report.
R12.07.01.02-T05 Enable Spanish subtitles and stereo audio track during playback
Original instruction
In the StreamBox app clone on the Android TV emulator, start the movie 'Harbor Lights', open the player's settings overlay, select Spanish subtitles and the 'English Stereo' audio track (not 5.1), and resume playback.
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ExoPlayer state exported by the app clone: selected text track language=es, audio track id = English stereo, playing=true. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
40% subtitles; 40% audio track; 20% resumed.
R12.07.01.02-T06 Change display resolution and HDR format in TV settings
Original instruction
On the Android TV emulator (TV image with display mode simulation), set Settings > Display & Sound > Advanced display settings to resolution 1080p 60 Hz, disable 'Match content frame rate', and set HDR format selection to 'Manual' with Dolby Vision disabled.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV Settings
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Settings global: user_preferred_resolution 1920x1080@60, match_content_frame_rate=never, hdr format override excludes DOLBY_VISION. Key-only input. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
40% resolution; 30% frame rate setting; 30% HDR selection.
R12.07.01.02-T07 Uninstall an app from TV app management
Original instruction
On the Android TV emulator, uninstall 'PuzzleTV' using Settings > Apps (not adb), and also clear the cache (not data) of 'StreamBox'.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV Settings
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
pm list packages: PuzzleTV absent; StreamBox cache dir empty while its data (login/prefs file) intact. Agent has no adb shell.
Proposed partial credit
50% uninstall; 40% cache cleared; 10% data intact.
R12.07.01.02-T08 Create a kid profile with content restrictions on a TV streaming app
Original instruction
In the StreamBox app clone on the Android TV emulator, create a profile 'Kids' with maturity limit 'TV-Y7', autoplay of previews off, and a 4-digit profile lock for the main profile entered by the user (hand off for the PIN; do not choose one).
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutstype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
App DB: profile Kids with maturity TV-Y7, preview_autoplay=false; main profile lock set after ASK_USER (agent sent no digit keycodes on PIN screen). The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
40% kid profile; 20% autoplay; 40% PIN handoff and lock set.
R12.07.01.02-T09 Rearrange the tvOS home screen top row
Original instruction
On the tvOS Simulator, using only the Siri Remote simulator (arrow keys, select, menu/back, play/pause), move 'TV' app to the first position of the top shelf row and put 'Photos' and 'Music' into a new folder named 'Media'.
- Platforms
ios· iOS/iPadOS- Applications
- tvOS Simulator
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
SpringBoard (PineBoard) icon state plist from simulator data: TV at index 0; folder named 'Media' containing Photos and Music. Input audit: remote events only. The agent sends Siri Remote events only and has no host shell, so simctl and the simulator's icon-state plist are not reachable; the layout must be changed on screen.
Proposed partial credit
40% TV first; 40% folder with 2 apps; 20% folder name.
R12.07.01.02-T10 Continue watching an episode from where another profile left off
Original instruction
In the StreamBox app clone on Android TV, the user's partner watched 'Sea Stories' S2E4 on profile 'Ana' up to 23:10. On the user's profile 'Ben', start S2E4 and seek to 23:10 using the remote's scrub (left/right with acceleration), then pause.
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsav· Audio/video over timevis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Player state on profile Ben: episode S2E4, position 1390 s ±5, paused; Ana's progress unchanged. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
30% correct profile/episode; 50% position; 20% Ana unchanged.
R12.07.01.02-T11 Configure accessibility: screen reader and high-contrast text on TV
Original instruction
On the Android TV emulator, turn on TalkBack, then with TalkBack active (which changes navigation feedback but keeps D-pad focus), turn on high-contrast text and set the speech rate to 1.25x in TalkBack settings. Leave TalkBack on.
- Platforms
and· Android- Applications
- Android TV emulator · TalkBack
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
settings secure: enabled_accessibility_services includes TalkBack; high_text_contrast_enabled=1; TalkBack speech rate pref = 1.25. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
30% TalkBack on; 35% high contrast; 35% speech rate.
R12.07.01.02-T12 Find a movie through nested category rows and add it to a watchlist
Original instruction
In the StreamBox clone (Android TV), without using search, navigate the home rows to 'Genres' > 'Documentaries' > 'Nature' and add the highest-rated title in that row (rating shown on the focused card only) to 'My List'.
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Watchlist contains the seeded top-rated Nature documentary; search activity never launched (activity log); no other watchlist additions. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
50% correct title; 30% no search used; 20% no extra additions.
R12.07.01.02-T13 Set up network proxy and private DNS on an Android TV via remote
Original instruction
On the Android TV emulator, configure the Wi-Fi network 'TestNet' to use a manual proxy 10.0.2.2:3128 with bypass for '*.local', and set Private DNS hostname 'dns.example.test'. Enter all text with the on-screen keyboard via D-pad.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV Settings
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutstype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
WifiConfigStore: proxy host/port/exclusion list exact; settings global private_dns_mode=hostname, specifier exact. Key-only input. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
50% proxy settings; 50% private DNS.
R12.07.01.02-T14 Pair a Bluetooth remote and remap a button on Android TV
Original instruction
On the Android TV emulator with the virtual Bluetooth remote 'Remote-X' in pairing mode (harness), pair it via Settings > Remotes & Accessories, then in the 'Button Mapper' app clone remap its 'Star' button to open the StreamBox app.
- Platforms
and· Android- Applications
- Android TV emulator · Button Mapper clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgmentmulti· Cross-app- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Bonded devices include Remote-X; Button Mapper prefs map KEYCODE_BOOKMARK (star) → launch StreamBox package; harness presses star and StreamBox foregrounds. The agent sends key events only through the harness and has no adb shell, so bluetooth and Button Mapper prefs are not reachable by command; pairing and mapping must be done on screen.
Proposed partial credit
40% paired; 40% mapping saved; 20% functional test.
R12.07.01.02-T15 Install an app from the TV store and sign in with a device code handoff
Original instruction
On the Android TV emulator, install 'PodTV' from the store clone, open it, and choose 'Sign in on another device'. Hand off the displayed device code to the user via ASK_USER (the user completes sign-in on their phone), then set PodTV's default playback speed to 1.25x.
- Platforms
and· Android- Applications
- Android TV emulator · TV store clone · PodTV clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
xdevice· Cross-device- Proposed difficulty
- 3 / 5
Proposed verifier
Package installed; ASK_USER message contains the exact device code shown; after simulated activation, app logged in; prefs default speed 1.25. The agent sends key events only through the harness and has no adb shell, so pm install and app prefs are not reachable; install and settings must be done on screen.
Proposed partial credit
25% installed; 40% correct code handoff; 15% logged in; 20% speed setting.
R12.07.01.02-T16 Navigate a non-standard grid UI with wrapping focus to select a channel
Original instruction
In the live-TV guide app clone on Android TV (EPG grid where focus wraps at row ends and skips disabled cells), set a recording for the program 'City Council Live' airing on channel 41 tonight at 19:00, recording series 'new episodes only'.
- Platforms
and· Android- Applications
- Android TV emulator · EPG app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
DVR schedule DB: series rule for that program, channel 41, new_only=true. Recording a different airing or channel fails. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
50% correct program/channel; 30% series rule; 20% new-only.
R12.07.01.02-T17 Enable tvOS parental controls with restricted ratings and app allowlist
Original instruction
On the tvOS Simulator, turn on Restrictions with a passcode the user provides (hand off for it), allow movies up to PG-13 and TV up to TV-14, disallow in-app purchases, and restrict apps to the 5 listed in fixtures/allowed_apps.txt.
- Platforms
ios· iOS/iPadOS- Applications
- tvOS Simulator
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Restrictions plist (simulator data): enabled; movie rating 300 (PG-13), TV 600; IAP disabled; app allowlist equals file; passcode set after ASK_USER (no agent digit entry).
Proposed partial credit
20% passcode handoff; 30% ratings; 20% IAP; 30% app allowlist.
R12.07.01.02-T18 Reorder streaming-app home rows and hide two categories
Original instruction
In the StreamBox clone's 'Customize home' screen on Android TV, reorder rows to: Continue Watching, My List, New Releases, Comedy, then the rest in original order; hide 'Horror' and 'Reality TV'. Rows are moved with a focus-and-move mode (center to pick up, up/down, center to drop).
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutsdrag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
App prefs: row order equals expected list; hidden set exactly {Horror, Reality TV}. The agent sends key events only through the harness and has no adb shell, so app prefs and content providers are not reachable; changes must be made by remote navigation.
Proposed partial credit
60% row order; 40% hidden rows.
R12.07.01.02-T19 Watch a short clip and answer what appears at a given time with the remote
Original instruction
In the StreamBox clone on Android TV, play 'Kitchen Tips #12', and using only the remote, report the ingredient shown in the on-screen text at 01:47 and the number of eggs used in the recipe (visible on the counter during 02:10-02:30). Give both in your final answer.
- Platforms
and· Android- Applications
- Android TV emulator · Leanback video app clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
key· Shortcutsav· Audio/video over timevis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Final answer: ingredient string exact and egg count exact (ground truth). Media session position log shows playback/seek near those times. adb shell is unavailable and the video file is not reachable from the agent, so both values must be read on screen during playback.
Proposed partial credit
50% ingredient; 50% egg count.
R12.07.01.02-T20 Set up a new TV end to end: account, apps, picture and accessibility
Original instruction
On a factory-reset Android TV emulator image, complete setup with remote only: language English (UK), connect to 'TestNet' (password from fixtures/wifi_card.png), skip Google sign-in, install StreamBox and PodTV from the store clone, set picture mode 'Movie', captions on, and pin StreamBox first in favorites.
- Platforms
and· Android- Applications
- Android TV emulator · Android TV setup · TV store clone
- Interfaces
hmi· Device/HMI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
key· Shortcutstype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Locale en-GB; Wi-Fi TestNet saved/connected; no Google account; both packages installed; picture mode pref Movie; captions enabled; favorites[0]=StreamBox. Key-only input audit. The agent sends key events only through the harness and has no adb shell, so settings put, am start and pm install are not reachable; changes must be made by remote navigation.
Proposed partial credit
35% locale and Wi-Fi; 25% apps installed; 25% picture mode and captions; 15% favorite.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.