Computer-Use Agent Taxonomy中文
R01.11.02.02 · TASK CLASS

Long-press, selection & drag reordering

Use long-press menus, multi-select, text selection and drag to reorder lists or move items.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Tasks · Files · Launcher · Notes
Class GUI-need label
G3 · GUI-only

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R01.11.02.02-T01 Tasks app: drag 'Pay rent' to the top and 'Gym' to the bottom

Original instruction

In the Tasks.org app list 'Home' (9 tasks, manual sort), drag 'Pay rent' to the first position and 'Gym' to the last, keeping the relative order of the other seven tasks.

Platforms
and · Android
Applications
Tasks
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Tasks DB manual order field: Pay rent first, Gym last, remaining seven in original relative order; no tasks edited or completed. adb and root shell are withheld from the agent (screen input only), so run-as DB edits are unavailable; tasks must be dragged in the list.

Proposed partial credit

40% Pay rent first; 40% Gym last; 20% others' order kept.

R01.11.02.02-T02 Android Files grid: drag a PDF into the Archive folder

Original instruction

In the Files app grid view of Documents, long-press report_2025.pdf and drag it onto the 'Archive' folder icon to move it there. Do not use the overflow 'Move to' menu.

Platforms
and · Android
Applications
Files
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP09 · Files & folders
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Documents/Archive/report_2025.pdf exists with original md5 and is gone from Documents; input log shows a drag gesture ending on the folder and no 'Move to' activity.

Proposed partial credit

70% file moved intact; 30% drag route used.

R01.11.02.02-T03 Calendar day view: drag an event from 10:00 to 14:00

Original instruction

In Simple Calendar day view for tomorrow, long-press 'Sync with Ops' (10:00–10:45) and drag it to start at 14:00, keeping its 45-minute duration and all other events unchanged.

Platforms
and · Android
Applications
Simple Calendar
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
touch · Touch gestures drag · Drag & drop geom · Precise geometry
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Calendar DB: dtstart 14:00, dtend 14:45 same day; other events identical to snapshot. adb and root shell are withheld from the agent (screen input only), so content update on the Calendar provider is unavailable; the event must be dragged in day view.

Proposed partial credit

50% start 14:00; 30% duration kept; 20% others unchanged.

R01.11.02.02-T04 Music player: reorder a 12-track playlist to a target order

Original instruction

In Retro Music, reorder playlist 'Run' (12 tracks) to the order listed in the Markor note run_order.md by dragging track handles. Do not remove or re-add tracks.

Platforms
and · Android
Applications
Retro Music · Markor
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app
Proposed difficulty
3 / 5

Proposed verifier

Playlist table order equals the note order; track row IDs unchanged (no delete/re-add). Score by Kendall tau distance. adb and root shell are withheld from the agent (screen input only), so run-as DB edits are unavailable; tracks must be dragged in the playlist.

Proposed partial credit

Score = 1 − normalised Kendall tau distance (90%); 10% no re-added rows.

R01.11.02.02-T05 Long-press select a full sentence and paste it into an SMS

Original instruction

In Markor note meeting.md, long-press inside paragraph 2, extend the selection handles to cover exactly the sentence that begins 'The vendor will', copy it, and send it as an SMS to Alex (+1-555-0110).

Platforms
and · Android
Applications
Markor · Messages
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop multi · Cross-app
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app risk · Irreversible/risky
Proposed difficulty
3 / 5

Proposed verifier

mmssms.db outgoing SMS to +15550110 whose body equals the target sentence exactly (including final period, no leading/trailing extra text). adb and root shell are withheld from the agent (screen input only), so content insert and am send intents are unavailable; the SMS must be built by copy-paste.

Proposed partial credit

30% SMS sent to Alex; 70% body equals exact sentence.

R01.11.02.02-T06 Gallery: drag-select 15 photos across rows and add them to an album

Original instruction

In Gallery's grid, long-press the first photo from 2026-09-20 and drag across to select all 15 photos from that day (they span 4 rows), then add them to a new album 'Lake'.

Platforms
and · Android
Applications
Gallery
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop vis · Visual judgment
Work product
WP09 · Files & folders
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Album Lake membership equals the set of photos with DATE_TAKEN on 2026-09-20; no other photos added. adb and root shell are withheld from the agent (screen input only), so MediaStore and DB edits are unavailable; photos must be drag-selected in Gallery.

Proposed partial credit

Score = Jaccard(album, target) (90%); 10% album named correctly.

R01.11.02.02-T07 Long-press Chrome icon to open a new incognito tab

Original instruction

From the home screen, long-press the Chrome icon and use its 'New Incognito tab' shortcut, then load http://fixtures.local/hello.html in that tab.

Platforms
and · Android
Applications
Android launcher · Chrome
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Chrome DevTools protocol (grader-side) lists an incognito tab with the URL; launch intent recorded as the app shortcut. adb and root shell are withheld from the agent (screen input only), so am start intents and DevTools forwarding are unavailable; the tab must come from the icon shortcut.

Proposed partial credit

60% incognito tab via shortcut; 40% URL loaded.

R01.11.02.02-T08 Delete one specific message from a 40-message thread

Original instruction

In Messages, open the thread with 'Dana' (40 messages). Long-press and delete only the message 'Door code is 4471' sent on 3 Oct; keep all others.

Platforms
and · Android
Applications
Messages
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
risk · Irreversible/risky
Proposed difficulty
3 / 5

Proposed verifier

mmssms.db: exactly that row removed; all other 39 rows unchanged. Deleting the whole thread or another message scores 0. adb and root shell are withheld from the agent (screen input only), so content delete on mmssms is unavailable; the message must be deleted in the thread.

Proposed partial credit

70% correct message deleted; 30% nothing else deleted.

R01.11.02.02-T09 Markor: select a span across lines with handles and bold it

Original instruction

In Markor note draft.md, select exactly the text from 'cost overrun' (line 4) through 'the board' (line 6) using long-press and handle drags, then apply bold from the toolbar.

Platforms
and · Android
Applications
Markor
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP01 · Document
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

File content equals original with '**' inserted exactly around the specified span; no other edits (diff check). adb and root shell are withheld from the agent (screen input only), so file pushes and run-as edits are unavailable; the span must be selected with handles.

Proposed partial credit

40% bold applied; 60% span boundaries exact.

R01.11.02.02-T10 Nextcloud Deck mobile: move and reorder cards across columns

Original instruction

In Nextcloud Deck (Android) board 'Release 4.2', drag cards 'API freeze' and 'Docs pass' from To do to In progress, put 'Docs pass' above 'Perf tests' there, and move 'Hotfix 4.1.3' to Done at the top.

Platforms
and · Android
Applications
Nextcloud Deck
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Nextcloud Deck API (grader) returns stack/order for each card as specified; other cards unchanged. adb is withheld and the agent holds no Nextcloud app password or API credentials (the app login is pre-provisioned), so cards must be moved in Deck.

Proposed partial credit

30% two cards in In progress; 30% order there; 30% hotfix in Done top; 10% others untouched.

R01.11.02.02-T11 iOS Reminders: drag to reorder and nest subtasks

Original instruction

In Reminders on the iOS 18 Simulator, list 'Trip': drag 'Book hotel' to the top, and make 'Passport' and 'Adapter' subtasks of 'Pack' by dragging each onto it.

Platforms
ios · iOS/iPadOS
Applications
Reminders
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Reminders store (CloudKit-less local SQLite) shows Book hotel first in display order and parent of Passport/Adapter = Pack. Host shell and xcrun simctl are withheld from the agent, so the Reminders store cannot be edited; items must be dragged in Reminders.

Proposed partial credit

40% Book hotel first; 60% Passport and Adapter nested under Pack (30 each).

R01.11.02.02-T12 iOS Notes: select the third sentence and turn it into a checklist

Original instruction

In Notes on the iOS 18 Simulator, open note 'Ideas', select exactly the third sentence of the first paragraph by double-tap and handle drag, cut it, and paste it as a new checklist item at the end of the note.

Platforms
ios · iOS/iPadOS
Applications
Notes
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP01 · Document
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

NoteStore.sqlite: paragraph 1 no longer contains the sentence; last paragraph is a checklist item with exactly that text. Host shell and xcrun simctl are withheld from the agent, so NoteStore.sqlite cannot be edited; the sentence must be moved in Notes.

Proposed partial credit

40% sentence removed cleanly; 40% checklist item text exact; 20% rest of note unchanged.

R01.11.02.02-T13 Calendar: long-press multi-select and delete five 'Standup' events

Original instruction

In Simple Calendar's agenda view, long-press to multi-select the five 'Standup' events in the week of 9 Nov 2026 and delete them, leaving Standups in other weeks.

Platforms
and · Android
Applications
Simple Calendar
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
risk · Irreversible/risky
Proposed difficulty
3 / 5

Proposed verifier

Calendar DB: zero Standup rows in 9–15 Nov; all other rows identical. Deleting a recurring series root scores 0. adb and root shell are withheld from the agent (screen input only), so content delete on the Calendar provider is unavailable; events must be multi-selected in the app.

Proposed partial credit

70% five target events removed; 30% no collateral deletions.

R01.11.02.02-T14 Collabora Office Android: select a cell range with handles and paste to new sheet

Original instruction

Open Documents/budget.ods in Collabora Office on Android. Long-press B2 and drag the selection handle to D9, copy, add sheet 'Q4', and paste at A1.

Platforms
and · Android
Applications
Collabora Office
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Saved ODS: sheet Q4 A1:C8 equals source B2:D9 values and formulas (relative references adjusted); source sheet unchanged. adb and root shell are withheld from the agent (screen input only), so host file pushes are unavailable; the range must be copied with handles and saved in Collabora.

Proposed partial credit

30% sheet added; 50% exact range pasted; 20% saved and source intact.

R01.11.02.02-T15 Joplin mobile: drag notes between notebooks and set custom order

Original instruction

In Joplin (Android) with custom sort enabled, drag notes 'Q3 plan' and 'Hiring' into notebook 'Archive', then reorder notebook 'Active' so 'Roadmap' is first and 'Retro' second.

Platforms
and · Android
Applications
Joplin
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Joplin SQLite: parent_id of both notes = Archive id; 'order' field in Active puts Roadmap then Retro first; other notes unchanged. adb and root shell are withheld from the agent (screen input only), so run-as SQLite edits are unavailable; notes must be moved and ordered in Joplin.

Proposed partial credit

50% notes moved; 40% custom order; 10% no other changes.

R01.11.02.02-T16 Chrome: long-press select three quotes and paste them in order

Original instruction

In Chrome open /essay.html (2,000 words). Using long-press selection only, copy the three sentences marked with superscripts [1], [2], [3] (each starts mid-paragraph) and paste them in order into Markor quotes.md as a numbered list.

Platforms
and · Android
Applications
Chrome · Markor
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop multi · Cross-app
Work product
WP01 · Document
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

quotes.md parses as 3 list items whose text equals the target sentences exactly (superscript markers excluded); clipboard history shows three copy events from Chrome. adb and root shell are withheld from the agent (screen input only), so file pushes are unavailable; quotes must be copied in Chrome and pasted into Markor.

Proposed partial credit

90% three quotes exact (30 each); 10% list format and order.

R01.11.02.02-T17 Reorder a 30-item list with auto-scroll to match a target

Original instruction

In Tasks list 'Backlog' (30 items, manual order, 7 visible per screen), reorder items to match the order in fixture image backlog_target.png (shown in Gallery). Long drags must auto-scroll; there is no move-to-position menu.

Platforms
and · Android
Applications
Tasks · Gallery
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop nav · Pan/zoom/scroll vis · Visual judgment
Work product
WP08 · App/system state
Conditions
long · Long-horizon multiapp · Multi-app
Proposed difficulty
5 / 5

Proposed verifier

Tasks DB order vs target: score = 1 − normalised Kendall tau; no tasks created, deleted or edited. adb and root shell are withheld from the agent (screen input only), so run-as DB edits are unavailable; items must be dragged with auto-scroll.

Proposed partial credit

Score = 1 − normalised Kendall tau distance (90%); 10% no edits.

R01.11.02.02-T18 Mail replica: long-press multi-select nine invoices across scroll and label

Original instruction

In MailDemo inbox (120 messages), long-press to start selection and select every message from billing@vendor.test with 'Invoice' in the subject (9, spread through the list), then apply label 'AP-2026' and archive them.

Platforms
and · Android
Applications
MailDemo
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures nav · Pan/zoom/scroll
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Mail server state: exactly the 9 messages have label AP-2026 and are archived; no other messages labelled or moved. Search box is disabled in this build.

Proposed partial credit

Score = F1 of selected set (80%); 20% label and archive both applied.

R01.11.02.02-T19 AppFlowy mobile: restructure a page by dragging and nesting blocks

Original instruction

In AppFlowy (Android) page 'Launch plan', drag blocks so sections appear in the order of the outline in the page's first callout, and nest 'Press list' and 'Embargo time' under 'PR' as child blocks by dragging them right.

Platforms
and · Android
Applications
AppFlowy
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop nav · Pan/zoom/scroll
Work product
WP01 · Document
Conditions
long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

Grader exports the page document JSON: top-level block order equals outline; the two blocks are children of PR; no text content changed (hash of each block's text). No adb shell, emulator console or AppFlowy API token is issued and cloud sync is off, so the page can change only through the app's UI session.

Proposed partial credit

50% section order (Kendall); 30% nesting correct; 20% no content edits.

R01.11.02.02-T20 iOS Files: drag three files onto a folder in list view, keep one

Original instruction

In Files › On My iPhone › Scans (list view), pick up scan_01.pdf by long-press, tap scan_03.pdf and scan_04.pdf to add them to the drag stack, and drop the stack on folder 'Filed'. scan_02.pdf stays.

Platforms
ios · iOS/iPadOS
Applications
Files
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP09 · Files & folders
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Simulator file tree: Filed contains exactly scan_01/03/04 with original md5; scan_02 remains in Scans. Host shell and xcrun simctl are withheld from the agent, so host file moves are unavailable; files must be dragged in the Files list view.

Proposed partial credit

90% scan_01/03/04 moved with md5 intact (30 each); 10% scan_02 untouched.

Mapped research references

No benchmark reference is assigned to this class in the current mapping.

In this workflow family