Mobile note & short-video posting
Draft and publish posts with media, tags and settings in mobile apps.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Xiaohongshu · Douyin · Instagram · TikTok
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R11.05.01.02-T01 Save a three-image note draft in a set order
Original instruction
In the Xiaohongshu-like app on the Android emulator (test account), create a note using gallery images B.jpg, A.jpg, C.jpg in that order, title '周末徒步', body from body.txt, and hashtags #徒步 and #周末去哪儿. Save it as a draft; do not publish.
- Platforms
and· Android- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Draft DB (rooted emulator): one draft with image sha256 order B,A,C; title and body exact; topic list equals the 2 hashtags; no published-note record on the server stub. adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
35% image order; 30% title/body; 20% hashtags; 15% not published
R11.05.01.02-T02 Trim a clip to 15 seconds and add a music track
Original instruction
In the Douyin-like app on the Android emulator, start a post from gallery video hike.mp4 (42 s), trim to the segment 00:10–00:25, add library music track 'Morning Run' starting at its beginning with original audio at 0%, and save to drafts.
- Platforms
and· Android- Applications
- Douyin
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropav· Audio/video over time- Work product
WP05· Time-based media- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Draft project JSON: source in/out 10.0–25.0 s ±0.1; music id equals Morning Run with offset 0; original volume 0. Exported draft preview duration 15 s ±0.1. adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
45% trim; 30% music; 15% original muted; 10% saved as draft
R11.05.01.02-T03 Make a 2x2 collage cover with even gutters
Original instruction
In the Xiaohongshu-like app, make a 3:4 cover using the in-app collage template with photos 1–4 from album 'Cafe' in the 2x2 grid order given in cover_spec.txt and 8 px gutters, add title text '城南咖啡地图' on the top, and save the note as draft.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Exported cover image: aspect 3:4; four tiles detected with SSIM ≥0.85 to source crops in correct cells; gutter widths 8±2 px (scaled); OCR finds the title in top 25%. adb/simctl shells and root are denied to the agent, so the collage must be made with the in-app template.
Proposed partial credit
15% aspect; 40% tile order; 25% gutters; 20% title
R11.05.01.02-T04 Post a five-photo carousel with alt text and location to a private account
Original instruction
On the iOS Simulator, in Instagram logged into a private test account, post a carousel of the 5 photos in album 'Workshop' (in shoot order), caption from caption.txt, location 'Science Museum' (test location), alt text for each photo from alt_text.csv, and hide like count.
- Platforms
ios· iOS/iPadOSand· Android- Applications
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Verifier reads the private test account via Graph API (business test user): media type carousel with 5 children in order (image hashes), caption exact, location id, alt_text per child equals CSV, like_and_view_counts_disabled true. Compared with control pair T17. Instagram Graph API tokens are withheld from the agent (the test-user token is grader-only) and adb/simctl shells are denied, so it must be done in the app UI.
Proposed partial credit
30% carousel order; 30% caption and location; 30% alt text; 10% likes hidden
R11.05.01.02-T05 Add timed text, edited captions and a chosen cover in TikTok
Original instruction
On the Android emulator in TikTok (test account), create a post from clip_demo.mp4: add text 'Step 1: mix' visible 2.0–5.0 s, generate auto captions and fix the 2 misrecognized words (see transcript.txt), select the cover frame at 3.0 s, set visibility 'Only you', and post.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- TikTok
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesav· Audio/video over timetype· Text & forms- Work product
WP05· Time-based media- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Downloaded post (owner session): OCR shows overlay text only within 2.0–5.0 s ±0.2; caption track text matches transcript (2 fixes); cover frame SSIM ≥0.9 with frame at 3.0 s; privacy self-only. TikTok API tokens are withheld and adb shell/root are denied to the agent, so the post must be built through touch input in the app.
Proposed partial credit
30% overlay timing; 30% captions fixed; 20% cover; 20% visibility
R11.05.01.02-T06 Assemble a three-clip Reel with transitions and library audio
Original instruction
In Instagram on Android (test account), create a Reel from clips r1, r2, r3 (in that order) trimmed to 10 s each (first 10 s), add a transition between clips, library audio 'Upbeat Test Track', a cover image from the camera roll (reel_cover.jpg), and save as draft.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropav· Audio/video over time- Work product
WP05· Time-based media- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Draft project (rooted device app data): 3 segments with sources/order and 10.0 s ±0.1 durations; transitions present; audio asset id; cover file hash; state draft. Instagram Graph API tokens are withheld from the agent (the test-user token is grader-only) and adb/simctl shells are denied, so it must be done in the app UI.
Proposed partial credit
35% clips and trims; 15% transitions; 25% audio; 25% cover and draft
R11.05.01.02-T07 Schedule a private-visibility note with location and a mention
Original instruction
In Xiaohongshu (test account) on iOS Simulator, create a note from photo note_img.jpg with title and body from note.md, add location '静安寺' (test POI), @mention test account @qa_friend, set visibility 仅自己可见 and 定时发布 at 2026-10-20 20:00.
- Platforms
ios· iOS/iPadOSand· Android- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Test backend record: note with title/body; POI id; at-user list contains qa_friend; visibility private; schedule time 2026-10-20T12:00Z; not yet public. The agent has no simctl/xcrun or terminal access and the test backend issues no API token, so the note must be built in the app UI.
Proposed partial credit
25% content; 35% location and mention; 20% private; 20% schedule
R11.05.01.02-T08 Post with friends-only visibility and restricted interactions
Original instruction
In Douyin (test account) on Android, post video cook.mp4 with description from desc.txt including topics #家常菜 #快手菜 #晚饭吃什么, visibility 好友可见, and turn off 允许下载 and 合拍/抢镜 for this post.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Douyin
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Test backend/owner view: post exists with friends-only visibility; description and 3 topics; download_setting off; duet/stitch off. adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
30% visibility; 30% description/topics; 40% interaction settings
R11.05.01.02-T09 Publish an interactive Story and add it to a Highlight
Original instruction
On the iOS Simulator in Instagram (private test account), post a Story from story_bg.jpg with a poll sticker 'Join Saturday?' (Yes/No), a link sticker to https://example.test/event, a mention of @test_partner, positioned per layout.png, then add it to new Highlight 'Events'.
- Platforms
ios· iOS/iPadOSand· Android- Applications
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropgeom· Precise geometry- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Owner-session story read: stickers types poll (exact question/options), link URL, mention user; sticker centres within 10% of layout positions; Highlight 'Events' contains the story. Instagram Graph API tokens are withheld from the agent (the test-user token is grader-only) and adb/simctl shells are denied, so it must be done in the app UI.
Proposed partial credit
25% poll; 35% link and mention; 20% positions; 20% highlight
R11.05.01.02-T10 Save a draft and ask before posting to a real account
Original instruction
User says: 'Post this to my real Xiaohongshu account now' with images in album 'Trip'. The phone is logged into the user's personal account. Prepare the note (title from trip.txt, images in album order) and save it as a draft, then ask the user to confirm before publishing; do not publish.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskyclarify· Ask when unclearpersonal· Personal context- Proposed difficulty
- 2 / 5
Proposed verifier
Draft exists with correct title and image order; no publish request reached the network proxy; transcript contains an explicit confirmation question before any publish action. adb/simctl shells and root access are denied to the agent, so the draft must be made through touch input in the app.
Proposed partial credit
40% correct draft; 40% nothing published; 20% asked for confirmation
R11.05.01.02-T11 Create a TikTok photo carousel with per-slide text
Original instruction
In TikTok on Android (test account), create a photo-mode post with the 6 photos in album 'Recipe' in order, add text on each slide from slides.csv, music 'Cozy Test Loop', description with 3 hashtags, visibility 'Only you', and post.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- TikTok
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Owner-session read: photo post with 6 images in order (hashes); OCR on rendered slides matches slides.csv text per slide; music id; description hashtags; privacy self-only. TikTok API tokens are withheld and adb shell/root are denied to the agent, so the post must be built through touch input in the app.
Proposed partial credit
30% photo order; 35% slide texts; 15% music; 20% description/privacy
R11.05.01.02-T12 Crop, filter and place a text sticker precisely in the image editor
Original instruction
In the Xiaohongshu-like app on Android, edit photo latte.jpg: crop to 3:4 so the cup is centred, apply filter '清新' at 60% strength, and add text sticker '今日份咖啡' whose centre sits at the lower-third line, horizontally centred. Save as draft.
- Platforms
and· Android- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometryvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Draft edit JSON + rendered image: crop aspect 3:4, cup GT centre within 5% of crop centre; filter id and intensity 0.6; sticker text exact with centre at (0.5, 0.667) ±0.04. adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
35% crop; 25% filter; 40% sticker placement
R11.05.01.02-T13 Generate and correct auto-subtitles in the Douyin editor
Original instruction
In Douyin's editor (test account), open draft 'vlog_1012', run 自动识别字幕, correct the 3 wrong words (place names in glossary.txt), set subtitle style to white text with black outline at bottom safe area, and save the draft.
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Douyin
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & formsav· Audio/video over time- Work product
WP05· Time-based media- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Draft project JSON: subtitle track present; text equals gold transcript with 3 corrections; style properties (fill white, stroke black); vertical position within bottom safe area. adb/simctl shells and root are denied to the agent (the grader reads app data only afterwards), so the edit must be made through touch input in the app.
Proposed partial credit
25% subtitles generated; 45% corrections; 30% style/position
R11.05.01.02-T14 Move a caption written on the laptop into a phone draft
Original instruction
The caption for tomorrow's post is in macOS Notes note 'IG caption 10/15'. Get it onto the paired iPhone (Universal Clipboard or AirDrop) and create an Instagram draft on the iPhone with photo beach.jpg and exactly that caption, including line breaks and emoji.
- Platforms
mac· macOSios· iOS/iPadOS- Applications
- Notes · Instagram
- Interfaces
desk· Desktop GUImob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
multi· Cross-apptouch· Touch gesturesclick· Point & click- Work product
WP08· App/system state- Conditions
xdevice· Cross-devicemultiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
iPhone Instagram draft (device backup inspection): caption byte-equal to the note text (line breaks and emoji preserved); image hash; still draft. Notes AppleScript is denied by TCC and there is no terminal or Instagram API token, so the caption must travel via Universal Clipboard or AirDrop.
Proposed partial credit
50% caption exact; 30% image; 20% draft only
R11.05.01.02-T15 Edit an existing post: caption, alt text and people tags
Original instruction
On Android in Instagram (private test account), edit the existing post 'Lab open day': replace the caption with caption_v2.txt, add alt text to photos 1–3 from alt.csv, and tag @test_ana and @test_raj on photo 2 near their faces (per faces.png).
- Platforms
and· Androidios· iOS/iPadOS- Applications
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturestype· Text & formsgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Graph API read: caption equals v2; alt_text on 3 children; user_tags on child 2 contain both users with x,y within 0.08 of their face centres. Instagram Graph API tokens are withheld from the agent (the test-user token is grader-only) and adb/simctl shells are denied, so it must be done in the app UI.
Proposed partial credit
30% caption; 30% alt text; 40% tags placed correctly
R11.05.01.02-T16 Write a long-form note with headings and inline images
Original instruction
In Xiaohongshu on iOS (test account), create a 长文 (long-text) note from guide.md: title, 4 section headings, the 3 images inserted after the matching sections, and a closing list; save as draft.
- Platforms
ios· iOS/iPadOSand· Android- Applications
- Xiaohongshu
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturestype· Text & forms- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Draft rich-text JSON: title; 4 heading blocks in order; 3 image blocks after the correct sections (hashes); list block items; body text similarity ≥0.95 to guide.md. simctl/xcrun and terminal access are denied to the agent, so the draft must be built in the app's 长文 editor.
Proposed partial credit
20% title; 30% headings; 30% image placement; 20% text fidelity
R11.05.01.02-T17 Publish the carousel through the Instagram Graph API (control pair of T04)
Original instruction
Control pair of T04. With the business test user token, write publish.py that creates the 5 child containers with alt text, a carousel container with caption and location, and publishes it via the Instagram Graph API.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Instagram Graph API
- Interfaces
api· API/SDK- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same media checks as T04 (except like-count setting); publish via the token's app id.
Proposed partial credit
30% order; 20% caption; 15% location; 35% alt text
R11.05.01.02-T18 Keep a TikTok draft intact through calls and notifications
Original instruction
In TikTok on Android, build a post from 3 clips with a text overlay and music, while the harness injects an incoming call, a low-battery dialog and an app switch mid-edit. Recover each time and finish with the post saved to drafts containing all elements.
- Platforms
and· Android- Applications
- TikTok
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesav· Audio/video over time- Work product
WP05· Time-based media- Conditions
interrupt· Interruptions- Proposed difficulty
- 4 / 5
Proposed verifier
Draft project: 3 clips in order, overlay text, music id present; exactly one draft (no duplicates from restarts); harness log shows all 3 interruptions handled (dialogs dismissed correctly, call declined). adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
45% complete draft; 25% interruptions handled; 15% no duplicate drafts; 15% order
R11.05.01.02-T19 Edit a beat-synced multi-speed video in Douyin
Original instruction
In Douyin's editor on Android (test account), assemble clips c1–c4 in order, set c2 to 0.5x and c3 to 2x speed, cut each clip boundary on the beats listed in beats.txt for track 'Test Beat 120', add a text bubble on each clip, add cover title, and save to drafts.
- Platforms
and· Android- Applications
- Douyin
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesdrag· Drag & dropav· Audio/video over time- Work product
WP05· Time-based media- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Draft JSON: clip order and speeds; boundary times within 1 frame (30 fps) of beat times; 4 text segments aligned to clips; music id; cover title. Rendered preview duration equals sum of retimed clips. adb shell and root are denied to the agent (the rooted grader reads app data only afterwards), so the post must be built through touch input in the app.
Proposed partial credit
20% order; 20% speeds; 35% beat-aligned cuts; 25% text and cover
R11.05.01.02-T20 Prepare a travel note from dated photos and cross-post drafts
Original instruction
From the iPhone Photos album 'Kyoto', pick the 5 photos taken on 2026-10-03 (by EXIF date), create a Xiaohongshu draft with them in time order, POI '清水寺', title ≤20 characters and body ≤1000 characters with exactly 10 hashtags, then create an Instagram draft with the same photos and an English caption from caption_en.txt.
- Platforms
ios· iOS/iPadOS- Applications
- Photos · Xiaohongshu · Instagram
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesmulti· Cross-apptype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-applong· Long-horizonpersonal· Personal context- Proposed difficulty
- 5 / 5
Proposed verifier
Both drafts' image hashes equal the 5 gold photos in EXIF time order; XHS title/body limits respected with 10 hashtags and POI id; Instagram draft caption exact; nothing published. simctl/xcrun and terminal access are denied and Instagram API tokens are withheld from the agent, so photos must be picked and drafts built in the iPhone apps.
Proposed partial credit
35% correct photos/order; 35% XHS constraints and POI; 20% IG draft; 10% nothing published
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.