Computer-Use Agent Taxonomy中文
R12.05.01.02 · TASK CLASS

Fitness & health records

Log, correct and export personal health and fitness data.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Apple Health · fitness apps
Class GUI-need label
G2 · GUI-essential

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R12.05.01.02-T01 Log a blood-pressure reading with time in Apple Health

Original instruction

In the Health app on the iOS Simulator (synthetic profile), add a blood pressure reading of 128/82 mmHg taken today at 07:45, and a heart rate of 64 bpm at the same time.

Platforms
ios · iOS/iPadOS
Applications
Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
1 / 5

Proposed verifier

HealthKit store (read via test-host app): correlation BloodPressure 128/82 at 07:45 today, HeartRate 64 at 07:45, source = Health app (manual entry). No other new samples. The agent drives only the Simulator window: it has no host shell, so simctl and Xcode test hosts are not reachable to write HealthKit samples directly.

Proposed partial credit

60% BP values and time; 40% heart rate.

R12.05.01.02-T02 Record a walking activity with a name in OpenTracks

Original instruction

In OpenTracks on the Android emulator, record a 30-minute walking activity named 'Morning walk' by starting a recording, playing fixtures/walk.gpx through the emulator location player, then stopping. Set the activity type to 'Walking' and add the description 'with Biscuit'.

Platforms
and · Android
Applications
OpenTracks · Android emulator extended controls
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures multi · Cross-app type · Text & forms
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app
Proposed difficulty
2 / 5

Proposed verifier

OpenTracks DB: track name, activity walking, description, moving time 30 min ±2, distance within 5% of GPX length. walk.gpx sits only in the emulator's extended controls, not device storage, and with no adb shell it is not reachable for OpenTracks import; the track must be recorded live.

Proposed partial credit

50% track recorded correctly; 30% name/type; 20% description.

R12.05.01.02-T03 Delete a duplicated workout and fix a wrong distance

Original instruction

In the Strava-style web clone, the user's 2026-10-11 run was uploaded twice (watch + phone). Delete the phone copy (fewer GPS points, shown in the activity details), then correct the remaining run's name to 'Sunday Long Run' and mark it as 'Long Run' workout type.

Platforms
web · Any OS (web)
Applications
Strava-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click vis · Visual judgment type · Text & forms
Work product
WP08 · App/system state
Conditions
risk · Irreversible/risky
Proposed difficulty
2 / 5

Proposed verifier

Activities DB: one activity for 10-11 remaining, the one with more GPS points; name and workout_type exact. Wrong deletion = 0. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

60% correct duplicate deleted; 20% name; 20% workout type.

R12.05.01.02-T04 Log a day of meals with portions from a food diary photo

Original instruction

fixtures/food_diary.jpg is a handwritten diary of yesterday's meals (breakfast, lunch, dinner, 2 snacks). In the MyFitnessPal-style app clone (Android emulator), log each food item with the closest database match and the stated portion in grams or servings, under the right meal.

Platforms
and · Android
Applications
calorie tracker clone
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
vis · Visual judgment touch · Touch gestures type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Diary entries for yesterday: 12 items mapped to accepted food ids (allowlist per item), portions within ±5%, meal assignment correct; total kcal within ±5% of reference. The agent gets screen input only (no adb shell, no backend API token), so entries must be logged in the app; the harness input log confirms UI taps.

Proposed partial credit

40% items matched; 30% portions; 20% meal slots; 10% total kcal.

R12.05.01.02-T05 Export Apple Health step data for a date range and chart it

Original instruction

On a macOS + iOS Simulator setup, export the Health data from the simulator (Health > profile > Export All Health Data), copy export.zip to the Mac, extract daily step totals for September 2026 into steps_sep.csv (date, steps), and make a bar chart in Numbers saved as steps.numbers.

Platforms
ios · iOS/iPadOS mac · macOS
Applications
Apple Health · Numbers · Terminal
Interfaces
mob · Mobile touch desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
touch · Touch gestures code · Code & commands multi · Cross-app
Work product
WP02 · Records/table
Conditions
xdevice · Cross-device multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

steps_sep.csv: 30 rows equal per-day sums of HKQuantityTypeIdentifierStepCount from the seeded data (dedup by source priority as Health does); steps.numbers contains a bar chart with 30 points.

Proposed partial credit

20% export obtained; 50% daily sums correct; 30% chart.

R12.05.01.02-T06 Correct a mis-logged weight unit and remove an outlier entry

Original instruction

In Apple Health on the iOS Simulator, a weight sample of 172 kg was logged on 2026-10-03 by mistake (it was 172 lb). Delete it and add the correct value in kg (to 0.1 kg). Also delete the 9.0 kg entry on 10-07 (a pet weighing). Keep all other weight samples.

Platforms
ios · iOS/iPadOS
Applications
Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures type · Text & forms vis · Visual judgment
Work product
WP08 · App/system state
Conditions
risk · Irreversible/risky
Proposed difficulty
3 / 5

Proposed verifier

HealthKit: 172 kg sample removed, new 78.0 kg sample on 10-03 (same time), 9.0 kg sample removed; other samples unchanged (count and values). The agent drives only the Simulator window: it has no host shell, so simctl and Xcode test hosts are not reachable to write HealthKit; samples must carry the Health app as source.

Proposed partial credit

40% wrong sample corrected; 30% outlier removed; 30% others intact.

R12.05.01.02-T07 Build a Shortcut that logs water intake after asking for the amount

Original instruction

In the Shortcuts app on an iPhone (iOS Simulator with Health), create a shortcut 'Log Water' that asks for a number (prompt 'How many ml?'), logs it as Dietary Water in Health in mL, and shows a notification 'Logged N ml'. Run it once with 350.

Platforms
ios · iOS/iPadOS
Applications
Shortcuts · Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures drag · Drag & drop type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Shortcut exported by the grader (simctl, grader side) contains: Ask for Input (number, prompt), Log Health Sample (Dietary Water, mL, variable), Show Notification with variable. HealthKit has a 350 mL water sample whose source is Shortcuts. The agent has no shell access (host or device), simctl is blocked and Shortcuts file import is disabled, so the shortcut must be built in the Shortcuts editor.

Proposed partial credit

50% action chain correct; 20% prompt/notification text; 30% run produced sample.

R12.05.01.02-T08 Set heart-rate zones and a training plan in a Garmin Connect-style web app

Original instruction

In the Garmin Connect-style web clone, set the user's heart-rate zones using max HR 186 and resting HR 52 with the heart-rate-reserve method (zones 1-5 at 50/60/70/80/90%), then add the 'Half Marathon - Intermediate' training plan ending on race day 2027-03-14 with runs scheduled Tue/Thu/Sat/Sun.

Platforms
web · Any OS (web)
Applications
Garmin Connect-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
type · Text & forms click · Point & click drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

User settings: HR zone boundaries equal HRR computation ±1 bpm; plan id with race date and day mask exact. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

50% HR zones; 30% plan + race date; 20% schedule days.

R12.05.01.02-T09 Merge two sleep data sources and set source priority

Original instruction

In Apple Health on the iOS Simulator, the user's sleep is double-counted from 'SleepTracker' and the watch. Set data-source priority for Sleep so the watch is first and SleepTracker second, and turn off SleepTracker's permission to write Heart Rate (keep its Sleep write permission).

Platforms
ios · iOS/iPadOS
Applications
Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

HealthKit source-priority table for sleepAnalysis: watch above SleepTracker (drag in Edit mode); authorization: SleepTracker heartRate write=denied, sleep write=allowed. The agent has no host shell or simctl, so healthdb and TCC files cannot be edited directly; a file-write audit flags writes by any process other than Health or Settings.

Proposed partial credit

60% priority order; 40% permission changes.

R12.05.01.02-T10 Create a habit tracker routine and backfill a week of check-ins

Original instruction

In Loop Habit Tracker (Android emulator), create habit 'Stretch' with a 5-times-per-week target and reminder at 21:30, then backfill check-ins for last week per fixtures/stretch_log.txt (Mon, Tue, Thu, Fri, Sun). Ensure the habit's score chart shows the week complete.

Platforms
and · Android
Applications
Loop Habit Tracker
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures type · Text & forms
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Loop DB: habit with freq 5/7, reminder 21:30; repetitions table rows for the 5 dates exactly. The agent has no adb shell or push, so a crafted Loop backup cannot be placed for import and the DB is not reachable; check-ins must be tapped in the app.

Proposed partial credit

30% habit config; 20% reminder; 50% backfilled days (equal weight per day).

R12.05.01.02-T11 Crop a GPS track to remove the drive home and recompute stats

Original instruction

In the Strava-style web clone, today's bike ride includes 6 km of driving home at the end (speeds > 60 km/h on the speed chart). Use the 'Crop' tool's slider to trim the ride's end to where the driving begins, then save. Report the new distance and average speed.

Platforms
web · Any OS (web)
Applications
Strava-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
drag · Drag & drop vis · Visual judgment av · Audio/video over time
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Activity end index within ±15 s of the ground-truth drive start; recomputed distance within 1%; reported values match DB. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

60% crop point accuracy; 20% saved; 20% report.

R12.05.01.02-T12 Import a watch FIT file and attach it to the correct gear

Original instruction

Import fixtures/ride_2026-10-12.fit into the Garmin Connect-style web clone via the upload page, set its gear to the 'Gravel bike' (not 'Road bike'), set privacy 'Followers only', and correct the auto-detected activity type from 'Cycling' to 'Gravel cycling'.

Platforms
mac · macOS win · Windows web · Any OS (web)
Applications
Garmin Connect-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click multi · Cross-app
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Activity exists with FIT hash; gear=Gravel bike id; privacy=followers; type=gravel_cycling. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

40% imported; 20% gear; 20% privacy; 20% type.

R12.05.01.02-T13 Reconcile step counts between a fitness band app and Health Connect

Original instruction

On the Android emulator, Gadgetbridge (simulated band) and Health Connect disagree on steps for 2026-10-05..10-11. Configure Gadgetbridge to sync steps to Health Connect, set Health Connect's app priority so Gadgetbridge wins for Steps, and report the corrected weekly total shown in Health Connect.

Platforms
and · Android
Applications
Gadgetbridge · Health Connect
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures drag · Drag & drop multi · Cross-app
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

Health Connect priority list: Gadgetbridge first for Steps; Gadgetbridge HC sync enabled with Steps permission; reported weekly total equals aggregate computed via HC aggregate API in harness. The agent has no adb shell, so pm grant and Health Connect permission or priority changes are not reachable by command; they must be set in the apps' UI.

Proposed partial credit

30% sync enabled; 40% priority; 30% total reported.

R12.05.01.02-T14 Build a monthly running log spreadsheet from exported activities

Original instruction

Export the user's September activities from the Strava-style web clone (bulk export CSV via Settings), then in Excel build running_log.xlsx with one row per run (date, distance km, moving time, pace min/km), a weekly total table, and highlight runs faster than 5:00 min/km. Exclude walks and rides.

Platforms
mac · macOS win · Windows
Applications
Strava-style clone · Excel
Interfaces
brow · Browser desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
multi · Cross-app code · Code & commands type · Text & forms
Work product
WP02 · Records/table
Conditions
multiapp · Multi-app
Proposed difficulty
3 / 5

Proposed verifier

running_log.xlsx rows equal reference (runs only), pace exact to second; weekly totals exact; conditional formatting rule or fills on exactly the fast runs.

Proposed partial credit

20% export; 40% run rows; 20% weekly totals; 20% highlighting.

R12.05.01.02-T15 Set up a cycle and symptom log with a medication reminder in Health

Original instruction

In Apple Health on the iOS Simulator, log menstrual flow for 2026-10-08..10-12 with the intensities in fixtures/cycle.txt, add symptoms 'Headache' on 10-08 and 'Cramps' on 10-09, and add the medication 'Ibuprofen 200 mg tablet' as needed (no schedule) in the Medications feature.

Platforms
ios · iOS/iPadOS
Applications
Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
touch · Touch gestures click · Point & click
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

HealthKit: menstrualFlow samples per day with intensity exact; symptom category samples on correct dates; medication entry exists with strength/form and as-needed schedule. The agent drives only the Simulator window: it has no host shell, so simctl and Xcode test hosts are not reachable to write HealthKit; samples must carry the Health app as source.

Proposed partial credit

40% flow entries; 30% symptoms; 30% medication.

R12.05.01.02-T16 Share specific health categories with a family member via Health Sharing

Original instruction

In Apple Health on the iOS Simulator, share only Heart Rate, Falls (Walking Steadiness) and Medications with contact 'Jordan Lee', not steps or sleep. Then verify the sharing summary lists exactly those categories and stop before sending the invitation if a confirmation appears; hand off.

Platforms
ios · iOS/iPadOS
Applications
Apple Health
Interfaces
mob · Mobile touch
GUI need · task setting
G3 · GUI-only
GUI need · ordinary access
G3 · GUI-only
Interaction primitives
touch · Touch gestures vis · Visual judgment
Work product
WP08 · App/system state
Conditions
risk · Irreversible/risky personal · Personal context
Proposed difficulty
3 / 5

Proposed verifier

Sharing draft (simulator Health sharing store) with recipient Jordan Lee and exactly 3 categories; invitation not sent; ASK_USER at confirmation. The agent drives only the Simulator window: it has no host shell, so simctl and the Health sharing store are not reachable; sharing must be set up in the Health app.

Proposed partial credit

60% exact category set; 20% recipient; 20% handoff before send.

R12.05.01.02-T17 Correct a misattributed workout heart-rate spike by trimming sensor data

Original instruction

In the Garmin Connect-style web clone, the 2026-10-09 interval workout has an optical HR spike to 220 bpm for ~40 s (sensor error) that set a new max HR. Use the chart editor to select and remove the erroneous HR segment, then reset the user's max HR back to 186 in settings.

Platforms
web · Any OS (web)
Applications
Garmin Connect-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
drag · Drag & drop vis · Visual judgment av · Audio/video over time
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Activity HR stream: removed range covers seeded spike (IoU >= 0.8 in time) and removes < 10 s of valid data; user max HR=186; activity max HR recomputed < 190. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

50% spike removal IoU; 20% valid data preserved; 30% max HR reset.

R12.05.01.02-T18 Migrate a fitness history from one app to another via export and import

Original instruction

Export all activities from the Runkeeper-style web clone (ZIP of GPX + CSV), then import them into the Strava-style clone, preserving names and activity types (the import tool defaults types to 'Run'). Fix types for the 7 non-run activities after import. Do not create duplicates of the 3 activities already present.

Platforms
mac · macOS win · Windows
Applications
Runkeeper-style clone · Strava-style clone · Finder
Interfaces
brow · Browser desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
multi · Cross-app click · Point & click drag · Drag & drop
Work product
WP08 · App/system state
Conditions
multiapp · Multi-app long · Long-horizon
Proposed difficulty
4 / 5

Proposed verifier

Strava clone DB: activity set equals union by start time (no duplicates), names preserved, types correct for all 7 non-runs.

Proposed partial credit

40% all activities imported; 20% no duplicates; 30% types fixed; 10% names.

R12.05.01.02-T19 Compile a doctor-ready summary of three months of home vitals

Original instruction

From Apple Health on the iOS Simulator (synthetic data), prepare for a cardiology visit: export BP and resting HR for Jul-Sep 2026, then in Numbers on the Mac produce a sheet with weekly averages, the count of readings >= 140/90, and a line chart, exported as vitals_q3.pdf (1 page). Attach it to a new note in Notes titled 'Cardiology visit'.

Platforms
ios · iOS/iPadOS mac · macOS
Applications
Apple Health · Numbers · Notes
Interfaces
mob · Mobile touch desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
multi · Cross-app code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
xdevice · Cross-device multiapp · Multi-app long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

vitals_q3.pdf: 1 page; extracted weekly averages equal reference ±0.5; high-reading count exact; chart present (vector paths). Notes DB has note with that PDF attachment.

Proposed partial credit

20% export; 40% weekly stats; 20% chart/1 page; 20% note attachment.

R12.05.01.02-T20 Configure a smartwatch workout with custom intervals and sync it to the device

Original instruction

In the Garmin Connect-style web clone, build a structured workout 'VO2 6x800': 15 min warm-up, 6 x (800 m at 3:50-4:00 min/km pace target, 400 m jog recovery), 10 min cool-down, using the drag-and-drop step editor and repeat block. Schedule it for next Tuesday and send it to the paired simulated watch.

Platforms
web · Any OS (web)
Applications
Garmin Connect-style clone
Interfaces
brow · Browser
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
drag · Drag & drop type · Text & forms click · Point & click
Work product
WP08 · App/system state
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Workout JSON: step sequence with repeat count 6, distance and pace targets exact, durations exact; calendar entry next Tuesday; device sync queue contains workout id. The clone issues no API token (no OAuth app) and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.

Proposed partial credit

50% step structure; 20% targets; 15% scheduled; 15% sent to device.

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

In this workflow family