Patient portals & insurance self-service
Book visits, message, download records and file claims in portals (test data).
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- MyChart · insurer portals
- Class GUI-need label
G2· GUI-essential
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.05.01.01-T01 Request a prescription refill and choose the pharmacy in a patient portal
Original instruction
In the MyChart-style patient portal clone (synthetic patient 'Alex Rivera'), request a 90-day refill of the active lisinopril prescription (not the discontinued metformin), to be sent to the saved pharmacy 'Walgreens #4471', and add the comment 'switching to 90-day supply'.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
Portal DB: refill request for the active lisinopril order id, days_supply=90, pharmacy id 4471, comment exact; no request on discontinued meds. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct medication; 20% 90-day; 25% pharmacy; 15% comment.
R12.05.01.01-T02 Book a follow-up visit with the same provider within a date window
Original instruction
In the patient portal clone, schedule a 'Follow-up 20 min' visit with Dr. Patel (the provider of the last visit) between 2026-11-02 and 11-13, on a Tuesday or Thursday afternoon, in-person at the Main Street clinic. Answer the pre-visit questionnaire using fixtures/symptoms.txt.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrolltype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Appointment: provider Patel, type, location, slot Tue/Thu PM in range (earliest such not required), questionnaire answers equal symptoms.txt mapping. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% provider/type; 30% slot constraints; 15% location; 25% questionnaire.
R12.05.01.01-T03 Download lab results as PDF and report out-of-range values
Original instruction
In the patient portal clone, open the most recent comprehensive metabolic panel, download the results PDF to ~/Documents/labs/CMP_2026-10.pdf, and list in labs.txt every analyte flagged out of range with value and reference range as displayed.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
PDF saved at exact path (hash equals portal export); labs.txt lists the seeded 3 flagged analytes with exact values/ranges (flags shown as colored icons, not text). No FHIR or API client credentials are issued, so results and the PDF are reachable only through the portal's browser session (session id + CSRF).
Proposed partial credit
40% PDF saved correctly; 60% flagged analytes (one third each).
R12.05.01.01-T04 Message the care team with an attachment and the right message category
Original instruction
In the patient portal clone, send a message to Dr. Patel's care team under the category 'Medication question' asking whether the user can take ibuprofen with current meds, listing the current medication names from the portal's Medications page, and attach fixtures/rash_photo.jpg.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & clickmulti· Cross-app- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Message row: recipient pool = Patel care team, category=medication_question, body contains all active med names (set match), attachment hash. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
25% recipient; 25% category; 30% med list; 20% attachment.
R12.05.01.01-T05 File an out-of-network claim with itemized receipt on an insurer portal
Original instruction
On the insurer portal clone (test member), file a medical claim for an out-of-network physiotherapy visit using fixtures/superbill.pdf (CPT codes, ICD-10, provider NPI, dates, charges). Attach the superbill and proof of payment, choose reimbursement by direct deposit, and stop before 'Submit claim' for user review.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- insurer portal clone · PDF viewer
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Claim draft: patient, NPI, dates, 3 service lines with CPT/ICD/charge exact, attachments 2 (hashes), payout=direct_deposit; status=draft; ASK_USER present. The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% service lines (per field); 20% provider/NPI; 20% attachments; 10% stopped for review.
R12.05.01.01-T06 Find a covered in-network specialist accepting new patients
Original instruction
On the insurer portal clone, find an in-network dermatologist accepting new patients within 10 miles of ZIP 98103 under the member's specific plan 'Silver PPO 2026' (network differs by plan), with the lowest estimated visit cost. Save them as 'favorite provider' and report name and estimated cost.
- Platforms
web· Any OS (web)- Applications
- insurer portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Favorite provider id = seeded answer (in-network for that plan, accepting, within radius, min cost); report matches. Providers in-network only for other plans are traps. The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% correct provider; 30% saved favorite; 20% cost reported.
R12.05.01.01-T07 Appeal a denied claim with the correct reason and supporting letter
Original instruction
On the insurer portal clone, claim CLM-55102 was denied for 'missing prior authorization'. The prior auth number is in fixtures/pa_letter.pdf. File an appeal for that claim citing the PA number, attach the letter, choose appeal type 'Standard', and provide a 2-sentence explanation. Stop before final submit.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- insurer portal clone · PDF viewer
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Appeal draft for CLM-55102: type standard, PA number exact (read from PDF), attachment hash, explanation <= 2 sentences mentioning the PA; status=draft. The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% correct claim; 30% PA number; 20% attachment; 20% explanation + not submitted.
R12.05.01.01-T08 Complete eCheck-in with insurance card images and updated address
Original instruction
In the patient portal app clone on the iOS Simulator, complete eCheck-in for tomorrow's appointment: update the home address to the one in fixtures/new_address.txt, upload front and back images of the new insurance card (fixtures/card_front.jpg, card_back.jpg) and enter member ID and group number from the card, confirm medications and allergies unchanged, and sign the consent with a drawn signature.
- Platforms
ios· iOS/iPadOS- Applications
- patient portal app clone (iOS)
- Interfaces
mob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturespath· Continuous pathtype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Portal DB: address exact; coverage with member/group IDs read from images; 2 card images attached; med/allergy review timestamps; signature stroke data present (non-empty path, length > threshold). The agent drives only the Simulator window: it has no shell on the host (simctl and the app container are not reachable) and no API token, so eCheck-in must be done in the app.
Proposed partial credit
20% address; 40% insurance fields + images; 20% reviews; 20% signature.
R12.05.01.01-T09 Grant proxy access to a family member with limited permissions
Original instruction
In the MyChart-style portal clone, grant proxy access for the user's adult daughter (email in fixtures/proxy.txt) with permissions: view appointments and message the care team, but not view test results or billing. Set expiry to one year from today.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Proxy grant row: email, scopes = {appointments, messaging}, results/billing false, expiry date exact; invitation queued in mail sink. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
60% exact scopes; 20% expiry; 20% invitation.
R12.05.01.01-T10 Estimate out-of-pocket cost for a scheduled MRI and pick the cheapest facility
Original instruction
On the insurer portal clone's cost estimator, compare the member's out-of-pocket cost for 'MRI lower back without contrast' (CPT 72148) at the 3 in-network imaging centers within 15 miles, accounting for the remaining deductible shown on the Benefits page. Report the cheapest facility and estimate, and download its estimate PDF.
- Platforms
web· Any OS (web)- Applications
- insurer portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Report facility and amount equal to estimator outputs with the member's deductible state; PDF downloaded for that facility (estimate id). The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% correct facility; 30% amount; 20% PDF.
R12.05.01.01-T11 Pay a medical bill with an HSA card after applying the insurance adjustment
Original instruction
In the hospital billing portal clone, the user has 3 open statements. Pay only the one for the 2026-08-14 ER visit after verifying the insurance adjustment matches the EOB (fixtures/eob_0814.pdf). If the patient-responsibility amount differs from the EOB, do not pay; instead open a billing inquiry citing both amounts.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- hospital billing portal clone · PDF viewer
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appvis· Visual judgmenttype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Seeded mismatch ($112 vs $87): no payment row; billing inquiry referencing the statement with both amounts. Variant with match: payment with HSA test card equal to amount. The billing portal issues no API token and accepts payments and inquiries only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct comparison; 40% correct action per variant; 20% inquiry content.
R12.05.01.01-T12 Order a replacement member ID card and update the mailing address only
Original instruction
On the insurer portal clone, order a replacement ID card for the dependent child 'Mia', mailed to the new mailing address in fixtures/address.txt. Update only the mailing address, not the residential address (which affects network eligibility).
- Platforms
web· Any OS (web)- Applications
- insurer portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Card order row for dependent Mia; member mailing_address updated exactly; residential address unchanged. The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct card order; 40% mailing address; 20% residential untouched.
R12.05.01.01-T13 Reconcile EOBs against provider bills and flag overbilling
Original instruction
Download all EOBs for Q3 2026 from the insurer portal clone and all statements from the hospital billing portal clone. In a sheet q3_reconcile.xlsx, match each statement to its EOB by date of service and provider, and flag statements where billed patient responsibility exceeds the EOB amount. Open a billing inquiry for each flagged statement.
- Platforms
mac· macOSwin· Windows- Applications
- insurer portal clone · hospital billing portal clone · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Sheet rows match seeded 9 statements to EOBs; flags exactly on seeded 2 overbilled; inquiries exist for those 2 with amounts.
Proposed partial credit
40% matching; 30% flags; 30% inquiries.
R12.05.01.01-T14 Submit a prior-authorization status check and add a missing document
Original instruction
On the insurer portal clone, check the status of the prior authorization for the user's upcoming knee surgery. It shows 'Pending - additional information required'. Read the request detail, upload the right document from docs/ (3 candidates: MRI report, PT notes, X-ray report) that satisfies it, and add a note. Report the new status.
- Platforms
web· Any OS (web)- Applications
- insurer portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentmulti· Cross-app- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
PA record attachment = the document matching the requested type (seeded: PT notes covering 6 weeks), note present; status transitions to 'In review'; report matches. The insurer portal issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% correct document; 20% note; 30% status reported.
R12.05.01.01-T15 Book a telehealth visit and run the device test before the visit
Original instruction
In the patient portal clone, book the earliest same-day telehealth 'Urgent care video visit', complete the intake (symptom checklist from fixtures/intake.json), and run the pre-visit audio/video device test (uses the virtual camera and mic), confirming both pass. Do not join the visit.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Appointment telehealth earliest same-day slot; intake answers exact; device-test result records camera=pass, mic=pass (browser permission granted); no join event. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% booking; 30% intake; 30% device test; 10% did not join.
R12.05.01.01-T16 Request medical records release to another provider with a date range
Original instruction
In the patient portal clone, request release of records from 2024-01-01 to 2026-09-30 (visit notes, imaging reports, labs; exclude mental-health notes) to the new orthopedist in fixtures/new_provider.txt via secure fax. Sign the HIPAA authorization with a typed signature and stop before submit.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Release draft: date range, record types set exactly (mental health excluded), recipient name/fax from file, delivery=fax, signature present; status=draft. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% record types; 20% dates; 30% recipient; 20% signature + not submitted.
R12.05.01.01-T17 Enroll in a pharmacy mail-order program and transfer three prescriptions
Original instruction
On the insurer's pharmacy-benefit portal clone, enroll in mail order, transfer the 3 maintenance prescriptions listed in fixtures/meds.txt from the retail pharmacy, choose the cheapest shipping, and set auto-refill for 2 of them (not the one with a dose change pending). Use the test HSA card on file.
- Platforms
web· Any OS (web)- Applications
- pharmacy benefit portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Enrollment row; 3 transfer requests matching meds (name, strength); auto-refill on exactly the 2 correct ones; shipping=cheapest; payment method HSA. The benefit portal issues no API token and accepts enrollment and transfers only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
20% enrollment; 40% transfers; 30% auto-refill selection; 10% shipping/payment.
R12.05.01.01-T18 Compare two plans during open enrollment and select the cheaper expected cost
Original instruction
On the employer benefits portal clone, compare 'Gold HMO' and 'HDHP + HSA' for the household's expected usage in fixtures/usage.json (visits, prescriptions, one MRI), including premiums, deductible, coinsurance, OOP max and employer HSA contribution. Select the plan with lower expected annual cost, add both dependents, and stop before final election.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- benefits portal clone · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appvis· Visual judgmentcode· Code & commands- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Expected cost computed by reference model from plan documents (portal-only); election draft plan = lower cost; dependents both covered; status=draft.
Proposed partial credit
40% correct plan; 30% cost computation (via answer file); 20% dependents; 10% not finalized.
R12.05.01.01-T19 Update allergies and medication list after a hospital discharge summary
Original instruction
fixtures/discharge_summary.pdf lists new allergies, stopped medications and new prescriptions. In the patient portal clone, use 'Review my health info' to report changes: add the 2 new allergies with reactions, mark the 2 stopped meds as 'no longer taking', and add the 3 new prescriptions with dose and frequency. Submit the changes for clinician review.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- MyChart-style portal clone · PDF viewer
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Pending-review change set: allergies (substance, reaction) exact; 2 meds marked stopped; 3 meds added (name, dose, frequency) exact; submitted. No FHIR or API client credentials are issued; the portal accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% allergies; 25% stopped meds; 35% new meds; 10% submitted.
R12.05.01.01-T20 Coordinate a referral, prior auth and appointment across portal and insurer
Original instruction
The user needs a sleep study. In the patient portal clone, request a referral from the PCP (message with symptoms), then once the referral appears (scripted), confirm on the insurer portal clone that prior auth was auto-approved, find an in-network sleep center within 20 miles, and book the earliest study slot through the portal's scheduling ticket. Record all IDs in sleep_study.txt.
- Platforms
web· Any OS (web)- Applications
- MyChart-style portal clone · insurer portal clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrollclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Portal: referral request message; scheduled study at in-network center (plan-specific) with earliest slot; sleep_study.txt contains referral id, PA id, appointment id equal to DB. Neither portal issues API or FHIR credentials; writes are accepted only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
20% referral request; 20% PA verified; 40% booking constraints; 20% IDs recorded.
Mapped research references
No benchmark reference is assigned to this class in the current mapping.