Chart review & data retrieval
Find and reconcile facts across a patient chart.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Epic (test) · OpenEMR · OpenMRS
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R09.03.01.01-T01 Find a patient by name and date of birth and report the MRN in OpenEMR
Original instruction
In the sandbox OpenEMR (synthetic Synthea patients; login in input/creds.txt), find the patient 'Marisol Delgado-Ruiz' born 1987-03-14 (two patients share the name) and write her external ID/MRN to output/answer.json.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
Answer exact vs seeded DB. The agent account has no API/FHIR client credentials and direct DB access is firewalled; web access logs confirm GUI navigation only.
Proposed partial credit
30% correct patient's chart opened (per web access log); 70% exact MRN reported (0 overall if the namesake's MRN)
R09.03.01.01-T02 Report the most recent magnesium within 24 hours in OpenEMR
Original instruction
In sandbox OpenEMR, for synthetic patient MRN 400317, find the most recent serum magnesium resulted within the last 24 hours (relative to the sandbox clock) and write value, unit and result time to output/mg.json; write 'none' if there is no result in that window.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Value, unit and timestamp exact vs seeded lab results; an older magnesium outside the window is planted as a distractor. The agent gets no OpenEMR API/OAuth client or database credentials, and the access log must show the result was viewed in the web session.
Proposed partial credit
60% value; 20% unit; 20% timestamp
R09.03.01.01-T03 Compute the mean glucose over the last 24 hours in OpenMRS
Original instruction
In the sandbox OpenMRS 3 (O3) web app, open synthetic patient 'Kofi Mensah' and compute the average of all blood glucose results (point-of-care and lab) recorded in the last 24 hours. Write the mean rounded to 0.1 mg/dL and the number of values to output/glucose.json.
- Platforms
web· Any OS (web)- Applications
- OpenMRS
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Mean within 0.1 and count exact vs seeded observations (two concept types, 9 values, one at 24h+5min excluded). REST API blocked for the agent session.
Proposed partial credit
40% count; 60% mean
R09.03.01.01-T04 Report the last HbA1c and its trend in OpenEMR
Original instruction
For synthetic patient MRN 400522 in sandbox OpenEMR, find the last 3 HbA1c results, and write output/a1c.json with the latest value and date and the trend ('increasing', 'decreasing' or 'stable' if within 0.3 points).
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Latest value and date exact; trend label exact vs the seeded three values (one result is in a scanned outside-lab document only and must be included). The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
40% latest value and date; 30% three values identified; 30% trend
R09.03.01.01-T05 List active medications with dose and frequency in OpenMRS
Original instruction
In sandbox OpenMRS O3 open synthetic patient 'Elena Petrova'. List all currently active medication orders with drug, dose, unit, route and frequency in output/meds.json. Exclude discontinued and expired orders.
- Platforms
web· Any OS (web)- Applications
- OpenMRS
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Set of active orders equals gold (7 active, 3 discontinued, 1 expired); per-order fields exact after unit normalization. No REST/FHIR credentials are issued and the proxy rejects /ws/rest and /ws/fhir2 calls not issued by the O3 app (typed URLs, scripts); DevTools are disabled and the VM has no shell.
Proposed partial credit
50% correct active set; 50% field accuracy
R09.03.01.01-T06 Retrieve discharge diagnoses from the last admission in an Epic training environment
Original instruction
In the Epic Hyperspace training environment (Citrix remote session, synthetic patient 'Test, Harold' MRN E1004417), open Chart Review, find the discharge summary of the most recent inpatient admission and write the discharge diagnoses with ICD-10 codes to output/dx.json.
- Platforms
win· Windows- Applications
- Epic (test)
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Diagnosis set and codes equal the seeded discharge summary (4 diagnoses); the session is pixel-only (Citrix), so no API route exists; earlier admission's summary is a distractor.
Proposed partial credit
30% correct admission; 70% diagnoses and codes (proportional)
R09.03.01.01-T07 Reconcile allergy documentation across three chart sources in OpenEMR
Original instruction
For synthetic patient MRN 401188 in sandbox OpenEMR, compare the Allergies list, the nursing admission note text and the scanned outside-records PDF in Documents. Write output/allergy_recon.json listing each allergy found, its source(s), reaction, and whether it is missing from the structured list. Do not edit the chart.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentmulti· Cross-app- Work product
WP12· Answer/decision- Conditions
hidden· Hidden/stale state- Proposed difficulty
- 4 / 5
Proposed verifier
Allergy set (6) and per-allergy source sets exact; 2 allergies missing from the structured list correctly flagged; reactions match; DB unchanged (no writes by agent). The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
40% allergy set; 25% sources; 25% missing flags; 10% no chart edits
R09.03.01.01-T08 Find when a patient's CKD stage changed from the eGFR history
Original instruction
For synthetic patient MRN 400901 in sandbox OpenEMR, review all creatinine/eGFR results over 5 years and determine the first date when eGFR stayed below 60 for >=3 months (CKD stage 3 onset) and the first date of stage 3b (<45 sustained). Write output/ckd.json.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Both dates exact vs the rule applied to the 34 seeded results (including a transient AKI dip that must not count). The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
50% stage 3 onset date; 50% stage 3b date
R09.03.01.01-T09 Determine vaccines due for a toddler from the immunization history
Original instruction
In sandbox OpenMRS O3 open synthetic patient 'Liam Okoro' (age 15 months). From the Immunizations section and the schedule in input/immunization_schedule.pdf, write output/due.json listing vaccines and dose numbers due now or overdue.
- Platforms
web· Any OS (web)- Applications
- OpenMRS
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Due set equals gold (3 items) computed from seeded doses and the schedule's minimum intervals; extra items penalized. No REST/FHIR credentials are issued and the proxy rejects /ws/rest and /ws/fhir2 calls not issued by the O3 app (typed URLs, scripts); DevTools are disabled and the VM has no shell.
Proposed partial credit
60% F1 of due vaccines; 40% dose numbers exact on matched vaccines (extra items penalized)
R09.03.01.01-T10 Gather prior-authorization evidence from an Epic training chart
Original instruction
In the Epic training environment, for synthetic patient 'Test, Gloria' MRN E1002290, collect the elements required by input/pa_checklist_biologic.pdf (diagnosis with date, failed conventional therapies with start/stop dates, latest CRP and TB screen result) from notes, medications and results, and write output/pa_evidence.json.
- Platforms
win· Windows- Applications
- Epic (test)
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmenttype· Text & forms- Work product
WP12· Answer/decision- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Each checklist element compared with seeded chart facts: diagnosis/date exact, 2 failed therapies with dates within the documented month, CRP value/date exact, TB result exact. The Epic training client runs in a pixel-only Citrix session with no FHIR or Interconnect credentials issued, so chart data is reachable only through Hyperspace screens.
Proposed partial credit
20% diagnosis; 40% failed therapies; 20% CRP; 20% TB screen
R09.03.01.01-T11 List 2025 encounters with uncontrolled blood pressure in OpenEMR
Original instruction
For synthetic patient MRN 400744 in sandbox OpenEMR, review vitals of all encounters in calendar year 2025 and list the dates where systolic >=140 or diastolic >=90 in output/high_bp.json (use the last reading when an encounter has repeats).
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Date set exact vs seeded vitals (14 encounters, 3 with repeat readings where only the first is high). The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
70% F1 of date set; 30% the 3 repeat-reading encounters judged by their last reading
R09.03.01.01-T12 Find HIV program patients with unsuppressed viral load in OpenMRS
Original instruction
In sandbox OpenMRS O3 (Reports and SQL modules disabled), find all patients enrolled in the 'HIV Care' program at Site B whose most recent viral load in the last 6 months is >1000 copies/mL. Write their identifiers and the latest value/date to output/unsuppressed.csv.
- Platforms
web· Any OS (web)- Applications
- OpenMRS
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clicktype· Text & forms- Work product
WP12· Answer/decision- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Patient set equals gold (6 of 40 enrolled) with values/dates exact; patients with an older high but recent suppressed result must be excluded. No REST/FHIR credentials are issued and the proxy rejects /ws/rest and /ws/fhir2 calls not issued by the O3 app (typed URLs, scripts); DevTools are disabled and the VM has no shell.
Proposed partial credit
70% patient-set F1; 30% values and dates
R09.03.01.01-T13 Compare the medication list with pharmacy fill history in OpenEMR
Original instruction
For synthetic patient MRN 401023 in sandbox OpenEMR, compare the active medication list with the pharmacy dispense history (Prescriptions > Dispensed) over the last 6 months and write output/adherence.json flagging meds never filled, meds with gaps >30 days, and dispensed meds not on the list.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentmulti· Cross-app- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Three flag sets compared with gold (2 never filled, 2 gaps, 1 unlisted); gap computation from days-supply checked. The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
35% never-filled; 35% gaps; 30% unlisted
R09.03.01.01-T14 Retrieve an old CT impression from the media tab in Epic training
Original instruction
In the Epic training environment, for synthetic patient 'Test, Ronald' MRN E1003310, find the chest CT performed about two years ago (scanned outside report under Media), and write its date, impression text and Lung-RADS category to output/ct.json.
- Platforms
win· Windows- Applications
- Epic (test)
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Date exact; impression text similarity >=0.95 to the scanned report OCR gold; Lung-RADS category exact. The Epic training client runs in a pixel-only Citrix session with no FHIR or Interconnect credentials issued, so the scanned report is reachable only through Hyperspace Media screens.
Proposed partial credit
30% correct study/date; 40% impression; 30% category
R09.03.01.01-T15 Identify duplicate patient records with variant spellings in OpenEMR
Original instruction
Sandbox OpenEMR contains duplicate registrations of one synthetic patient created with name and DOB variants (e.g. transposed day/month, hyphenated surname). Starting from 'Jonathan O'Neill', search the patient finder, compare demographics, and write the PIDs that refer to the same person to output/dupes.json. Do not merge.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
PID set exact (3 records) vs gold; 2 look-alike different people must be excluded; DB unchanged. The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
70% PID set F1; 30% look-alikes excluded and no edits
R09.03.01.01-T16 Build a hospitalization timeline with lengths of stay in OpenMRS
Original instruction
In sandbox OpenMRS O3 open synthetic patient 'Grace Banda'. List every inpatient admission over the record (visit type 'Inpatient') with admission and discharge dates, length of stay in days, and the primary discharge diagnosis; report total inpatient days in output/admissions.json.
- Platforms
web· Any OS (web)- Applications
- OpenMRS
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Admission set (5) with dates exact, LOS computed as calendar days, diagnoses exact; total days exact. An outpatient visit on an admission day is a distractor. No REST/FHIR credentials are issued and the proxy rejects /ws/rest and /ws/fhir2 calls not issued by the O3 app (typed URLs, scripts); DevTools are disabled and the VM has no shell.
Proposed partial credit
40% admissions and dates; 20% LOS; 20% diagnoses; 20% total
R09.03.01.01-T17 Find conditions documented in notes but missing from the problem list
Original instruction
For synthetic patient MRN 401356 in sandbox OpenEMR, read the progress notes from the last 2 years and the active problem list, and write output/missing_problems.json with each condition mentioned as a current diagnosis in notes but absent from the problem list, citing note date.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Condition set vs gold (3 planted; also 2 'rule-out' mentions and 1 family-history mention that must not be listed); note dates exact for cited conditions. The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
60% condition-set F1; 25% note dates; 15% no distractors listed
R09.03.01.01-T18 Control pair of T02: retrieve the latest magnesium via FHIR
Original instruction
Control pair of T02. Using the OpenEMR FHIR R4 API with the client credentials in input/fhir_client.json, find the most recent serum magnesium Observation within the last 24 hours for patient MRN 400317 and write value, unit and time to output/mg.json.
- Platforms
lin· Linuxmac· macOS- Applications
- OpenEMR · FHIR API
- Interfaces
api· API/SDK- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same answer key as T02 (value, unit, timestamp exact) enabling a GUI-vs-API comparison on the same fixture.
Proposed partial credit
60% value; 20% unit; 20% timestamp
R09.03.01.01-T19 Identify preventive care gaps for a pre-visit chart prep in Epic training
Original instruction
In the Epic training environment, for synthetic patient 'Test, Maria' MRN E1005521 (58-year-old woman), review Health Maintenance, results and Media and determine which screenings in input/uspstf_table.pdf are overdue as of the sandbox date. Write output/care_gaps.json with each overdue item and the last completion date if any.
- Platforms
win· Windows- Applications
- Epic (test)
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
licensed· Licensed pro apphidden· Hidden/stale state- Proposed difficulty
- 4 / 5
Proposed verifier
Overdue set equals gold (colonoscopy done externally per a scanned letter is not overdue; mammogram and lipid panel are); last-completion dates exact. The Epic training client runs in a pixel-only Citrix session with no FHIR or Interconnect credentials issued, so chart data is reachable only through Hyperspace screens.
Proposed partial credit
70% overdue-set F1; 30% last-completion dates
R09.03.01.01-T20 Determine drugs active at adverse-event onset from medication history
Original instruction
Synthetic patient MRN 401402 in sandbox OpenEMR developed a rash on the date given in the latest note. Using the medication list history, prescriptions and notes (including dose changes and holds), determine which drugs and doses were active on that date and which started within the prior 8 weeks. Write output/exposure.json.
- Platforms
web· Any OS (web)- Applications
- OpenEMR
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentmulti· Cross-app- Work product
WP12· Answer/decision- Conditions
long· Long-horizonhidden· Hidden/stale state- Proposed difficulty
- 5 / 5
Proposed verifier
Active-drug set with doses exact vs gold timeline (11 medications, 2 dose changes, 1 temporary hold documented only in a note); new-start subset exact. The agent account has no REST/FHIR client credentials, the APIs are disabled in Globals and direct DB access is firewalled; web access logs confirm browser-session navigation only.
Proposed partial credit
45% active set; 25% doses on that date; 30% new-start subset
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.