Computer-Use Agent Taxonomy中文
R09.05.01.03 · TASK CLASS

CDISC SDTM mapping

Map raw trial data to SDTM and pass validation.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Pinnacle 21 · SAS · R
Class GUI-need label
CL · CLI-sufficient

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R09.05.01.03-T01 Map raw demographics to an SDTM DM dataset in R

Original instruction

fixtures/raw/dm_raw.csv (synthetic study ABC-301, 48 subjects) and fixtures/rand.csv hold demographics and arm assignment. Write an R script that produces sdtm/dm.xpt (SAS v5 transport) with STUDYID, DOMAIN, USUBJID, SUBJID, RFSTDTC, RFENDTC, SITEID, AGE, AGEU, SEX, RACE, ETHNIC, ARMCD, ARM, ACTARMCD, ACTARM, COUNTRY per SDTMIG 3.4 and fixtures/ct_package.xlsx.

Platforms
mac · macOS lin · Linux
Applications
R · haven
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Hidden gold dm.xpt compared keyed by USUBJID (values, labels, types, lengths <= 200); CDISC CORE engine run on the output with 0 Error-level DM findings. Script re-run from a clean checkout must reproduce the file byte-identically.

Proposed partial credit

50% values match gold; 25% labels/types/lengths; 25% zero DM validator errors.

R09.05.01.03-T02 Reshape wide vital signs into a long VS domain with baseline flags

Original instruction

fixtures/raw/vitals_wide.csv has one row per visit with SYSBP, DIABP, PULSE, TEMP, WEIGHT, HEIGHT columns and a dosing time. Produce sdtm/vs.xpt with VSTESTCD/VSTEST, VSORRES/VSORRESU, VSSTRESC/VSSTRESN/VSSTRESU, VISITNUM, VISIT, VSDTC, VSDY and VSBLFL (last non-missing value on or before first dose per test). Use R or SAS.

Platforms
lin · Linux win · Windows
Applications
R · SAS
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Row count and key-based diff vs gold vs.xpt (USUBJID, VSTESTCD, VISITNUM, VSDTC); VSBLFL set exactly on gold rows; VSDY has no day 0; CORE engine 0 Error-level VS findings.

Proposed partial credit

40% reshape and values; 30% VSBLFL exact; 15% VSDY; 15% validator clean.

R09.05.01.03-T03 Map adverse events with MedDRA coding to AE and SUPPAE

Original instruction

fixtures/raw/ae_raw.csv (EDC export) and fixtures/ae_coded.csv (MedDRA coding output) for study ABC-301. Produce sdtm/ae.xpt (AETERM, AEDECOD, AEBODSYS, AESOC, AESEV, AESER and the SAE criteria flags, AEREL, AEACN, AEOUT, AESTDTC, AEENDTC, AESTDY, AEENDY, AESEQ) and sdtm/suppae.xpt for the two sponsor-defined fields listed in fixtures/spec_ae.xlsx.

Platforms
mac · macOS lin · Linux
Applications
R
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Gold diff on AE keyed by USUBJID+AESPID and on SUPPAE keyed by USUBJID+IDVARVAL+QNAM; QNAM <= 8 chars, QLABEL <= 40, IDVAR=AESEQ links resolve; CORE engine 0 Error-level AE/SUPPAE findings.

Proposed partial credit

45% AE values; 25% SUPPAE structure and linkage; 15% study days; 15% validator clean.

R09.05.01.03-T04 Map concomitant medications with partial ISO 8601 dates and WHODrug fields

Original instruction

fixtures/raw/cm_raw.csv contains 612 medication rows with free-text partial dates ('UNK-MAR-2025', '2025', 'ONGOING') and fixtures/whodrug_coded.csv. Produce sdtm/cm.xpt with CMTRT, CMDECOD, CMCLAS/CMCLASCD (ATC level 4), CMDOSE/CMDOSU, CMROUTE, CMSTDTC/CMENDTC as truncated ISO 8601, CMENRTPT/CMENTPT for ongoing meds, and CMSTDY only when the date is complete.

Platforms
lin · Linux
Applications
R · SAS
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Key-based diff vs gold cm.xpt; all --DTC values match ISO 8601 regex with no imputation; ongoing rows use CMENRTPT='ONGOING' with CMENTPT; CORE engine 0 Error-level CM findings.

Proposed partial credit

35% treatment/coding fields; 35% partial dates exact; 15% ongoing handling; 15% validator clean.

R09.05.01.03-T05 Fix all Error-level Pinnacle 21 findings on a staged SDTM package

Original instruction

sdtm_staged/ holds 9 XPT datasets and define.xml for study ABC-301. Open Pinnacle 21 Community, validate against SDTMIG 3.4 with the define.xml and the provided CT package, then correct the data or metadata behind every Error-level issue (do not delete records to silence them). Re-validate and save the final report as output/p21_final.xlsx.

Platforms
win · Windows mac · macOS
Applications
Pinnacle 21 Community · R
Interfaces
desk · Desktop GUI term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
click · Point & click code · Code & commands nav · Pan/zoom/scroll
Work product
WP02 · Records/table
Conditions
multiapp · Multi-app
Proposed difficulty
3 / 5

Proposed verifier

Grader re-runs the P21 validator (CLI) on the edited package: Error count 0, Warning count not higher than baseline; record counts per dataset unchanged; output/p21_final.xlsx exists with the matching configuration (standard, CT version) read from its summary sheet.

Proposed partial credit

50% Error count to 0; 25% no record deletion or new warnings; 25% final report with correct configuration.

R09.05.01.03-T06 Build a complete SDTM package with define.xml 2.1 for a small study

Original instruction

Study XYZ-102 (synthetic, 30 subjects) has raw EDC exports in fixtures/raw/ (11 CSVs), the protocol synopsis input/protocol.pdf and the annotated CRF input/acrf.pdf. Produce DM, AE, CM, VS, LB, EX, DS, SV, MH and trial design domains TA, TE, TV, TI, TS as XPT files plus define.xml 2.1 with value-level metadata for VS and LB, all in sdtm/.

Platforms
mac · macOS lin · Linux
Applications
R · Pinnacle 21 Community
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

Per-domain key-based diff vs gold (pass >= 98% cell agreement per domain); CORE engine and P21 CLI: 0 Errors; define.xml validates against the Define-XML 2.1 schema and its ItemDefs match dataset variables 1:1.

Proposed partial credit

40% subject-level domains match gold; 20% trial design domains; 20% define.xml schema-valid and consistent; 20% zero validator errors.

R09.05.01.03-T07 Derive study day variables for a lab dataset

Original instruction

sdtm_staged/lb.xpt is missing LBDY and dm.xpt has RFSTDTC. Add LBDY to lb.xpt using the SDTM convention (no day 0; days before reference start are negative; leave blank when either date is partial) and keep every other variable and attribute unchanged. Save as sdtm/lb.xpt.

Platforms
lin · Linux
Applications
R
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
1 / 5

Proposed verifier

LBDY equals gold for all 4,212 rows (blank where gold is blank); all other columns byte-equal to input; variable label 'Study Day of Specimen Collection' and numeric type.

Proposed partial credit

70% LBDY exact; 15% partial-date blanks; 15% other columns untouched.

R09.05.01.03-T08 Standardize lab units to SI and derive reference-range indicators in LB

Original instruction

fixtures/raw/lab_raw.csv mixes conventional and SI units from two central labs; fixtures/unit_conversions.xlsx gives factors and fixtures/ref_ranges.csv gives ranges by lab, sex and age band. Produce sdtm/lb.xpt with original results/units in LBORRES/LBORRESU, standardized LBSTRESN/LBSTRESU in SI, LBSTNRLO/LBSTNRHI and LBNRIND (LOW/NORMAL/HIGH).

Platforms
lin · Linux win · Windows
Applications
SAS · R
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

LBSTRESN within 0.1% relative of gold for all numeric rows; LBSTRESU exact; LBNRIND exact on 100% of rows with ranges; character results ('<0.5') kept in LBSTRESC with LBSTRESN null; CORE engine clean for LB.

Proposed partial credit

40% SI conversion; 25% reference ranges joined; 20% LBNRIND; 15% character results handled.

R09.05.01.03-T09 Generate define.xml 2.1 with value-level metadata from a spec in Pinnacle 21

Original instruction

input/define_spec_draft.xlsx (Pinnacle 21 Community spec template) for study ABC-301 lacks codelists for VSTESTCD/LBTESTCD and value-level metadata for VSORRESU and LBORRESU by test. Complete the spec in Excel, generate define.xml in Pinnacle 21 Community with the CDISC stylesheet, and confirm the rendered HTML shows working links from VS variables to their value-level rows.

Platforms
win · Windows mac · macOS
Applications
Pinnacle 21 Community · Excel
Interfaces
desk · Desktop GUI
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
click · Point & click type · Text & forms vis · Visual judgment
Work product
WP02 · Records/table
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

define.xml validates against the Define-XML 2.1 schema; codelists and ValueListDefs match gold (set equality on test codes and where-clauses); P21 define validation 0 Errors; grader renders it with the stylesheet in headless Chromium and checks every VLM hyperlink resolves.

Proposed partial credit

30% codelists; 40% value-level metadata and where-clauses; 30% schema-valid and links resolve.

R09.05.01.03-T10 Move non-standard demographics variables into SUPPDM

Original instruction

fixtures/raw/dm_raw.csv includes three sponsor fields not in SDTMIG DM: handedness, years of education and a genetic-substudy consent flag. Build sdtm/suppdm.xpt with RDOMAIN, USUBJID, IDVAR (blank), QNAM, QLABEL, QVAL, QORIG and QEVAL per fixtures/spec_dm.xlsx, and make sure DM itself is unchanged.

Platforms
mac · macOS lin · Linux
Applications
R
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Key-based diff vs gold suppdm.xpt; QNAM values exact and <= 8 chars; QLABEL <= 40 chars; no blank QVAL rows; dm.xpt checksum unchanged; CORE engine 0 Error-level SUPPDM findings.

Proposed partial credit

50% SUPPDM rows; 25% QNAM/QLABEL/QORIG metadata; 25% DM untouched and validator clean.

R09.05.01.03-T11 Collapse a daily dosing log into EX records with interruptions

Original instruction

fixtures/raw/dosing_log.csv has one row per subject per day with dose taken (mg), dose modifications and reasons. Produce sdtm/ex.xpt where consecutive days at the same dose and frequency collapse into one record (EXSTDTC/EXENDTC), interruptions start a new record, and EXDOSE, EXDOSU, EXDOSFRQ, EXROUTE, EXADJ are populated per fixtures/spec_ex.xlsx.

Platforms
lin · Linux
Applications
R · SAS
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Record-level diff vs gold ex.xpt keyed by USUBJID+EXSTDTC (pass: 100% intervals match); total dose per subject recomputed from EX equals the raw log total; EXADJ populated only where a modification occurred; CORE engine clean.

Proposed partial credit

45% interval collapsing; 25% dose totals reconcile; 15% EXADJ; 15% validator clean.

R09.05.01.03-T12 Build trial design domains for a three-period crossover from the protocol

Original instruction

input/protocol_xover.pdf describes a 3-period, 3-sequence crossover with screening, 7-day washouts and follow-up, plus inclusion/exclusion criteria. Produce TA, TE, TV, TI and TS XPTs (TS must include the required parameters listed in fixtures/ts_required.csv) and write the same values into the define.xml in sdtm_staged/.

Platforms
mac · macOS lin · Linux
Applications
R · Pinnacle 21 Community
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
5 / 5

Proposed verifier

TA arms/elements/epochs match gold exactly (order-sensitive on TAETORD); TE start/end rules and TV planned visits match gold; TI has every criterion with IETESTCD <= 8 chars; TS covers all required TSPARMCD; P21 CLI 0 Errors on trial design domains.

Proposed partial credit

35% TA/TE structure; 20% TV visits; 20% TI criteria; 25% TS parameters and validator clean.

R09.05.01.03-T13 Map medical history with MedDRA coding in SAS

Original instruction

fixtures/raw/mh_raw.sas7bdat and fixtures/mh_coded.csv hold medical history for study ABC-301. In SAS write mh_map.sas that creates sdtm/mh.xpt with MHTERM, MHDECOD, MHBODSYS, MHCAT, MHPRESP/MHOCCUR for pre-specified conditions, MHSTDTC, and MHENRTPT='ONGOING' with MHENTPT = screening date for ongoing conditions.

Platforms
win · Windows lin · Linux
Applications
SAS
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

mh_map.sas runs in batch SAS without errors in the log; key-based diff vs gold mh.xpt; pre-specified rows carry MHPRESP='Y' and correct MHOCCUR; CORE engine 0 Error-level MH findings.

Proposed partial credit

45% values; 25% pre-specified handling; 15% ongoing variables; 15% clean SAS log and validator.

R09.05.01.03-T14 Convert raw date strings to ISO 8601 across three domains

Original instruction

sdtm_staged/ae.xpt, cm.xpt and ds.xpt store --DTC values in raw EDC formats ('12MAR2025 14:30', 'UN-JAN-2025', '2025/02/--'). Rewrite every --DTC variable to ISO 8601 (keeping the original precision, never imputing) and save to sdtm/ without changing any other variable.

Platforms
lin · Linux
Applications
R · Python
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

All --DTC values match gold strings exactly and the ISO 8601 regex; non-DTC columns byte-equal to inputs; row counts unchanged.

Proposed partial credit

70% DTC values exact; 30% no other changes.

R09.05.01.03-T15 Triage a 40-issue Pinnacle 21 report: fix true errors, explain the rest in the cSDRG

Original instruction

input/p21_report.xlsx lists 40 issues on sdtm_staged/. Fix every issue caused by a mapping error (in the data or define.xml) after reviewing it in Pinnacle 21 Community, then add an entry to output/csdrg_issues.xlsx (columns: Rule ID, Dataset, Count, Explanation) for each issue that is justified by study design and remains.

Platforms
win · Windows mac · macOS
Applications
Pinnacle 21 Community · Excel · R
Interfaces
desk · Desktop GUI term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
click · Point & click code · Code & commands type · Text & forms
Work product
WP01 · Document
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

P21 CLI re-run: all 11 seeded mapping errors gone, no new issues; the set of rule/dataset pairs in csdrg_issues.xlsx equals the remaining-issue set exactly; Explanation quality scored by an LLM rubric (only that part, 15%).

Proposed partial credit

50% seeded mapping errors fixed; 35% remaining issues listed exactly; 15% explanation quality (LLM rubric).

R09.05.01.03-T16 Map 12-lead ECG results to the EG domain

Original instruction

fixtures/raw/ecg_central.csv is a central-reader export with PR, QRS, QT, RR, QTcF intervals in ms and an overall interpretation, one row per triplicate tracing. Produce sdtm/eg.xpt with EGTESTCD/EGTEST from the CT package, EGORRES/EGSTRESN in msec, EGTPT/EGTPTNUM for the triplicates, EGEVAL='INDEPENDENT ASSESSOR' and EGBLFL.

Platforms
mac · macOS lin · Linux
Applications
R
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Key-based diff vs gold eg.xpt (USUBJID, EGTESTCD, VISITNUM, EGTPTNUM); interpretation stored as EGTESTCD='INTP' with character result; EGBLFL exact; CORE engine 0 Error-level EG findings.

Proposed partial credit

45% interval values; 20% interpretation rows; 20% timepoints and baseline flag; 15% validator clean.

R09.05.01.03-T17 Annotate a blank CRF with SDTM mappings in Acrobat Pro

Original instruction

input/blankcrf.pdf (42 pages, study ABC-301) and the mapping spec fixtures/spec_all.xlsx. In Adobe Acrobat Pro add SDTM annotations per the CDISC Metadata Submission Guidelines: a domain box per form, a variable text box next to each collected field, 'NOT SUBMITTED' for unmapped fields, consistent fonts/colors per domain, and bookmarks by visit and by form. Save as output/acrf.pdf.

Platforms
win · Windows mac · macOS
Applications
Adobe Acrobat Pro
Interfaces
desk · Desktop GUI
GUI need · task setting
G2 · GUI-essential
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
click · Point & click type · Text & forms geom · Precise geometry drag · Drag & drop
Work product
WP10 · Annotations
Conditions
licensed · Licensed pro app long · Long-horizon
Proposed difficulty
5 / 5

Proposed verifier

Grader parses FreeText annotations: each of 318 CRF fields has an annotation with the gold variable text within 40 pt of the field bbox (IoU-free proximity); domain boxes per page match gold; colors consistent per domain; both bookmark trees present. Acrobat's JavaScript console is disabled and no PDF libraries are installed on the VM; annotations without Acrobat /RC rich-text streams (scripted insertion) are rejected.

Proposed partial credit

45% variable annotations correct and placed; 20% domain boxes and colors; 15% NOT SUBMITTED; 20% bookmarks.

R09.05.01.03-T18 Map subject disposition and protocol milestones to DS

Original instruction

fixtures/raw/ds_raw.csv holds consent, randomization, end-of-treatment and end-of-study pages, with free-text reasons for discontinuation. Produce sdtm/ds.xpt with DSCAT ('PROTOCOL MILESTONE' vs 'DISPOSITION EVENT'), DSSCAT for epoch-level disposition, DSDECOD from the NCOMPLT codelist in the CT package, DSTERM verbatim, DSSTDTC and EPOCH.

Platforms
lin · Linux
Applications
R · SAS
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Key-based diff vs gold ds.xpt; every DSDECOD is in the codelist or an 'OTHER' row with verbatim DSTERM; each randomized subject has exactly one end-of-treatment and one end-of-study disposition; CORE engine clean for DS.

Proposed partial credit

35% milestone rows; 35% disposition events and DSDECOD; 15% one-per-epoch rule; 15% validator clean.

R09.05.01.03-T19 Upgrade controlled terminology to a newer CDISC CT package

Original instruction

sdtm_staged/ (8 domains) and define.xml use an older CDISC CT package; fixtures/ct_new.xlsx is the newer package and fixtures/ct_changes.csv its change log. Recode every affected value, add sponsor extensions only for extensible codelists, update codelist versions and terms in define.xml, and list unresolved non-extensible conflicts in output/ct_conflicts.csv.

Platforms
mac · macOS lin · Linux
Applications
R · Python
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Value-level diff vs gold datasets; define.xml codelists equal gold (including ExtendedValue flags); ct_conflicts.csv equals the 4 seeded conflicts; P21 CLI with the new CT: 0 terminology Errors except the listed conflicts.

Proposed partial credit

40% recoded values; 25% define.xml codelists; 20% conflicts listed exactly; 15% validator result.

R09.05.01.03-T20 Produce a dataset-level change log between two SDTM deliveries

Original instruction

delivery_v1/ and delivery_v2/ hold two transfers of the same 10 SDTM domains for study ABC-301. Produce output/sdtm_changes.csv with one row per change (DOMAIN, USUBJID, key, VARIABLE, OLD, NEW, CHANGE_TYPE in {added, deleted, modified}) using natural keys from define.xml, plus a per-domain summary in output/summary.csv.

Platforms
mac · macOS lin · Linux
Applications
R · Python
Interfaces
term · Terminal
GUI need · task setting
CL · CLI-sufficient
GUI need · ordinary access
CL · CLI-sufficient
Interaction primitives
code · Code & commands
Work product
WP02 · Records/table
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

sdtm_changes.csv compared as a set with the gold change log (F1 >= 0.99 on DOMAIN+key+VARIABLE+CHANGE_TYPE, values exact); summary counts equal gold; --SEQ renumbering alone must not be reported as modifications.

Proposed partial credit

60% change-log F1; 20% summary counts; 20% no false changes from --SEQ renumbering.

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

In this workflow family