Audit testing & regulatory filings
Test controls, sample transactions and assemble regulatory filings.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
Previous class address: R05.04.01.02 · Original proposal IDs are retained.
- Typical applications
- Excel · AuditBoard · regulator portals
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R05.04.01.02-T01 Draw a seeded monetary-unit sample of 25 AP invoices in Excel
Original instruction
Excel workbook input/ap_population_fy26.xlsx (3,412 invoices). Per input/sampling_memo.pdf, draw a monetary-unit sample of 25 using systematic selection with interval = population total / 25 and random start 0.6180 x interval; list selected invoice IDs (top-stratum items selected once) on 'Sample' with cumulative amounts.
- Platforms
mac· macOSwin· Windows- Applications
- Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Selected IDs equal key in order; interval and start cells equal key; cumulative-amount column is formula-driven (grader perturbs one amount and checks propagation).
Proposed partial credit
30% interval and start; 60% selected IDs; 10% formulas
R05.04.01.02-T02 Record test results and evidence for three controls in AuditBoard
Original instruction
AuditBoard test tenant, SOX module, FY2026 project. For controls REV-03, AP-07 and ITGC-12, attach the evidence files in input/evidence/, complete the test attribute steps per each control's test sheet (A-D), and set test conclusion 'Effective' where all attributes pass; REV-03 attribute C fails, so conclude 'Deficient' with a comment.
- Platforms
web· Any OS (web)- Applications
- AuditBoard
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
licensed· Licensed pro app- Proposed difficulty
- 2 / 5
Proposed verifier
AuditBoard export: 3 test records with attachments; attribute results and conclusions equal key; REV-03 comment present. AuditBoard API tokens are not issued to the agent; records must show the agent's web session in the activity log.
Proposed partial credit
30% evidence attached; 40% attributes recorded; 30% conclusions
R05.04.01.02-T03 Compute planning materiality from draft financial statements
Original instruction
Excel. input/draft_fs_fy26.pdf contains the draft income statement. Per input/materiality_policy.pdf, overall materiality is 5% of profit before tax from continuing operations, performance materiality 75% of overall, and clearly-trivial 5% of overall. Write the three amounts (rounded down to the nearest 1,000) to output/materiality.txt.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Adobe Acrobat Reader
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & forms- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
Three amounts equal key exactly; benchmark must exclude discontinued operations (decoy line).
Proposed partial credit
50% overall materiality; 25% performance; 25% trivial threshold
R05.04.01.02-T04 Run journal-entry risk criteria in IDEA and export exceptions
Original instruction
CaseWare IDEA (Windows). Import input/gl_fy26.csv (184k lines) with the given field definitions, then run criteria from input/je_testing_plan.pdf: entries posted on weekends, after the close date, round amounts >= 10,000 ending in 000, and by users not in the finance list. Export each exception set and a summary count table to output/je_exceptions.xlsx.
- Platforms
win· Windows- Applications
- IDEA · Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Exception sets equal key per criterion (entry-ID set equality); summary counts match; IDEA project history log must show the extractions (equations recorded).
Proposed partial credit
20% import with correct field types; 60% four exception sets (equal shares); 20% summary
R05.04.01.02-T05 Run a first-two-digits Benford test on disbursements
Original instruction
Excel or Galvanize ACL. input/disbursements_fy26.csv has 48,201 payments. Exclude amounts under 10.00, compute first-two-digit frequencies (10-99) vs Benford expectations, mean absolute deviation and its conformity class (Nigrini thresholds), and list the 3 digit pairs with highest z-statistic in output/benford.xlsx.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Galvanize ACL
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Observed counts per pair equal key; MAD within 0.0001; conformity class exact; top-3 pairs equal key.
Proposed partial credit
40% observed frequencies; 30% MAD and class; 30% top z pairs
R05.04.01.02-T06 Three-way match a sample of 40 purchases and document attributes
Original instruction
Workpaper template input/wp_purchases.xlsx lists 40 sampled purchases; PDFs of POs, goods receipts and invoices are in input/docs/ (mixed scans). For each item, test attributes A (PO approved within limit), B (receipt qty = invoice qty), C (price matches PO), D (correct GL period), mark P/F, and summarize exceptions with projected misstatement.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Adobe Acrobat Reader
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgmentmulti· Cross-app- Work product
WP02· Records/table- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 3 / 5
Proposed verifier
Per-item attribute results equal key (160 cells, >= 97% for full credit); exception list and projected misstatement equal key. Agent VM has no terminal, Python or OCR tools and Excel macros are disabled, so source documents must be read in Acrobat and the workpaper saved by Excel (docProps/app.xml checked).
Proposed partial credit
60% attribute accuracy; 20% exceptions summarized; 20% projection
R05.04.01.02-T07 Tie bank confirmations to year-end reconciliations
Original instruction
Excel. input/confirms/ has 6 bank confirmation PDFs (one scanned) and input/bank_recs_fy26.xlsx has the client's reconciliations at 2026-09-30. Agree confirmed balances to the recs, investigate differences using the outstanding items list, and record agreed/unagreed status, differences and explanation per account on 'Tie-out'.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Adobe Acrobat Reader
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Per-account confirmed balance, difference and status equal key; the one unexplained difference (key) is flagged; loan balance and covenant fields from confirms captured.
Proposed partial credit
50% balances agreed; 30% difference status; 20% loan/covenant fields
R05.04.01.02-T08 Find terminated users who still hold ERP access
Original instruction
SOX ITGC user-access review. input/erp_users.csv (role export), input/ad_disabled.csv and input/hr_terminations.xlsx. Identify users terminated on or before 2026-09-30 with active ERP access and any SoD conflicts per input/sod_matrix.xlsx, then log each finding as an issue in AuditBoard (test tenant) under control ITGC-04 with severity per rubric.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- Excel · AuditBoard
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
licensed· Licensed pro appmultiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
AuditBoard issues under ITGC-04 equal key set (user IDs), severities per rubric; no issues for users with name-only matches (decoy). AuditBoard API tokens are not issued to the agent; records must show the agent's web session in the activity log.
Proposed partial credit
50% finding set exact; 25% SoD conflicts identified; 25% issue severity
R05.04.01.02-T09 Fix Inline XBRL validation errors in a quarterly filing
Original instruction
input/q3_10q_ixbrl.zip is an Inline XBRL 10-Q that fails Arelle validation (EFM and calculation 1.1) with 5 errors: wrong context period on a duration fact, a sign error, a missing dei element, a calculation inconsistency and a duplicate fact. Fix them in the HTML (any editor) and save output/q3_10q_fixed.zip.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Arelle · VS Code
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Arelle EFM + calc 1.1 validation returns zero errors; fact values unchanged except the 5 corrections; rendered HTML text unchanged.
Proposed partial credit
40% errors 1-2 fixed (half each); 40% errors 3-4 fixed (half each); 20% error 5 fixed
R05.04.01.02-T10 Import a trial balance and post adjustments in CaseWare Working Papers
Original instruction
CaseWare Working Papers 2025 (Windows). Create a file for 'Kestrel Metals FY26', import input/tb_fy26.xlsx, map accounts to the standard lead sheets per input/mapping_guide.pdf, post the 4 AJEs in input/aje.xlsx, and produce the adjusted lead schedule for Cash, AR and Inventory as PDF.
- Platforms
win· Windows- Applications
- CaseWare Working Papers · Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
CaseWare file opened by grader: mapping codes per account equal key; 4 AJEs posted; adjusted balances on lead sheets equal key; PDFs generated from CaseWare. No automation SDK available to agent.
Proposed partial credit
20% import; 35% mapping; 25% AJEs; 20% lead schedules
R05.04.01.02-T11 Test an inventory count sample and project the misstatement
Original instruction
Excel. input/count_sheets/ (scanned tag sheets) and input/perpetual_2026-09-30.xlsx. For the 30 sampled SKUs listed on 'Sample', record counted vs book qty and unit cost, compute value differences, then project misstatement to the population using ratio estimation, and conclude against tolerable misstatement 85,000.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Adobe Acrobat Reader
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsvis· Visual judgment- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Counted qty per SKU equal key (handwritten tags); differences and projected misstatement within 0.5% of key; conclusion matches key. Agent VM has no terminal, Python or OCR tools and Excel macros are disabled, so tags must be read in Acrobat and the workpaper saved by Excel (docProps/app.xml checked).
Proposed partial credit
40% counted quantities; 30% value differences; 20% projection; 10% conclusion
R05.04.01.02-T12 Recalculate depreciation for a fixed-asset sample and evaluate the variance
Original instruction
Excel. input/fa_register_fy26.xlsx (1,140 assets) and input/fa_policy.pdf (straight-line, half-year convention, useful lives by class, 5,000 capitalization threshold). For the 30 assets on 'Sample', recalculate FY26 depreciation and accumulated depreciation, compute differences vs the register, and flag any asset below threshold that was capitalized.
- Platforms
mac· macOSwin· Windows- Applications
- Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Recalculated FY26 and accumulated depreciation per asset equal key to the cent; differences and flagged set equal key; total variance cell equals key.
Proposed partial credit
50% recalculations; 25% differences; 25% threshold flags
R05.04.01.02-T13 Complete a call-report loan schedule in a regulatory filing sandbox
Original instruction
FFIEC-style call report sandbox app (web). From input/loan_tape_2026-09-30.csv, classify loans per input/rc_c_instructions.pdf (real estate types, C&I, consumer, other), aggregate to Schedule RC-C Part I items, enter them, run the edit checks to zero validity errors and document quality-edit explanations. Save the draft; do not submit.
- Platforms
web· Any OS (web)- Applications
- Call report sandbox · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskymultiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Sandbox DB: RC-C items equal key; validity edits pass; quality-edit explanations present for the key's 2 triggered edits; status draft (not submitted).
Proposed partial credit
40% item classification totals; 30% edits clean; 20% explanations; 10% not submitted
R05.04.01.02-T14 Test revenue cut-off around year end and quantify misstatement
Original instruction
Excel. input/cutoff/ has 20 sales invoices dated 2026-09-24 to 10-07 with bills of lading, carrier proofs of delivery and contract incoterms. Determine for each whether revenue belongs in FY26 or FY27 (control transfer per incoterm), compute the cut-off misstatement and proposed AJE, and fill input/wp_cutoff.xlsx.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Adobe Acrobat Reader
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsvis· Visual judgmentmulti· Cross-app- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Per-invoice period conclusion equals key; misstatement amount and AJE lines equal key. Agent VM has no terminal, Python or OCR tools and Excel macros are disabled, so invoices and shipping documents must be read in Acrobat and the workpaper saved by Excel (docProps/app.xml checked).
Proposed partial credit
60% per-invoice conclusions; 25% misstatement; 15% AJE
R05.04.01.02-T15 Build and validate an ESEF annual-report package
Original instruction
input/esef_ar_2025.xhtml is an annual report with partial iXBRL tags. Tag the primary statements to the IFRS 2025 taxonomy, anchor every extension element to a wider IFRS element, mark the block-tagged notes, package as an ESEF report package (ZIP with META-INF) and validate with Arelle's ESEF plugin. Save output/esef_package.zip.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Arelle · VS Code
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Arelle ESEF validation: zero errors; primary-statement facts mapped to key concepts (>= 95%); all extensions anchored; package structure valid.
Proposed partial credit
40% statement tagging; 20% anchoring; 20% block tags; 20% clean validation
R05.04.01.02-T16 Compute broker-dealer net capital for a FOCUS Part II filing draft
Original instruction
FINRA eFOCUS-style sandbox (web). From input/bd_trial_balance.xlsx and input/haircut_schedule.pdf, compute net capital: deduct non-allowable assets, apply securities haircuts by category, compute aggregate indebtedness and minimum requirement, enter the computation in the FOCUS Part II form and save as draft without submitting.
- Platforms
web· Any OS (web)- Applications
- FOCUS sandbox · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskymultiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Sandbox form values: non-allowables, haircuts, net capital, AI and excess net capital equal key; status draft.
Proposed partial credit
25% non-allowables; 30% haircuts; 30% net capital and excess; 15% draft only
R05.04.01.02-T17 Roll forward a quarterly report and tie it out in Workiva
Original instruction
Workiva test workspace. Roll forward the Q2 FY26 10-Q document to Q3: update period labels, link the statements to the new TB spreadsheet (input/tb_q3_fy26.xlsx imported to a Workiva sheet), update MD&A variance tables via links, and mark every linked amount tied. Leave the XBRL tags in place.
- Platforms
web· Any OS (web)- Applications
- Workiva · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Workiva export: all statement amounts are links (no hard-typed numbers) to the Q3 sheet with values equal key; period labels updated; XBRL facts' contexts roll to Q3; tie-out marks present. Workiva API not available to agent.
Proposed partial credit
30% roll forward of periods; 40% linked amounts correct; 20% XBRL contexts; 10% tie-out marks
R05.04.01.02-T18 Execute a SOX revenue-process control test end to end in AuditBoard
Original instruction
AuditBoard test tenant. For control REV-01 (credit approval before shipment), document the walkthrough from input/walkthrough_notes.docx, select 25 samples from input/rev_population.xlsx using the tool's random sampling with seed 2026, test each against attachments in input/rev_docs/, evaluate deficiencies per the severity rubric, and create issues.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- AuditBoard · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-appvis· Visual judgment- Work product
WP08· App/system state- Conditions
licensed· Licensed pro applong· Long-horizonmultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
AuditBoard export: walkthrough narrative present; sample IDs equal key (seeded); per-sample results equal key; deficiency severity and issue records equal key. AuditBoard API tokens are not issued to the agent; records must show the agent's web session in the activity log.
Proposed partial credit
15% walkthrough; 20% sample; 45% per-sample results; 20% deficiency evaluation
R05.04.01.02-T19 Evaluate a summary of audit differences against materiality
Original instruction
Excel. input/sad_fy26.xlsx lists 14 identified misstatements (factual, judgmental, projected) with prior-year turnaround effects. Using input/materiality.txt, classify each, compute iron-curtain and rollover effects on income and equity, decide which must be corrected per input/policy.pdf, and prepare the AJE list and the summary memo table.
- Platforms
mac· macOSwin· Windows- Applications
- Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Classification and per-item correct/waive decisions equal key; iron-curtain and rollover totals within 1 unit; AJE lines equal key.
Proposed partial credit
30% classification; 30% iron-curtain and rollover; 25% decisions; 15% AJE list
R05.04.01.02-T20 Assemble a 10-K XBRL instance that passes EDGAR validation
Original instruction
input/10k_fy26/ has the HTML 10-K (no tags), the TB, and the prior-year XBRL. Produce an Inline XBRL 10-K: dei cover-page tags, primary statements and parentheticals tagged to US-GAAP 2026 taxonomy, calculation and presentation linkbases, and policy text blocks. Validate with Arelle EFM until no errors; stop before any EDGAR transmission.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Arelle · VS Code · Excel
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Arelle EFM validation zero errors; dei facts exact; >= 95% of statement facts mapped to key concepts with correct contexts/units; calculation relationships consistent; no submission made.
Proposed partial credit
20% dei; 40% statement tagging; 20% linkbases; 20% clean validation
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.