Email → spreadsheet → report pipelines
Gather data from mail and attachments, analyze it in a sheet and produce a report.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Outlook · Excel · Word
- Class GUI-need label
G2· GUI-essential
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R04.07.01.03-T01 Tabulate twelve emailed vendor quotes and highlight the cheapest
Original instruction
In Outlook (test mailbox), folder 'Vendor quotes' has 12 quote emails, some with the price in the body and some in a PDF attachment. Fill input/quotes.xlsx with Vendor, Unit Price (USD), MOQ and Lead Time (days), sort by price ascending, and fill the cheapest row yellow.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appvis· Visual judgmenttype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 2 / 5
Proposed verifier
Rows equal key (12 vendors; numbers exact); sort order; only the cheapest row has fill FFFF00; mailbox unchanged (no moves/deletes) via IMAP read. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
60% values (pro-rata over 12 vendors); 20% sort; 20% highlight.
R04.07.01.03-T02 Tally RSVP replies into a sheet and write a one-paragraph summary
Original instruction
In the test mailbox, 34 replies to 'Team offsite RSVP' say yes/no/maybe in free text, some with dietary needs. Record each person's latest answer in input/rsvp.xlsx (Name, Response, Dietary), and write output/rsvp_summary.docx with one paragraph giving counts by response and each dietary need count.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 2 / 5
Proposed verifier
Sheet rows equal key (latest reply wins for 3 people who changed answers); docx paragraph contains the exact counts (regex for numbers next to yes/no/maybe and each diet). Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
50% responses; 20% dietary; 30% summary counts.
R04.07.01.03-T03 Consolidate five regional sales emails into a Word report with a chart
Original instruction
Five regional managers emailed weekly CSVs (subject 'W41 sales – <region>'). Combine them in input/w41_sales.xlsx (one table, region column), compute totals and week-over-week % vs input/w40_totals.csv, and fill input/weekly_report_template.docx with the summary table and a column chart of revenue by region.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Workbook combined table equals union of CSVs (row count, sums); WoW % equals key (0.1 pp); docx table values equal key; docx contains a chart part (or linked chart) with region series equal totals. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
25% consolidation; 25% totals/WoW; 30% report table; 20% chart.
R04.07.01.03-T04 Log emailed receipts against policy and write an exceptions memo
Original instruction
Folder 'Receipts – Sept' holds 22 emails from staff with receipt images/PDFs. Enter each in input/expenses.xlsx (employee, date, merchant, amount, category), flag violations of input/travel_policy.pdf (meal caps, missing itemization, weekend), and write output/exceptions_memo.docx listing each flagged item with the rule broken.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appvis· Visual judgmenttype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Sheet rows equal key (amounts exact); violation flags equal key (precision/recall); memo lists exactly the flagged items with rule names (regex per item). Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
45% receipts entered; 30% flags; 25% memo.
R04.07.01.03-T05 Categorize a month of complaints and file the processed emails
Original instruction
The shared mailbox 'support@' (test) has 57 complaint emails in Inbox for September. Classify each into input/taxonomy.csv categories in input/complaints.xlsx (sender, date, order no., category), build a pivot of counts by category × week, write output/complaints_sept.docx with the top 3 categories, and move processed emails to folder 'Processed/2026-09'.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
HY· Hybrid- Interaction primitives
multi· Cross-appdrag· Drag & droptype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Category accuracy vs key ≥90% for full credit; pivot exists with counts consistent with the sheet; memo names the correct top 3 with counts; IMAP: 57 emails in Processed/2026-09, none lost. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
40% classification; 20% pivot; 20% memo; 20% emails filed.
R04.07.01.03-T06 Turn membership renewal emails into a Numbers table and Pages letters
Original instruction
In Apple Mail (test account), 18 members replied to 'Renewal 2027' with their tier and payment method. Record them in input/members.numbers, then in Pages produce one confirmation letter per member from input/letter_template.pages (merge name, tier, fee from the tier table) saved as output/letters.pdf.
- Platforms
mac· macOS- Applications
- Apple Mail · Numbers · Pages
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Grader AppleScript export of the Numbers table equals key; letters.pdf has 18 pages (one per member) with name/tier/fee text equal key; fees derived from the tier table. Agent VM has no IMAP credentials and osascript/JXA is blocked; mail is reachable only in Apple Mail, and the files must be saved by Numbers and Pages.
Proposed partial credit
40% table; 45% letters merged correctly; 15% PDF page count.
R04.07.01.03-T07 Compile emailed bug reports into a Calc tracker and Writer summary
Original instruction
In Thunderbird (test profile), folder 'QA reports' has 26 bug-report emails with logs attached. Record ID, component, severity, build and reporter in input/bugs.ods (LibreOffice Calc), count by component × severity, and write output/qa_summary.odt in Writer with the count table and the 5 critical bugs listed.
- Platforms
lin· Linuxwin· Windowsmac· macOS- Applications
- Thunderbird · LibreOffice
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
ODS rows equal key; count table consistent; ODT contains the table values and the 5 critical bug IDs and titles. Agent VM has no IMAP credentials, and Thunderbird add-ons and LibreOffice macros/UNO are disabled; mail is reachable only in Thunderbird.
Proposed partial credit
45% tracker rows; 20% counts; 35% summary.
R04.07.01.03-T08 Combine eight site survey workbooks into a templated report
Original instruction
Eight site leads emailed survey result XLSX files with different column orders and header spellings. Combine them in input/survey_all.xlsx with a site column, compute per-site and overall satisfaction means and response rates, and fill the 14 «placeholders» in input/survey_report_template.docx with the computed values.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Combined table row count and per-column sums equal key; means equal key (0.01); all 14 placeholders replaced with values equal key; no « » remains. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
35% combination; 30% computed stats; 35% report placeholders.
R04.07.01.03-T09 Age emailed invoices and prepare reminder drafts without sending
Original instruction
From folder 'Invoices sent' (31 emails with invoice PDFs) and payment confirmations in 'Payments', build input/ar_aging.xlsx with buckets (current, 1–30, 31–60, 60+) as of 2026-10-14, then mail-merge a reminder from input/reminder.docx into Outlook drafts for each overdue client. Do not send.
- Platforms
win· Windowsmac· macOS- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Aging table equals key; IMAP Drafts contains one reminder per overdue client (9) with correct amounts/days in body; Sent folder unchanged (no sends). Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
40% aging; 40% drafts correct; 20% nothing sent.
R04.07.01.03-T10 Build a board KPI report from tables pasted inside department emails
Original instruction
Six department heads sent September KPIs as tables inside the email body (no attachments). Copy them into input/kpi_model.xlsx tabs, let the model compute variances, then update input/board_report.docx: refresh the 3 linked charts, fill the KPI table and write each department's one-line variance note from the model.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Model input ranges equal email table values (exact); computed variances equal key; docx KPI table equals model outputs; linked chart caches updated to new values; 6 notes contain the variance figures. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
35% email tables transferred; 20% model results; 25% report table + charts; 20% notes.
R04.07.01.03-T11 Extract forty order confirmations from webmail into a spend memo
Original instruction
In the sandbox Roundcube webmail, 40 HTML order-confirmation emails from 6 vendors arrived this quarter. Record order no., date, vendor, items count, subtotal, tax, total in input/orders_q3.xlsx, compute spend per vendor, and write output/spend_memo.docx with a table of vendors over $5,000.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- Roundcube · Excel · Word
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Sheet rows equal key (amounts exact); vendor totals equal key; memo table lists exactly vendors > $5,000 with totals. Agent has no IMAP/API credentials; mail is reachable only through the webmail UI.
Proposed partial credit
55% orders entered; 20% vendor totals; 25% memo table.
R04.07.01.03-T12 Apply superseding timesheet emails before building a payroll summary
Original instruction
Folder 'Timesheets W40–W41' has 48 emails; some later emails correct earlier ones ('Correction:' subject or reply). Use only the latest submission per employee-week, fill input/payroll_hours.xlsx, and produce output/payroll_summary.docx with total regular and overtime hours per department.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-apphidden· Hidden/stale state- Proposed difficulty
- 3 / 5
Proposed verifier
Hours per employee-week equal key (latest-wins applied to 7 corrected cases); department totals equal key; summary docx numbers match. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
40% hours; 25% corrections applied; 20% department totals; 15% summary.
R04.07.01.03-T13 Chart emailed QC results and flag Westgard rule violations
Original instruction
The lab's daily QC results arrive as PDF tables attached to 30 emails. Enter the glucose control values into input/qc_glucose.xlsx, build a Levey-Jennings chart with mean and ±1/2/3 SD lines from input/qc_targets.csv, flag 1-3s, 2-2s and R-4s violations, and write output/qc_report.docx with the chart and violation list.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appvis· Visual judgmenttype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Values equal key; chart series present with 7 reference lines at correct values; violation list equals rule-computed key (dates and rule codes); report contains chart image/chart part and the list. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
35% values; 25% chart lines; 25% violations; 15% report.
R04.07.01.03-T14 Score five vendors from a quarter of POs, deliveries and complaints
Original instruction
From Outlook folders 'POs' and 'Deliveries' plus complaint emails in 'Quality' (Jul–Sep), build input/vendor_scorecard.xlsx: per vendor on-time delivery %, defect rate and average lead time; then generate 5 one-page Word reports from input/vendor_report.docx and a summary table ranking vendors.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Scorecard metrics equal key per vendor (0.1 pp / 0.1 day); 5 report files with values equal scorecard; ranking table order equals key. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
45% metrics; 35% five reports; 20% ranking.
R04.07.01.03-T15 Track data-room documents announced by email into a diligence memo
Original instruction
Twenty data-room notification emails link to documents in the sandbox data room. Open each document, extract counterparty, term, change-of-control and termination clauses into input/dd_tracker.xlsx, and write output/dd_memo.docx summarizing contracts with change-of-control provisions and termination notice ≥90 days.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- Outlook · Excel · Word · browsers
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Tracker fields equal key (yes/no flags exact, notice days exact); memo lists exactly the qualifying contracts (set match); no documents missed (20 rows). Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
20% all documents opened; 45% tracker fields; 35% memo set.
R04.07.01.03-T16 Rebuild an incident timeline from 200 alert emails across time zones
Original instruction
Folder 'Alerts – INC-2291' holds 200 monitoring emails (duplicates, re-fires, mixed UTC/PDT/IST timestamps in bodies). Build input/inc2291_timeline.xlsx with deduplicated events in UTC, compute time to detect, acknowledge and resolve, and fill input/postmortem_template.docx timeline table and metrics.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-applong· Long-horizonhidden· Hidden/stale state- Proposed difficulty
- 5 / 5
Proposed verifier
Timeline events equal key (dedupe precision/recall ≥0.95; UTC times exact to the minute); TTD/TTA/TTR equal key; postmortem table rows equal timeline. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
40% dedupe and UTC conversion; 25% metrics; 35% postmortem filled.
R04.07.01.03-T17 Control pair of T03: run the sales pipeline with Graph, pandas and python-docx
Original instruction
Control pair of T03. Using the Microsoft Graph API (test tenant token), pandas and python-docx from the terminal, fetch the five 'W41 sales' CSV attachments, compute totals and WoW %, and fill the same report template with table and chart image.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Graph API · pandas · python-docx
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same value checks as T03 (chart may be an image here); compared with the GUI variant to measure modality gap.
Proposed partial credit
25% consolidation; 25% totals/WoW; 30% report table; 20% chart.
R04.07.01.03-T18 Track candidates from application emails into a weekly hiring report
Original instruction
Shared mailbox 'jobs@' has 45 application emails for 3 roles with resumes attached and stage tags in later replies ('[Screen]', '[Onsite]', '[Offer]'). Build input/pipeline.xlsx (candidate, role, latest stage, source, days in stage) and write output/hiring_weekly.docx with counts per role × stage and the 5 longest-waiting candidates.
- Platforms
mac· macOSwin· Windows- Applications
- Outlook · Excel · Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-apphidden· Hidden/stale state- Proposed difficulty
- 4 / 5
Proposed verifier
Tracker rows equal key (latest stage from thread replies); days in stage as of 2026-10-14; report counts and longest-waiting list equal key. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
45% tracker; 15% days in stage; 25% count table; 15% longest-waiting list.
R04.07.01.03-T19 Summarize newsletter sign-up emails in Google Sheets and Docs
Original instruction
In the test Gmail account, 60 'New sign-up' notification emails list name, email, city and interests. Put them in Google Sheet 'Signups Oct' (dedupe by email), add a count-by-city table, and write Google Doc 'Signup summary' with total unique sign-ups and the top 3 interests with counts.
- Platforms
web· Any OS (web)- Applications
- Gmail · Google Sheets · Google Docs
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Sheets API: unique rows equal key (52); city counts table correct; Docs API: summary contains total 52 and top-3 interests with counts; no emails sent or deleted. Agent has no IMAP/API credentials; mail is reachable only through the webmail UI.
Proposed partial credit
45% sheet rows; 20% city table; 35% summary doc.
R04.07.01.03-T20 Merge results letters to PDFs and stage them as email drafts
Original instruction
Using input/assessment_results.xlsx (24 candidates), mail-merge input/result_letter.docx into one PDF per candidate named output/letters/<CandidateID>.pdf, then create an Outlook draft to each candidate's email with their PDF attached and subject 'Assessment result – <ID>'. Do not send.
- Platforms
win· Windowsmac· macOS- Applications
- Word · Excel · Outlook
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
24 PDFs with correct names and merged text equal key; IMAP Drafts has 24 messages, each to the right address with the matching PDF attached (hash) and exact subject; Sent unchanged. Agent VM has no Graph/IMAP credentials, scripting host or VBA; mail is reachable only through the client UI.
Proposed partial credit
40% PDFs correct; 40% drafts with right attachments; 20% nothing sent.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.