Filing-based company research
Extract and reconcile figures from filings into a memo or dataset.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
Previous class address: R05.02.01.02 · Original proposal IDs are retained.
- Typical applications
- SEC EDGAR · Excel
- Class GUI-need label
AP· API-sufficient
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R05.02.01.02-T01 Pull five years of reported revenue for five issuers from XBRL company facts
Original instruction
input/ciks.csv lists 5 CIKs. Using the SEC companyfacts JSON on the frozen EDGAR mirror (env EDGAR_BASE, User-Agent required), extract annual revenue (10-K, full-year duration) for FY2021-FY2025, choosing the issuer's revenue concept when several exist, and write output/revenue.csv (cik, fy, concept, value_usd, accession).
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
25 rows; value and accession equal key for each cik-fy; concept choice must match the face income statement (custom-concept issuer included). Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
70% values exact (per row); 20% correct accession; 10% schema
R05.02.01.02-T02 Report cover-page shares outstanding and its as-of date from the latest 10-K
Original instruction
For the issuer whose CIK is in input/cik.txt, open its most recent 10-K on the frozen EDGAR mirror and report the cover-page number of common shares outstanding and the as-of date (dei:EntityCommonStockSharesOutstanding). Write 'shares;yyyy-mm-dd' to output/answer.txt; dual-class issuers report each class on its own line.
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR
- Interfaces
api· API/SDKbrow· Browser- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsnav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
Exact match on share count(s) and date to the key's 10-K. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
70% share count(s); 30% date
R05.02.01.02-T03 Decide whether a quarter beat the prior EPS guidance midpoint
Original instruction
Issuer in input/cik.txt. From its earnings 8-K exhibits (EX-99.1) on the frozen EDGAR mirror, find the adjusted EPS guidance range it gave for Q2 FY2026 and the actual adjusted EPS it reported. Write output/answer.json {"guidance_mid":x,"actual":y,"result":"beat|miss|inline","delta_cents":z}.
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR
- Interfaces
api· API/SDKbrow· Browser- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsnav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
JSON fields equal key (mid to 0.005, delta to 1 cent); guidance must come from the earlier-quarter release (the later release restates the range and is a decoy). Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
30% guidance midpoint; 30% actual; 40% result and delta
R05.02.01.02-T04 Compute FY2025 adjusted EBITDA from the 10-K non-GAAP reconciliation
Original instruction
Issuer in input/cik.txt reports non-GAAP measures in its FY2025 10-K MD&A. Rebuild adjusted EBITDA from net income using the company's reconciliation items, cross-check D&A and stock-based compensation against the cash flow statement, and write output/adj_ebitda.xlsx with each item, source page and the total.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Excel
- Interfaces
api· API/SDKterm· Terminaldesk· Desktop GUI- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Total within 0.5% of key; each reconciling item equals key; cross-check cells equal CF statement values. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
50% total within tolerance; 30% items correct; 20% CF cross-check
R05.02.01.02-T05 Rank the three largest segments by FY2025 operating income
Original instruction
For the issuer in input/cik.txt, use the FY2025 10-K segment note (ASC 280) to list the three segments with the highest segment operating income, with values and the FY2024 comparatives. Corporate/unallocated is not a segment. Write output/segments.csv (segment, fy2025, fy2024).
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Segment names (normalized) and values equal key; order by FY2025 descending. The XBRL dimension members use custom names, so naive tag matching yields wrong labels. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
40% correct three segments in order; 60% values exact (both years)
R05.02.01.02-T06 Discount a lease maturity table and reconcile it to the reported liability
Original instruction
For the issuer in input/cik.txt, extract the operating-lease maturity table from the FY2025 10-K lease footnote into Excel, discount it using the disclosed weighted-average discount rate and remaining term, and compare with the balance-sheet operating lease liability (current + non-current). Explain any difference over 2% in a cell comment.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Maturity rows (incl. 'thereafter' and imputed interest) equal key; computed PV and reported liability equal key values; difference % within 0.1pp of key. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
40% maturity table exact; 30% PV computation; 30% reconciliation to BS
R05.02.01.02-T07 Find 2026 8-K filings disclosing a material weakness across a watchlist
Original instruction
input/watchlist.csv lists 60 CIKs. Using EDGAR full-text search on the frozen mirror (UI or efts endpoint), find every 8-K or 8-K/A filed 2026-01-01 to 2026-09-30 by those issuers whose text contains 'material weakness'. Write output/mw_8k.csv (cik, accession, filed, item_numbers).
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR full-text search · Python
- Interfaces
api· API/SDKbrow· Browser- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commandsnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Set equality with key on accession (precision and recall); item numbers per filing equal key. Pagination beyond 100 hits is required to reach full recall. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
70% F1 on accession set; 30% item numbers correct
R05.02.01.02-T08 Build a remaining-performance-obligation dataset for six SaaS peers
Original instruction
input/saas_peers.csv lists 6 CIKs. From each FY2025 10-K (or latest 10-Q for off-cycle FYs as listed), extract total RPO, the percentage expected to be recognized within 12 months, and current RPO where disclosed. Write output/rpo.csv with value, unit and source accession.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Values equal key per peer; percentages to 1 decimal; issuers reporting only 'approximately X%' must be marked approx=true. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% RPO values; 30% percentages; 10% approx flags
R05.02.01.02-T09 Locate Hong Kong annual reports via HKEXnews and extract headline figures
Original instruction
input/hk_watchlist.csv lists 4 HKEX stock codes. Using the HKEXnews title search UI (frozen snapshot site), find each company's 2025 annual report PDF, then extract total revenue and profit attributable to owners (currency and units as stated) into output/hk.csv with the document's news ID.
- Platforms
web· Any OS (web)- Applications
- HKEXnews · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Values, currency/unit and news ID equal key; supplementary announcements and interim reports are decoys in search results. Snapshot site has no search API; grader logs show UI queries.
Proposed partial credit
30% correct documents; 50% values and units; 20% news IDs
R05.02.01.02-T10 Read segment backlog values off image-only investor-deck charts
Original instruction
Issuer in input/cik.txt filed its investor day slides as EX-99.2 where pages are flattened images. Slide 14 shows a stacked bar chart of backlog by segment for 4 quarters with no data labels. Estimate the 8 segment values from the chart axes and write output/backlog.csv; also record the stated total from the slide footnote.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR · Adobe Acrobat Reader
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Each value within +-3% of the key (derived from the source chart data); totals consistent with footnote within 1%. PDF has no text layer for the chart; table-extraction tools return nothing. Terminal, Python and OCR tools are disabled in the agent VM, so values must be read visually in Acrobat or the browser.
Proposed partial credit
70% values within tolerance (per value); 30% footnote total
R05.02.01.02-T11 Write a peer gross-margin memo with filing citations
Original instruction
Three peers in input/peers.csv. Write output/gm_memo.md (<= 400 words) comparing FY2023-FY2025 gross margin, citing each figure with accession number and statement name, and include a table 'Peer | FY2023 | FY2024 | FY2025' in percent to one decimal. State which peer expanded margin most.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Table cells parsed and equal key to 0.1pp; every figure has a valid accession citation that contains it; named top expander equals key. LLM rubric only for 'no unsupported claims' (10%).
Proposed partial credit
60% table values; 20% citations valid; 10% conclusion; 10% rubric
R05.02.01.02-T12 Tabulate say-on-pay vote results from 8-K Item 5.07 filings
Original instruction
input/watchlist_20.csv lists 20 CIKs. For each issuer's 2026 annual meeting, find the 8-K Item 5.07 and extract say-on-pay votes for, against, abstain and broker non-votes. Compute support % = for/(for+against). Write output/sop.csv; issuers without a 2026 meeting by 2026-09-30 get status 'none'.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Row-level equality on vote counts and support % (1 decimal) with key; 'none' status for the key's 2 issuers; 8-K/A amendments supersede originals. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% vote counts; 25% support %; 15% none/amendment handling
R05.02.01.02-T13 Find an XBRL scaling error by comparing tagged values to the rendered table
Original instruction
For the 5 10-K filings in input/accessions.txt, compare XBRL-tagged balance-sheet debt facts against the HTML-rendered balance sheet. One filing has a tagging scale/sign error. Identify the accession, concept, tagged value and rendered value, and write output/xbrl_issue.json; open the inline viewer to confirm.
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR · Inline XBRL Viewer · Python
- Interfaces
api· API/SDKbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
JSON fields equal key exactly (accession, concept, both values). Decoy: a legitimately different 'total debt' non-GAAP figure in MD&A. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
40% correct filing; 30% concept; 30% both values
R05.02.01.02-T14 Compute covenant headroom from a credit agreement exhibit
Original instruction
Issuer in input/cik.txt filed its amended credit agreement as EX-10.1 (scanned PDF, 180 pages). Find the maximum net leverage and minimum interest coverage covenants and their definitions (EBITDA add-backs), then compute headroom using the latest 10-Q figures. Write output/covenants.xlsx with thresholds, computed ratios and headroom %.
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR · Adobe Acrobat Reader · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrolltype· Text & formsvis· Visual judgment- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Thresholds equal key (incl. step-down for the current quarter); computed ratios within 2% of key using defined add-backs; headroom % within 1pp. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
30% thresholds incl. step-down; 40% ratios per definitions; 30% headroom
R05.02.01.02-T15 List line items recast between the FY2024 and FY2025 10-K
Original instruction
Issuer in input/cik.txt recast prior-year figures after a segment reorganization and a discontinued operation. Compare FY2024 income statement and segment figures as originally reported (FY2024 10-K) against the comparative column in the FY2025 10-K, and write output/recast.csv (line_item, original, recast, delta, reason_note_ref).
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Set of changed line items equals key (no false positives from rounding < 1 unit); values exact; note references point to the right footnote. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
50% changed-item set F1; 35% values; 15% note references
R05.02.01.02-T16 Derive twelve quarters of revenue for ten companies with fiscal-year mismatches
Original instruction
input/universe.csv lists 10 CIKs with different fiscal year ends. Build output/quarterly_rev.csv for calendar quarters 2023Q4-2026Q3: use 10-Q three-month figures, derive Q4 as FY minus nine-month YTD, map fiscal to calendar quarters by period end date, and flag 52/53-week years.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
HY· Hybrid- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
120 cells equal key; derived Q4s match key; 53-week flags correct; recast comparatives must not overwrite originally filed values unless the key says so. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% values (per cell); 25% Q4 derivations; 15% calendar mapping and flags
R05.02.01.02-T17 Build a debt maturity ladder across six issuers from their footnotes
Original instruction
input/issuers_hy.csv lists 6 CIKs. From each FY2025 10-K debt footnote, extract principal maturities for 2026-2030 and thereafter, including finance leases only where the issuer includes them in debt, then build output/ladder.xlsx with a stacked column chart by year and a check against total principal.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrolltype· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Maturities per issuer-year equal key; totals reconcile to disclosed principal (diff 0); chart XML is stacked column with 6 series. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% maturities exact; 20% reconciliation; 20% chart
R05.02.01.02-T18 Build a reconciled free-cash-flow dataset for fifteen software companies
Original instruction
input/saas15.csv lists 15 CIKs. For FY2021-FY2025 compute FCF (CFO minus capex incl. capitalized software) and SBC-adjusted FCF, resolving custom XBRL concepts and restated years (use the latest filing presenting each year). Output output/fcf.csv plus output/method.md noting every concept mapping decision.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
75 issuer-years compared to key (tolerance 0.5%); capitalized-software inclusion per issuer correct; method.md lists a mapping for each custom concept used (parsed). Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% FCF values; 20% SBC-adjusted; 20% mapping documentation
R05.02.01.02-T19 Assemble a forensic red-flag memo from a 10-K and three 10-Qs
Original instruction
Issuer in input/cik.txt. From its FY2025 10-K and 2026 Q1-Q3 10-Qs, determine: DSO trend, allowance for credit losses as % of AR, changes in revenue-recognition policy, any going-concern or substantial-doubt language, auditor change (8-K 4.01) and late-filing notices (NT 10-Q). Write output/redflags.json and a 1-page memo.
- Platforms
lin· Linuxmac· macOSwin· Windowsweb· Any OS (web)- Applications
- SEC EDGAR · Python
- Interfaces
api· API/SDKbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
JSON checklist of 6 facts equals key (numeric within 1%, booleans exact, accession cited); memo checked only for consistency with JSON (LLM rubric, 10%). Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
60% six red-flag facts (equal weight); 30% citations valid; 10% memo rubric
R05.02.01.02-T20 Normalize EBITDA between an IFRS 20-F filer and a US GAAP peer
Original instruction
Compare the IFRS filer (20-F, input/ifrs_cik.txt) with its US GAAP peer (10-K). Restate the US GAAP peer's EBITDA as if IFRS 16 applied (add back operating lease cost, using the lease footnote) and convert the IFRS filer to USD at the average rate in input/fx_2025.csv. Output output/normalized.xlsx with both EBITDA bases and EV/EBITDA at the snapshot EVs.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- SEC EDGAR · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrolltype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Operating lease cost add-back equals key; converted IFRS EBITDA within 0.5%; normalized multiples within 0.1x of key. Key built from a frozen EDGAR mirror pinned to listed accession numbers, so later filings cannot change the answer.
Proposed partial credit
30% lease add-back; 30% FX conversion; 40% normalized multiples
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.