BI dashboard authoring
Build dashboards, calculated fields and filters in BI tools to a spec.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Tableau · Power BI Desktop · Looker · Metabase
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R03.02.01.02-T01 Build a filtered executive KPI dashboard in Metabase
Original instruction
In the Metabase sandbox (SQLite 'retail' DB loaded), create dashboard 'Exec KPIs' with 6 cards per spec/exec_kpis.md (revenue, AOV, conversion, revenue by month line, top categories bar, new vs returning), a Region filter wired to all 6 cards, and default date range 2025. Export the 6 values for Region=EMEA to output/metrics.json.
- Platforms
web· Any OS (web)- Applications
- Metabase
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & droptype· Text & forms- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader reads the dashboard via an admin session: 6 cards, filter mapped to each card's region field, defaults set; executes each card with Region=EMEA and compares to gold. metrics.json exact. Agent's account has API keys disabled; card creation events in the audit log must come from the web UI session.
Proposed partial credit
30% cards present with correct queries; 30% filter wired to all; 20% card results correct; 20% metrics.json
R03.02.01.02-T02 Control pair of T01: create the KPI dashboard via the Metabase API
Original instruction
Control pair of T01. Same spec/exec_kpis.md, but you have only a terminal and a Metabase API key: create the 'Exec KPIs' dashboard, 6 cards and the Region filter through the REST API and export EMEA values to output/metrics.json.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Metabase API
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same checks as T01 on dashboard structure, filter mappings and values; reported as the API half of the T01 pair to measure the GUI gap.
Proposed partial credit
30% cards; 30% filter mappings; 20% results; 20% metrics.json
R03.02.01.02-T03 Model a star schema with time-intelligence measures in Power BI Desktop
Original instruction
In Power BI Desktop, load fixtures/sales_star/*.csv, create relationships (Sales to Date, Product, Store, single direction), mark Date as a date table, write measures Total Sales, Sales YoY %, Rolling 3M Sales, and build page 'Overview' with a Year slicer, a bar of Sales by Category and a line of Rolling 3M. Save output/sales.pbix.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropcode· Code & commands- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader opens sales.pbix with pbi-tools and queries the model via the local AS instance: relationships and date-table flag exact; DAX results for 6 filter contexts equal gold ±0.01; Layout JSON has the 3 visuals with correct fields. pbi-tools, Tabular Editor and DAX Studio are grader-only, the AS port is firewalled from agent processes, and PBIP save is disabled.
Proposed partial credit
25% relationships/date table; 40% measure values; 25% visuals and fields; 10% slicer works
R03.02.01.02-T04 Build a dual-axis sales and profit-ratio dashboard in Tableau
Original instruction
In Tableau Desktop, connect to fixtures/superstore.xlsx and build a sheet with Sales (bars) and Profit Ratio (line, calculated) by month on a synchronized dual axis, then a dashboard with that sheet and a Region map where a Region filter applies to all worksheets. Save output/sales.twbx.
- Platforms
mac· macOSwin· Windows- Applications
- Tableau Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & click- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses the .twbx: calculated field SUM([Profit])/SUM([Sales]); dual axis with synchronized axes; filter applies to all worksheets using the data source; then publishes to the sandbox Tableau Server and exports crosstab per month for Region=West, matching gold. Document API, tabcmd and Python are absent, and .twbx writes by non-Tableau processes are detected.
Proposed partial credit
25% calculated field; 25% dual axis sync; 25% filter scope; 25% exported values
R03.02.01.02-T05 Build a cohort retention heatmap with LOD expressions in Tableau
Original instruction
In Tableau Desktop with fixtures/orders.hyper, create a cohort analysis: cohort month = FIXED first order month per customer, months since first order, and % of cohort active, shown as a heatmap of 2024 cohorts × months 0–11 with a color legend from 0–60%. Save output/cohorts.twbx.
- Platforms
mac· macOSwin· Windows- Applications
- Tableau Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clickcode· Code & commands- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Grader extracts the workbook, publishes to the sandbox server and downloads the crosstab: all 12×12 cells equal gold ±0.1 pp. Calculations use {FIXED [Customer ID]: MIN(...)} (XML check). Color range fixed 0–0.6. Tableau Document API, tabcmd and Python are absent, no server token is issued, and .twbx writes by processes other than Tableau Desktop are detected.
Proposed partial credit
25% cohort LOD; 15% months-since calc; 45% cell values; 15% formatting
R03.02.01.02-T06 Add row-level security roles to a Power BI report
Original instruction
In fixtures/regional.pbix (Power BI Desktop), create roles RegionManager (filter Store[Region] = USERPRINCIPALNAME lookup via table Managers) and Exec (no filter). Use 'View as' to confirm that manager anna@contoso.test sees only EMEA, then save.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickcode· Code & commands- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader evaluates the model via the local AS instance with EffectiveUserName/roles: anna sees 412 stores (EMEA only), Exec sees all 1,380; DAX filter references the Managers table, not a hard-coded region. Tabular Editor, DAX Studio and pbi-tools are absent for the agent, the local AS port is firewalled from agent processes, and PBIP save is disabled.
Proposed partial credit
40% RegionManager filter correct; 20% dynamic via Managers table; 20% Exec role; 20% saved model valid
R03.02.01.02-T07 Create a blended report with date controls in Looker Studio
Original instruction
In Looker Studio (sandbox Google account), build report 'Campaign ROI' from the Google Sheet 'ad_spend' and the BigQuery table 'shop.orders': blend on campaign_id and date, calculated field ROAS = revenue / spend, a scorecard, a time series, a table by campaign sorted by ROAS, and a date range control defaulting to last 28 days.
- Platforms
web· Any OS (web)- Applications
- Looker Studio
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clicktype· Text & forms- Work product
WP03· Visual layout- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Grader opens the report in a headless browser at a fixed date, sets the range to 2025-09-01..30, and reads rendered scorecard and table values: equal gold ±0.01. Blend join keys checked by inspecting the data panel. No Google API credentials are issued and Linking API template copies are rejected; Drive activity must show the report built in the agent's browser session.
Proposed partial credit
25% blend correct; 20% ROAS field; 35% rendered values; 20% date control default
R03.02.01.02-T08 Create a cross-filtering Superset dashboard on a virtual dataset
Original instruction
In Apache Superset, create a virtual dataset from the SQL in fixtures/orders_enriched.sql, then charts: Big Number with trendline (weekly orders), stacked bar (orders by status by week), and a table of top 20 customers. Assemble dashboard 'Ops' with native filters for country and date and cross-filtering enabled.
- Platforms
web· Any OS (web)- Applications
- Apache Superset
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & droptype· Text & forms- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader reads dashboard/chart metadata from the Superset metadata DB: dataset SQL, viz types, metrics, filter scopes. Then renders via Playwright with country=DE and checks chart values vs gold. No API token or metadata-DB access is issued; Superset REST writes need the browser session cookie and CSRF token.
Proposed partial credit
20% dataset; 30% charts configured; 20% native filters; 30% values with filter
R03.02.01.02-T09 Configure drill-through and report-page tooltips in Power BI
Original instruction
In fixtures/service.pbix, add a drill-through page 'Ticket Detail' keyed on Agent and Priority with a back button, and a report-page tooltip showing SLA % and median resolution time when hovering bars on 'Summary'. Save the file.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & drophover· Hover & context- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader parses Layout JSON: drill-through filters on the two fields, back button action, tooltip page type and assignment to the Summary bar visual. Renders via the sandbox Power BI Report Server: hovering a bar shows tooltip values equal gold. PBIP save is disabled and .pbix writes by any process other than PBIDesktop.exe are detected (Sysmon) and fail.
Proposed partial credit
35% drill-through config; 15% back button; 30% tooltip page and assignment; 20% rendered values
R03.02.01.02-T10 Build a parameter-driven Top N with an 'Other' bucket in Tableau
Original instruction
In Tableau Desktop with fixtures/sales.hyper, create an integer parameter Top N (5–20, default 10) and a sheet showing the top N products by sales with all remaining products grouped as 'Other', sorted descending with Other last, and a dynamic title 'Top <N> products'. Show the parameter control on a dashboard.
- Platforms
mac· macOSwin· Windows- Applications
- Tableau Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clickcode· Code & commands- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Grader publishes the workbook to the sandbox server and exports crosstabs with N=5 and N=15 via URL parameters: rows equal gold including the Other total; title text exact; Other row last. Tableau Document API, tabcmd and Python are absent, no server token is issued, and .twbx writes by processes other than Tableau Desktop are detected.
Proposed partial credit
25% parameter; 35% Top N + Other logic; 20% sort with Other last; 20% dynamic title
R03.02.01.02-T11 Author a Qlik Sense sheet with master items and set analysis
Original instruction
In Qlik Sense (sandbox), add master measures Sales, Sales LY (set analysis on the previous year relative to selected year), and Margin %, a master dimension Product Hierarchy (drill-down Category > Subcategory > Product), and a sheet with a KPI, a combo chart and a filter pane. Publish the app.
- Platforms
web· Any OS (web)win· Windows- Applications
- Qlik Sense
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clickcode· Code & commands- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Grader uses the Qlik Engine API (read-only) to list master items and evaluates measures with Year=2025 and Category=Bikes: values equal gold. Sheet has the 3 objects using master items. No Engine/QRS API certificates are issued and the local Desktop engine port is firewalled from agent processes, so master items must be built in the Qlik Sense UI.
Proposed partial credit
30% master measures correct; 15% drill-down dimension; 35% evaluated values; 20% sheet objects
R03.02.01.02-T12 Restyle a Power BI report to a corporate style guide
Original instruction
fixtures/hr_report.pbix uses default colors and misaligned visuals. Apply input/style_guide.pdf: import the theme JSON, align the 8 visuals to the 12-column grid with 16 px gutters, set titles to Segoe UI Semibold 14, and create a phone layout with the 4 key visuals stacked.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometryvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses Layout JSON: theme applied; visual x/width snap to grid ±2 px with 16 px gutters; title formatting; mobile layout contains the 4 visuals in order. Rendered page SSIM ≥0.85 vs the style-guide mock. PBIP save is disabled and .pbix writes by any process other than PBIDesktop.exe are detected (Sysmon) and fail.
Proposed partial credit
20% theme; 35% grid alignment; 15% titles; 30% phone layout and visual match
R03.02.01.02-T13 Add a derived table, measures and a dashboard tile in Looker
Original instruction
In the Looker sandbox (development mode), add a LookML derived table customer_ltv (orders aggregated per customer), dimension tiers, measures avg_ltv and customers_count, join it to the orders explore, validate, commit to the dev branch, and add a tile 'LTV by tier' to dashboard 'Customer Health'.
- Platforms
web· Any OS (web)- Applications
- Looker
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsclick· Point & click- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader reads the dev branch LookML (git), runs the LookML validator (no errors), runs the explore query via the API with admin credentials and compares tier values to gold; dashboard element exists with that query.
Proposed partial credit
30% derived table; 20% measures/dimension; 30% query values; 20% dashboard tile
R03.02.01.02-T14 Define a Metabase model with metadata and a custom metric
Original instruction
In Metabase, turn the saved question 'Orders enriched' into a model, set display names, semantic types (currency, category, entity key) and descriptions per spec/model_meta.csv, then create a question using a custom expression for net margin % grouped by category.
- Platforms
web· Any OS (web)- Applications
- Metabase
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader reads the card (type=model) and result_metadata: semantic types and display names equal spec; question uses the model as source with the custom expression and its results equal gold. No Metabase API key or app-DB access is issued, so model metadata must be set in the browser session (revision history shows the agent user).
Proposed partial credit
30% model created; 30% metadata; 40% question results
R03.02.01.02-T15 Add target-based KPI cards and conditional formatting in Power BI
Original instruction
In fixtures/branch_kpis.pbix, add KPI visuals for Revenue, NPS and Churn against targets from the Targets table (with correct direction for Churn), and conditional formatting on the branch table: red/amber/green background by % to target using rules in spec/rag.md.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Grader parses Layout JSON for KPI visuals (indicator, target, trend axis) and conditional formatting rules (thresholds exact). Renders via Report Server and samples cell background colors for 5 branches: RAG colors match expected classes. PBIP save is disabled and .pbix writes by any process other than PBIDesktop.exe are detected (Sysmon) and fail.
Proposed partial credit
35% KPI visuals; 15% churn direction; 35% formatting rules; 15% rendered colors
R03.02.01.02-T16 Author a Grafana dashboard with variables and thresholds
Original instruction
In the Grafana sandbox (Prometheus data source), create dashboard 'Checkout Service' with variables env and instance (chained), panels for request rate, p95 latency (histogram_quantile), error ratio with thresholds at 1% and 5%, and a stat for SLO burn rate, plus row repeats per instance. Save it to folder 'Payments'.
- Platforms
web· Any OS (web)- Applications
- Grafana
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader fetches the dashboard JSON with an admin token: variables chained, PromQL for each panel returns gold values at a fixed time, thresholds exact, repeat by instance configured, folder correct. No API token or service account is issued; version history is checked, and a single-save JSON import or JSON-model paste is detected and fails.
Proposed partial credit
20% variables; 40% panel queries; 20% thresholds; 20% repeats and folder
R03.02.01.02-T17 Map custom sales territories in Tableau
Original instruction
In Tableau Desktop, create sales territories by grouping US states per input/territories.csv (8 territories), show a filled map colored by territory sales with a second map layer of store points sized by revenue, and label each territory with its total.
- Platforms
mac· macOSwin· Windows- Applications
- Tableau Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgment- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Grader parses the .twbx group definition (all 50 states assigned per CSV), map layers (filled + points), and size encoding; publishes to the sandbox server and exports territory totals equal gold. Tableau Document API, tabcmd and Python are absent, no server token is issued, and .twbx writes by processes other than Tableau Desktop are detected.
Proposed partial credit
35% territory groups; 25% dual layers; 25% totals; 15% labels
R03.02.01.02-T18 Reproduce a dashboard from a screenshot in Power BI
Original instruction
input/target_dashboard.png shows a finished 'Store Performance' page (7 visuals, custom layout and colors) built on fixtures/stores.xlsx. Rebuild it in Power BI Desktop so the visuals, fields, sort orders, numbers and layout match, and save output/store_perf.pbix.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometryvis· Visual judgment- Work product
WP03· Visual layout- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Grader renders the page via Report Server: SSIM ≥0.80 vs target; per-visual type and fields from Layout JSON; OCR of displayed numbers equals the target's values; visual positions IoU ≥0.8. PBIP save is disabled and .pbix writes by any process other than PBIDesktop.exe are detected (Sysmon) and fail.
Proposed partial credit
25% visual types/fields; 30% numbers match; 25% layout IoU; 20% overall SSIM
R03.02.01.02-T19 Repoint a Tableau workbook to a migrated data source
Original instruction
fixtures/finance.twbx points to a retired Excel source; the data now lives in Postgres 'fin' with renamed fields (map in docs/field_map.csv). In Tableau Desktop, replace the data source, fix the 14 broken calculated fields and 3 sets, and confirm all 9 sheets render the same numbers as before. Save output/finance_pg.twbx.
- Platforms
mac· macOSwin· Windows- Applications
- Tableau Desktop · PostgreSQL
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgment- Work product
WP03· Visual layout- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Grader publishes both old (with old extract) and new workbooks to the sandbox server and compares crosstab exports of all 9 sheets: identical values. No red-pill (missing) fields in XML; connection class postgres. Tableau Document API, tabcmd and Python are absent, no server token is issued, and .twbx writes by processes other than Tableau Desktop are detected.
Proposed partial credit
20% data source replaced; 30% calcs fixed; 10% sets; 40% sheet outputs identical
R03.02.01.02-T20 Build a multi-page sales performance app in Power BI from requirements
Original instruction
From spec/requirements.md (10 numbered requirements: 4 pages, navigation buttons, bookmarks for 'Plan vs Actual' toggles, dynamic RLS, a what-if parameter, incremental refresh policy on Sales), build output/sales_app.pbix in Power BI Desktop over fixtures/sales_db.sqlite via ODBC.
- Platforms
win· Windows- Applications
- Power BI Desktop
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropcode· Code & commands- Work product
WP03· Visual layout- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Grader checks each requirement with a scripted test: Layout JSON (pages, buttons, bookmarks), model via local AS (RLS, what-if table, measures), refresh policy in the model's TMSL. Measure values for 10 contexts equal gold. Tabular Editor, DAX Studio and pbi-tools are grader-only, the AS port is firewalled from agent processes, and non-Desktop .pbix writes are detected.
Proposed partial credit
60% requirements met (6 pts each); 25% measure values; 15% navigation/bookmarks render
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.