Reading dashboards & rendered charts
Extract exact KPIs and trends from interactive dashboards and charts.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Tableau · Power BI · Grafana
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R03.02.01.03-T01 Read a p95 latency and peak error hour from Grafana panels
Original instruction
In the Grafana sandbox, open dashboard 'API Latency' (viewer role, no Explore or query inspector). Report the p95 latency at 2025-10-01 14:00 UTC and the hour on that day with the highest error rate, in output/answer.json as {p95_ms, peak_error_hour}.
- Platforms
web· Any OS (web)- Applications
- Grafana
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Exact gold from the seeded Prometheus data: p95 within ±2%, hour exact. Viewer role blocks Explore, panel inspect and the data source API; the request log is checked for /api/ds/query calls not originating from dashboard panels.
Proposed partial credit
60% p95 value; 40% peak hour
R03.02.01.03-T02 Find the region with the largest quarter-on-quarter margin drop
Original instruction
Open the published Power BI report 'Exec Margin' in the sandbox (export and 'show as table' disabled). Using its slicers and visuals, find the region whose gross margin fell most from Q2 to Q3 2025 and the drop in percentage points, and write output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Power BI Service
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickhover· Hover & contextvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Gold computed from the hidden dataset: region exact, drop ±0.1 pp. Tenant settings disable export, Analyze in Excel and XMLA for the agent's account, so values must be read from the rendered report.
Proposed partial credit
50% region; 50% drop value
R03.02.01.03-T03 Read an outlier's exact values from a scatter-plot tooltip
Original instruction
On the embedded Tableau dashboard 'Store Efficiency' (sandbox Tableau Server, download disabled), find the store that is the farthest outlier above the trend line and report its name, sales per sq ft and staff hours shown in its tooltip in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Tableau Server
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Name exact; both values exact as displayed (tooltip formatting). Download/crosstab permissions are denied and no Tableau REST/Metadata API token is issued; the outlier is not labeled, so it must be identified visually and hovered.
Proposed partial credit
40% correct store; 30% sales per sq ft; 30% staff hours
R03.02.01.03-T04 Return the ranking of regions from a bar chart
Original instruction
In the Apache Superset dashboard 'Revenue Overview' (sandbox, viewer role), read the 'Revenue by region' bar chart for FY2025 and return the regions sorted by revenue descending as a JSON list in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Apache Superset
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Exact order vs gold (6 regions; two close bars differ by 1.5%, resolvable via hover). Viewer role is denied SQL Lab, view-query and CSV download, and no API token is issued, so the order must be read from the chart.
Proposed partial credit
70% order correlation (Kendall tau scaled); 30% exact full order incl. the close pair
R03.02.01.03-T05 Extract plotted values from a figure in a PDF report
Original instruction
input/q3_report.pdf page 7 has Figure 3, a grouped bar chart of quarterly operating margin for 3 business units (Q1–Q4) with gridlines but no data labels. Extract the 12 values in percent and write output/fig3.csv (unit, quarter, margin_pct).
- Platforms
mac· macOSwin· Windows- Applications
- Adobe Acrobat Reader · Preview
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Values compared to the generating data: all 12 within ±0.3 pp scores full; per-value partial. The PDF embeds the chart as vector paths without text data, so text extraction gives no numbers. No Python, PDF libraries or Ghostscript are installed, so the paths cannot be parsed; values must be read from the rendered chart.
Proposed partial credit
33% first unit's 4 quarters; 33% second unit's 4 quarters; 34% third unit's 4 quarters (each value within ±0.3 pp)
R03.02.01.03-T06 Count server errors per service in Kibana for a fixed window
Original instruction
In the Kibana sandbox (Discover/Dashboard access only), set the time range to 2025-10-12 00:00–24:00 UTC and report the number of HTTP 5xx responses per service from the 'Web Logs' dashboard, plus the service with the highest 5xx rate, in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Kibana
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Counts per service exact vs gold; top service exact. Agent's Kibana role is denied Dev Tools and no Elasticsearch API key is issued; relative time ranges ('last 24h') produce wrong counts because data continues to stream.
Proposed partial credit
70% per-service counts; 30% top-rate service
R03.02.01.03-T07 Explain a KPI change with a Power BI decomposition tree
Original instruction
In the published Power BI report 'Churn Drivers' (sandbox), use the Decomposition Tree to find the path (dimension values, in order of highest value split) that explains the largest share of churned MRR in September 2025, and report the path and its MRR in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Power BI Service
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Gold path computed from data using the tree's 'High value' splits: the 3-level path exact (e.g., Plan=Pro > Region=APAC > Tenure<6m) and MRR ±1%. Export and Analyze in Excel are disabled and no Power BI REST/XMLA token is issued.
Proposed partial credit
60% path (20 pts per level); 40% MRR value
R03.02.01.03-T08 Answer five filter-combination questions on a Tableau dashboard
Original instruction
On the sandbox Tableau Server dashboard 'Shipping Performance', answer the 5 questions in input/questions.md (e.g., late-shipment % for Corporate segment by Second Class in 2024) by setting filters and reading the highlight table. Write answers to output/answers.json.
- Platforms
web· Any OS (web)- Applications
- Tableau Server
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Each answer compared to gold exactly as displayed (one decimal). Crosstab download disabled and no Tableau REST token is issued; filter states left on the dashboard are not graded.
Proposed partial credit
40% answers 1–2; 40% answers 3–4; 20% answer 5 (20 pts per answer)
R03.02.01.03-T09 Link a latency spike to the preceding deployment annotation
Original instruction
In the Grafana sandbox dashboard 'Checkout', find the largest p99 latency spike in the past 7 days (fixed snapshot), the deployment annotation that preceded it, and the spike's peak value. Report {annotation_text, deploy_time, peak_ms} in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Grafana
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Annotation text and time exact; peak within ±3%. Three annotations appear near spikes; only one precedes the largest. Viewer role blocks the annotations API and panel inspect.
Proposed partial credit
40% correct annotation; 20% time; 40% peak value
R03.02.01.03-T10 Compute true change from a chart with a truncated axis
Original instruction
The dashboard 'Weekly Signups' in the Metabase sandbox shows two bars with a y-axis starting at 9,500. Read both values (hover where needed) and report the actual percentage change and whether the visual exaggerates it, with the visual ratio of bar heights, in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Metabase
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Values exact; % change ±0.1; exaggerates true; visual height ratio ±5% of the rendered pixel ratio measured by the grader. Viewer group is denied question editing, downloads and API keys.
Proposed partial credit
40% values; 30% % change; 30% exaggeration ratio
R03.02.01.03-T11 Read the hottest bucket from a latency heatmap
Original instruction
In the Grafana sandbox dashboard 'Queue Latency', the heatmap panel shows latency buckets over time. For 2025-10-10 between 09:00 and 10:00 UTC, report the latency bucket with the highest count, its count, and the 1-minute time slot where it occurs.
- Platforms
web· Any OS (web)- Applications
- Grafana
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Bucket bounds exact, count exact, time slot exact vs seeded data. Requires zooming the time range and hovering cells; inspect and API disabled.
Proposed partial credit
35% bucket; 35% count; 30% time slot
R03.02.01.03-T12 Recover series values from a stacked area chart image
Original instruction
input/channel_mix.png is a stacked area chart of monthly traffic by 4 channels over 2024 with gridlines every 10k. Recover each channel's monthly values and write output/channel_mix.csv (month, channel, sessions).
- Platforms
mac· macOSwin· Windows- Applications
- Preview · Image viewer
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgment- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Compared to generating data: per value within ±1.5k scores full credit; per-month totals within ±1k. Stacked areas require subtracting layer boundaries, which naive digitizing gets wrong. No Python, OpenCV or digitizer tools are installed and network is off, so values must be read visually in the viewer.
Proposed partial credit
60% individual values; 25% monthly totals; 15% CSV schema
R03.02.01.03-T13 Drill into a KPI on the Power BI mobile app
Original instruction
On the iOS Simulator with the Power BI mobile app signed into the sandbox tenant, open report 'Field Sales' in its phone layout, tap through the KPI 'Units vs Target' to the drill-through page, and report the three reps furthest below target in October 2025 with their gaps.
- Platforms
ios· iOS/iPadOS- Applications
- Power BI Mobile
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Rep names exact and in order; gaps exact as displayed. The phone layout hides the table on desktop view. The tenant disables export and XMLA, and no Power BI API token is issued to the agent.
Proposed partial credit
50% correct reps in order; 50% gap values
R03.02.01.03-T14 List counties above a threshold from a choropleth map
Original instruction
On the sandbox Tableau Server dashboard 'Broadband Access', the choropleth colors counties of Ohio by % households without broadband in 5 legend bins. List all counties in the top bin (≥25%) and their exact values from tooltips in output/answer.csv.
- Platforms
web· Any OS (web)- Applications
- Tableau Server
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Set of counties vs gold: precision and recall; values exact as displayed. Small counties require zooming; download is disabled and no Tableau REST token is issued.
Proposed partial credit
70% county set F1; 30% values
R03.02.01.03-T15 Compare two dashboard snapshots and flag big KPI moves
Original instruction
Dashboards 'Ops Weekly — W40' and 'Ops Weekly — W41' (Metabase snapshots) each show 18 KPIs. List every KPI that changed by more than 10% week over week with both values and the % change in output/moves.csv.
- Platforms
web· Any OS (web)- Applications
- Metabase
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & click- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Set of flagged KPIs exact (7 expected) with values and % change ±0.1. Downloads are disabled and the agent's group is denied native SQL and API keys.
Proposed partial credit
60% correct KPI set; 40% values and changes
R03.02.01.03-T16 Find rows in a paginated Looker Studio table
Original instruction
In the sandbox Looker Studio report 'Partner Payouts', the table shows 340 partners across pages (100 per page, sorted by name). Find the payout amounts for the 6 partner IDs in input/ids.txt and write output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Looker Studio
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Six amounts exact. The report has no search control, data download is disabled, and the agent's account can view the report only, not its data source, so pagination or re-sorting is needed.
Proposed partial credit
34% amounts 1–2; 33% amounts 3–4; 33% amounts 5–6
R03.02.01.03-T17 Reconcile revenue between a Finance and a Sales dashboard
Original instruction
The Finance Power BI report and the Sales Tableau dashboard (both sandbox, view-only) disagree on 2025 revenue. Find the month(s) where they differ, the size of each difference, and the cause shown in the dashboards' definitions or filters (e.g., refunds, timing), writing output/recon.json.
- Platforms
web· Any OS (web)- Applications
- Power BI Service · Tableau Server
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickhover· Hover & contextvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
multiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
Gold: two months differ (refund timing and an excluded channel). Grader checks months exact, differences ±0.5%, cause category exact for each. Exports, Analyze in Excel and crosstab downloads are disabled and no Power BI or Tableau API token is issued, so values and definitions must be read in the browser.
Proposed partial credit
40% months; 30% differences; 30% causes
R03.02.01.03-T18 Pinpoint the exact timestamp of a metric minimum
Original instruction
In the Grafana sandbox dashboard 'Fleet Battery', find the exact minute (UTC) when vehicle V-2271's state of charge reached its minimum during 2025-10-08, and the value. The panel defaults to a 30-day range with downsampling.
- Platforms
web· Any OS (web)- Applications
- Grafana
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollhover· Hover & contextvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Minute and value exact vs raw data. At the default range downsampling hides the true minimum, so the agent must zoom in until 1-minute resolution. Viewer role lacks Explore and no Grafana API token or datasource credentials are issued, so raw points come only via the panel.
Proposed partial credit
60% timestamp; 40% value
R03.02.01.03-T19 Decline to invent a metric the dashboard does not show
Original instruction
A manager asks: 'What was our net revenue retention for Q3 from the Customer Health dashboard?' The sandbox Metabase dashboard shows gross churn, expansion MRR and logo retention but not NRR or starting MRR. Answer in output/answer.json.
- Platforms
web· Any OS (web)- Applications
- Metabase
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
clarify· Ask when unclear- Proposed difficulty
- 3 / 5
Proposed verifier
Pass if the answer states NRR is not shown and cannot be computed from the visible values (starting MRR missing), names the closest available metrics with their Q3 values exactly, and gives no fabricated NRR number. Agent's Metabase group is denied native SQL and data-model access and gets no API key, so starting MRR cannot be pulled from the DB.
Proposed partial credit
50% no fabricated value + correct reason; 50% nearest metrics with values
R03.02.01.03-T20 Fill a 25-field weekly ops review from four dashboards
Original instruction
Fill input/ops_review_template.xlsx (25 fields: uptime, p95s, incident counts, revenue, conversion, NPS, backlog) for week 41 using the sandbox Grafana, Kibana, Power BI and Metabase dashboards (view-only, exports disabled). Save to output/ops_review_w41.xlsx.
- Platforms
web· Any OS (web)- Applications
- Grafana · Kibana · Power BI Service · Metabase · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickhover· Hover & contextvis· Visual judgmentmulti· Cross-app- Work product
WP02· Records/table- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Each field compared to gold with per-field tolerance (exact for counts, ±1% for rates). Template structure unchanged. Exports are disabled on all four dashboards and no Grafana, Kibana, Power BI or Metabase API token is issued, so values must be read from the dashboards in the browser.
Proposed partial credit
32% fields 1–8; 32% fields 9–16; 36% fields 17–25 (4 pts per field)
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.