Computer-Use Agent Taxonomy中文
R02.04.01.01 · TASK CLASS

Design or screenshot to working page

Turn a design file, screenshot or sketch into a page that matches it visually and interactively.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications
Figma · browser · Vite
Class GUI-need label
HY · Hybrid

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R02.04.01.01-T01 Turn a pricing-page screenshot into HTML and CSS

Original instruction

`input/pricing.png` (1440x2100) shows a 3-tier pricing page with a toggle for monthly/yearly, feature lists and a FAQ accordion. Build `out/index.html` + `styles.css` (no frameworks) that matches the screenshot at 1440 px wide, with the toggle switching prices and the accordion working. Use the text from `input/copy.json`.

Platforms
web · Any OS (web)
Applications
browser · VS Code
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Headless render at 1440 px: Design2Code-style block-match >= 0.9, text similarity >= 0.95, CLIP >= 0.9, SSIM >= 0.8 vs screenshot; Playwright checks toggle switches all 3 prices to yearly values and accordion items expand.

Proposed partial credit

50% visual metrics (scaled); 15% text; 20% toggle; 15% accordion

R02.04.01.01-T02 Implement a responsive React page from a view-only Figma file

Original instruction

The sandbox Figma file `Clinic Landing` (view-only link in `input/figma_url.txt`) has Desktop 1440, Tablet 834 and Mobile 390 frames. Inspect it in the Figma web app (dev mode is off; no API token) and implement a React + CSS page in `/work/clinic-landing` that matches all three breakpoints, using the exported assets in `input/assets/`.

Platforms
web · Any OS (web)
Applications
Figma · Vite · React
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment nav · Pan/zoom/scroll code · Code & commands
Work product
WP03 · Visual layout
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

Renders at 1440/834/390 compared to grader exports of the frames: SSIM >= 0.85 and per-element bbox IoU >= 0.8 for 25 tagged elements (data-testid map provided); fonts/colors ΔE2000 <= 3. Figma REST is blocked for the agent.

Proposed partial credit

40% desktop match; 30% tablet; 30% mobile

R02.04.01.01-T03 Rebuild a dense analytics dashboard from a screenshot

Original instruction

`input/dashboard.png` shows an admin dashboard: sidebar nav, 4 KPI cards, a sortable 12-column data table with 25 rows, and a line chart. Recreate it in `out/` with HTML/CSS/JS (any chart library) using data from `input/data.json`; the table must sort by clicking headers.

Platforms
web · Any OS (web)
Applications
browser · Chart.js
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Render at 1600x1000: block-match recall >= 0.85 (dense tables are the hard part), SSIM >= 0.8; table cell text matches data.json; header click sorts ascending/descending (Playwright); chart canvas present with 12 data points (exposed via data attribute).

Proposed partial credit

40% layout metrics; 25% table content; 20% sorting; 15% chart

R02.04.01.01-T04 Turn a hand-drawn mobile sketch into a working page

Original instruction

`input/sketch.jpg` is a photo of a hand-drawn mobile screen for a habit tracker (header, weekly streak grid, list of habits with checkboxes, floating add button). Build a 390x844 mobile web page following the sketch's layout and `input/notes.md` (colors, font), where checking a habit updates the streak grid.

Platforms
web · Any OS (web)
Applications
browser · VS Code
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment code · Code & commands
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Layout check on 6 annotated regions (bbox IoU >= 0.6 vs sketch annotation scaled to 390 px); colors from notes within ΔE2000 5; Playwright: checking a habit marks today's cell; add button opens a form. VLM rubric for overall fidelity (documented, 20% weight).

Proposed partial credit

40% region layout; 30% interactions; 10% colors; 20% VLM fidelity rubric

R02.04.01.01-T05 Build a three-screen flow with modal and tabs from mockups

Original instruction

`input/mockups/` has 3 PNG screens (list, detail with tabs, edit modal) and a flow diagram. Implement them as one SPA in `/work/flow` where clicking a list item opens the detail, tabs switch content, `Edit` opens the modal with focus trapped, and saving updates the list. Match each mockup at 1280x800.

Platforms
web · Any OS (web)
Applications
browser · Vite
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Per-screen SSIM >= 0.85 at 1280x800 after scripted navigation; Playwright journeys: navigation, tab switching, modal focus trap (Tab cycles inside), Esc closes, save persists in list; axe-core no critical violations.

Proposed partial credit

45% visual per screen (a third each); 40% interaction journeys; 15% accessibility

R02.04.01.01-T06 Code a responsive email template that renders in major clients

Original instruction

Implement `input/newsletter.png` (600 px wide email design) as table-based HTML email in `out/newsletter.html` with inline CSS, dark-mode support and a mobile stack below 480 px. It must render consistently in the sandbox render farm (Outlook desktop, Gmail web, Apple Mail).

Platforms
web · Any OS (web) win · Windows mac · macOS
Applications
browser · MJML · Litmus-like sandbox
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

Render farm screenshots: SSIM >= 0.85 vs design in Apple Mail and Gmail, >= 0.75 in Outlook (Word engine); mobile render stacks columns; no external CSS; all images have alt; HTML size < 100 KB.

Proposed partial credit

45% client renders (a third per client); 20% mobile stack; 15% dark mode; 20% email constraints

R02.04.01.01-T07 Implement a Figma design with auto-layout and tokens in Tailwind

Original instruction

The exported Figma frame `input/settings.fig.png` plus `input/tokens.json` (colors, spacing, radii) define a settings page. Implement it with Tailwind CSS in `/work/settings-ui`, mapping tokens into `tailwind.config.js` (no arbitrary hex values in markup), matching the design at 1440 and 768 px.

Platforms
web · Any OS (web)
Applications
Tailwind CSS · Vite
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

SSIM >= 0.88 at both widths; tailwind config contains all tokens; markup has no arbitrary color/spacing values (regex for [#..] and [..px]); computed styles of 15 elements match token values.

Proposed partial credit

45% visual; 25% token usage; 30% computed style checks

R02.04.01.01-T08 Recreate a scroll-driven hero animation from a screen recording

Original instruction

`input/hero_scroll.mp4` records a landing page where, while scrolling 0-1200 px, a product image scales and rotates, a headline fades word by word, and the background color interpolates. Recreate the page and animation timing in `out/` (CSS scroll-driven animations or JS), matching the recording frames.

Platforms
web · Any OS (web)
Applications
browser · VS Code
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
av · Audio/video over time vis · Visual judgment code · Code & commands
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
5 / 5

Proposed verifier

Grader scrolls the page in 12 steps and captures frames; compares to corresponding video frames: mean SSIM >= 0.8; image transform per step within 5% scale / 5 deg of reference; background ΔE2000 <= 5 per step.

Proposed partial credit

40% frame SSIM; 30% transform curve; 15% headline fade timing; 15% background

R02.04.01.01-T09 Build an accessible form from a design including error and success states

Original instruction

`input/form_states/` has 4 designs for a contact form: default, focus, validation errors and success. Implement the form in `out/` with client-side validation that produces exactly the designed error states and messages, ARIA wiring for errors, and the success state after submit to the sandbox endpoint.

Platforms
web · Any OS (web)
Applications
browser · VS Code
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Playwright drives each state; screenshot SSIM >= 0.85 per state; error messages exact; aria-invalid/aria-describedby set; axe-core zero serious violations; sandbox endpoint receives valid payload.

Proposed partial credit

40% states visual; 25% validation logic; 20% accessibility; 15% submit

R02.04.01.01-T10 Implement light and dark themes from paired screenshots

Original instruction

`input/light.png` and `input/dark.png` show the same blog article page in two themes. Build the page in `out/` with a theme switch that follows `prefers-color-scheme` by default, using CSS custom properties, matching both screenshots.

Platforms
web · Any OS (web)
Applications
browser
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Render with emulated light and dark: SSIM >= 0.88 each; switch toggles and persists; all colors via custom properties (no hard-coded colors in component rules, CSS parse); contrast ratio >= 4.5 for body text in both.

Proposed partial credit

35% light match; 35% dark match; 15% switch + preference; 15% contrast/custom properties

R02.04.01.01-T11 Rebuild a chart from a screenshot as an interactive SVG

Original instruction

`input/chart.png` is a stacked bar chart (5 series, 12 months) with legend and axis labels; `input/values.csv` holds the underlying data. Build an interactive SVG chart in `out/chart.html` (D3 or plain SVG) that matches the screenshot and shows a tooltip with series and value on hover; clicking a legend item toggles the series.

Platforms
web · Any OS (web)
Applications
D3 · browser
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment hover · Hover & context
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

SSIM >= 0.85 vs screenshot; SVG rect heights proportional to CSV values (±1 px); colors ΔE2000 <= 3 vs screenshot palette; Playwright hover shows correct tooltip text; legend toggle hides series rects.

Proposed partial credit

40% visual + colors; 25% data accuracy; 20% tooltip; 15% legend toggle

R02.04.01.01-T12 Convert a PDF brochure page into a responsive HTML page

Original instruction

`input/brochure.pdf` page 2 is a two-column print layout with a pull quote, an image with a caption and a sidebar table. Recreate it as a responsive HTML page that matches the PDF at 1200 px and reflows to one column at 600 px with the pull quote after paragraph 2.

Platforms
web · Any OS (web)
Applications
browser · Preview/Acrobat
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment code · Code & commands
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

1200 px render vs PDF raster: SSIM >= 0.82, text similarity >= 0.97; 600 px: single column and DOM/visual order puts the quote after paragraph 2; table is a real <table>; image alt equals caption.

Proposed partial credit

45% desktop match; 30% mobile reflow; 15% semantics; 10% text

R02.04.01.01-T13 Build a SwiftUI screen from an iOS screenshot

Original instruction

`input/profile_ios.png` is an iPhone 15 screenshot of a profile screen (avatar, stats row, segmented control, grid of 9 thumbnails). Implement it as a SwiftUI view in `/work/ProfileUI` (Xcode project provided) that matches in the iOS Simulator with light mode and Dynamic Type default, and the segmented control switches the grid.

Platforms
mac · macOS ios · iOS/iPadOS
Applications
Xcode · SwiftUI · iOS Simulator
Interfaces
desk · Desktop GUI mob · Mobile touch
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

XCUITest launches the view on iPhone 15 simulator and screenshots: SSIM >= 0.85 vs input; element frames for 8 accessibility identifiers within 8 pt; segmented control switch changes grid images (hash).

Proposed partial credit

45% visual; 30% element frames; 25% interaction

R02.04.01.01-T14 Implement a Figma prototype's hover, drawer and carousel interactions

Original instruction

The sandbox Figma prototype (view link in `input/proto_url.txt`) for a furniture store home page includes hover states on cards, a slide-in cart drawer and an auto-advancing carousel (4 s). Play the prototype in the Figma web app to observe interactions and implement them in `/work/store-home` with matching visuals and timing.

Platforms
web · Any OS (web)
Applications
Figma · Vite
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment hover · Hover & context code · Code & commands
Work product
WP03 · Visual layout
Conditions
multiapp · Multi-app
Proposed difficulty
4 / 5

Proposed verifier

Static SSIM >= 0.85 at 1440; hover state screenshot per card style SSIM >= 0.85; drawer slides from right with duration 300 ms ±50; carousel advances every 4 s ±0.3; behavior specs only observable by playing the prototype.

Proposed partial credit

35% static match; 20% hover; 25% drawer; 20% carousel timing

R02.04.01.01-T15 Lay out a newspaper-style front page with CSS Grid from a scan

Original instruction

`input/frontpage.png` is a newspaper front page with a 6-column grid, spanning headlines, a lead image spanning 4 columns, column rules, drop caps and a below-the-fold teaser row. Recreate it with CSS Grid in `out/` at 1280 px, using `input/articles.json` text, with text flowing in CSS multi-column where the scan shows it.

Platforms
web · Any OS (web)
Applications
browser
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

SSIM >= 0.8 and block-match >= 0.85; grid-template-columns has 6 tracks; lead image spans 4; drop caps via ::first-letter; column rules present; text content equals articles.json.

Proposed partial credit

45% visual metrics; 30% grid structure; 15% typographic details; 10% content

R02.04.01.01-T16 Build a data-heavy admin table UI with filtering from a design spec

Original instruction

`input/admin_users.png` plus `input/spec.md` describe a user-management page: filter bar (role, status, date range), bulk-select with action bar, paginated table of 1,000 users from `input/users.json`, column resizing and a row detail side panel. Implement in `/work/admin-users` with any framework.

Platforms
web · Any OS (web)
Applications
React · Vite · Playwright
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment drag · Drag & drop
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
4 / 5

Proposed verifier

SSIM >= 0.85 vs design at 1440; Playwright: filters produce expected row counts for 10 combos; bulk select selects only filtered rows; column resize by drag changes width ±5 px; side panel shows correct user.

Proposed partial credit

30% visual; 30% filters; 25% bulk select and resize; 15% side panel

R02.04.01.01-T17 Turn a grayscale wireframe into a styled page with given content

Original instruction

`input/wireframe.png` is a low-fidelity grayscale wireframe of an event page (hero, schedule, speakers grid, venue map placeholder, footer). Implement it at 1280 px using `input/content.json` and the brand colors/fonts in `input/brand.md`, preserving the wireframe's layout proportions.

Platforms
web · Any OS (web)
Applications
browser
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
G2 · GUI-essential
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
2 / 5

Proposed verifier

Section bounding boxes IoU >= 0.75 vs wireframe annotation; content strings present in correct sections; brand colors used (computed styles within ΔE2000 3); speakers grid has 3 columns.

Proposed partial credit

40% layout IoU; 25% content placement; 20% brand styles; 15% grid

R02.04.01.01-T18 Implement a game HUD overlay from a mockup

Original instruction

`input/hud.png` is a game HUD mockup: health/armor bars with segmented ticks, minimap frame, ammo counter, ability cooldown radial timers. Implement as an HTML/CSS/JS overlay on a 1920x1080 transparent page driven by `window.hud.update(state)` per `input/api.md`.

Platforms
web · Any OS (web)
Applications
browser
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
code · Code & commands vis · Visual judgment
Work product
WP03 · Visual layout
Conditions
None declared
Proposed difficulty
3 / 5

Proposed verifier

Grader calls update with 6 scripted states and screenshots: SSIM >= 0.85 vs pre-rendered expected frames; radial cooldown arc angle within 3 deg; bar widths within 2 px; background transparent (alpha 0).

Proposed partial credit

40% static match; 30% state-driven accuracy; 20% radial timers; 10% transparency

R02.04.01.01-T19 Build a five-page marketing site from a Figma file with shared components

Original instruction

The sandbox Figma file `Atlas Marketing` (view-only) has 5 pages (Home, Product, Pricing, About, Contact) at desktop and mobile, built from shared components (nav, footer, cards, CTA). Implement the site in `/work/atlas-site` with reusable components, client-side routing, and the contact form posting to the sandbox endpoint.

Platforms
web · Any OS (web)
Applications
Figma · Next.js/Astro · Playwright
Interfaces
brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment nav · Pan/zoom/scroll code · Code & commands
Work product
WP03 · Visual layout
Conditions
long · Long-horizon multiapp · Multi-app
Proposed difficulty
5 / 5

Proposed verifier

10 renders (5 pages × 2 widths) vs grader frame exports: mean SSIM >= 0.85; nav/footer DOM identical across pages (component reuse check); links route correctly; form posts valid payload; Lighthouse accessibility >= 90.

Proposed partial credit

50% visual (a tenth per render); 15% component reuse; 15% routing; 20% form and accessibility

R02.04.01.01-T20 Produce a pixel-precise page from a Sketch design on macOS

Original instruction

Open `input/Checkout.sketch` in Sketch on the Mac, inspect measurements, text styles and exported slices, and build the checkout page in `out/` so that at 1440x1024 it differs from Sketch's own PNG export by under 2% of pixels (anti-aliasing tolerance), including the payment-method radio cards and order summary.

Platforms
mac · macOS
Applications
Sketch · browser
Interfaces
desk · Desktop GUI brow · Browser term · Terminal
GUI need · task setting
HY · Hybrid
GUI need · ordinary access
HY · Hybrid
Interaction primitives
vis · Visual judgment geom · Precise geometry code · Code & commands
Work product
WP03 · Visual layout
Conditions
licensed · Licensed pro app multiapp · Multi-app
Proposed difficulty
5 / 5

Proposed verifier

Pixel diff (pixelmatch threshold 0.1) between render and Sketch export < 2% of pixels; text styles (font size/line-height/letter-spacing) of 12 elements equal Sketch values; radio cards selectable.

Proposed partial credit

60% pixel diff (linear from 15 down to 2 pct of pixels); 25% text styles; 15% interaction

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

In this workflow family