Store & catalog administration
Manage products, variants, prices, inventory and storefront in commerce admins.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Shopify admin · WooCommerce · Magento
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R05.08.01.01-T01 Create a three-variant product with inventory in a Shopify dev store
Original instruction
Shopify development store 'brightline-dev'. Create product 'Merino Beanie' (vendor Brightline, type Hats) with option Size (S, M, L), SKUs MB-S/M/L, price 34.00, compare-at 42.00, weight 0.1 kg, and inventory 20/35/15 at location 'Main Warehouse'. Upload input/beanie.jpg with alt text and set status Active on Online Store.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Admin GraphQL read (grader token): product fields, 3 variants with SKUs/prices/weights, inventory levels per variant at location, image alt, status and publication. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
40% variants and prices; 30% inventory; 20% media/alt; 10% active and published
R05.08.01.01-T02 Schedule a category sale price in WooCommerce
Original instruction
WooCommerce admin on the staging shop. Put all products in category 'Towels' (simple and variable) on sale at 20% off regular price, scheduled 2026-11-27 00:00 to 2026-11-30 23:59 store time, rounding to 2 decimals. Do not change products outside the category.
- Platforms
web· Any OS (web)- Applications
- WooCommerce
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
DB: _sale_price and schedule dates for every product/variation in Towels equal key; others unchanged. WooCommerce REST keys, WP-CLI and the CSV importer are disabled for the agent; changes must come from admin screens.
Proposed partial credit
60% sale prices exact; 30% schedule dates; 10% no collateral changes
R05.08.01.01-T03 Report the best-selling product of last month from Magento reports
Original instruction
Magento Open Source 2.4 admin (seeded store, like the WebArena shopping admin). Using Reports > Products > Bestsellers for September 2026 (monthly period, default store), write the top product's name, SKU and quantity ordered to output/answer.txt.
- Platforms
web· Any OS (web)- Applications
- Magento
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
Answer equals key from the refreshed bestsellers aggregate; stale statistics must be refreshed first (a decoy product leads before refresh).
Proposed partial credit
40% product; 30% SKU; 30% quantity
R05.08.01.01-T04 Raise prices 5% on a collection except MAP-protected items in Shopify
Original instruction
Shopify development store. For collection 'Winter 2026', increase price and compare-at price of every variant by 5% (round to .99 endings per input/pricing_rules.txt), excluding products tagged 'MAP'. Use the admin bulk editor.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Variant prices and compare-at prices equal key for in-scope variants; MAP-tagged and out-of-collection variants unchanged. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin. Control pair: T12.
Proposed partial credit
60% in-scope prices exact; 25% exclusions honored; 15% compare-at prices
R05.08.01.01-T05 Build a rule-based automated collection and publish it
Original instruction
Shopify development store. Create automated collection 'Gifts under 50' with conditions: price less than 50 AND product tag equals 'giftable' AND inventory stock greater than 0; sort by best selling; SEO title and description from input/collection_seo.txt; image input/gifts.jpg; publish to Online Store only.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Collection rule set (all conditions, disjunctive false), sort order, SEO fields, image and publications equal spec; member products equal key. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
50% rules; 20% sort; 20% SEO/image; 10% publication
R05.08.01.01-T06 Create a variable product with twelve variations and color images in WooCommerce
Original instruction
WooCommerce admin. Create variable product 'Canvas Tote' with global attributes Color (Natural, Black, Olive) and Size (S, M, L, XL) used for variations, generate all 12 variations, set prices per input/tote_prices.csv, stock per variation, and assign the color-specific images from input/totes/ to each variation.
- Platforms
web· Any OS (web)- Applications
- WooCommerce
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
DB: 12 variations with attribute pairs, SKUs, prices, stock and image IDs equal key (image file hash per color). WooCommerce REST keys, WP-CLI and the CSV importer are disabled for the agent; changes must come from admin screens.
Proposed partial credit
30% attributes and variations; 30% prices; 20% stock; 20% images
R05.08.01.01-T07 Create a configurable product with visual swatches in Magento
Original instruction
Magento 2.4 admin. Create configurable product 'Trail Jacket' from attribute set 'Apparel' with configurations Color (visual swatches: hex codes in input/colors.txt) and Size (S-XL), price 129, qty 10 per child, images per color, and enable it on the default website.
- Platforms
web· Any OS (web)- Applications
- Magento
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
catalog_product_entity: parent with child simple products for all combos; swatch option values and hex codes; prices, stock, images per color; website assignment. Integration tokens are revoked for the agent and the admin action log must show the changes from the admin panel.
Proposed partial credit
30% configurations; 25% swatches; 25% child price/stock; 20% images
R05.08.01.01-T08 Rename collections and keep menus and old URLs working in Shopify
Original instruction
Shopify development store. Rename collections 'Hats' to 'Headwear' and 'Bags' to 'Carry' including their URL handles, create URL redirects from the old /collections/ paths, update the 'Main menu' items and nest 'Carry' under a new parent item 'Accessories' by dragging in the menu editor.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & droptype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Admin read: collection titles and handles; 2 redirects old->new; menu tree structure equals key; storefront requests to old URLs return 301 to new. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
30% renames; 30% redirects; 40% menu structure
R05.08.01.01-T09 Add product options with price-adjusting modifier rules in BigCommerce
Original instruction
BigCommerce sandbox store. For product 'Custom Doormat', add a shared variant option Size (60x90, 90x150) with SKU-level prices and a modifier 'Monogram' (text, max 3 chars) plus a checkbox 'Weatherproof coating' that adds 12.00 via a rule; mark monogram optional. Preview the product page.
- Platforms
web· Any OS (web)- Applications
- BigCommerce
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Catalog read by grader token: variant options and SKU prices, modifiers (types, constraints), adjuster rule +12.00; storefront add-to-cart test with coating adds 12.00. Agent has no API account.
Proposed partial credit
30% variants; 30% modifiers; 30% price rule; 10% storefront test
R05.08.01.01-T10 Receive an incoming inventory transfer with damaged units in Shopify
Original instruction
Shopify development store with two locations. Incoming transfer #T0007 from supplier lists 6 SKUs. Per input/receiving_note.pdf, receive the counted quantities (two short), mark 3 units of SKU MB-M as rejected, and adjust 'Damaged' inventory state accordingly. Leave the transfer partially received.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Admin read: transfer line received/rejected quantities equal key; inventory available per SKU at destination equal key; transfer status partial. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
50% received quantities; 25% rejections; 25% resulting inventory
R05.08.01.01-T11 Configure weight-based shipping zones and tax classes in WooCommerce
Original instruction
WooCommerce admin. Create shipping zones US-West (CA, OR, WA), US-Rest and Canada with flat-rate methods using weight formulas from input/shipping_rates.pdf (e.g. 6 + 1.5 * [qty]), free shipping over 150 in US zones, and add tax class 'Reduced' with rates from input/tax_rates.csv applied to the 'Books' category products.
- Platforms
web· Any OS (web)- Applications
- WooCommerce
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
DB: zones with locations and methods/cost formulas equal key; free-shipping min amount; tax class rates; Books products set to Reduced. Grader runs 3 cart checkouts in a headless browser and checks shipping/tax totals. WooCommerce REST keys, WP-CLI and the CSV importer are disabled for the agent; changes must come from admin screens.
Proposed partial credit
40% zones and methods; 20% free shipping; 25% tax class; 15% cart tests
R05.08.01.01-T12 Apply the same collection price increase via the Admin GraphQL API (control pair of T04)
Original instruction
Control pair of T04. A second dev store with a custom-app token in env SHOPIFY_TOKEN. Use the Admin GraphQL API (productVariantsBulkUpdate) to apply the identical +5% rule with .99 rounding and MAP exclusions to 'Winter 2026'.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Shopify Admin API · Python
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Same checks as T04; compared with T04 to measure the GUI penalty.
Proposed partial credit
60% prices; 25% exclusions; 15% compare-at
R05.08.01.01-T13 Lay out a seasonal homepage in the Shopify theme editor
Original instruction
Shopify development store, Dawn theme duplicate 'Fall 2026'. In the theme editor: add an image banner with input/hero_fall.jpg, set the focal point on the model's face (marked in input/focal.json), heading/button text from input/home_copy.md, add a featured collection 'Gifts under 50' with 8 products, reorder sections by drag to banner > collection > newsletter, and check the mobile view. Do not publish.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clickgeom· Precise geometryvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Theme JSON (templates/index.json) read by grader: section order, settings, image focal point within 3% of target; theme unpublished; mobile screenshot shows face visible in crop (IoU check). Admin API tokens and Shopify CLI theme access are withheld from the agent, so templates/index.json can only be changed through the theme editor.
Proposed partial credit
30% sections and order; 30% focal point; 20% copy/settings; 20% not published
R05.08.01.01-T14 Create catalog and cart price rules and verify them on the storefront
Original instruction
Magento 2.4 admin. Create catalog price rule 'Outdoor 15' (15% off category Outdoor for NOT LOGGED IN and General groups, 2026-11-01 to 11-30, priority 1) and cart price rule 'SHIP50' (coupon SHIP50: free shipping when subtotal >= 50 and items from category Outdoor). Apply rules, then place a test order as guest using the coupon.
- Platforms
web· Any OS (web)- Applications
- Magento
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
catalogrule/salesrule rows equal spec; indexers applied; test order shows discounted prices and free shipping; coupon usage 1. Integration tokens are revoked for the agent and the admin action log must show the changes from the admin panel.
Proposed partial credit
35% catalog rule; 35% cart rule and coupon; 30% test order totals
R05.08.01.01-T15 Set country-specific pricing and duties for EU and UK markets in Shopify
Original instruction
Shopify development store with Markets. Create markets 'EU' and 'UK' with local currencies, price adjustments (+8% EU, +5% UK), rounding rules, fixed prices for 3 hero products from input/fixed_prices.csv, and duties/import taxes collected at checkout; disable market 'Rest of world'.
- Platforms
web· Any OS (web)- Applications
- Shopify admin
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Markets config read by grader: regions, currencies, price adjustments, rounding, fixed prices per product, duties setting, RoW disabled; storefront price checks in EUR/GBP for 3 products. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
30% markets/currencies; 25% adjustments and rounding; 25% fixed prices; 20% duties and RoW
R05.08.01.01-T16 Translate ten products for a French store view in Magento
Original instruction
Magento 2.4 admin with store view 'French'. For the 10 products in input/fr_translations.xlsx, switch scope to the French store view and set name, short description, URL key and meta title to the provided French values without changing the default-scope values.
- Platforms
web· Any OS (web)- Applications
- Magento · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
EAV rows with store_id = French equal sheet; default-scope (store 0) values unchanged; FR storefront URLs resolve. Integration tokens are revoked for the agent and the admin action log must show the changes from the admin panel.
Proposed partial credit
60% translated attributes at store-view scope; 30% defaults untouched; 10% URLs resolve
R05.08.01.01-T17 Normalize vendors, types, tags and material metafields on forty products
Original instruction
Shopify development store. Forty products imported from a marketplace have inconsistent vendor names, product types and tags. Normalize them to input/taxonomy.xlsx (canonical vendor, Shopify standard product category, tag set) and fill metafield 'custom.material' from each description, using the bulk editor where possible.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- Shopify admin · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Per-product vendor, category, tags and material metafield equal key (>= 95% field accuracy for full credit). No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
30% vendors; 25% categories; 25% tags; 20% metafields
R05.08.01.01-T18 Restructure a category tree while preserving URLs and layered navigation
Original instruction
Magento 2.4 admin. Per input/new_category_tree.pdf, create the new tree (3 levels), move 60 products into new categories, make leaf categories anchor with layered navigation on Color, Size and Price, keep old category URLs working via permanent URL rewrites, and reindex. Remove the emptied old categories.
- Platforms
web· Any OS (web)- Applications
- Magento
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & droptype· Text & forms- Work product
WP08· App/system state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Category tree equals key; product-category assignments equal key; is_anchor and filterable attributes set; old URLs 301 to new; no orphan products. Integration tokens are revoked for the agent and the admin action log must show the changes from the admin panel.
Proposed partial credit
25% tree; 30% product assignments; 20% layered nav; 25% URL rewrites
R05.08.01.01-T19 Launch a seasonal collection from a PDF line sheet in Shopify
Original instruction
Shopify development store. input/linesheet_fall26.pdf lists 15 products (variants, prices, SKUs, fabric) with photos in input/photos/. Create all products with variants, alt text and SEO, a manual collection 'Fall 26' with the sort order on the line sheet, add it to the main menu, create discount FALL10 (10%, collection only, ends 2026-11-30), and schedule publishing for 2026-10-20 09:00.
- Platforms
web· Any OS (web)- Applications
- Shopify admin · Adobe Acrobat Reader
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsdrag· Drag & drop- Work product
WP08· App/system state- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Per-product fields/variants equal line sheet (>= 95% accuracy full credit); collection order; menu item; discount config; scheduled publication time. No custom-app Admin API token for the agent; product/collection timelines must show changes made in Shopify admin.
Proposed partial credit
40% products and variants; 15% media/SEO; 15% collection order; 30% discount and schedule
R05.08.01.01-T20 Reconcile store inventory with a supplier feed and set backorder rules
Original instruction
WooCommerce admin. input/supplier_feed_2026-10-14.csv gives availability for 80 SKUs. Update stock for each product/variation, enable backorders ('notify') only for items the feed marks 'restock < 14 days', hide other out-of-stock items from the catalog, and set low-stock thresholds per input/thresholds.xlsx. Write output/changes.csv.
- Platforms
web· Any OS (web)- Applications
- WooCommerce · Excel
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
long· Long-horizonmultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
DB: stock quantities, backorder settings and per-product low-stock thresholds equal key; catalog visibility setting for OOS; changes.csv matches DB diff. WooCommerce REST keys, WP-CLI and the CSV importer are disabled for the agent; changes must come from admin screens.
Proposed partial credit
40% stock levels; 25% backorder rules; 20% thresholds; 15% change log
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.