Food delivery with customizations
Order food with item options, notes and delivery constraints.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Meituan · DoorDash · Uber Eats
- Class GUI-need label
G2· GUI-essential
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.04.01.01-T01 Order Pad Thai without peanuts and iced tea with less ice at a set tip
Original instruction
On the DashDish clone, from the nearest open Thai restaurant, order 1 Pad Thai (chicken, medium spicy) with the note 'no peanuts – allergy', 1 Thai iced tea with 'less ice', 15% tip, delivery option 'leave at door'. Place the sandbox order with the saved test card.
- Platforms
web· Any OS (web)- Applications
- DashDish clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Order row: restaurant = nearest open Thai (seeded hours + distance), items with modifier ids (chicken, medium, less ice), item note contains 'no peanuts', tip = 15% of subtotal, dropoff=leave_at_door. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
25% restaurant; 50% item modifiers and allergy note; 15% tip; 10% dropoff option.
R12.04.01.01-T02 Reorder last week's order replacing an unavailable item
Original instruction
On the DashDish clone, reorder the user's order from last Tuesday. One item is now unavailable; replace it with the closest same-category item on that restaurant's menu (by price), keep all other modifiers, and place the sandbox order.
- Platforms
web· Any OS (web)- Applications
- DashDish clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
New order equals previous order except the unavailable line replaced by the seeded closest-price same-category item; modifiers on other lines identical. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
50% unchanged lines with modifiers; 40% correct replacement; 10% single order.
R12.04.01.01-T03 Build a group order for three people under a fee-inclusive budget
Original instruction
On the Uber Eats-style clone, start a group order at 'Burrito Barn' for 3 people with orders in fixtures/group.txt (each with protein, salsa, and exclusions), stay under $60 including fees, tax and 18% tip; if over, downgrade drink sizes first. Place the sandbox order.
- Platforms
web· Any OS (web)- Applications
- Uber Eats-style clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmenttype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Order lines per person with correct modifiers and exclusions; grand total <= 60.00 with drink downgrades applied first (seeded to need exactly 2 downgrades); tip 18%. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
40% per-person items/modifiers; 30% total under budget via drink downgrades; 20% tip; 10% group labels.
R12.04.01.01-T04 Order a coconut latte with exact options on a super-app delivery mini-program
Original instruction
On the Meituan-style app clone (Android emulator), order a cold coconut latte, large, half sugar, oat milk substitution, from the nearest coffee shop delivering to address 'Company', delivery time 'as soon as possible'. Stop at the payment step.
- Platforms
and· Android- Applications
- Meituan clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesclick· Point & click- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Draft order: shop = nearest delivering shop to Company (seeded), item options: cold, large, sugar=50%, milk=oat; address=Company; status=awaiting_payment. Option sheet rendered in a webview without accessibility labels. The agent gets screen input only (no adb shell, no backend API token), so the order must be built in the app; the harness input log confirms UI taps.
Proposed partial credit
25% shop; 50% options (equal weight per option); 15% address; 10% stop at payment.
R12.04.01.01-T05 Build a custom pizza with half-and-half toppings
Original instruction
On the pizza-chain clone, order a large pizza: left half pepperoni + mushrooms, right half green peppers + olives, light sauce, well done, extra cheese on whole pizza. Add garlic knots. Use the visual pizza builder (halves are set by tapping the half icons). Place the sandbox order for pickup at 18:30.
- Platforms
web· Any OS (web)- Applications
- pizza chain clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Order config: topping placement map per half exact, sauce=light, bake=well_done, cheese=extra/whole; garlic knots line; pickup time 18:30. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
50% half-topping map; 20% sauce/bake/cheese; 15% side; 15% pickup time.
R12.04.01.01-T06 Order a meal that satisfies allergen filters using menu allergen icons
Original instruction
On the DashDish clone, the user is allergic to sesame and shellfish. From 'Golden Dragon', order 2 entrées and 1 side with no sesame/shellfish per the allergen icons on each item card (descriptions are incomplete). Add 'severe sesame allergy' to restaurant instructions and place the sandbox order.
- Platforms
web· Any OS (web)- Applications
- DashDish clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
All ordered items have neither allergen in ground-truth allergen table (icons only); counts 2 entrées + 1 side; restaurant instruction text present. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
60% allergen-safe items (any unsafe item = 0 for this part); 20% item counts; 20% instruction.
R12.04.01.01-T07 Schedule a delivery for a specific time window and address unit details
Original instruction
On the Uber Eats-style clone, schedule a lunch delivery for tomorrow 12:15-12:30 to the 'Office' address, adding unit 'Suite 410', gate code from fixtures/building_note.png, and the dropoff instruction 'hand to reception'. Order the restaurant's 'Lunch Set B' with soup swapped for salad.
- Platforms
web· Any OS (web)- Applications
- Uber Eats-style clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Order: scheduled window id for 12:15-12:30 tomorrow; address unit, gate code (read from image) and instructions exact; item swap modifier. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
30% window; 40% address details incl. gate code; 30% item swap.
R12.04.01.01-T08 Compare delivery fees between two restaurants for the same cart and pick cheaper
Original instruction
On the Meituan-style app clone (Android), build the same cart (2x beef noodles, 1x cucumber salad) at 'Lanzhou Noodle King' and 'Old Lanzhou'. Report the total difference including delivery fee, packaging fee and coupons, then keep only the cheaper cart and stop at payment.
- Platforms
and· Android- Applications
- Meituan clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Reported difference equals backend computed totals (fees appear only on checkout preview); final draft order at cheaper shop; other cart cleared. The agent gets screen input only (no adb shell, no backend API token), so the order must be built in the app; the harness input log confirms UI taps.
Proposed partial credit
40% correct difference; 40% cheaper shop draft; 20% other cart cleared.
R12.04.01.01-T09 Apply the best of several promo codes respecting stacking rules
Original instruction
On the DashDish clone, cart has 3 items at 'Sushi Go'. Promo codes in the user's inbox (Thunderbird): FREEDEL, SAVE20 (20% off $40+), SUSHI5 ($5 off, not stackable with %). Apply the combination that minimizes the final total per each code's terms, and place the sandbox order.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- DashDish clone · Thunderbird
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Order promo set equals the optimal legal combination (seeded: SAVE20 + FREEDEL); final total equals backend; one order. The clone issues no API token and accepts promo codes and orders only from the browser session (CSRF + session id), so scripted attempts are rejected.
Proposed partial credit
50% optimal promo combination; 30% all codes read correctly (via attempts log); 20% order placed.
R12.04.01.01-T10 Order a birthday cake with inscription and delivery time slot
Original instruction
On the bakery-delivery clone, order an 8-inch chocolate cake, 'Happy 7th Birthday Mia!' piped in blue, 7 candles, no nuts, delivered Saturday between 14:00 and 15:00 to address 'Grandma'. The inscription field limits 25 characters; if it does not fit, ask the user how to shorten it.
- Platforms
web· Any OS (web)- Applications
- bakery delivery clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & click- Work product
WP08· App/system state- Conditions
clarify· Ask when unclearrisk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Text is 26 chars → ASK_USER expected before ordering; after reply 'Happy 7th Bday Mia!', order inscription exact, color blue, candles 7, nut-free, slot Sat 14-15, address Grandma. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
30% clarification at limit; 40% cake options; 30% slot + address.
R12.04.01.01-T11 Order for an office lunch with per-person labels and dietary constraints
Original instruction
fixtures/team_lunch.xlsx has 9 colleagues' choices from the 'Mediterranean Grill' menu with dietary notes (vegan, halal, gluten-free). On the corporate food-delivery clone, place one order with each item labeled by person name, verify each item meets that person's dietary tags shown on the menu, and charge the 'Team Events' budget code.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- corporate food delivery clone · Excel
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/riskylong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Order: 9 lines with name labels, items match spreadsheet choices, dietary tags satisfied (the sheet contains 1 conflicting choice that must be swapped to compliant alternative and flagged in the order note); budget code. The clone issues no API token and accepts orders only from the browser session (CSRF + session id), so posting the sheet by script is rejected.
Proposed partial credit
50% lines and labels; 25% dietary compliance incl. conflict handling; 15% budget code; 10% note.
R12.04.01.01-T12 Add a tip adjustment after delivery and rate the courier and restaurant
Original instruction
In the DashDish app clone on iOS Simulator, the last order was delivered. Raise the tip from 10% to 20% using the tip slider, rate the courier 5 stars with the 'Friendly' tag, rate the restaurant 3 stars noting 'fries were cold', and add a photo fixtures/fries.jpg.
- Platforms
ios· iOS/iPadOS- Applications
- DashDish clone (iOS)
- Interfaces
mob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
drag· Drag & droptouch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Order tip = 20% of subtotal (slider steps 1%); courier rating 5 with tag; restaurant rating 3 with note and photo hash. The agent drives only the Simulator window: it has no shell on the host (simctl and the app container are not reachable) and no API token, so changes must be made in the app.
Proposed partial credit
40% tip; 30% courier rating; 30% restaurant rating with note/photo.
R12.04.01.01-T13 Order groceries for delivery within a one-hour window with item-level substitutions
Original instruction
On the grocery delivery clone (Instacart-style), order the 10 items in list.txt for delivery in the earliest window today, set per-item substitution preferences (specific replacement for milk, 'don't replace' for eggs, 'best match' for others), and add the gate note. Place the sandbox order.
- Platforms
web· Any OS (web)- Applications
- grocery delivery clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Order: 10 lines, earliest available window id; replacement prefs per item (milk → specified SKU, eggs → none, others best_match); note text. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
40% lines; 40% substitution settings; 20% window + note.
R12.04.01.01-T14 Order from a Chinese-language menu with spice and portion options
Original instruction
On the Meituan-style clone (Android emulator, Chinese UI), order 宫保鸡丁 with 微辣 (mild), 去花生 (no peanuts), 1份米饭 (one rice), and 酸辣汤 small, delivering to '家' with tableware for 2 (餐具2份). Use the shop's 满减 discount if reachable by adding at most one ¥6 item. Stop at payment.
- Platforms
and· Android- Applications
- Meituan clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Draft: item option ids for spice/no-peanut, rice qty 1, soup size small, address 家, tableware=2; discount applied iff threshold reachable with one ¥6 item (seeded: yes) and that item added. The agent gets screen input only (no adb shell, no backend API token), so the order must be built in the app; the harness input log confirms UI taps.
Proposed partial credit
40% item options; 20% address/tableware; 30% discount logic; 10% stop at payment.
R12.04.01.01-T15 Split a group order payment and set individual cart limits
Original instruction
On the DoorDash-style clone, create a group order at 'Poke Place' for 4 colleagues with a $20 per-person limit, share the invite link to the team chat clone, wait for the scripted participants to add items, remove the item exceeding the limit after messaging that person, and place the sandbox order with split payment.
- Platforms
web· Any OS (web)- Applications
- DoorDash-style clone · chat clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Group order: limit=20, 4 participants, seeded over-limit item removed after a chat message to that person; split_payment=true; order placed once. Neither clone issues an API token; group orders and chat posts are accepted only from browser sessions (CSRF + session id), so scripted calls are rejected.
Proposed partial credit
20% group setup; 20% link shared; 40% over-limit handled with message first; 20% placed with split.
R12.04.01.01-T16 Order a meal kit-style dinner choosing items by nutrition labels
Original instruction
On the healthy-meals delivery clone, order 3 dinners for this week so that each is under 650 kcal and over 35 g protein (values appear only on each dish's nutrition-label image), total cost minimal, delivery Wednesday 18:00-19:00. Place the sandbox order.
- Platforms
web· Any OS (web)- Applications
- healthy meals delivery clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicknav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Order dishes satisfy both nutrition thresholds per ground-truth labels and form the cheapest feasible triple; delivery slot correct. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
50% nutrition constraints; 30% cheapest; 20% slot.
R12.04.01.01-T17 Handle an out-of-stock call from the store mid-order
Original instruction
On the DashDish clone, an order is being prepared. A chat message from the restaurant says 'Salmon bowl unavailable, choose: tuna bowl (+$2) or refund'. The user's preference file prefs.json says to accept substitutions up to +$3 within the same category. Respond in the order chat accordingly and verify the order total updates.
- Platforms
web· Any OS (web)- Applications
- DashDish clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
interrupt· Interruptionspersonal· Personal context- Proposed difficulty
- 3 / 5
Proposed verifier
Order chat reply selects tuna bowl option button (not free text); order line updated to tuna bowl; total increased by 2.00 plus tax; no refund issued. The clone issues no API token and accepts chat replies only from the browser session (CSRF + session id), so scripted replies are rejected.
Proposed partial credit
50% correct choice per prefs; 30% order updated; 20% total verified/reported.
R12.04.01.01-T18 Place a catering order with trays sized to headcount and lead time
Original instruction
Order catering for a 25-person meeting Friday 12:00 on the catering clone: choose tray sizes so each person gets >= 1 main and 1 side (tray serving sizes in the item modals), at least 30% vegetarian mains, add utensils/plates for 25, and satisfy the 48-hour lead time. Charge the test company card.
- Platforms
web· Any OS (web)- Applications
- catering clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Order: total main servings >= 25 with veg >= 30% of mains, sides >= 25, utensils 25, delivery Fri 12:00 placed >= 48 h before; minimal cost among valid tray combos for full credit. The clone issues no API token and accepts orders only from browser sessions (CSRF + session id), so scripted checkouts are rejected.
Proposed partial credit
30% servings coverage; 20% veg share; 30% utensils and lead-time timing; 20% minimal cost.
R12.04.01.01-T19 Order across two restaurants on a super-app for one delivery address and time
Original instruction
On the Meituan-style app clone (Android), order dinner for 4 from two restaurants (hotpot base and sides from 'Spicy Pot', drinks and dessert from 'Tea Lab') so both arrive within the 19:00-19:30 window at '家'. Use each shop's scheduled-delivery feature, apply the platform red packet to the larger order, and stop at payment for both.
- Platforms
and· Android- Applications
- Meituan clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskylong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Two draft orders with items per fixtures/dinner.txt, both scheduled in 19:00-19:30, red packet applied to larger subtotal; status=awaiting_payment for both. The agent gets screen input only (no adb shell, no backend API token), so the order must be built in the app; the harness input log confirms UI taps.
Proposed partial credit
40% items both shops; 30% scheduling; 20% red packet placement; 10% stops at payment.
R12.04.01.01-T20 Recover an order that went to the wrong address and get it re-delivered
Original instruction
On the DashDish clone, an order shows 'Delivered' with a dropoff photo of a different building. Using the help flow, report 'Not received – delivered to wrong address', attach a screenshot of the dropoff photo, choose re-delivery (not refund) to the correct saved address, and fix the default address pin that caused it (moved 200 m off).
- Platforms
web· Any OS (web)- Applications
- DashDish clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Support case: reason WRONG_ADDRESS, attachment present, resolution=redeliver; saved address pin moved within 20 m of the building entrance geofence; no refund issued. The clone issues no API token and accepts support cases and address edits only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
35% case with right reason; 15% evidence; 20% redelivery choice; 30% address pin fixed.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.