Flight booking & changes
Book or change flights with fare rules and seat maps (sandbox).
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- airline sites · Google Flights · Trip.com
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.02.01.01-T01 Book a nonstop round trip with aisle seats in a row range on both legs
Original instruction
On the FlyUnified clone, book a round trip SFO-ORD departing 2026-11-20 and returning 2026-11-23, nonstop only, cheapest Economy fare (not Basic Economy), aisle seats in rows 10-20 on both legs, 1 checked bag, passenger from fixtures/traveler.json. Pay with the sandbox card.
- Platforms
web· Any OS (web)- Applications
- FlyUnified clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Booking record: 2 segments, nonstop flights matching cheapest Economy (fare family check), seats in rows 10-20 with seat letter in aisle set per aircraft map (C/D), bags=1, passenger fields exact. Seat map is a canvas without DOM seat labels. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% correct flights; 20% fare family; 30% seats both legs; 20% bag + passenger.
R12.02.01.01-T02 Hold the earliest-arriving itinerary under a price cap without purchasing
Original instruction
On the airline clone, find the itinerary BOS to DEN on 2026-12-05 that arrives earliest with at most 1 stop and total under $450, and use the 72-hour 'Hold fare' option (free). Do not purchase. Report flight numbers and the hold expiry time.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Hold record with correct itinerary (earliest arrival among feasible set); no ticketed booking; reported flight numbers and expiry equal DB. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% correct itinerary; 30% hold created, no purchase; 20% report accurate.
R12.02.01.01-T03 Change a booked flight to the next day in the same fare class and pay the difference
Original instruction
On the FlyUnified clone, change PNR ABC123's outbound (SFO-IAD 2026-11-12 08:00) to the same time slot on 2026-11-13 in the same fare class, keep the original seat letter type (window), pay the fare difference with the sandbox card, and report the total change cost.
- Platforms
web· Any OS (web)- Applications
- FlyUnified clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
PNR ABC123: outbound replaced by the 11-13 flight at the same departure time, fare class unchanged, seat is a window seat; return segment unchanged; reported cost equals fare diff + change fee in DB. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct new flight; 20% fare class; 20% window seat; 20% cost reported, return intact.
R12.02.01.01-T04 Book a family of four with adjacent seats and an infant in lap
Original instruction
On the airline clone, book JFK-LAX on 2026-12-20 for 2 adults, 1 child (age 7) and 1 lap infant from fixtures/family.json, on the cheapest nonstop with Main Cabin fare. Seat the child next to an adult, and both adults together in the same row; the infant's adult must be in a row with an extra oxygen mask (shown by an icon). Pay with the sandbox card.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clickgeom· Precise geometry- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Booking: 4 passengers (infant linked to adult 1); seats: 3 adjacent in one row in a block containing the infant-allowed icon (seat attribute only rendered as icon), child adjacent to an adult; fare family Main.
Proposed partial credit
30% passengers & infant linkage; 40% seat constraints; 20% fare/flight; 10% paid once.
R12.02.01.01-T05 Find the cheapest flexible fare that permits free changes and book it
Original instruction
On the OTA clone, book one adult LHR-CDG on 2026-11-06 departing 07:00-10:00 with a fare that allows free date changes and a carry-on. Compare fare-family grids across airlines (rules are in hover tooltips and 'Fare rules' modals). Pay with the sandbox card.
- Platforms
web· Any OS (web)- Applications
- OTA clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextclick· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Booking fare basis equals the cheapest option among departures in window whose rules have change_fee=0 and carry_on=true (computed from seeded rules). Rules not exposed in search results JSON.
Proposed partial credit
40% fare rule satisfied; 40% cheapest among valid; 20% time window and booking.
R12.02.01.01-T06 Book a multi-city itinerary with minimum connection times respected
Original instruction
On the OTA clone, book a multi-city trip for one adult: NYC to Lisbon on 2026-11-10, Lisbon to Rome on 11-14, Rome back to NYC on 11-19. Each leg at most one stop, no connection shorter than 75 minutes or longer than 4 hours, total under $1,600, cheapest qualifying. Pay with the sandbox card.
- Platforms
web· Any OS (web)- Applications
- OTA clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskylong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Booking: 3 legs with dates; each connection duration in [75, 240] min; total <= 1600 and equals min over feasible itineraries (enumerated from seeded schedule). Airport choice JFK/EWR/LGA allowed. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% three legs correct dates; 30% connection constraints; 30% cheapest; 10% booked once.
R12.02.01.01-T07 Select the exact seat from a seat-map screenshot sent by a colleague
Original instruction
A colleague sent fixtures/seat_pick.png (a marked screenshot of the A321 seat map) showing the seat they want next to theirs on flight UA 512 (PNR KX9P2Q). On the airline clone, assign that seat to the passenger 'Dana Wu' in the PNR. Seat maps on the site may render in a different zoom level and cabin layout orientation.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- airline clone · Preview
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clickgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
PNR seat for Dana Wu equals the marked seat id (e.g. 23E) derived from row markers in the screenshot; the site's map is rotated 90° vs the screenshot. Other passenger seats unchanged. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
70% exact seat; 20% correct passenger; 10% others unchanged.
R12.02.01.01-T08 Cancel a refundable segment and request a refund to the original card
Original instruction
On the airline clone, PNR M7TQ2Z has a refundable return segment ORD-SFO on 2026-11-30. Cancel only the return segment (keep the outbound), choose refund to the original form of payment rather than travel credit, and report the refund amount shown on the confirmation. Stop at the final 'Confirm cancellation' click and hand off.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Cancellation draft for return segment only; refund_method=original_fop (default is travel credit); status=pending_confirmation; ASK_USER before confirm. Outbound intact. Reported amount equals draft amount. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct segment; 30% refund method; 20% stopped with handoff; 10% amount.
R12.02.01.01-T09 Book a train ticket with seat class and seat-position preferences on a mobile app
Original instruction
On the 12306-style rail app clone (Android emulator), book the cheapest second-class seat from Beijing South to Shanghai Hongqiao on 2026-11-02 arriving before 12:00, for passenger from the saved profile, choosing a window seat (letter A or F) in the seat-selection diagram. Stop at the payment-password screen.
- Platforms
and· Android- Applications
- rail app clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgmentclick· Point & click- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Draft order: train id equals cheapest second-class arriving < 12:00 (seeded timetable), seat letter in {A, F}, passenger profile id, status=awaiting_payment; no payment input events. The agent gets screen input only (no adb shell, no backend API token), so the order must be drafted in the app; the harness input log confirms UI taps.
Proposed partial credit
50% correct train; 30% window seat; 20% stopped at payment.
R12.02.01.01-T10 Add extras to an existing booking: bag, priority boarding and meal
Original instruction
On the low-cost airline clone, for booking QZ3K9L (2 passengers), add 1 x 20 kg checked bag for passenger 2 only, priority boarding for both on the outbound only, and a vegetarian meal for passenger 1 on both legs. Decline insurance and car-hire upsells. Pay with the sandbox card.
- Platforms
web· Any OS (web)- Applications
- low-cost airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Booking ancillaries match exactly (per passenger and segment); no insurance/car-hire lines (pre-ticked in UI); single payment. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% bag; 25% priority; 25% meals; 20% upsells declined.
R12.02.01.01-T11 Rebook after a schedule change notification within the airline's waiver rules
Original instruction
The airline clone has emailed (in the Thunderbird test inbox) a schedule change for PNR TW8R4N that breaks the user's connection in DEN. Per the waiver in the email, rebook free to any flight on the same day or the next with the same origin/destination. Choose the option arriving SEA before 18:00 with the shortest total travel time; keep seats in the same zone.
- Platforms
mac· macOSwin· Windows- Applications
- airline clone · Thunderbird
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appvis· Visual judgmentclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
PNR rebooked with fee=0 (waiver code applied from email), itinerary equals shortest-duration option arriving < 18:00, seats in same cabin zone as originally; original segments cancelled. The airline clone issues no API token and accepts PNR changes only from the browser session (CSRF + session id), so scripted rebooking is rejected.
Proposed partial credit
25% waiver code used; 45% correct itinerary; 20% seat zone; 10% old segments cancelled.
R12.02.01.01-T12 Use frequent-flyer miles plus cash to book an award ticket
Original instruction
On the airline clone logged in to a test loyalty account, book a one-way award ticket SFO-HNL on 2026-12-14 using the 'miles + cash' slider so that miles used do not exceed 18,000 and cash is minimized. Pick an aisle seat in Economy Plus if available. Taxes go on the sandbox card.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropclick· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Award booking: miles_used <= 18000 and cash equals the minimum achievable at that bound (slider snaps to 1,000-mile steps); seat aisle in Economy Plus if available per seeded map; loyalty balance decremented accordingly. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
50% miles/cash split optimal; 20% correct flight; 20% seat; 10% single booking.
R12.02.01.01-T13 Book the cheapest itinerary that meets a corporate travel policy PDF
Original instruction
Book Seattle to Austin for 2026-11-17 returning 11-19 for one employee. Corporate policy fixtures/travel_policy.pdf requires: lowest logical fare within a 2-hour window around preferred times (09:00 out, 17:00 back), no Basic Economy, preferred carriers when within $50. Book on the corporate OTA clone and enter the justification code required by the booking tool.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- corporate OTA clone · PDF viewer
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appvis· Visual judgmenttype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Booking itinerary equals the policy-compliant option computed by a rules engine over seeded fares; justification code matches the policy table entry for 'preferred carrier'. The OTA clone issues no API token and accepts bookings only from browser sessions (CSRF + session id), so scripted bookings are rejected.
Proposed partial credit
60% compliant itinerary (half per direction); 20% fare family; 20% justification code.
R12.02.01.01-T14 Choose seats for a group of six split across two rows in an exit-row-restricted cabin
Original instruction
On the airline clone, PNR GRP6XT has 6 passengers including a 14-year-old (cannot sit in an exit row) and one passenger needing an aisle. Assign seats so the group occupies two consecutive rows, 3 + 3 adjacent, exit rows excluded for the minor, aisle for the marked passenger, and no seat-fee over $25 per seat.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clickgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Predicate check over seat graph: 2 consecutive rows, adjacency 3+3, minor not in exit row, aisle constraint, per-seat fee <= 25 (fees shown only on hover). Seats saved in PNR. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% group adjacency; 20% minor constraint; 20% aisle; 20% fee cap.
R12.02.01.01-T15 Book a flight using a travel voucher and pay the remainder
Original instruction
The user has a travel credit (voucher code in fixtures/credit_email.eml, partly expired: one of two credits ends 2026-10-31). On the airline clone, book MIA-ATL 2026-11-08 cheapest nonstop Main fare and apply only the unexpired credit; pay the remainder with the sandbox card.
- Platforms
mac· macOSwin· Windows- Applications
- airline clone · Mail
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Booking payment: unexpired credit applied in full, expired credit not attempted (log), remainder on card equals fare - credit; flight correct. The airline clone issues no API token and accepts payments and credits only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct credit applied; 30% flight; 30% payment split.
R12.02.01.01-T16 Detect an infeasible booking request and ask instead of guessing
Original instruction
User: 'Book me the 6 am nonstop Denver to Boise next Monday, window seat.' On the airline clone, there is no 6 am nonstop that day (only a 6:40 one-stop and a 9:15 nonstop). Do not book; ask the user which alternative they prefer with times and prices, then book the chosen option (reply: 'the nonstop') with a window seat.
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
clarify· Ask when unclearrisk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
No booking before ASK_USER; message lists both alternatives with correct times and prices; final booking 9:15 nonstop with window seat letter (A/F). The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% correct clarification with details; 40% booking correct; 20% window seat.
R12.02.01.01-T17 Book an open-jaw trip comparing nearby airports by total door-to-door cost
Original instruction
Plan and book a trip from Chicago (ORD or MDW) to Florence, returning from Rome, 2026-11-12 to 11-22, one adult. Include in total cost each airport's ground transfer cost from fixtures/transfers.csv. Pick the minimal door-to-door cost with <= 1 stop each way and book on the OTA clone with the sandbox card.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- OTA clone · Numbers
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appvis· Visual judgmentclick· Point & click- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 5 / 5
Proposed verifier
Booked itinerary equals argmin(fare + transfers) over feasible set (FLR or PSA inbound, FCO/CIA outbound) from seeded inventory; open-jaw structure correct. The OTA clone issues no API token and accepts bookings only from browser sessions (CSRF + session id), so scripted bookings are rejected.
Proposed partial credit
30% open-jaw structure; 50% optimal choice; 20% booking completed.
R12.02.01.01-T18 Upgrade with a bid on the airline's upgrade auction slider
Original instruction
For PNR UPG771 on the airline clone, place an upgrade bid to Premium on the outbound only: the bid slider ranges $80-$400; set the bid to the minimum value that shows 'Strong' likelihood on the meter, and confirm with the sandbox card authorization (no charge until accepted).
- Platforms
web· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Bid record: segment=outbound, amount equals the lowest slider step whose meter state is 'Strong' (meter rendered as canvas gradient), card auth present, return segment no bid. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
60% correct bid amount; 20% correct segment; 20% auth confirmed.
R12.02.01.01-T19 Complete online check-in with passport data and a seat change
Original instruction
On the airline clone, check in passenger 'Leah Park' for international flight LH 431 on PNR QWJ2L8: enter passport data from fixtures/passport.jpg (MRZ visible), confirm no dangerous goods, move from middle seat 32E to any open window seat in rows 28-35, and save the boarding pass PDF to ~/Downloads/bp.pdf.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- airline clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmenttype· Text & formsclick· Point & click- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Check-in record: passport number, expiry, nationality, DOB exact (from MRZ); seat in rows 28-35 window; bp.pdf exists with PDF417 barcode decoding to PNR+seat. The airline sandbox issues no API token and accepts booking writes only from browser sessions (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
40% passport fields; 30% seat; 30% boarding pass saved.
R12.02.01.01-T20 Re-plan a disrupted multi-passenger trip after a cancellation and rebook all
Original instruction
PNR FAM55D (3 passengers, SEA-JFK-BOS) had its JFK-BOS leg cancelled. On the airline clone's self-service rebooking, find the option that gets all 3 to BOS together soonest (train vouchers count: an Amtrak option appears via partner link), rebook, keep the three seated together, and request the hotel voucher if arrival is next day.
- Platforms
web· Any OS (web)- Applications
- airline clone · rail partner clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentmulti· Cross-app- Work product
WP08· App/system state- Conditions
long· Long-horizoninterrupt· Interruptionsmultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
Final PNR: all 3 passengers on the earliest-arrival option (including partner rail if earliest), seats adjacent, hotel voucher requested iff arrival next day; original cancelled segment removed. Neither clone issues an API token; rebooking and voucher requests are accepted only from browser sessions (CSRF + session id), so scripted calls are rejected.
Proposed partial credit
40% earliest option chosen; 20% all 3 together; 20% seats; 20% voucher logic.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.