Multi-leg itinerary planning
Plan a multi-city trip that satisfies budget, timing and preference constraints across flights, stays and transport.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Google Flights · Booking · Maps · spreadsheets
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.02.01.05-T01 Plan a 9-day Lisbon-Madrid-Barcelona-Paris trip for two under 1,800 EUR
Original instruction
Using the frozen replicas flights.local, rail.local and stays.local, plan Lisbon > Madrid > Barcelona > Paris for 2 adults, 2026-11-12 to 11-21: at least 2 nights per city, at most one flight, total <= 1,800 EUR including lodging. Fill plan.xlsx from the template (sheet Itinerary: leg, mode, ID, depart, arrive, price; sheet Stays).
- Platforms
mac· macOSwin· Windows- Applications
- Chrome · Excel · flights.local · rail.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Checker resolves every ID in the replica inventory (times and prices must match), checks continuity (no lodging gap/overlap, >= 45 min connections), constraints and total; full credit needs total within 5% of the grader's MILP optimum.
Proposed partial credit
30% valid legs and IDs; 25% hard constraints met; 25% stays continuous; 20% cost within five percent of optimum
R12.02.01.05-T02 Find the cheapest SFO-Tokyo / Osaka-SFO open-jaw fare
Original instruction
On the Google Flights-style replica flights.local (frozen fares), find the cheapest economy open-jaw for 1 adult: SFO to Tokyo (HND or NRT) on 2027-03-20 and Osaka (KIX or ITM) to SFO on 2027-03-30, at most 1 stop each way. Write both flight numbers and the total fare to answer.txt. Do not book.
- Platforms
web· Any OS (web)- Applications
- flights.local (Google Flights-style replica)
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
answer.txt flight numbers and fare equal the ground-truth cheapest multi-city result computed from the fare snapshot; no booking or hold rows created. flights.local issues no API token and serves searches only to browser sessions (CSRF token + session id), so fares are reachable only through its UI.
Proposed partial credit
40% outbound correct; 40% return correct; 20% fare exact
R12.02.01.05-T03 Hold a three-leg NYC-Rome-Athens-NYC multi-city ticket with aisle seats
Original instruction
On flights.local, search multi-city for 1 adult: JFK/EWR to Rome on 2026-12-05, Rome to Athens 12-09, Athens to JFK/EWR 12-14; each leg <= 1 stop, every arrival before 22:00 local, total <= $1,650. Take the cheapest valid option, pick aisle seats on every segment from the seat maps, and place a 24-hour hold. Stop before payment.
- Platforms
web· Any OS (web)- Applications
- flights.local (Google Flights-style replica)
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Replica DB: one HELD record with no payment, segments equal to the cheapest valid itinerary under the constraints, aisle seat on each segment. Any purchase scores 0. flights.local issues no API token and accepts holds only from browser sessions (CSRF token + session id).
Proposed partial credit
50% cheapest valid itinerary (half credit if valid but not cheapest); 25% aisle seats; 25% held without payment
R12.02.01.05-T04 Plan a conference-plus-vacation trip and split business vs personal costs
Original instruction
Plan Boston to Berlin for a conference (2026-11-17 to 19, venue in fixtures/venue.txt), then 4 personal days in Prague and Vienna, flying home from Vienna, using flights.local, rail.local and stays.local. Follow fixtures/travel_policy.pdf (hotel cap 180 EUR/night within 2 km of venue) and fill expense_split.xlsx using its mixed-trip allocation rule.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · flights.local · rail.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-apptutorial· Follow docs/tutorial- Proposed difficulty
- 3 / 5
Proposed verifier
Checker validates IDs/prices, venue distance via maps.local, policy caps, and allocation: business share = policy comparison round-trip BOS-BER fare; sums exact to the cent.
Proposed partial credit
30% valid feasible itinerary; 20% policy-compliant hotel; 35% correct business/personal allocation; 15% workbook format
R12.02.01.05-T05 Plan a 5-day pet-friendly self-drive loop with daily driving limits
Original instruction
Plan a 5-day trip for 2 adults and a dog from Denver via Santa Fe and Moab back to Denver: self-drive only, <= 6 h driving per day (maps.local times), pet-friendly lodging from stays.local, a different cuisine at each dinner from eats.local, total <= $2,000. Output plan.json in the provided schema.
- Platforms
web· Any OS (web)- Applications
- maps.local · stays.local · eats.local
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
TravelPlanner-style checkers: delivery, commonsense (valid cities, no repeated restaurants, consistent days) and hard constraints (budget, pet rule, cuisine diversity, drive time per day). Every entity must exist in the replica DBs. Control pair: T15.
Proposed partial credit
20% schema/delivery; 40% commonsense checks; 40% hard constraints (equal weight)
R12.02.01.05-T06 Build a day-layered trip map with driving routes in Google My Maps
Original instruction
In Google My Maps (test account), create map 'Scotland 6 days' from fixtures/stops.csv: one layer per day named Day 1 to Day 6, each stop as a pin in its day's layer, hotels styled with the bed icon, and one driving-route line per day through that day's stops in order. Export it as scotland.kml.
- Platforms
web· Any OS (web)- Applications
- Google My Maps
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropnav· Pan/zoom/scroll- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
KML parse: 6 folders named exactly; pins within 100 m of CSV coordinates in the right folder; hotel icon style; one directions line per day with waypoints in stop order. CSV import allowed for pins only; routes must exist. My Maps has no API; KML import yields plain lines, not directions layers, so routes must come from the Add directions tool in the browser session.
Proposed partial credit
20% layers; 35% pins in correct layers; 10% hotel icons; 35% per-day routes in order
R12.02.01.05-T07 Decide JR Pass versus point-to-point tickets for a 10-day Japan route
Original instruction
For 1 adult travelling Tokyo > Kanazawa > Kyoto > Hiroshima > Osaka > Tokyo on the dates in fixtures/japan_dates.txt, pick trains on rail.local (frozen timetables and fares) departing before 20:00, then compare a 7-day JR Pass (price in fixtures, Nozomi excluded) plus uncovered legs against point-to-point fares. Record legs and the cheaper option in japan.xlsx.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · rail.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsmulti· Cross-app- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Legs must match replica train IDs and times; pass coverage rules applied per window; totals exact; the verdict equals ground truth computed by the grader.
Proposed partial credit
35% valid legs; 30% pass coverage applied correctly; 20% totals exact; 15% correct verdict
R12.02.01.05-T08 Plan a step-free family rail trip across three cities
Original instruction
Plan London > Amsterdam > Brussels for 2 adults, 2 children and one wheelchair user, 2026-12-20 to 12-28: rail only, step-free stations, <= 2 transfers per travel day, rooms with a roll-in shower on stays.local, each hotel within 800 m step-free walk of a station (maps.local accessible routing). Save plan.xlsx.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · rail.local · stays.local · maps.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & click- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker verifies train IDs, station step-free flags, transfers <= 2, room accessibility attributes, accessible walking distance, and nightly continuity for the whole party.
Proposed partial credit
30% valid rail legs; 25% accessibility flags; 25% hotel distances; 20% continuity and format
R12.02.01.05-T09 Repair a multi-leg itinerary after a cancelled leg
Original instruction
itinerary.xlsx and the replicas show leg 2 (Madrid to Marrakesh, 2026-11-14) cancelled. Re-plan legs 2-4 so the traveler still uses the 3 non-refundable hotel nights in fixtures/bookings.json and reaches Lisbon for the 11-20 flight home at minimum added cost. Hold new segments, cancel only refundable items that must go, and update the workbook.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · flights.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/riskymultiapp· Multi-app- Proposed difficulty
- 5 / 5
Proposed verifier
Checker: holds exist and are feasible; non-refundable nights still used; arrival before the flight home; added cost within 3% of MILP minimum. Cancelling a non-refundable item caps score at 30%.
Proposed partial credit
25% feasible new legs; 25% non-refundable nights kept; 30% near-optimal added cost; 20% workbook updated
R12.02.01.05-T10 Hold gap-free lodging for three consecutive Italian cities
Original instruction
On stays.local (Booking-style replica), hold stays for 2 adults in Florence 2027-04-02 to 05, Bologna 04-05 to 07 and Venice 04-07 to 10: free cancellation only, <= 140 EUR per night, review score >= 8.5, within 1 km of the main station. Pick the cheapest valid property per city and stop at the payment page.
- Platforms
web· Any OS (web)- Applications
- stays.local (Booking-style replica)
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Replica DB: three held reservations with contiguous dates, each the cheapest property meeting all filters at the free-cancellation rate; no payment records. stays.local issues no API token and accepts holds only from browser sessions (CSRF token + session id).
Proposed partial credit
30% correct dates; 45% correct property per city, equal shares; 15% free-cancellation rate; 10% no payment
R12.02.01.05-T11 Assemble a chronological itinerary from seven confirmations
Original instruction
fixtures/confirmations/ holds 7 PDFs and emails (2 flights, 1 train, 3 hotels, 1 car rental). Build itinerary.xlsx with one row per segment in chronological order with local times, UTC times and confirmation codes, plus a 'Problems' sheet naming the one night without lodging and the one connection shorter than its minimum connection time.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · Preview
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsmulti· Cross-app- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Rows vs ground truth (times exact including time-zone conversion); Problems sheet must name the gap-night date and the short-connection airport; extra false problems penalized.
Proposed partial credit
50% segment rows (time/UTC/code); 20% order; 15% gap night; 15% short connection
R12.02.01.05-T12 Converge three travelers from three continents on Lisbon
Original instruction
Travelers start in Chicago, Sao Paulo and Nairobi and must all land at LIS on 2026-12-28 within one 3-hour window, share 4 nights in one stays.local apartment, and fly home on 2027-01-01. Minimize total cost on flights.local with <= 1 stop per direction. Record the plan and per-person costs in group_trip.xlsx.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · flights.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Checker: arrivals within a 3 h window, valid IDs, apartment sleeps >= 3, all constraints met; total within 3% of grader MILP optimum; per-person sums exact.
Proposed partial credit
30% valid flights; 20% arrival window; 20% apartment; 30% near-optimal total
R12.02.01.05-T13 Plan a Paris-Vienna-Budapest-Munich loop under a CO2 cap
Original instruction
Plan a 7-day loop Paris > Vienna > Budapest > Munich > Paris for 1 adult under 900 EUR and <= 150 kg CO2 in total, using rail.local (night trains included) and flights.local emission figures. A night train counts as that night's lodging. Save plan.xlsx with each leg's CO2 and a totals row.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · rail.local · flights.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Checker: legs exist with matching emissions and prices; CO2 <= 150 kg; cost <= 900 EUR; all nights covered; full credit needs cost within 5% of the cheapest feasible plan.
Proposed partial credit
30% valid legs; 30% CO2 cap met; 20% budget met; 20% near-optimal cost
R12.02.01.05-T14 Route a multi-leg trip around transit-visa restrictions
Original instruction
Plan Delhi > Toronto > Mexico City > Delhi on flights.local for the traveler in fixtures/traveler.json. fixtures/visa_rules.csv lists countries where this passport needs a transit visa; no connection may be in those countries (Canada is already covered). Minimize total fare with <= 2 stops per leg and hold the tickets without paying.
- Platforms
web· Any OS (web)- Applications
- flights.local (Google Flights-style replica)
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Checker: no connection airport in a restricted country; holds valid and unpaid; fare within 3% of the constrained optimum.
Proposed partial credit
40% no restricted transit; 30% valid holds without payment; 30% near-optimal fare
R12.02.01.05-T15 Control pair of T05: solve the road trip with the tool API only
Original instruction
Control pair of T05. Same constraints, but use only the sandbox's documented REST tools (search_stays, search_restaurants, distance_matrix) from a terminal, with no browser. Produce plan.json in the same schema.
- Platforms
lin· Linux- Applications
- TravelPlanner-style tool API · curl
- Interfaces
api· API/SDKterm· Terminal- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Same checkers as T05; the score is compared with T05 to isolate the cost of the GUI.
Proposed partial credit
20% schema/delivery; 40% commonsense checks; 40% hard constraints
R12.02.01.05-T16 Schedule two city days around opening hours and closures
Original instruction
For Rome on 2026-12-07 (a Monday) and Naples on 12-08, fit the 6 attractions per city in fixtures/wishlist.csv into hour-by-hour plans in plan.xlsx: respect opening hours and Monday closures from places.local, use maps.local transit times between stops, include a 60-minute lunch between 12:00 and 14:30, and if infeasible drop the fewest attractions and list them.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · places.local · maps.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & forms- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker simulates each day: visits within opening hours, travel gaps >= maps.local times, lunch window respected, number of dropped attractions equals the grader's optimum.
Proposed partial credit
40% feasible visits; 20% travel times; 15% lunch; 25% minimal drops
R12.02.01.05-T17 Build a round-the-world itinerary under alliance fare rules
Original instruction
Build a round-the-world economy itinerary from London covering 5 continents (cities in fixtures/rtw_wishlist.txt) on flights.local alliance carriers only: continuous eastbound travel, <= 16 segments, <= 39,000 total miles, one Atlantic and one Pacific crossing, ground time per fixtures. Output rtw.xlsx with segments, miles and dates.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · flights.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formscode· Code & commands- Work product
WP02· Records/table- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Checker recomputes great-circle miles, direction, ocean crossings, alliance membership, segment count and dates vs replica schedules; every rule must pass for full credit.
Proposed partial credit
30% valid segments; 25% direction and crossings; 25% mileage and segment caps; 20% dates and ground time
R12.02.01.05-T18 Convert an itinerary budget into per-day EUR and JPY totals
Original instruction
itinerary.xlsx lists 22 costs in USD, EUR and JPY across 8 days. Add a 'Budget' sheet with per-day totals in EUR and JPY using the fixed rates in fixtures/rates.json, a red-fill conditional format on days over the 220 EUR daily cap, and a trip total row.
- Platforms
mac· macOSwin· Windows- Applications
- Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsclick· Point & click- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
openpyxl: per-day totals equal expected to 0.01 and are formulas, not constants; conditional-format rule with red fill on the EUR total column; flagged days equal expected.
Proposed partial credit
50% per-day totals; 20% formulas used; 20% conditional formatting; 10% trip total
R12.02.01.05-T19 Put a seven-segment itinerary into Calendar with correct time zones
Original instruction
From itinerary.xlsx (5 flights, 2 trains), create events in the macOS Calendar 'Trips' calendar with each start and end in its local time zone (turn on time-zone support), titled '<carrier code+number> <from>-<to>' with the departure airport or station as location.
- Platforms
mac· macOS- Applications
- Calendar · Excel
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsmulti· Cross-app- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 2 / 5
Proposed verifier
Grader exports the Trips calendar to ICS: 7 VEVENTs with correct TZIDs and exact UTC instants, exact titles and locations.
Proposed partial credit
50% UTC instants exact; 20% TZIDs; 20% titles; 10% locations
R12.02.01.05-T20 Plan a 14-day Thailand-Laos-Vietnam loop by air and bus and hold it
Original instruction
Plan Bangkok > Chiang Mai > Luang Prabang > Hanoi > Ho Chi Minh City > Bangkok for 2 adults, 14 days from 2027-01-10, under $2,200, using flights.local, bus.local and stays.local: no departures before 07:00, a rest day after any leg over 8 h, >= 2 nights per city, rooms with AC rated >= 8. Fill trip.xlsx and hold every bookable item without paying.
- Platforms
mac· macOSwin· Windows- Applications
- Excel · flights.local · bus.local · stays.local
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP02· Records/table- Conditions
long· Long-horizonmultiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 5 / 5
Proposed verifier
Checker: all IDs/prices valid; constraints and continuity pass; holds exist and match the workbook; total within 5% of MILP optimum; no payments.
Proposed partial credit
25% valid legs; 25% constraints; 20% workbook and holds agree; 30% near-optimal cost
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.