Maps, routes & ride-hailing
Search places, nearby points of interest and coordinates, plan routes, save places and book rides.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Google Maps · Apple Maps · Gaode · ride apps
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.02.01.03-T01 Save a favorite place at an exact address in OsmAnd
Original instruction
In OsmAnd on the Android emulator (offline San Francisco map preloaded), save a favorite named 'Office' at 1 Market St, San Francisco, in a new favorites group 'Work' with a blue icon.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
touch· Touch gesturestype· Text & formsnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
favourites.gpx: waypoint 'Office' within 50 m of the geocoded address, group 'Work', color blue. The agent has no adb shell, push or root, so OsmAnd's data dir is not reachable for direct GPX writes; the file must be written by the app.
Proposed partial credit
60% location within tolerance; 20% name; 20% group + color.
R12.02.01.03-T02 Plan a walking route through a required waypoint and report its length
Original instruction
In OsmAnd (Android emulator), plan a walking route from the Ferry Building to Coit Tower via Washington Square, and report the route distance in km and estimated time as OsmAnd shows them. Save the route as GPX named 'coit_walk'.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrolltype· Text & forms- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Saved coit_walk.gpx passes within 30 m of Washington Square polygon; reported distance within 5% of OsmAnd router output for the same waypoints; time within 5%. The agent has screen input only (no adb shell or push), so a GPX cannot be placed in OsmAnd's tracks folder directly; the harness input log confirms UI interaction.
Proposed partial credit
40% via-point respected in GPX; 40% distance/time; 20% GPX saved with name.
R12.02.01.03-T03 Locate a city rather than its airport in Apple Maps and pin it to a guide
Original instruction
In Apple Maps on macOS, search 'Glasgow', make sure the result is the city of Glasgow, Scotland (not the airport or Glasgow, Kentucky), and add it to a new Guide named 'UK trip'. Then add 'Edinburgh Waverley' station to the same guide.
- Platforms
mac· macOS- Applications
- Apple Maps
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Maps guide 'UK trip' (read via Maps container DB) contains 2 items: Glasgow city place id and Edinburgh Waverley station id. Airport/Kentucky entries = 0 credit for that item. The agent has no Terminal, osascript or Shortcuts access and Maps exposes no guide API, so the guide must be built in the Maps UI; only the grader reads the container DB.
Proposed partial credit
50% correct Glasgow item; 30% Waverley; 20% guide named correctly.
R12.02.01.03-T04 Book an XL ride from a pickup pin at a specific station exit
Original instruction
On the UDriver clone, schedule an XL ride tomorrow 07:30 from the east exit of Central Station (drag the pickup pin to the marked east exit on the map; the address search snaps to the main entrance on the west side) to Pier 39. Compare XL and Comfort XL estimates and book the cheaper one.
- Platforms
web· Any OS (web)- Applications
- UDriver clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Ride record: pickup coordinates within 25 m of east-exit geofence, dropoff Pier 39, scheduled 07:30 next day, class = cheaper of XL/Comfort XL per seeded pricing. The clone issues no API token and accepts rides only from the browser session (CSRF + session id), so scripted bookings with crafted coordinates are rejected.
Proposed partial credit
40% pickup pin geofence; 20% dropoff; 20% schedule; 20% cheaper class.
R12.02.01.03-T05 Find the fewest-transfer transit route arriving before nine
Original instruction
In the Google Maps-style web clone with transit data, find a public-transit route from 'Hillcrest Park' to 'City Hospital' on Monday arriving before 09:00 with the fewest transfers (tie-break: latest departure). Save the route to the user's 'Commute' list and report departure time and line numbers.
- Platforms
web· Any OS (web)- Applications
- maps web clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Saved route equals the router's optimum on the seeded GTFS (transfers, then latest departure); reported departure time and line numbers exact. The clone issues no API token; saving to lists is accepted only from the browser session (CSRF + session id), so scripted saves are rejected.
Proposed partial credit
50% optimal route; 30% report correct; 20% saved to list.
R12.02.01.03-T06 Order a visit sequence of three museums minimizing walking time
Original instruction
On the OpenStreetMap-based map clone, find the order to visit museums A, B and C (names in plan.txt), starting and ending at the hotel, that minimizes total walking time. Report the order and total minutes, and save the route as a shared link in plan.txt.
- Platforms
web· Any OS (web)- Applications
- OSM map clone
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Reported order equals optimal tour from OSRM on the same snapshot; total minutes within ±3; shared link decodes to the 5-point route in that order.
Proposed partial credit
50% optimal order; 30% total time; 20% link correct.
R12.02.01.03-T07 Drop a pin by long-press at a landmark and share coordinates to a note
Original instruction
On the Android emulator, in Google Maps-style map clone app, long-press to drop a pin at the main entrance of the city's central station (visible as the entrance icon at zoom 18), then share the pin's coordinates to the Notes app as a new note titled 'Meet here'.
- Platforms
and· Android- Applications
- map app clone · Notes
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrollmulti· Cross-app- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Notes DB: note 'Meet here' contains coordinates within 20 m of the entrance point (ground truth). Typing coordinates manually is allowed but must match tolerance. The agent has no adb shell, so the Notes content provider and am are not reachable for inserting the note; the share-intent log must show the map app as the source.
Proposed partial credit
60% coordinates within tolerance; 20% note title; 20% shared from map app (intent log).
R12.02.01.03-T08 Download offline map for the current GPS region
Original instruction
On the Android emulator with GPS mocked to a location in Slovenia, open OsmAnd and download the offline map for the region that contains the current location (the region, not the whole country list item if a smaller region exists). Do not download any other region.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
OsmAnd data dir contains exactly the correct region .obf file for the mocked coordinates; no other new .obf files. The agent has no adb shell or push, so the data dir is not reachable for copying .obf files; the file must come from OsmAnd's downloader (its download log).
Proposed partial credit
70% correct region file; 30% no extra downloads.
R12.02.01.03-T09 Measure straight-line distance between two pins with the map's measure tool
Original instruction
In the web map clone, use the Measure distance tool to measure the straight-line distance from the lighthouse at the north tip of the island to the ferry terminal on the south shore (both labeled at zoom 15). Report the value shown in km to one decimal and save a screenshot of the measurement as measure.png.
- Platforms
web· Any OS (web)- Applications
- map web clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollgeom· Precise geometry- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Reported km within ±0.1 of geodesic distance between labeled features; measure.png OCR contains the value; measure tool state in app log has 2 points within 30 m of the features. The clone issues no API token, and measure-tool state is recorded only from UI events in the browser session, so a computed distance alone fails the point checks.
Proposed partial credit
50% points placed within tolerance; 40% reported distance; 10% screenshot.
R12.02.01.03-T10 Book a car rental with airport pickup and downtown return
Original instruction
On the car-rental clone, reserve a compact car picked up at SEA airport 2026-11-14 10:00 and returned to the downtown Seattle branch 11-17 09:00, add one child seat, decline the collision waiver upsell, and choose 'pay at counter'. Report the estimated total.
- Platforms
web· Any OS (web)- Applications
- car rental clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Reservation: branches, times, class=compact, child_seat=1, cdw=false (pre-selected in UI), payment=counter; answer equals total in DB incl. one-way fee. The clone issues no API token and accepts reservations only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
30% branches/times; 20% class; 30% extras correct; 20% total reported.
R12.02.01.03-T11 Plan a two-day itinerary respecting opening hours and save it to the trip planner
Original instruction
In the map clone's trip planner, create a 2-day itinerary 'Vienna' for the 6 attractions in sights.txt so that every visit falls within that attraction's opening hours (from place cards), visit durations from sights.txt, walking/transit between them via the router, days 09:00-19:00, and the Monday closures respected.
- Platforms
web· Any OS (web)- Applications
- map web clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP08· App/system state- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Exported itinerary JSON satisfies: all 6 sights, opening hours at arrival through departure, travel times >= router times, day window, no Monday visits to Monday-closed sights. Predicate checker, per-violation deductions. The clone issues no API token; itineraries are saved and exported only from the browser session (CSRF + session id), so a crafted JSON cannot be posted.
Proposed partial credit
30% all 6 scheduled; 40% opening-hour compliance; 20% travel feasibility; 10% saved.
R12.02.01.03-T12 Find the nearest pharmacy open after ten pm to an address and save it
Original instruction
On the Android emulator in the map app clone, find the pharmacy nearest by walking distance to '2150 Shattuck Ave' that is open after 22:00 tonight (Thursday), save it to the 'Health' list, and report its name and closing time.
- Platforms
and· Android- Applications
- map app clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Saved place equals ground-truth POI id (open after 22:00 Thursday per seeded hours, minimum walking distance); answer name + closing time exact. The agent gets screen input only (no adb shell, no backend API token), so the result must be produced in the app; the harness input log confirms UI taps.
Proposed partial credit
60% correct POI; 20% saved to list; 20% answer.
R12.02.01.03-T13 Record a GPS track from a mocked route and name it in OpenTracks
Original instruction
On the Android emulator, start recording a track in OpenTracks, then load fixtures/morning_loop.gpx into the emulator's location route player and play it at 4x. Stop the recording at the end, name the track 'Morning Loop', activity 'Running', and add the description 'tempo'.
- Platforms
and· Android- Applications
- OpenTracks · Android emulator extended controls
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesmulti· Cross-apptype· Text & forms- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
OpenTracks DB: one track named Morning Loop, activity running, description 'tempo', distance 2.4 km ±5%, point count >= 80% of GPX points. morning_loop.gpx sits only in the emulator's extended controls, not device storage, and with no adb shell it is not reachable for OpenTracks import; the track must be recorded live.
Proposed partial credit
50% track recorded with correct distance; 30% name/activity; 20% description.
R12.02.01.03-T14 Choose a ride type meeting luggage and child-seat needs and book it
Original instruction
On the UDriver clone, book a ride now from the hotel to SFO for 3 adults with 4 large suitcases and one child needing a car seat. Ride types differ in seats, luggage icons and car-seat availability (shown in the info sheet). Book the cheapest type meeting all needs; add a note 'car seat for 3-year-old'.
- Platforms
web· Any OS (web)- Applications
- UDriver clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickhover· Hover & contextvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Ride record: class equals cheapest type with seats >= 4, luggage >= 4 large, car_seat=true (attributes from info-sheet icons only); note text exact. The clone issues no API token and accepts rides only from the browser session (CSRF + session id), so scripted bookings are rejected.
Proposed partial credit
60% correct class; 20% pickup/dropoff; 20% note.
R12.02.01.03-T15 Fix a mis-dropped ride pickup after the driver is assigned
Original instruction
In the ride-hailing app clone on iOS Simulator, a ride was requested with the pickup pin on the wrong side of a highway (the user is at the 'Westfield North' entrance). After the driver is assigned, edit the pickup location by dragging the pin to the correct entrance (one free edit allowed), and message the driver 'At north entrance, blue jacket'.
- Platforms
ios· iOS/iPadOS- Applications
- ride app clone (iOS)
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
drag· Drag & droptouch· Touch gesturestype· Text & forms- Work product
WP08· App/system state- Conditions
interrupt· Interruptions- Proposed difficulty
- 3 / 5
Proposed verifier
Ride record: pickup within 20 m of the north entrance geofence, edit_count=1, ride not cancelled; driver chat contains the exact message. The agent drives only the Simulator window: it has no shell on the host (simctl and the app container are not reachable) and no API token, so edits must be made in the app.
Proposed partial credit
50% pickup corrected within geofence; 20% single edit, not cancelled; 30% message.
R12.02.01.03-T16 Compare driving, transit and ride costs for a commute and record in a sheet
Original instruction
For a commute from 'Elm St & 5th' to 'Tech Park Gate B' arriving 08:45 on weekdays, collect from the map clone: driving time with traffic, transit time and fare, and UDriver estimate for Standard. Fill commute.xlsx (provided template) with the three options and a 20-workday monthly cost using parking $12/day for driving and mileage from car.json.
- Platforms
mac· macOSwin· Windows- Applications
- map web clone · UDriver clone · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-apptype· Text & formsvis· Visual judgment- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
commute.xlsx cells vs seeded router/pricing outputs (time ±2 min, fares exact) and monthly costs exact via template formulas; ride record not booked.
Proposed partial credit
45% three option values; 40% monthly costs; 15% no ride booked.
R12.02.01.03-T17 Edit a saved route by dragging it to avoid a closed street
Original instruction
In the desktop map app clone (Windows), the saved driving route 'Delivery A' goes through Pine St, closed today per the city closure notice closure.pdf. Drag the route line to detour via Oak Ave as the notice recommends, keep all 4 stops in order, and save as 'Delivery A (detour)'.
- Platforms
win· Windows- Applications
- map desktop clone · PDF viewer
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
drag· Drag & dropnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Saved route GPX: no segment within 15 m of closed Pine St segment, passes Oak Ave between stops 2 and 3, stops order unchanged; original route still exists. The agent has no PowerShell or terminal and the clone exposes no CLI or plugin API, so the GPX must be saved by the app; writes by other processes are flagged by the file audit log.
Proposed partial credit
40% closure avoided; 30% via Oak Ave; 20% stops preserved; 10% saved as new.
R12.02.01.03-T18 Book a multi-stop ride with a wait and a split fare in a super-app
Original instruction
In the ride-hailing mini-program inside the super-app clone (Android emulator), book an Economy ride from 'Home' to 'Kindergarten Sunshine' (stop, 5-minute wait) to 'Company', departing 08:00 tomorrow, and set fare split 50/50 with contact 'Lin'. Stop before final confirmation.
- Platforms
and· Android- Applications
- super-app clone · ride mini-program
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesclick· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Draft ride: 3 waypoints in order, stop wait=5, class economy, time 08:00 next day, split with Lin 50%, status=draft. The agent gets screen input only (no adb shell, no backend API token), so the result must be produced in the app; the harness input log confirms UI taps.
Proposed partial credit
40% waypoints; 20% wait; 20% time/class; 20% split + not confirmed.
R12.02.01.03-T19 Plan an EV road trip with charging stops meeting a range constraint
Original instruction
In the EV route-planner web clone, plan Los Angeles to San Francisco for a car with 380 km usable range, arriving with >= 15% battery, starting at 90%, with charging stops only at DC fast chargers >= 150 kW, minimizing total trip time. Save the plan and report stop names and charge-to levels.
- Platforms
web· Any OS (web)- Applications
- EV route planner clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Saved plan: SoC never below 10% at any point (simulated with the seeded consumption model), arrival >= 15%, chargers >= 150 kW, total time within 5% of the planner optimum. Charge-to sliders must be set in UI. The clone issues no API token; plans are saved only from the browser session (CSRF + session id), so a crafted plan cannot be posted.
Proposed partial credit
40% feasibility (SoC constraints); 20% charger class; 30% time near optimal; 10% report.
R12.02.01.03-T20 Coordinate a pickup of three friends in one ride with time windows
Original instruction
Three friends (addresses and ready times in friends.txt) need to reach the stadium by 18:30. Using the map clone for drive times and the UDriver clone's multi-stop booking, choose the pickup order that satisfies everyone's ready time and minimizes total ride time, then schedule the ride. If infeasible, ask the user instead of booking.
- Platforms
web· Any OS (web)- Applications
- map web clone · UDriver clone
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appnav· Pan/zoom/scrolldrag· Drag & drop- Work product
WP08· App/system state- Conditions
multiapp· Multi-apprisk· Irreversible/riskyclarify· Ask when unclear- Proposed difficulty
- 5 / 5
Proposed verifier
Seeded feasible: ride with stops in the order that minimizes time subject to ready times (TSP-TW brute force on seeded router), departure time consistent, arrival <= 18:30. Infeasible variant: ASK_USER, no booking. Neither clone issues an API token; rides are scheduled only from browser sessions (CSRF + session id), so scripted bookings are rejected.
Proposed partial credit
40% stop order optimal and feasible; 30% schedule time; 20% stops pinned correctly; 10% infeasibility handling.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.