Pinch, rotate & map manipulation
Zoom, rotate and position maps, photos and documents to a target view.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Maps · OsmAnd · Gallery
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R01.11.02.03-T01 OsmAnd: pinch-zoom until the scale bar reads 100 m
Original instruction
In OsmAnd centred on the Ferry Building, San Francisco, pinch to zoom until the on-screen ruler/scale bar reads exactly 100 m. Do not use the +/− buttons or volume keys.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gestures- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
OsmAnd map state (prefs last_known_map_zoom and fractional zoom) maps to a 100 m scale step; input log shows multi-touch pinch events and no zoom-button taps.
Proposed partial credit
70% zoom at the 100 m step; 30% pinch route only.
R01.11.02.03-T02 OsmAnd: rotate map so north points right, then restore north-up
Original instruction
In OsmAnd with map rotation set to 'manual', two-finger rotate the map so north points to the right (bearing 270° ±5°) and hold it for 3 s, then rotate it back to north-up (±2°) by gesture, not the compass button.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
OsmAnd instrumentation log of map bearing: a ≥3 s interval within 270±5°, final bearing 0±2°; compass button tap not logged. adb and root shell are withheld from the agent (screen input only), so geo: intents and the OsmAnd AIDL API are unavailable; the map must be rotated by gesture.
Proposed partial credit
50% intermediate bearing held; 50% final north-up by gesture.
R01.11.02.03-T03 PDF reader: zoom page 4 to 200% and screenshot the table
Original instruction
In Docreader, open Documents/annual.pdf, go to page 4, pinch-zoom to 200% (±10%) and pan so the 'Regional totals' table fills the screen, then take a screenshot.
- Platforms
and· Android- Applications
- Docreader
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
App state zoom 1.8–2.2 on page 4; newest screenshot's OCR contains all 6 row labels and the table header; table bbox covers ≥ 70% of the screenshot width. adb and root shell are withheld from the agent (screen input only), so intents and screencap are unavailable; zoom and screenshot must be done on the device.
Proposed partial credit
40% zoom in range on correct page; 40% table fully visible; 20% screenshot taken.
R01.11.02.03-T04 Gallery: pinch the grid to five columns
Original instruction
In the Gallery app's photo grid, use a pinch-out/pinch-in gesture to change the grid to exactly 5 columns, then open the photo in row 3, column 5 and mark it as favourite.
- Platforms
and· Android- Applications
- Gallery
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Gallery prefs grid_column_count=5; favourite flag set on the photo at index 14 of the current sort order and on no other photo. adb and root shell are withheld from the agent (screen input only), so run-as pref edits are unavailable; the grid must be pinched.
Proposed partial credit
50% five columns; 50% correct photo favourited.
R01.11.02.03-T05 OsmAnd: frame Coit Tower at zoom 17 with bearing 45°
Original instruction
In OsmAnd (manual rotation), set the map view so Coit Tower is at screen centre (±40 px), zoom level 17 (±0.2), and bearing 45° (±5°), using gestures only. Leave it on screen.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometrynav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
OsmAnd map state dump: centre lat/lon within ~40 px equivalent of Coit Tower, zoom 16.8–17.2, bearing 40–50°; search activity never opened. adb and root shell are withheld from the agent (screen input only), so geo: intents and the OsmAnd AIDL API are unavailable; the map must be framed by gesture.
Proposed partial credit
35% centre; 35% zoom; 30% bearing.
R01.11.02.03-T06 Photo editor: straighten a tilted horizon with the rotate gesture
Original instruction
Open Pictures/sea_tilt.jpg (horizon tilted 7.3°) in the Gallery editor's crop tool, straighten it with the two-finger rotate gesture or the angle dial so the horizon is level, and save a copy.
- Platforms
and· Android- Applications
- Gallery editor
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
touch· Touch gesturesgeom· Precise geometryvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hough-line horizon angle in the saved copy within ±0.5° of level; image not flipped; resolution ≥ 80% of source. adb and root shell are withheld from the agent (screen input only), so file pushes are unavailable; the copy must be straightened and saved in the editor.
Proposed partial credit
30% saved copy; 50% horizon within ±0.5° (25 pts if ±1.5°); 20% no excessive crop.
R01.11.02.03-T07 MapLibre demo: match a target camera with pitch and bearing
Original instruction
In the MapLibre Android demo app's 'Simple Map', match the camera in fixtures/target_view.png: centre, zoom 15.5, pitch 50°, bearing 120°. Use pinch, two-finger rotate and two-finger vertical drag for tilt.
- Platforms
and· Android- Applications
- MapLibre demo
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometryvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Camera position logged by the demo's instrumentation: centre within 50 m, zoom ±0.25, pitch ±4°, bearing ±5°. adb and root shell are withheld from the agent (screen input only), so am start camera extras are unavailable; the camera must be set by gesture.
Proposed partial credit
25% centre; 25% zoom; 25% pitch; 25% bearing.
R01.11.02.03-T08 OsmAnd: find a street name by panning with search disabled
Original instruction
In OsmAnd (search disabled in this build), the start view shows Union Square. Pan and zoom to the intersection marked by the red favourite 'X1' about 3 km away and write the two street names at that intersection into Markor x1.md.
- Platforms
and· Android- Applications
- OsmAnd · Markor
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
x1.md contains both street names (normalised case/abbreviations) from the OSM ground truth; no extra names. adb and root shell are withheld from the agent (screen input only), so geo: intents are unavailable and search is disabled; names must be found by panning.
Proposed partial credit
20% reached favourite; 80% both street names correct (40 each).
R01.11.02.03-T09 iOS MapKit: zoom and rotate to a target view in MapProbe
Original instruction
On the iOS 18 Simulator, MapProbe shows a MapKit map. Use pinch and two-finger rotate (Option-drag in Simulator) to show Golden Gate Bridge centred at a camera distance of 3,000 m (±10%) with heading 300° (±5°).
- Platforms
ios· iOS/iPadOS- Applications
- MapProbe · MapKit
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometry- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
MapProbe logs its MKMapCamera: centre within 150 m of the bridge midpoint, distance 2700–3300 m, heading 295–305°. Host shell and xcrun simctl are withheld from the agent, so camera deep links cannot be sent; the view must be set by gesture.
Proposed partial credit
35% centre; 35% distance; 30% heading.
R01.11.02.03-T10 iOS Photos: zoom to read a sign and record it
Original instruction
In Photos on the iOS 18 Simulator, open IMG_0102 (a street scene). Pinch-zoom to the small shop sign at the lower left until legible and type the shop's name and phone number into a new Notes note 'Shop'.
- Platforms
ios· iOS/iPadOS- Applications
- Photos · Notes
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Note text contains the exact shop name and the 10-digit phone number from ground truth (digits-only comparison). The sign is unreadable at fit-to-screen size. Host shell and xcrun simctl are withheld from the agent, so the photo cannot be pulled to the host; the sign must be read by zooming in Photos.
Proposed partial credit
50% shop name; 50% phone number.
R01.11.02.03-T11 Document scanner: drag four corner handles to the page corners
Original instruction
In OpenNoteScanner, load Pictures/contract_photo.jpg. Auto-detection is off; drag the four crop handles onto the paper's corners (use the magnifier) and save the rectified scan.
- Platforms
and· Android- Applications
- OpenNoteScanner
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesdrag· Drag & dropgeom· Precise geometry- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Grader recovers the applied quad from the app's log (or by homography fit of the output) and compares to annotated corners: mean corner error ≤ 1.5% of page diagonal; output text OCR similarity ≥ 0.9. adb and root shell are withheld from the agent (screen input only), so file pushes and intents are unavailable; corners must be dragged in the app.
Proposed partial credit
60% corner accuracy (scaled); 20% saved output; 20% OCR quality.
R01.11.02.03-T12 OsmAnd: measure a trail with the ruler by placing points along it
Original instruction
Using OsmAnd's Plan a route / measure tool in 'straight line' mode, trace the Lands End Trail from the parking lot to the labyrinth by placing points along the path (zooming as needed), save it as GPX 'landsend', and report its length in Markor.
- Platforms
and· Android- Applications
- OsmAnd · Markor
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturespath· Continuous pathgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Saved GPX: 95% of points within 20 m of the OSM trail polyline, endpoints within 30 m; reported length within 5% of the GPX length and of the trail length. adb and root shell are withheld from the agent (screen input only), so GPX file pushes are unavailable; points must be placed with the ruler.
Proposed partial credit
40% path fidelity; 20% endpoints; 20% saved GPX; 20% length reported.
R01.11.02.03-T13 Geofence app: draw a polygon around a park boundary
Original instruction
In GeofenceDemo (MapLibre-based), create a polygon geofence 'Dolores Park' by tapping vertices around the park's boundary (zoom and pan as needed), with at most 24 vertices, and save it.
- Platforms
and· Android- Applications
- GeofenceDemo
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturespath· Continuous pathgeom· Precise geometry- Work product
WP04· 3D/spatial model- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Saved polygon vs OSM park polygon: IoU ≥ 0.85 for full credit (scaled down to 0.6); vertex count ≤ 24; name exact. adb and root shell are withheld from the agent (screen input only), so run-as file edits are unavailable; the polygon must be drawn in the app.
Proposed partial credit
70% IoU (scaled); 15% vertex limit; 15% saved with name.
R01.11.02.03-T14 Tablet labeling app: zoom and place six keypoints within 5 px
Original instruction
On the Pixel Tablet AVD, open LabelDemo image hand_017.png (4000×3000). Place the six requested keypoints (wrist, thumb tip, index tip, middle tip, ring tip, pinky tip), zooming in so each lands within 5 px at native resolution, and save.
- Platforms
and· Android- Applications
- LabelDemo
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometryvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Saved annotation JSON vs ground truth: per-keypoint error in native pixels; full credit ≤ 5 px, partial ≤ 15 px; labels must match. adb and root shell are withheld from the agent (screen input only), so run-as file edits are unavailable; keypoints must be placed in the app.
Proposed partial credit
50% keypoints 1-3 (one sixth each); 50% keypoints 4-6 (one sixth each); per point full credit ≤ 5 px, half ≤ 15 px.
R01.11.02.03-T15 Collage app: rotate, scale and place a photo tile
Original instruction
In CollageDemo, place Pictures/logo.png on the canvas and use two-finger gestures to rotate it 30° clockwise (±2°), scale it to 1.5× (±0.05), and centre it at the marked crosshair (±10 px), then export.
- Platforms
and· Android- Applications
- CollageDemo
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesgeom· Precise geometry- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Exported project JSON transform: rotation, scale and centre within tolerances; exported PNG matches the transform (template match). adb and root shell are withheld from the agent (screen input only), so run-as project edits are unavailable; the tile must be placed by gesture.
Proposed partial credit
35% rotation; 35% scale; 30% position.
R01.11.02.03-T16 OsmAnd: download offline map of the area currently in view
Original instruction
Pan and zoom OsmAnd until the downtown Oakland city-hall area is in view, then download the offline map region that contains it using the map's 'download this region' prompt, not the list.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
OsmAnd data dir contains the region file covering Oakland city hall; download initiated from the map context (OsmAnd log event). adb and root shell are withheld from the agent (screen input only), so host file pushes are unavailable; the region must be downloaded from the map prompt.
Proposed partial credit
40% correct region in view; 60% region downloaded via map prompt.
R01.11.02.03-T17 Gallery: zoom-compare two near-identical photos and report the difference
Original instruction
Pictures/pair_a.jpg and pair_b.jpg differ in one small detail (a single object changed). Pinch-zoom and pan both in Gallery to find it and write which object differs and in which quadrant into Markor diff.md.
- Platforms
and· Android- Applications
- Gallery · Markor
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
diff.md matched against the ground-truth object label (synonym list) and quadrant (TL/TR/BL/BR). Detail is < 1% of image area. adb and root shell are withheld from the agent (screen input only), so photos cannot be pulled for pixel diffing; the difference must be found by zooming.
Proposed partial credit
60% object identified; 40% quadrant correct.
R01.11.02.03-T18 OsmAnd: locate the highest summit in a region from contours
Original instruction
With OsmAnd contour lines enabled and search disabled, find the highest summit inside the red polygon 'Area B' (Marin Headlands) by panning and zooming, long-press to drop a favourite named 'Summit' on it, and note its elevation.
- Platforms
and· Android- Applications
- OsmAnd
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
favourites.gpx point 'Summit' within 30 m of the DEM maximum inside Area B; recorded elevation within 5 m (in description). adb and root shell are withheld from the agent (screen input only), so file pushes and the OsmAnd AIDL API are unavailable; the favourite must be placed on the map.
Proposed partial credit
60% favourite within 30 m (30 pts within 100 m); 20% named; 20% elevation.
R01.11.02.03-T19 iOS Photos: crop and rotate a document photo to a straight A4 page
Original instruction
In Photos on the iOS 18 Simulator, edit IMG_0300 (a tilted receipt): use crop with free aspect and the rotation dial to straighten it to within 1°, and crop tightly to the paper edges.
- Platforms
ios· iOS/iPadOS- Applications
- Photos
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesgeom· Precise geometrydrag· Drag & dropvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Rendered edited asset: text-line angle within ±1° (Hough on OCR boxes); paper occupies ≥ 90% of area with < 5% background on any edge. Host shell and xcrun simctl are withheld from the agent, so Photos.sqlite and host image tools are unavailable; the edit must be made in Photos.
Proposed partial credit
50% rotation within tolerance; 50% tight crop.
R01.11.02.03-T20 Control pair of T05: set the OsmAnd view via a geo intent
Original instruction
Control pair of T05. With adb on the same AVD, open OsmAnd at Coit Tower, zoom 17, using an Android intent/URL scheme, and set bearing 45° via OsmAnd's API or settings, without touching the screen.
- Platforms
and· Androidlin· Linux- Applications
- adb · OsmAnd
- Interfaces
term· Terminalapi· API/SDK- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same map-state checks as T05; input log shows no touch events. Reported as the CLI baseline for the gesture task.
Proposed partial credit
35% centre; 35% zoom; 30% bearing.
Mapped research references
No benchmark reference is assigned to this class in the current mapping.