Photo library organization & light edits
Make albums, tag people, fix dates and lightly edit photos.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Photos · Google Photos
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.09.01.01-T01 Create a trip album from GPS and date in Apple Photos
Original instruction
The test macOS user's Photos library holds 1,200 seeded photos. Create album 'Kyoto 2024' containing exactly the photos taken within 15 km of Kyoto Station between 2024-04-03 and 2024-04-09 (local time). Do not include photos from Osaka taken on the same days.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrolldrag· Drag & drop- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Photos.sqlite read after quitting the app: album membership must equal the ground-truth UUID set (F1 = 1.0 for full credit). Automation permission for osascript/Terminal is denied and Photos scripting events are logged; a scripted album creation scores 0.
Proposed partial credit
20% album exists with exact name; 80% scaled by membership F1
R12.09.01.01-T02 Name and merge face clusters for a family member
Original instruction
In Apple Photos (test library), the People view shows Grandma split across three unnamed clusters, and one cluster mixes her with her sister. Name Grandma's clusters 'Rosa Alvarez', merge them, and remove the 14 misassigned photos of the sister from her person (use 'This is not Rosa'). Mark Rosa as a favorite person.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentdrag· Drag & drop- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Photos.sqlite person and face tables: one person 'Rosa Alvarez' whose face set has F1 ≥ 0.95 against the seeded identity labels, favorite flag set, and none of the 14 sister photos attached. Faces are not in Photos' AppleScript dictionary, and Terminal lacks Photos library access (TCC), so the library database cannot be edited outside the app.
Proposed partial credit
20% person named; 30% clusters merged; 30% misassigned faces removed; 20% favorite flag
R12.09.01.01-T03 Fix a camera clock offset on one camera's photos
Original instruction
52 photos in the Photos library were shot on a Canon EOS R6 whose clock was set 1 year and 3 hours behind. Select exactly those photos (other cameras were shot the same week) and use Adjust Date and Time to shift them so the Canon shots interleave correctly with the iPhone shots of the same events.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Photos.sqlite asset dates: the 52 Canon assets must each be shifted by +1 y +3 h ±1 min, and all other assets unchanged. Original files must keep their EXIF (Photos stores adjustments in the DB); edits made with exiftool on originals are detected by file hash. Automation permission for osascript/Terminal to control Photos is denied, so dates cannot be set by script.
Proposed partial credit
30% exactly the Canon set selected; 50% correct offset on all 52; 20% no other asset changed
R12.09.01.01-T04 Straighten a horizon and crop to 4:5 non-destructively
Original instruction
In Apple Photos, open IMG_2041 (a seascape with a tilted horizon). Straighten it so the horizon is level and crop to a 4:5 portrait aspect that keeps the lighthouse fully in frame. Keep the edit non-destructive (Revert to Original must remain available) and export a JPEG to ~/Desktop/IMG_2041_edit.jpg.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
geom· Precise geometrydrag· Drag & dropvis· Visual judgment- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Exported JPEG: aspect 0.800 ±0.005; horizon angle measured by Hough line on the export within ±0.3°; lighthouse bbox (ground truth transformed) fully inside the crop. Photos adjustment data must exist for the asset, so a file edited elsewhere and re-imported fails; Terminal lacks Photos library access (TCC), so that data cannot be written directly.
Proposed partial credit
35% horizon level; 30% aspect 4:5; 20% lighthouse inside crop; 15% edit stored non-destructively
R12.09.01.01-T05 Archive all screenshots in Google Photos on Android without deleting
Original instruction
On the Android emulator, Google Photos is signed in to a test account whose library has 300 items, 40 of them screenshots mixed into the camera timeline (some saved from other apps and not in the Screenshots folder). Archive all 40 screenshots so they leave the Photos view; do not delete anything and do not archive camera photos.
- Platforms
and· Android- Applications
- Google Photos
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP09· Files & folders- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Test account state read via the Google Photos Library API: archived set must equal the 40 seeded screenshot IDs (F1), trash must be empty and item count must remain 300. The archive flag is not writable by the Library API, and the agent holds no credentials or OAuth token for the test account, so it must be set in the app.
Proposed partial credit
70% scaled by archive F1; 30% no deletions
R12.09.01.01-T06 Make a Windows Photos album and set its cover
Original instruction
In the Windows Photos app on the test account, create album 'Lake House 2025' with the 18 photos from the folder Pictures\LakeHouse that show the lake (the folder also has 6 indoor shots). Set the photo of the dock at sunset as the album cover and mark it as a favorite.
- Platforms
win· Windows- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Photos app database (MediaDb .v1.sqlite in the app LocalState) read after closing: album exists with exactly the 18 lake photos, cover item equals the dock photo, favorite flag set. The app exposes no scripting interface, and Python, PowerShell and sqlite3 are blocked by AppLocker, so MediaDb cannot be edited directly.
Proposed partial credit
20% album created; 50% membership F1; 15% cover set; 15% favorite flag
R12.09.01.01-T07 Merge true duplicates only in the Photos Duplicates album
Original instruction
Apple Photos lists 31 duplicate sets in its Duplicates album. 18 sets are true duplicates (same image, different files); 13 are burst variants or edits that the user wants kept separately (see input/keep_rules.txt). Merge only the 18 true duplicate sets. Deleted items go to Recently Deleted; do not empty it.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Photos.sqlite: the 18 duplicate sets must each collapse to one asset with highest-resolution file kept; the 13 keep sets must be intact; Recently Deleted must not be purged. Each wrong merge subtracts credit. Merging is not in Photos' AppleScript dictionary, and Terminal lacks Photos library access (TCC), so Photos.sqlite cannot be edited outside the app.
Proposed partial credit
60% scaled by correct merges; 30% no wrong merges; 10% Recently Deleted intact
R12.09.01.01-T08 Build a smart album with four rules in Apple Photos
Original instruction
Create a Smart Album 'Print 2025' in Apple Photos that matches all of: camera model 'iPhone 15 Pro', keyword 'print', date between 2025-01-01 and 2025-12-31, and not a screenshot. Then add the keyword 'print' to the 9 photos listed by filename in input/print_list.txt so the album shows them.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Photos.sqlite smart-album query blob decoded: 4 conditions with the exact values and 'all' logic. Keyword table: 'print' attached to exactly the 9 listed assets. Evaluated album membership must equal the expected set. Automation permission for osascript/Terminal to control Photos is denied and Terminal lacks Photos library access (TCC), so keywords cannot be set by script.
Proposed partial credit
40% four smart-album conditions correct; 30% keyword on exactly 9 photos; 30% evaluated membership correct
R12.09.01.01-T09 Assign locations to film scans by visual match with phone photos
Original instruction
Folder album 'Film Scans 2023' holds 24 scans without GPS. The same trip's iPhone photos (with GPS) show the same scenes in Hallstatt, Salzburg and Vienna. In Apple Photos, assign each scan the location of the matching city using Info > Assign a Location, and set each scan's date to the date of its matching iPhone scene.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmenttype· Text & formsclick· Point & click- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Photos.sqlite: each scan's location must be within 5 km of the ground-truth city and its date equal to the matching iPhone day. Ground truth was built by hand-matching scenes; there is no metadata link, so the match requires visual comparison. Automation permission for osascript/Terminal to control Photos is denied, so locations and dates cannot be set by script.
Proposed partial credit
50% scaled by correct locations; 40% scaled by correct dates; 10% no iPhone photos modified
R12.09.01.01-T10 Keyword-tag every photo that contains a dog
Original instruction
The Apple Photos test library has 500 photos; 63 contain a dog, some small or partly hidden. Add keyword 'dog' to every photo with a dog and to no others. Photos' own search for 'dog' returns several false positives and misses, so check results visually.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicknav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Photos.sqlite keyword joins compared to human-labelled ground truth: F1 ≥ 0.95 for full credit. Seeded hard cases (dog in background, dog toy, wolf statue) measure precision. Keywords applied through osascript are blocked by denied Automation permission.
Proposed partial credit
50% F1 rising linearly from 0.70 to 0.825; 50% F1 rising linearly from 0.825 to 0.95 (full credit at 0.95)
R12.09.01.01-T11 Apply red-eye removal and exposure fixes to five portraits
Original instruction
In the Windows Photos app, open the five portraits in Pictures\Party (P01-P05). Remove red-eye on every affected eye, raise exposure on the two underexposed shots so the face region's mean luminance is 110-140, and save each as a copy with suffix '_fixed'. Originals must stay unchanged.
- Platforms
win· Windows- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentgeom· Precise geometry- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
_fixed copies: red-eye regions (ground-truth pupil masks) have mean a* ≤ 20 in Lab; face-mask mean luminance 110-140 on the two dark shots; SSIM ≥ 0.9 elsewhere vs original. Originals' hashes unchanged. Python, PowerShell and cmd are blocked by AppLocker, and Sysmon logs the process that writes each _fixed file; it must be Photos.
Proposed partial credit
40% red-eye fixed on all eyes; 30% exposure targets met; 20% copies named correctly; 10% originals untouched
R12.09.01.01-T12 Hide photos of personal documents in iOS Photos
Original instruction
On the iOS Simulator, the Photos library has 220 images; 12 are photos of personal documents (passport, driver licence, bank card, tax letter). Move exactly those 12 to the Hidden album using the Photos app, and turn on 'Use Face ID' protection for Hidden and Recently Deleted.
- Platforms
ios· iOS/iPadOS- Applications
- Photos
- Interfaces
mob· Mobile touch- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP09· Files & folders- Conditions
personal· Personal context- Proposed difficulty
- 3 / 5
Proposed verifier
Simulator Photos.sqlite: hidden flag set on exactly the 12 ground-truth assets (F1); Settings plist shows protection enabled. No deletions allowed. The agent reaches the Simulator only through its window: xcrun simctl and the host shell are not exposed, so the simulator's library database cannot be edited directly.
Proposed partial credit
70% scaled by hidden-set F1; 20% protection setting on; 10% no deletions
R12.09.01.01-T13 Correct orientation of mis-rotated scans only
Original instruction
Folder Pictures/Scans1998 contains 30 scanned prints imported into Apple Photos; 11 are sideways or upside down. Rotate only the 11 wrong ones so people and horizons are upright. Leave correctly oriented scans untouched.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & click- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Photos.sqlite orientation/adjustment per asset compared to ground truth (exact quarter turns). Any rotation on a correct scan counts as an error. Rotating originals in Finder or with sips is detected by file hash change.
Proposed partial credit
80% scaled by correctly rotated scans; 20% no rotations on correct scans
R12.09.01.01-T14 Choose the key frame and loop effect for Live Photos
Original instruction
In Apple Photos, three Live Photos (LP_01-LP_03) of a group show people blinking in the default key frame. For each, scrub the Live Photo frames and set the key photo to a frame where all faces have open eyes. Then set LP_02's effect to Loop.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
av· Audio/video over timevis· Visual judgmentdrag· Drag & drop- Work product
WP05· Time-based media- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Photos.sqlite adjustment data: key-frame timestamp for each Live Photo must fall in a ground-truth eyes-open interval (labelled per frame); LP_02 playback style must be Loop and the others unchanged. Key photo and playback style are not in Photos' AppleScript dictionary, and Terminal lacks Photos library access (TCC), so Photos.sqlite cannot be edited outside the app.
Proposed partial credit
25% LP_01 key frame valid; 25% LP_02 key frame valid; 25% LP_03 key frame valid; 25% LP_02 set to Loop
R12.09.01.01-T15 Name a person and fix face assignments in a self-hosted photo server
Original instruction
In the Immich web UI (self-hosted, test account), people recognition produced two clusters for the child 'Leo' and put 6 of his cousin's photos into one of them. Merge Leo's clusters, name him 'Leo', set his birthday to 2019-05-14, reassign the 6 cousin photos to person 'Mateo', and hide the 'unknown' cluster of a statue.
- Platforms
web· Any OS (web)- Applications
- Immich
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmenttype· Text & forms- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Immich Postgres: person 'Leo' with birthDate 2019-05-14 whose faces have F1 ≥ 0.95 vs labels; 6 faces assigned to 'Mateo'; statue person isHidden = true. The API key page is disabled for the test user and API calls without the web session are logged.
Proposed partial credit
30% merge and name; 10% birthday; 35% cousin faces reassigned; 25% statue hidden
R12.09.01.01-T16 Write titles and captions from a CSV in digiKam
Original instruction
In digiKam, open album 'Exhibition'. Using input/captions.csv (filename, title, caption, 1-5 rating), set the title, caption and star rating for each of its 20 photos through the digiKam Captions and Rating panels, and have digiKam write the metadata to XMP sidecar files. This task is the GUI half of the pair with T17.
- Platforms
mac· macOSwin· Windows- Applications
- digiKam
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsclick· Point & clickmulti· Cross-app- Work product
WP10· Annotations- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
digikam4.db ImageComments and ImageInformation rows must match the CSV for all 20 images, and each .xmp sidecar must carry the same dc:title, dc:description and xmp:Rating. Sidecars without DB rows fail; the process writing the DB is logged (FSEvents on macOS, Sysmon on Windows), and any writer other than digiKam is detected.
Proposed partial credit
40% titles; 40% captions; 20% ratings (each scaled over 20 photos)
R12.09.01.01-T17 Control: write the same titles and captions with exiftool
Original instruction
Control pair of T16. In a terminal, use exiftool to write title, caption and rating from input/captions.csv into XMP sidecars for the 20 photos in ~/Pictures/Exhibition without modifying the original image files. Produce one .xmp per image.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- exiftool
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Each .xmp must carry dc:title, dc:description and xmp:Rating equal to the CSV row; original image hashes unchanged. Used as the CLI control for T16 to measure the GUI gap on the same data.
Proposed partial credit
40% titles; 40% captions; 20% ratings, scaled per image; 0 if any original changes
R12.09.01.01-T18 Export a wedding selects album to a web-ready spec
Original instruction
Export album 'Wedding selects' (48 photos) from Apple Photos to ~/Export/Wedding as JPEG, quality High, longest edge 2048 px, with location information removed and filenames as a sequence 'Wedding_001' to 'Wedding_048' in the album's manual sort order (not capture date).
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
48 JPEGs: max(width, height) = 2048; no GPS EXIF tags; filename index n must hash-match (perceptual) the nth photo of the album's custom order. Exports made from originals with sips lack the album order and are caught by the order check. Automation permission for osascript/Terminal to control Photos is denied, so the album cannot be exported by script.
Proposed partial credit
20% 48 files exported; 25% size spec; 20% GPS stripped; 35% names follow album order
R12.09.01.01-T19 Import a second library into folders and monthly albums without duplicates
Original instruction
An external drive folder 'Dad_iPhone_2019' holds 860 photos and videos, 140 of which are already in the Apple Photos library. Import it with 'Don't import duplicates', then create folder 'Family/2019' containing 12 albums '2019-01' to '2019-12', each holding exactly the imported items from that month (videos included).
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickdrag· Drag & dropnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Photos.sqlite: library gains exactly 720 assets; folder hierarchy Family/2019 with 12 albums; album membership by capture month equals ground truth (F1 per album). Duplicate imports reduce the import score. Automation permission for osascript/Terminal to control Photos is denied and Terminal lacks Photos library access (TCC), so imports and albums cannot be scripted.
Proposed partial credit
30% 720 new assets, no duplicates; 20% folder and 12 albums; 50% scaled by mean album F1
R12.09.01.01-T20 Curate twelve monthly albums of best shots for a family photo book
Original instruction
From the 2,000-photo 2025 library in Apple Photos, build albums 'Book-01' to 'Book-12', each with 10 photos from that month that are sharp, show at least one face with eyes open, and are not near-duplicates of each other. First fix the 70 photos from a camera with a wrong date (see input/notes.txt) so they land in the right month.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentclick· Point & clicknav· Pan/zoom/scrolldrag· Drag & drop- Work product
WP09· Files & folders- Conditions
long· Long-horizonhidden· Hidden/stale state- Proposed difficulty
- 5 / 5
Proposed verifier
Per album: 10 assets from the right month (after the date fix); each passes Laplacian-variance sharpness ≥ threshold and a face detector with eyes-open classifier; pairwise pHash distance > 8. Date fix checked in Photos.sqlite. Album creation by script is blocked and logged.
Proposed partial credit
20% date fix; 20% 12 albums × 10 photos in correct months; 40% quality checks pass rate; 20% no near-duplicates
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.