Content-aware curation
Sort, keep, delete or edit files according to what they show or say (blurry photos, duplicates with different names, the version in a given language).
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Finder · Photos · File Explorer
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R01.02.01.03-T01 Trash the blurry shots from an event folder
Original instruction
~/Pictures/Event has 120 photos; 27 are noticeably blurred (motion or missed focus). Move only the blurry ones to the Trash and leave the sharp ones in place. Slightly soft but usable shots (marked borderline in no file) should be kept.
- Platforms
mac· macOSwin· Windows- Applications
- Finder · Preview · File Explorer
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & click- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Trash/Recycle Bin contents vs annotator ground truth (27 blurry; 8 borderline labeled keep): F1 on removed set; borderline removals count as false positives. Kept file hashes unchanged.
Proposed partial credit
80% F1 on blurry set; 20% no borderline/sharp files removed
R01.02.01.03-T02 Separate receipt photos from personal photos
Original instruction
~/Phone_Dump has 400 images. Move photos and screenshots of receipts or invoices to ~/Receipts, leave everything else. Some receipts are partially visible on a table or photographed at an angle.
- Platforms
mac· macOSwin· Windows- Applications
- Finder · File Explorer · Photos
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentdrag· Drag & drop- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
~/Receipts set vs labeled ground truth (52 receipts; decoys: menus, tickets, notebooks). F1 scored; no files deleted (count+hash multiset unchanged).
Proposed partial credit
85% F1 on receipt set; 15% nothing deleted or altered
R01.02.01.03-T03 Merge duplicates in Photos keeping the best copy
Original instruction
The Photos library has a Duplicates album with 38 sets (some resized or recompressed copies with different names). Merge every set so the highest-resolution original survives, and confirm no non-duplicate photo was removed.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Photos.sqlite: each duplicate set leaves exactly one asset with max pixel count; merged copies in Recently Deleted; asset count drop equals expected; non-duplicates untouched. Terminal and Script Editor withheld; eslogger detects osxphotos/AppleScript deletes -> 0.
Proposed partial credit
60% sets merged correctly; 25% highest-res survivor; 15% no collateral deletions
R01.02.01.03-T04 Pick the signed final of each contract version chain
Original instruction
~/Contracts has 60 .docx/.pdf files for 8 contracts with names like 'MSA_acme_v3_final_FINAL2.docx'. For each contract, move the version that is final (signature page signed, no tracked changes, latest effective date) to ~/Contracts/Final; move all others to ~/Contracts/Drafts.
- Platforms
mac· macOSwin· Windows- Applications
- Finder · Word · Preview
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scrolldrag· Drag & drop- Work product
WP09· Files & folders- Conditions
licensed· Licensed pro app- Proposed difficulty
- 5 / 5
Proposed verifier
Final set equals the 8 ground-truth files (filename 'FINAL' is misleading on 5 decoys); Drafts holds the other 52; hashes unchanged. Per-contract credit.
Proposed partial credit
80% correct final per contract (equal share per contract); 20% all others in Drafts
R01.02.01.03-T05 Split product sheets by language
Original instruction
~/Sheets has 40 PDF product sheets in English, German, French and Spanish with language-neutral names. Keep the English ones in place and move the others into ~/Sheets/de, fr and es folders.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Finder · File Explorer · bash
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentdrag· Drag & dropcode· Code & commands- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Folder membership vs ground truth language labels (2 sheets are bilingual EN/DE: majority language counts). F1 per language; no files lost.
Proposed partial credit
25% English kept in place; 25% de folder; 25% fr folder; 25% es folder (bilingual cases weigh 2x within each)
R01.02.01.03-T06 Cull bursts to the single best frame
Original instruction
Photos contains 10 burst sequences of 8 frames each (portraits). For every burst keep only the frame with eyes open and the sharpest focus on the face: mark it Favorite, and delete the other 7 frames to Recently Deleted.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicknav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Photos.sqlite: per burst exactly one surviving asset, favorited; it must be in the annotators' accepted set (eyes open + top-2 sharpness by Laplacian variance on face crop). Deleted assets in Recently Deleted. Terminal and Script Editor withheld; eslogger detects osxphotos/AppleScript library edits -> 0.
Proposed partial credit
60% correct keeper per burst; 20% favorites set; 20% exactly 70 frames deleted
R01.02.01.03-T07 Fix upside-down and sideways scans
Original instruction
D:\Scans has 200 scanned pages as JPEG; 31 are rotated 90°, 180° or 270°. Using File Explorer's Rotate commands (large icons view), make every page upright. Do not touch already-upright scans.
- Platforms
win· Windows- Applications
- File Explorer · Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentclick· Point & clickhover· Hover & context- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
OCR orientation detection (Tesseract OSD) on each output (applying EXIF orientation) = upright for all 200; untouched files hash-equal to seed. PowerShell blocked by AppLocker and no ImageMagick installed; Sysmon detects scripted rotate tools -> 0.
Proposed partial credit
70% misrotated pages fixed (per-file); 30% upright pages untouched
R01.02.01.03-T08 Quarantine broken files and fix wrong extensions
Original instruction
In ~/dataset (1,500 files) move files that cannot be opened (truncated JPEG/PNG, zero-byte or corrupt PDFs, broken zips) to ~/dataset/_broken, and rename files whose extension does not match their content (e.g. a PNG named .jpg) to the correct extension.
- Platforms
mac· macOSlin· Linuxwin· Windows- Applications
- bash · file · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
_broken set equals ground truth (64 files); extension-fix renames equal expected (42); all other files unchanged; no file deleted.
Proposed partial credit
50% F1 on broken set; 40% correct extension fixes; 10% no collateral
R01.02.01.03-T09 Keep only bug-worthy screenshots
Original instruction
The Desktop has 300 screenshots. Move those showing an error dialog, crash report or red error banner to ~/Bugs, Trash those that are blank, all-desktop or accidental (menu bar only), and leave the rest.
- Platforms
mac· macOS- Applications
- Finder · Quick Look
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentkey· Shortcutsdrag· Drag & drop- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Three-way labels vs ground truth: Bugs F1 (46 files), Trash F1 (38 files), remainder unchanged. Trash restorable (no permanent delete).
Proposed partial credit
45% Bugs F1; 40% Trash F1; 15% remainder untouched
R01.02.01.03-T10 Collect whiteboard and document photos into an album
Original instruction
In Photos, find every photo of a document, receipt or whiteboard among 2,000 library items and add them to a new album 'Docs & Boards'. Do not add photos of people or screens.
- Platforms
mac· macOS- Applications
- Photos
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsvis· Visual judgmentclick· Point & click- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Photos.sqlite album 'Docs & Boards' membership vs ground truth (88 items; decoys: laptop screens, posters). F1 scored. Terminal and Script Editor withheld; eslogger detects AppleScript/osxphotos album edits -> 0.
Proposed partial credit
20% album created; 80% membership F1
R01.02.01.03-T11 Tag beach photos via Explorer properties
Original instruction
In C:\Users\tester\Pictures\2025 (350 JPEGs) add the tag 'beach' (Properties > Details > Tags) to every photo showing a beach or shoreline. Existing tags must be kept.
- Platforms
win· Windows- Applications
- File Explorer
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicktype· Text & forms- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
XMP/EXIF XPKeywords read by grader: 'beach' present on exactly the 61 ground-truth images (decoys: lakes, pools, desert); pre-existing tags preserved. No exiftool on the VM and PowerShell AppLocker-blocked; Sysmon detects property writes by processes other than explorer.exe -> 0.
Proposed partial credit
75% F1 on tagged set; 25% existing tags preserved
R01.02.01.03-T12 Delete empty voice memos
Original instruction
~/Recordings has 80 .m4a voice memos. Move to the Trash those that are shorter than 2 s or contain no speech (peak below -45 dBFS or only background noise); keep everything with speech.
- Platforms
mac· macOSlin· Linux- Applications
- bash · ffmpeg · sox
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandsav· Audio/video over time- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Trash set equals ground truth (17; decoys: quiet but audible speech at -38 dBFS, 3 s of speech); kept files unchanged.
Proposed partial credit
80% F1 on removed set; 20% speech files all kept
R01.02.01.03-T13 Find visually changed logo exports between versions
Original instruction
~/logos/v1 and ~/logos/v2 each have 140 PNG exports with the same names; v2 was re-exported, so all bytes differ. Copy to ~/logos/changed only the v2 files whose rendered image differs visibly from v1 (not just metadata or compression).
- Platforms
mac· macOSlin· Linuxwin· Windows- Applications
- Python · ImageMagick
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandsvis· Visual judgment- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
changed set equals the 19 truly modified files (pixel diff after alpha-premultiply; decoys: re-encoded identical pixels, 1-px antialias jitter below threshold). F1 scored.
Proposed partial credit
90% F1 on changed set; 10% v1/v2 untouched
R01.02.01.03-T14 Remove double-scanned invoices, keeping the clearer scan
Original instruction
D:\AP\scans has 260 invoice PDFs; some invoices were scanned twice (different bytes and names, same invoice number and amount). For each duplicate pair keep the clearer scan (higher DPI, then less skew) and move the other to D:\AP\dupes.
- Platforms
win· Windowsmac· macOS- Applications
- File Explorer · Acrobat Reader · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentcode· Code & commandsdrag· Drag & drop- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
dupes folder equals the 33 expected inferior copies; pairs identified by (invoice no, amount) ground truth; decoys share invoice number but differ in amount (credit notes) and must stay.
Proposed partial credit
50% pairs identified; 30% correct copy kept; 20% credit-note decoys untouched
R01.02.01.03-T15 Spot the misfiled report
Original instruction
~/Clients/Northwind/Reports has 25 PDF reports; exactly one belongs to a different client although its name looks the same. Move it to ~/Clients/_Misfiled.
- Platforms
mac· macOSwin· Windows- Applications
- Finder · Preview · File Explorer
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
_Misfiled contains exactly the one file whose cover page client name differs; others unchanged.
Proposed partial credit
50% correct file moved to _Misfiled; 50% no other file moved (both 0 if the correct file is not moved)
R01.02.01.03-T16 Keep only photos of my dog
Original instruction
Reference photos of my dog Miso are in ~/ref/miso. ~/Pictures/Park has 240 photos with several dogs: keep the photos in which Miso appears and move all others to ~/Pictures/Park/NotMiso. Photos' Pets recognition may help.
- Platforms
mac· macOS- Applications
- Photos · Finder
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clickdrag· Drag & drop- Work product
WP09· Files & folders- Conditions
personal· Personal context- Proposed difficulty
- 4 / 5
Proposed verifier
Remaining set vs ground truth (73 photos with Miso; decoys: same breed, different dog). F1 on kept set; no deletions. Terminal withheld; eslogger detects mv/Python file moves, and moves not attributed to Finder -> 0.
Proposed partial credit
85% F1 on kept set; 15% no files deleted
R01.02.01.03-T17 Select good takes by reading the slate
Original instruction
~/Footage has 50 clips; each begins with a slate card reading 'OK' or 'NG' plus scene/take. Move OK takes to ~/Footage/Selects and rename them 'S<scene>_T<take>.mov' from the slate. Leave NG takes in place.
- Platforms
mac· macOSwin· Windows- Applications
- QuickTime Player · Finder · ffmpeg
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
av· Audio/video over timevis· Visual judgmentcode· Code & commands- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Selects set equals ground truth OK takes (21); names match slate scene/take; slate on 4 clips appears after 1–3 s; hashes unchanged.
Proposed partial credit
50% correct OK/NG split; 40% correct names from slate; 10% NG untouched
R01.02.01.03-T18 Keep each student's latest complete draft
Original instruction
~/Essays has 30 .docx drafts from 9 students with inconsistent names. For each student keep the latest draft that is complete (has a 'Conclusion' heading and ≥1,500 words); move all others to ~/Essays/Superseded.
- Platforms
mac· macOSwin· Windows- Applications
- Word · Finder · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
nav· Pan/zoom/scrollcode· Code & commands- Work product
WP09· Files & folders- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Kept set equals 9 expected files (student identity from document author/cover, not filename; latest by in-document date, mtimes shuffled); Superseded has the remaining 21.
Proposed partial credit
80% correct keeper per student; 20% all others in Superseded
R01.02.01.03-T19 Quarantine files with personal data on a team share
Original instruction
Scan D:\TeamShare (900 synthetic files: docx, xlsx, pdf, jpg) and move any file containing personal data (US SSN patterns, passport scans, bank account numbers with names) into D:\Restricted, preserving relative paths. Write moved.csv with path and data type.
- Platforms
win· Windows- Applications
- File Explorer · PowerShell · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentcode· Code & commands- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Restricted set vs ground truth (47; includes 9 image-only passport scans; decoys: invalid SSN-like numbers, sample/test IBANs in docs); moved.csv data-type labels accuracy; relative paths preserved.
Proposed partial credit
60% F1 on restricted set; 20% image-only scans caught; 20% moved.csv labels
R01.02.01.03-T20 Curate a 100-photo print set from a family year
Original instruction
From ~/Family2025 (500 photos) remove near-duplicates, screenshots/memes, blurry and closed-eye shots, then pick the best 100 with 7–10 photos per month and at least 3 photos of each of the 4 family members (reference faces in ~/ref). Copy them to ~/Print.
- Platforms
mac· macOSwin· Windows- Applications
- Photos · Finder · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentcode· Code & commandsnav· Pan/zoom/scroll- Work product
WP09· Files & folders- Conditions
long· Long-horizonpersonal· Personal context- Proposed difficulty
- 5 / 5
Proposed verifier
~/Print: exactly 100 files; monthly counts 7–10; ≥3 per member (face match vs ref); zero members of the reject classes (labeled); quality score mean ≥ annotator median.
Proposed partial credit
20% count; 20% month balance; 20% member coverage; 40% no rejects and quality bar
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.