E-books & reading
Organize libraries, highlights and notes in reading apps.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Kindle · Apple Books · Calibre
- Class GUI-need label
G2· GUI-essential
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R12.07.01.03-T01 Create a Calibre collection by tag and set series metadata
Original instruction
In Calibre (desktop), add the tag 'book-club-2026' to the 5 books listed in fixtures/club.txt (match by title and author; two titles have multiple editions — choose the EPUB edition), and set series 'Book Club' with indices 1-5 in list order.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Calibre metadata.db: the 5 correct book ids carry the tag; series 'Book Club' with series_index 1-5 in order; no other books tagged. The agent has no terminal, and calibredb, calibre-debug and ebook-meta are removed from PATH, so metadata.db can only be changed through the Calibre GUI.
Proposed partial credit
40% correct books tagged; 40% series indices; 20% no extra tags.
R12.07.01.03-T02 Highlight a passage and add a note in Apple Books
Original instruction
In Apple Books on macOS, open 'Pride and Prejudice' (public-domain EPUB in the library), highlight in yellow the sentence beginning 'It is a truth universally acknowledged' (the full sentence), and add the note 'Opening irony'. Then add a bookmark on the first page of Chapter 3.
- Platforms
mac· macOS- Applications
- Apple Books
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
path· Continuous pathclick· Point & clicktype· Text & forms- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
AEAnnotation SQLite: highlight with CFI range covering exactly that sentence (±2 chars), style yellow, note text; bookmark annotation at Chapter 3 start location. The agent has no Terminal or sqlite3, so the Books library and annotation databases are not reachable for direct edits; changes must be made in the Books app.
Proposed partial credit
50% highlight range; 20% color; 15% note; 15% bookmark.
R12.07.01.03-T03 Convert an EPUB to Kindle format with a custom cover and send to a device folder
Original instruction
In Calibre, convert 'field_notes.epub' to AZW3 with the cover replaced by fixtures/cover.png, a generated table of contents from h2 headings, and margins of 5 pt, then save the AZW3 to the connected simulated Kindle (mounted folder /Volumes/Kindle/documents).
- Platforms
mac· macOSwin· Windows- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
AZW3 in device folder: cover image perceptual hash matches cover.png; NCX/TOC entries equal the 12 h2 headings; conversion log options margins=5. ebook-convert binary disabled outside the GUI.
Proposed partial credit
30% conversion to AZW3; 25% cover; 30% TOC; 15% on device.
R12.07.01.03-T04 Export highlights from a reading app and compile them into a note
Original instruction
In the Kindle for PC/Mac app (test account with sample annotations), export the notebook for 'Thinking in Systems' and create a Markdown file systems_notes.md listing every highlight (in book order) with its location and any attached note.
- Platforms
mac· macOSwin· Windows- Applications
- Kindle for PC/Mac
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
click· Point & clickmulti· Cross-app- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
systems_notes.md contains all 23 seeded highlights (fuzzy text match >= 0.95), locations exact, notes attached to the right highlight, order by location. The agent has no terminal and the web notebook is not reachable from the sandbox, so highlights must come from the Kindle app's Notebook export.
Proposed partial credit
50% highlights present; 20% locations; 20% notes; 10% order.
R12.07.01.03-T05 Fix wrong metadata and covers for a batch of books using a source download
Original instruction
In Calibre, 8 books imported from PDFs have filenames as titles and no authors. Use 'Download metadata' for each, but verify against the title page (first page preview) and reject wrong matches; set title, author, publisher, published date, and cover from the selected source.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentclick· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
metadata.db fields vs ground truth for 8 books (title, author exact; publisher/date per seeded source); covers present. Two seeded wrong-match traps (same title, different author). The agent has no terminal, and calibredb, calibre-debug and ebook-meta are removed from PATH, so metadata.db can only be changed through the Calibre GUI.
Proposed partial credit
50% title/author (per book); 20% publisher/date; 20% covers; 10% traps avoided.
R12.07.01.03-T06 Organize Apple Books into collections and mark finished books
Original instruction
In Apple Books on macOS, create collections 'Work' and 'Fiction 2026', move the 6 work-related PDFs and 9 novels accordingly (by content: check titles), mark the 4 novels at 100% progress as Finished, and remove the sample 'Untitled.pdf' from the library.
- Platforms
mac· macOS- Applications
- Apple Books
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
drag· Drag & dropclick· Point & clickvis· Visual judgment- Work product
WP09· Files & folders- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
BKLibrary SQLite: collection memberships exact; finished flags on the 4; Untitled.pdf removed; nothing else removed. The agent has no Terminal or sqlite3, so the Books library and annotation databases are not reachable for direct edits; changes must be made in the Books app.
Proposed partial credit
50% collection membership; 25% finished flags; 25% removal correct only.
R12.07.01.03-T07 Adjust reading layout for low vision in an e-reader app
Original instruction
In KOReader (Linux desktop build or Android emulator), open 'Moby Dick.epub' and set font 'Atkinson Hyperlegible' (installed), font size 26, line spacing 130%, margins narrow, dark mode with sepia-inverted night colors off, and make these the book's defaults only (not global).
- Platforms
lin· Linuxand· Android- Applications
- KOReader
- Interfaces
desk· Desktop GUImob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktouch· Touch gestures- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Book's .sdr metadata.lua: font face, size, line spacing, margins, night mode values; global settings.reader.lua unchanged. The agent has no terminal or adb shell, so metadata.lua and settings.reader.lua are not reachable for direct edits; settings must be made in KOReader's menus.
Proposed partial credit
60% per-book settings (equal weight per setting); 40% globals untouched.
R12.07.01.03-T08 Find a quote across a library with full-text search and annotate it
Original instruction
In Calibre's full-text search, find the book in the library containing the exact phrase 'the map is not the territory', open it in Calibre's viewer at that location, highlight the phrase in green with the note 'Korzybski', and report the book title and chapter.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formspath· Continuous pathvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Calibre annotations table: highlight in the correct book with selected text equal to the phrase, style green, notes 'Korzybski'; report title/chapter exact. The agent has no terminal, and calibredb and calibre-debug are removed from PATH, so search and annotations must be done in Calibre's GUI and viewer.
Proposed partial credit
30% correct book; 40% highlight range + color; 15% note; 15% report.
R12.07.01.03-T09 Lend a library ebook and return it early in a library app
Original instruction
In the Libby-style library app clone on the iOS Simulator, borrow 'Educated' (14-day loan) for reading in the app, place a hold on 'Circe' (waitlist) with auto-borrow off, then return 'Old Man and the Sea' early. Report the hold position for Circe.
- Platforms
ios· iOS/iPadOS- Applications
- Libby-style app clone (iOS)
- Interfaces
mob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesclick· Point & click- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Library backend: active loan Educated 14 days; hold on Circe with auto_borrow=false; loan for Old Man returned; reported position equals DB. The agent drives only the Simulator window: it has no shell on the host (simctl and the app container are not reachable) and no API token, so loans and holds must be made in the app.
Proposed partial credit
30% loan; 30% hold settings; 25% early return; 15% position.
R12.07.01.03-T10 Merge duplicate books in Calibre keeping all formats
Original instruction
In Calibre, 7 books exist twice (one EPUB-only record and one PDF-only record each). Merge each pair into one record keeping both formats, keep the metadata from the record with an ISBN, and do not merge 'Dune' and 'Dune Messiah' (similar titles).
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickvis· Visual judgment- Work product
WP09· Files & folders- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
metadata.db: 7 merged records each with EPUB and PDF formats and the ISBN-bearing metadata; Dune and Dune Messiah remain separate; total count reduced by 7. The agent has no terminal, and calibredb, calibre-debug and ebook-meta are removed from PATH, so metadata.db can only be changed through the Calibre GUI.
Proposed partial credit
60% correct merges; 20% metadata choice; 20% decoys untouched.
R12.07.01.03-T11 Transfer reading progress and highlights between two e-reader apps
Original instruction
The user read 'Middlemarch' in Foliate (Linux) to 47% with 9 highlights. Open the same EPUB in KOReader (Linux build) and recreate the 9 highlights at the same passages, then set KOReader's position to the same location.
- Platforms
lin· Linux- Applications
- Foliate · KOReader
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
multi· Cross-apppath· Continuous pathnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
KOReader metadata.lua: 9 highlights whose text equals Foliate's annotations (fuzzy >= 0.95); last position within 0.5% of Foliate's CFI location. The agent has no terminal, so KOReader's metadata.lua is not reachable for direct writes; highlights and position must be set in KOReader.
Proposed partial credit
70% highlights (per highlight); 30% position.
R12.07.01.03-T12 Edit an EPUB's table of contents and fix a broken chapter in Calibre's editor
Original instruction
In Calibre's 'Edit book' tool, fix 'handbook.epub': the TOC is missing chapters 7-9 and chapter 8's heading is styled as body text. Regenerate the TOC from h1/h2, set chapter 8's heading to h1, and run the built-in checker until no errors remain. Save.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
EPUB: nav/NCX contains all 12 chapters in order; chapter 8 heading tag h1; epubcheck reports 0 errors; body text otherwise unchanged (diff). The agent has no terminal, and calibre-debug and ebook-polish are removed from PATH, so the EPUB must be edited and saved in Calibre's Edit book tool.
Proposed partial credit
40% TOC complete; 25% heading fix; 25% checker clean; 10% no other changes.
R12.07.01.03-T13 Set up a reading goal and track a book's progress manually
Original instruction
In the Goodreads-style web clone, set the 2026 reading challenge to 30 books, mark 'The Overstory' as currently reading at page 212 of 502, add it to shelf 'nature', and rate the already-finished 'Piranesi' 4 stars with a 2-sentence review.
- Platforms
web· Any OS (web)- Applications
- Goodreads-style clone
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 1 / 5
Proposed verifier
DB: challenge goal 30; reading status with page 212; shelf membership; Piranesi rating 4 and review with exactly 2 sentences. The clone issues no API token and accepts writes only from the browser session (CSRF + session id), so scripted requests are rejected.
Proposed partial credit
20% goal; 30% progress; 20% shelf; 30% rating/review.
R12.07.01.03-T14 Annotate a PDF textbook chapter with precise highlights and margin notes
Original instruction
In the Kindle for PC/Mac app or Apple Books, open 'stats_ch4.pdf' and highlight the 5 definitions in Section 4.2 (each is a bolded term plus its definition sentence), adding a note to each with your own one-line paraphrase. Do not highlight examples.
- Platforms
mac· macOSwin· Windows- Applications
- Apple Books · Kindle for PC/Mac
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
path· Continuous pathtype· Text & formsvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Annotation DB: 5 highlights whose text overlaps each ground-truth definition span with IoU >= 0.8 (character offsets), each with a note; no highlights on example spans. The agent has no terminal or sqlite3, so the annotation databases are not reachable for direct edits; highlights must be made in the reader app.
Proposed partial credit
60% definition spans (equal weight per span); 25% notes; 15% no examples highlighted.
R12.07.01.03-T15 Send a set of articles to an e-reader as one compiled ebook
Original instruction
Use Calibre's 'Fetch news' custom recipe GUI to create a recipe 'Weekend Reads' from the RSS feed http://feeds.local/longreads.xml limited to the 5 newest articles, download it, then send the resulting ebook to the simulated Kobo device (mounted) in EPUB format.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Custom recipe saved with feed URL and max articles 5; library has a news book with the 5 newest article titles; device folder contains its EPUB. The agent has no terminal, and ebook-convert and calibredb are removed from PATH, so the recipe, download and device send must be done in the Calibre GUI.
Proposed partial credit
30% recipe; 40% book content; 30% sent to device.
R12.07.01.03-T16 Look up and save vocabulary words while reading in a mobile e-reader
Original instruction
In the Kindle app clone on the Android emulator, while reading 'Bleak House' chapter 1, look up the words 'implacable', 'megalosaurus' and 'Michaelmas' with the built-in dictionary by long-pressing each, add them to Vocabulary Builder, and create flashcards deck 'Dickens'.
- Platforms
and· Android- Applications
- Kindle app clone
- Interfaces
mob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Vocabulary DB: 3 lookups with book id and chapter-1 locations; flashcard deck 'Dickens' containing exactly those 3 words. The agent gets screen input only (no adb shell, no backend API token), so lookups and the deck must be made in the app; the harness input log confirms UI taps.
Proposed partial credit
60% lookups (one third each); 40% deck.
R12.07.01.03-T17 Clean up a digital comics library: series, reading order and page fixes
Original instruction
In Calibre (or YACReader on Windows), import 12 CBZ issues of 'Night Harbor' with messy filenames, set series and issue numbers from the cover images (issue number printed on each cover), fix issue 7 where pages 10 and 11 are swapped, and set reading order by issue.
- Platforms
win· Windowsmac· macOS- Applications
- Calibre · YACReader
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentdrag· Drag & dropclick· Point & click- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Library metadata: 12 records with series index from cover numbers (ground truth); issue 7 CBZ page order corrected (image hash sequence). The agent has no terminal and calibredb is removed from PATH, so metadata and the CBZ page order must be fixed in Calibre or YACReader.
Proposed partial credit
60% issue numbers; 30% page swap fixed; 10% series name.
R12.07.01.03-T18 Prepare an accessible audiobook-style reading with text-to-speech settings
Original instruction
In Apple Books on the iPad Simulator, open 'Treasure Island', start Speak Screen-based reading from the start of Chapter 2, set speaking rate to 1.2x via the speech controller, and set Books' auto-hyphenation off and 'Justify text' off for readability. Report the first sentence of Chapter 2.
- Platforms
ios· iOS/iPadOS- Applications
- Apple Books · iOS Accessibility
- Interfaces
mob· Mobile touch- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G3· GUI-only- Interaction primitives
touch· Touch gesturesvis· Visual judgment- Work product
WP08· App/system state- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Accessibility prefs: SpeakScreen enabled, rate 1.2; Books prefs hyphenation=false, justify=false; reported sentence exact. The agent has no host shell, so simctl and defaults writes to the simulator's prefs are not reachable; settings must be changed in Settings and Books.
Proposed partial credit
40% speech settings; 40% Books layout prefs; 20% sentence.
R12.07.01.03-T19 Build a research reading list with citations from highlighted ebooks
Original instruction
From the 4 Calibre books tagged 'thesis', export all highlights tagged with the colour purple into a single Markdown file thesis_quotes.md, each quote followed by a Chicago-style citation (author, title, publisher, year, page or location) using Calibre metadata. Sort by author.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Calibre
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickmulti· Cross-apptype· Text & forms- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
thesis_quotes.md: all 17 purple highlights (fuzzy >= 0.95), none of other colors; citations' fields match metadata.db; sorted by author surname.
Proposed partial credit
50% quotes; 30% citations; 20% sort/format.
R12.07.01.03-T20 Migrate a Kindle library to Calibre with collections and reading positions
Original instruction
Move the user's 40 DRM-free Kindle documents (personal docs folder on the simulated Kindle) into Calibre: import all, map the Kindle's 5 collections to Calibre tags, record last-read percentages from the Kindle's sidecar files into a custom column 'Progress', and make a virtual library 'Unfinished' for books under 100%.
- Platforms
mac· macOSwin· Windows- Applications
- Calibre · Finder · File Explorer
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
multi· Cross-appclick· Point & clicktype· Text & forms- Work product
WP09· Files & folders- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
metadata.db: 40 books; tags equal Kindle collection memberships; custom column #progress values equal sidecar percentages ±1; virtual library search expression yields exactly unfinished books.
Proposed partial credit
25% import; 25% tags; 30% progress values; 20% virtual library.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.