Manuscripts, LaTeX & citations
Fix and assemble manuscripts with figures, references and templates.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Overleaf · TeXstudio · Word
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R08.11.01.03-T01 Resolve [CITE] placeholders to existing BibTeX keys in TeXstudio
Original instruction
In TeXstudio open project input/paper/main.tex. Three '[CITE]' placeholders each have a LaTeX comment naming the cited paper's title. Replace each with a cite command using the matching key already in refs.bib (do not edit refs.bib), then compile with pdfLaTeX+BibTeX so the reference list shows the 3 entries.
- Platforms
mac· macOSwin· Windows- Applications
- TeXstudio
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Checker diffs main.tex: only the 3 placeholder lines changed, each now citing the gold key; refs.bib hash unchanged; latexmk compile succeeds with no undefined citations and the bbl lists the 3 keys.
Proposed partial credit
60% correct keys (three, equal shares); 20% compile clean; 20% no other edits
R08.11.01.03-T02 Fix six compile errors in an Overleaf manuscript
Original instruction
The Overleaf project 'MicrobiomeLetter' (local Overleaf CE instance, test account) fails to compile: a missing $ in a formula, a command from a removed package, unbalanced braces, a figure path with wrong case, a duplicated label and a malformed BibTeX field. Fix all errors in the Overleaf editor without removing content.
- Platforms
web· Any OS (web)- Applications
- Overleaf
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsnav· Pan/zoom/scrollclick· Point & click- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker downloads the project zip via the admin API and compiles with the same TeX Live: 0 errors, 0 undefined references, no duplicate-label warning; diff limited to the 6 faulty spots (+-2 lines); word count of body text within 1% of original.
Proposed partial credit
60% six errors fixed (equal shares); 20% clean compile; 20% content preserved
R08.11.01.03-T03 Port a manuscript from the article class to the IEEEtran template
Original instruction
input/draft/ is a paper in the generic article class. Move it into the provided IEEEtran journal template (input/ieee_template/): title, 4 authors with affiliations and IEEE membership, abstract, index terms, sections, 5 figures, 2 tables, and IEEEtran bibliography style. Produce output/ieee/main.tex compiling to output/ieee/main.pdf.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- TeXstudio · Overleaf
- Interfaces
desk· Desktop GUIbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsnav· Pan/zoom/scrollmulti· Cross-app- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Compile succeeds; documentclass IEEEtran journal; pdffonts shows Times; PDF text contains all section headings, figure/table captions and 4 authors; references use IEEEtran numeric style (bst check); body text similarity to draft >=0.98.
Proposed partial credit
20% compiles with template; 25% front matter (authors, abstract, index terms); 25% figures/tables; 30% bibliography style and text preserved (equal halves)
R08.11.01.03-T04 Convert typed author-year citations into live Zotero fields in Word
Original instruction
input/draft.docx contains 14 manually typed citations like '(Smith et al., 2021)'. Using the Zotero Word plugin and the Zotero library collection 'Thesis', replace each with a live Zotero citation for the correct item, switch the style to Vancouver, and insert the bibliography at the 'References' heading. Save output/draft_cited.docx.
- Platforms
mac· macOSwin· Windows- Applications
- Word · Zotero
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Checker parses docx field codes: 14 ZOTERO_ITEM CSL_CITATION fields whose item URIs match gold; no residual typed citations (regex); document style is Vancouver; bibliography field present with 14 entries in citation order. Word VBA macros are disabled by policy and the VM has no shell or Python, so field codes cannot be scripted; docx writes by any process other than Word are caught by file-access auditing.
Proposed partial credit
50% citations matched (proportional); 15% no typed residue; 15% style; 20% bibliography
R08.11.01.03-T05 Lay out two side-by-side subfigures with working cross-references
Original instruction
In Overleaf project 'GrowthPaper', Figure 2 currently stacks two images (figs/ctrl.pdf, figs/treat.pdf). Make them side-by-side subfigures with the subcaption package, each 0.48 linewidth, subcaptions '(a) Control' and '(b) Treated', labels fig:ctrl and fig:treat, and update the two text references that point to them.
- Platforms
web· Any OS (web)- Applications
- Overleaf
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsnav· Pan/zoom/scroll- Work product
WP03· Visual layout- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Compiled PDF: pdfplumber finds both images on the same page with equal y-positions (within 2 pt) and widths 0.48 linewidth (+-2%); subcaptions text present; refs resolve to '2a'/'2b' style; no '??'.
Proposed partial credit
40% side-by-side geometry; 25% subcaptions; 25% labels and refs resolve; 10% compile clean
R08.11.01.03-T06 Apply a journal's manuscript formatting rules in Word
Original instruction
Format input/manuscript.docx per input/author_guidelines.pdf: Times New Roman 12 pt, double spacing, 2.5 cm margins, continuous line numbering, page numbers bottom centre, Heading 1/Heading 2 styles applied to section and subsection titles (currently bold body text). Save output/manuscript_formatted.docx.
- Platforms
mac· macOSwin· Windows- Applications
- Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
docx XML checks: Normal style font/size/spacing; sectPr margins 1418 twips; lnNumType restart continuous; footer page field centred; the 9 gold headings use Heading1/Heading2 styles; text content unchanged. Word VBA macros are disabled by policy and the VM has no shell or Python; docx writes by any process other than Word are caught by file-access auditing and fail.
Proposed partial credit
20% font and spacing; 15% margins; 30% line numbers and page numbers (equal halves); 35% heading styles
R08.11.01.03-T07 Typeset a CSV table with booktabs and decimal-aligned columns
Original instruction
Convert input/results.csv (12 rows x 6 columns, mixed precision) into Table 3 of main.tex using booktabs rules, siunitx S columns aligned on the decimal point, caption above the table, units in the header row, and two table notes with threeparttable. Compile in TeXstudio.
- Platforms
mac· macOSwin· Windows- Applications
- TeXstudio
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formscode· Code & commands- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker extracts the table environment: booktabs rules present, numeric columns use S spec, caption before tabular; compiled PDF table cell text equals CSV values; decimal points aligned (x-coordinates of '.' within 0.5 pt per column).
Proposed partial credit
35% values correct; 25% S columns and decimal alignment; 20% booktabs and caption; 20% table notes
R08.11.01.03-T08 Apply reviewer edits and produce a latexdiff and response letter
Original instruction
Apply the 8 edits requested in input/reviews.pdf to the paper in input/v1/ (producing output/v2/), enable lineno, then generate output/diff.pdf with latexdiff between v1 and v2 and write output/response.tex answering each comment with the page and line numbers of the change in v2.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- TeXstudio · latexdiff
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formscode· Code & commandsmulti· Cross-app- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Checker verifies each of 8 edits in v2 via targeted text checks; diff.pdf compiles and contains latexdiff markup for each; response.tex compiles and each cited line number points to the changed text in v2 PDF (+-2 lines).
Proposed partial credit
40% 8 edits applied; 20% latexdiff PDF; 30% correct page/line references; 10% response compiles
R08.11.01.03-T09 Combine three co-authors' tracked changes in Word and resolve comments
Original instruction
Co-authors returned input/ms_A.docx, ms_B.docx and ms_C.docx with tracked changes against input/ms_base.docx. Use Word's Combine to merge them, accept all changes by A, accept B's changes only in Methods, reject all formatting-only changes, and reply 'Done' then resolve every comment. Save output/ms_merged.docx.
- Platforms
mac· macOSwin· Windows- Applications
- Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Final text compared with gold merged text (exact after whitespace normalization); no remaining w:ins/w:del/w:rPrChange; commentsExtended shows all comments done with a 'Done' reply by the user account. Word VBA macros are disabled by policy and the VM has no shell or Python; docx writes by any process other than Word are caught by file-access auditing and fail.
Proposed partial credit
45% accepted/rejected correctly (text match); 20% no pending revisions; 20% comment replies; 15% comments resolved
R08.11.01.03-T10 Clean a BibTeX file so biber reports no warnings
Original instruction
input/refs.bib has 60 entries with duplicates (same DOI, different keys), missing required fields, unprotected acronyms in titles and inconsistent month formats. Produce output/refs.bib with duplicates merged (keep the key used in main.tex), required fields filled from the provided input/crossref_dump.json, acronyms brace-protected, and update main.tex keys if needed.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- biber · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
biber --validate-datamodel on main.tex: 0 warnings; no two entries share a DOI; all cited keys resolve; 12 listed acronyms brace-protected; entry field values equal crossref_dump for the 15 incomplete entries.
Proposed partial credit
30% duplicates merged; 25% fields completed; 20% acronyms; 25% zero warnings and keys resolve
R08.11.01.03-T11 Replace eqnarray with align and number only referenced equations
Original instruction
In Overleaf project 'ModelPaper', convert all 7 eqnarray environments to align (keeping alignment points), and configure mathtools so only equations that are referenced in the text carry numbers. Keep all existing labels.
- Platforms
web· Any OS (web)- Applications
- Overleaf
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Source has no eqnarray; showonlyrefs active; compiled PDF shows equation numbers only for the 5 referenced labels and references resolve; equation content unchanged (normalized TeX diff).
Proposed partial credit
40% eqnarray converted; 30% showonlyrefs numbering; 30% refs resolve and content unchanged
R08.11.01.03-T12 Retype equation images as native Word equations
Original instruction
input/thesis_ch2.docx contains 10 equations pasted as images (originals in input/eq_images/). Replace each with a native Word equation (OMML) that reproduces it exactly, keeping the equation numbers right-aligned with the existing '(2.x)' scheme. Save output/thesis_ch2.docx.
- Platforms
mac· macOSwin· Windows- Applications
- Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsclick· Point & clickvis· Visual judgment- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Checker extracts OMML, converts to LaTeX and compares with gold after normalization (symbol-level match); no inline images remain in equation paragraphs; numbering text preserved. Word VBA macros are disabled by policy and the VM has no shell or Python to inject OMML; docx writes by other processes are caught by file-access auditing and fail.
Proposed partial credit
70% equations matched (equal share each); 15% images removed; 15% numbering kept
R08.11.01.03-T13 Export a Zotero collection to BibTeX and cite it in a LaTeX draft
Original instruction
In Zotero, export collection 'Chapter3' with Better BibTeX (keys authorTitleYear) to input/thesis/ch3.bib, then in TeXstudio insert citations at the 9 places marked '%CITE: <title>' in ch3.tex and compile the thesis with biblatex/biber.
- Platforms
mac· macOSwin· Windows- Applications
- Zotero · TeXstudio
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsmulti· Cross-app- Work product
WP01· Document- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
ch3.bib contains exactly the 23 items of the collection with Better BibTeX keys; 9 cite commands use gold keys; biber run has no missing entries.
Proposed partial credit
35% export correct; 45% nine citations (equal shares); 20% compile clean
R08.11.01.03-T14 Bring a conference paper under the 9-page limit without format hacks
Original instruction
Overleaf project 'NeurIPS-sub' compiles to 9.4 pages of main text; the limit is 9 pages before references. Move Figure 4 and its paragraph to the appendix, tighten the two paragraphs marked '% TIGHTEN' by at least 25% each, and fix widows. Do not change margins, font sizes, spacing commands or the style file.
- Platforms
web· Any OS (web)- Applications
- Overleaf
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsnav· Pan/zoom/scrollvis· Visual judgment- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Compiled PDF: references start on page <=10 with main text ending on page 9; style file hash unchanged; no negative vspace, savetrees, or font-size commands added; Figure 4 in appendix; marked paragraphs reduced >=25% words with key claims kept (LLM rubric 20%).
Proposed partial credit
35% page limit met; 20% no format hacks; 15% figure moved; 30% paragraphs tightened (two-thirds rubric)
R08.11.01.03-T15 Replace hard-coded figure numbers with cross-reference fields in Word
Original instruction
In input/report.docx the figures were reordered, so text like 'see Figure 3' is now wrong. Insert captions with Word's caption numbering where missing, then replace all 18 hard-coded figure and table references with cross-reference fields to the right captions. Save output/report_xref.docx.
- Platforms
mac· macOSwin· Windows- Applications
- Word
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
docx: all captions use SEQ fields; 18 REF fields point to bookmarks on the correct captions (per gold mapping); updated field results show correct numbers; no plain-text 'Figure N' references remain in body. Word VBA macros are disabled by policy and the VM has no shell or Python to inject fields; docx writes by other processes are caught by file-access auditing and fail.
Proposed partial credit
25% captions with SEQ fields; 55% references correct (proportional); 20% no plain-text leftovers
R08.11.01.03-T16 Anonymize a LaTeX submission for double-blind review
Original instruction
Prepare output/anon/ from input/camera/ for double-blind review: remove author names and affiliations, acknowledgments and funding, rephrase 3 self-citations ('our previous work [12]') to third person, clear PDF metadata (author, creator) and rename figure files that contain a lab name. It must still compile.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- TeXstudio · Overleaf
- Interfaces
desk· Desktop GUIbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
PDF text contains none of the names/affiliations/grant numbers in a hidden list; pdfinfo Author empty and Creator not revealing; 3 self-citations rephrased (no 'our previous'); figure filenames free of lab name; compile succeeds.
Proposed partial credit
30% names removed; 20% acknowledgments removed; 20% self-citations; 30% metadata, filenames and compile (equal halves)
R08.11.01.03-T17 Assemble a thesis with chapters, front matter and a glossary
Original instruction
In input/thesis/ assemble main.tex from the university template: include the 6 chapter files with include, add title page, abstract, table of contents, list of figures and tables, and a glossary of the 15 terms in input/glossary.csv using the glossaries package (makeglossaries). Compile in TeXstudio to output/thesis.pdf.
- Platforms
mac· macOSwin· Windows- Applications
- TeXstudio
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formscode· Code & commands- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Compile chain succeeds; PDF TOC lists 6 chapters in order; LoF and LoT entries match captions; glossary section contains 15 terms with definitions; first-use expansions appear in text.
Proposed partial credit
25% chapters included in order; 25% front matter lists; 35% glossary; 15% clean compile
R08.11.01.03-T18 Convert a Word manuscript into a LaTeX journal project
Original instruction
Convert input/manuscript.docx (5 tables, 4 figures, 30 Zotero citations, 12 equations) into an Overleaf project from the journal template input/elsarticle_template.zip. Equations must be LaTeX math, citations cite commands backed by a refs.bib, tables in tabular, figures from the docx media exported at original resolution.
- Platforms
web· Any OS (web)- Applications
- Overleaf · Word · Zotero
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
type· Text & formsmulti· Cross-appcode· Code & commands- Work product
WP01· Document- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Compiled PDF text similarity to docx body >=0.95; 30 citations resolve to bib entries matching the Zotero items (DOI match); 12 equations compared symbol-level to OMML-derived gold; 5 tables cell-exact; 4 figures at native pixel size.
Proposed partial credit
20% compiles in template; 20% text; 20% citations; 40% equations, tables and figures (3:3:2)
R08.11.01.03-T19 Prepare a LaTeX project for arXiv submission
Original instruction
Clean input/paper/ for arXiv into output/arxiv.tar.gz: strip comments, remove unused figures and files, include the generated .bbl, convert 2 EPS figures to PDF, keep total size under 10 MB, and make sure it compiles with pdfLaTeX in a TeX Live 2023 container without network.
- Platforms
mac· macOSlin· Linux- Applications
- latexmk · arxiv-latex-cleaner
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP09· Files & folders- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker extracts the tarball in the container: compiles without errors; no % comments in .tex; no unreferenced files; .bbl present; tarball <10 MB; PDF page count and text equal original compile.
Proposed partial credit
30% compiles offline; 20% comments stripped; 20% unused files removed; 30% bbl, size and EPS conversion (equal halves)
R08.11.01.03-T20 Split supplementary material with S-numbered cross-references
Original instruction
input/paper/ has main text and supplement in one file. Split into main.tex and supp.tex: supplement figures, tables and equations numbered S1, S2..., main-text references to supplement items rendered as 'Fig. S3' via the xr package, and supplement references back to main figures. Both must compile; submit output/main.pdf and output/supp.pdf.
- Platforms
mac· macOSwin· Windowsweb· Any OS (web)- Applications
- TeXstudio · Overleaf
- Interfaces
desk· Desktop GUIbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formscode· Code & commandsnav· Pan/zoom/scroll- Work product
WP01· Document- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Both PDFs compile with no '??'; supplement captions numbered S1..Sn; each of the 14 cross-document references resolves to the correct label (checked against gold mapping); main text contains no supplement content.
Proposed partial credit
20% split files compile; 25% S-numbering; 40% cross-document refs (proportional); 15% content separated
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.