Reimplementation from behavior
Rebuild a program from its binary, docs or observed behavior in a clean room.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- compilers · fuzzers · Docker
- Class GUI-need label
CL· CLI-sufficient
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R02.01.02.02-T01 Clean-room rebuild of a jq-like JSON query tool from its binary
Original instruction
`/work/jqx/bin/jqx` is a stripped binary of a jq-like tool with docs in `docs/manual.md`. In a clean room (no internet, no jq source), write `jqx` in any language that reproduces its stdout, stderr and exit codes for any filter and input, including error messages and number formatting.
- Platforms
lin· Linux- Applications
- cleanroom Docker · fuzzing harness
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Hidden differential suite: >200k fuzzed filter/input pairs compared on stdout, stderr and exit code; score = match fraction; build must not embed or exec the reference binary (strace/exec audit, binary hash scan).
Proposed partial credit
Score scaled by fraction of matching cases: 60% stdout, 20% exit codes, 20% stderr
R02.01.02.02-T02 Reverse-engineer a custom compression format and write a compatible compressor
Original instruction
`/work/squish/bin/unsquish` decompresses `.sq` files; there is no compressor and no spec. Sample `.sq` files and their originals are in `samples/`. Write `squish` that produces files `unsquish` decodes back to the original input, with compression ratio within 10% of the samples' ratio on text.
- Platforms
lin· Linux- Applications
- xxd · gdb · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden corpus (300 files, text/binary/empty/1-byte/large): unsquish(squish(x)) == x for all; mean ratio on text within 10% of reference; unsquish exits 0 and validates checksums.
Proposed partial credit
60% round-trip correctness (scaled); 25% checksum/header validity; 15% compression ratio
R02.01.02.02-T03 Reproduce the expression semantics of a closed CLI calculator
Original instruction
`/work/calcx/bin/calcx` evaluates arithmetic expressions with undocumented precedence quirks, implicit multiplication, percent semantics and rounding to 'display precision'. Probe it and write `calcx` (any language) matching outputs and error messages exactly.
- Platforms
lin· Linuxmac· macOS- Applications
- fuzzing harness
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden differential test on 50k generated expressions (incl. malformed ones): exact stdout and exit code match fraction; reference binary not called by the solution (exec audit).
Proposed partial credit
Scaled by match fraction: 70% valid expressions, 30% error cases
R02.01.02.02-T04 Recreate a wc-like utility including all flags and edge cases
Original instruction
`/work/wcx/bin/wcx` behaves like `wc` with extra flags (`-L`, `--files0-from`, `--total=auto|always|never`) and locale-aware multibyte counting. Implement `wcx` in C or Rust that matches its output formatting and column widths exactly.
- Platforms
lin· Linuxmac· macOS- Applications
- gcc · cargo
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Hidden differential tests: 2,000 file sets × flag combinations under C.UTF-8 and en_US.UTF-8; stdout, stderr and exit code exact; invalid UTF-8 inputs included.
Proposed partial credit
60% counts correct; 25% formatting/widths; 15% error handling
R02.01.02.02-T05 Write a save-file writer for a game from sample saves
Original instruction
`/work/savefmt/samples/` holds 40 save files from a small open-source-licensed game with matching JSON dumps of the in-game state (no format docs). Write `encode_save(json) -> bytes` producing files the game's own loader (`bin/loadcheck`) accepts and decodes to the same state, including its checksum.
- Platforms
lin· Linuxwin· Windows- Applications
- Python · ImHex · xxd
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden 200 JSON states (incl. fields absent from samples' common cases): loadcheck exits 0 and its JSON dump equals input; byte-identical output to the game's own writer for 50 states scores a bonus milestone.
Proposed partial credit
30% header/checksum; 50% loadcheck acceptance + state equality; 20% byte-identical subset
R02.01.02.02-T06 Implement a client for an undocumented protocol from packet captures
Original instruction
`/work/telemetry/captures/*.pcapng` contain sessions between a closed client and a sandbox server (custom binary TCP protocol with handshake, framing, heartbeats). Write a client library + CLI that can authenticate with the test key, upload readings and fetch configs from the sandbox server.
- Platforms
lin· Linux- Applications
- Wireshark · tshark · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden tests against the sandbox server: handshake accepted; 500 readings uploaded and visible in server DB; config fetch equals server state; heartbeats keep a 5-minute session alive; malformed frames never sent (server strict log).
Proposed partial credit
30% handshake; 30% uploads; 20% config fetch; 20% session keepalive
R02.01.02.02-T07 Reimplement a COBOL claims-rating program's logic in Java
Original instruction
`/work/rating-cbl/` has a compiled GnuCOBOL program `RATECLM` (source withheld) plus a user guide and 20 sample input/output pairs. Reimplement its rating logic as a Java 21 library + CLI with byte-identical output records for any valid input, discovering rounding and table-lookup rules by probing.
- Platforms
lin· Linux- Applications
- GnuCOBOL · Java · Maven
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden differential test on 5,000 generated claim records: byte-identical output fraction; solution does not exec the COBOL binary (exec audit); mvn test green.
Proposed partial credit
Scaled by match fraction: 70% standard records, 30% edge records (caps, ties, missing tables)
R02.01.02.02-T08 Recover a checksum algorithm from a stripped ELF and reimplement it
Original instruction
`/work/fwtool/bin/fwsign` appends a 32-bit checksum to firmware images. Disassemble or probe it (Ghidra and gdb are installed) to recover the algorithm, then implement `fwsum` in Python that produces identical checksums for any input.
- Platforms
lin· Linux- Applications
- Ghidra · gdb · Python
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Hidden 1,000 random inputs (0 B-8 MB): checksum equality; no invocation of fwsign by the solution; algorithm description in NOTES.md (keyword check for polynomial/init/xorout).
Proposed partial credit
80% checksum match fraction; 20% notes
R02.01.02.02-T09 Reproduce a Windows loan calculator's amortization from its GUI
Original instruction
`LoanPro.exe` is a closed Windows GUI calculator (no CLI, no docs). Explore it on the VM by entering values and reading the schedule grid, then write a Python function `schedule(principal, rate, years, freq, extra)` that reproduces its amortization table exactly, including its rounding and final-payment adjustment.
- Platforms
win· Windows- Applications
- LoanPro (closed app) · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
type· Text & formsvis· Visual judgmentcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden 300 parameter sets compared to schedules exported by the grader through UI automation: every row (payment, interest, principal, balance) equal to the cent; the app's binary is obfuscated and has no export, so probing must go through its GUI.
Proposed partial credit
Scaled by exact-row fraction: 50% regular rows, 30% final-payment adjustment, 20% extra-payment cases
R02.01.02.02-T10 Rebuild a black-box REST API from observed responses
Original instruction
A sandbox inventory API runs at `http://legacy.local:8080` (no docs, no source). By exercising it, reimplement a compatible service (any stack) that returns the same status codes, JSON bodies, headers that matter (ETag, Location, pagination Link) and validation errors for the endpoints you can discover.
- Platforms
lin· Linux- Applications
- curl · Docker
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden differential tests (1,500 recorded request sequences replayed against both): status, normalized JSON and key headers match fraction; unknown endpoints count too (14 total, 3 undocumented by any link).
Proposed partial credit
Scaled by match fraction: 60% bodies, 25% status codes, 15% headers
R02.01.02.02-T11 Recreate a Python package's source from bytecode-only distribution
Original instruction
`/work/geofence-pyc/` contains only `.pyc` files (Python 3.11) of an internal package whose source was lost. Recreate readable source with equivalent behavior and the same public API, plus docstrings where you can infer intent.
- Platforms
lin· Linuxmac· macOS- Applications
- Python · dis · pytest
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Hidden behavioral tests (300) pass when importing the agent's source instead of the pyc; public names/signatures identical (inspect); solution contains no marshal/pyc loading.
Proposed partial credit
70% behavior; 20% API identical; 10% no pyc loading
R02.01.02.02-T12 Reimplement a CLI tool's tokenizer and error reporting
Original instruction
`/work/cfglint/bin/cfglint` lints a custom config language and prints tokens with `--tokens`. Using only the binary and the 2-page user doc, reimplement its tokenizer and diagnostics (line:col, messages, exit codes) in TypeScript, matching output for any input.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Node · fuzzing harness
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden differential test: 30k fuzzed config files; --tokens output and diagnostics exact match fraction; CRLF and Unicode identifiers included.
Proposed partial credit
Scaled: 60% token streams, 40% diagnostics
R02.01.02.02-T13 Reproduce a sort utility's locale collation and options exactly
Original instruction
`/work/sortx/bin/sortx` is a sort utility with numeric, version, month and human-size modes, `-k` keys, `-t` separators and stable/unique options, using ICU collation for a given locale. Implement `sortx` matching output for any input and options.
- Platforms
lin· Linux- Applications
- Go · ICU
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden differential tests: 10k generated inputs × option sets × locales (en_US, de_DE, sv_SE, ja_JP); exact output fraction.
Proposed partial credit
Scaled: 50% key/mode handling, 30% locale collation, 20% stable/unique semantics
R02.01.02.02-T14 Recreate a compiled path tracer's C source under a size limit
Original instruction
`/work/pt/bin/pathtrace` renders a fixed scene to `out.ppm` deterministically given a seed. Write C source (at most 6 KB after gzip) that, when compiled with the provided flags, produces images matching the binary's output for any seed and resolution flag.
- Platforms
lin· Linux- Applications
- gcc · Ghidra · gdb
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Hidden 20 seeds × 3 resolutions: per-image PSNR >= 45 dB vs reference (bit-exact earns full); gzip size <= 6,144 bytes; solution does not embed the binary or images.
Proposed partial credit
60% PSNR threshold fraction; 25% bit-exact fraction; 15% size limit
R02.01.02.02-T15 Reimplement a macOS menu-bar timer app's behavior from the GUI
Original instruction
`FocusBar.app` is a closed macOS menu-bar Pomodoro app with undocumented rules (long breaks, skipping, pause carry-over, notifications). Observe it through its menu and notifications, then write a Swift package `FocusCore` whose state machine reproduces its event timeline for any sequence of user actions.
- Platforms
mac· macOS- Applications
- FocusBar (closed app) · Swift
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
click· Point & clickvis· Visual judgmentcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden 400 action scripts replayed on the real app (accelerated clock via test hook) vs FocusCore: emitted event sequence (state, timestamps ±1 s, notifications) equal; app internals are obfuscated, so behavior must be probed through the menu bar.
Proposed partial credit
Scaled by matching scripts: 50% core cycle, 30% pause/skip rules, 20% notifications
R02.01.02.02-T16 Port a DOS inventory program's logic by observing it in DOSBox
Original instruction
`/work/dosinv/INV.EXE` is a 1990s DOS inventory program run in DOSBox-X (provided). It reads `STOCK.DAT` and prints reorder reports through a text UI. By running it in the emulator, recover its report logic and the `STOCK.DAT` record format, and implement `inv_report.py` producing identical report text.
- Platforms
lin· Linuxwin· Windows- Applications
- DOSBox-X · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
key· Shortcutsvis· Visual judgmentcode· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Hidden 100 STOCK.DAT variants: report text equal to what INV.EXE prints (captured via emulator printer-to-file); solution doesn't run DOSBox. Partial on per-line match.
Proposed partial credit
30% record format parsed; 50% report lines match (scaled); 20% edge cases (empty, overflow)
R02.01.02.02-T17 Re-create a Java report generator from its JAR behavior in Kotlin
Original instruction
`/work/reportgen/reportgen.jar` turns order CSVs into fixed-layout text reports. Without copying decompiled code (decompiling to read logic is allowed), write a Kotlin CLI producing byte-identical reports, including page breaks, totals and locale-specific number formatting.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- Kotlin · Gradle · CFR
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden 500 CSV inputs: byte-identical report fraction; code-similarity check against decompiled source (token n-gram overlap < 30%); no JAR on classpath.
Proposed partial credit
Scaled: 70% identical reports; 15% page-break edge cases; 15% similarity check passes
R02.01.02.02-T18 Identify and reimplement a PRNG from its output stream
Original instruction
`/work/rng/bin/rng <seed> <n>` prints n 32-bit numbers. Determine the generator algorithm and parameters from outputs and/or the binary, then implement `rng.py` producing identical sequences for any 64-bit seed, plus `predict.py` that recovers the seed from the first 4 outputs.
- Platforms
lin· Linux- Applications
- Python · gdb · z3
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden 1,000 seeds × 10k outputs: identical sequences; predict.py recovers seeds for 100 hidden streams within 10 s each.
Proposed partial credit
60% sequence reproduction; 40% seed recovery
R02.01.02.02-T19 Turn a macro-driven Excel pricing workbook into a Python library
Original instruction
`/work/quote.xlsm` is a pricing workbook whose results come from VBA macros and hidden sheets. Reimplement the quote calculation as a Python function taking the input cells' values and returning the same outputs as the workbook, documenting each rule in `RULES.md`.
- Platforms
lin· Linuxwin· Windowsmac· macOS- Applications
- Python · olevba · openpyxl
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden 1,000 input vectors evaluated in Excel (grader-side) vs the function: output equality (to the cent) fraction; no Excel/COM/LibreOffice calls in the solution; RULES.md non-empty.
Proposed partial credit
Scaled: 80% matching outputs; 20% rules documented
R02.01.02.02-T20 Reimplement a make-like build tool's dependency semantics
Original instruction
`/work/mk/bin/mk` is a make-like tool (pattern rules, automatic variables, order-only prerequisites, `.PHONY`, timestamps, `-j`). Implement `mk` so that, for any Mkfile and file tree, the set and partial order of executed commands and the final file states match the reference.
- Platforms
lin· Linux- Applications
- Go · Rust
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden 400 generated Mkfile + tree scenarios (incl. touch-then-rebuild): executed command multiset equal, dependency order respected, exit codes equal; -j8 runs respect order; no calls to the reference binary.
Proposed partial credit
40% command sets; 30% ordering; 20% rebuild after touch; 10% exit codes
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.