Vulnerability reproduction & patching
Reproduce a known vulnerability in a sandbox and patch it.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- fuzzers · sanitizers · git
- Class GUI-need label
CL· CLI-sufficient
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R03.05.01.02-T01 Reproduce and patch a stack buffer overflow
Original instruction
The sandbox C project in fixtures/vuln/parser crashes on long records. Build with AddressSanitizer, write test/poc_overflow.c that reproduces the overflow on the pre-patch build, then patch the bounds handling and commit. Keep the public API unchanged.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch ASan build aborts on the PoC; post-patch build runs the PoC clean and passes the existing unit tests; git diff touches only parser.c bounds logic; no API symbol changes.
Proposed partial credit
40% PoC reproduces on pre-patch; 40% patch stops the overflow (ASan clean); 20% tests pass + scoped diff
R03.05.01.02-T02 Fix a use-after-free in a cache module
Original instruction
fixtures/vuln/cache frees an entry that a later code path still reads. Reproduce with ASan via test/poc_uaf.c, then patch the lifetime/ownership so the entry stays valid until last use, and commit. Do not leak memory.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ASan reports heap-use-after-free on the PoC pre-patch; post-patch ASan+LSan are clean on the PoC and the test suite; no new leaks reported.
Proposed partial credit
40% UAF reproduced; 40% lifetime fixed (ASan clean); 20% no leaks + tests pass
R03.05.01.02-T03 Patch an integer overflow before allocation
Original instruction
In fixtures/vuln/imgload, a width*height*bpp multiply overflows and undersizes a buffer. Add test/poc_intoflow that reproduces the heap overflow, patch with a checked multiply that rejects oversized dimensions, and commit.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch ASan overflows on the PoC; post-patch rejects the oversized input with an error and is ASan-clean; valid images still load; tests pass.
Proposed partial credit
40% overflow reproduced; 40% checked multiply added; 20% valid inputs unaffected
R03.05.01.02-T04 Fix a NULL dereference on malformed input
Original instruction
fixtures/vuln/config segfaults when a required key is missing. Write test/poc_null reproducing the crash, add the missing presence check so the parser returns a clean error, and commit. Keep error codes consistent with the header docs.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Pre-patch binary segfaults on the PoC; post-patch returns the documented ERR_MISSING_KEY and exits 0 on valid configs; ASan clean; tests pass.
Proposed partial credit
50% crash reproduced; 50% guarded + correct error code
R03.05.01.02-T05 Patch a heap out-of-bounds read
Original instruction
fixtures/vuln/tlv reads past a buffer when a length field exceeds the remaining bytes. Reproduce with ASan (test/poc_oob), patch to validate length against remaining input, and commit. Preserve parsing of well-formed TLV records.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ASan heap-buffer-overflow (read) on PoC pre-patch; post-patch ASan clean and rejects the malformed TLV; valid-record tests pass.
Proposed partial credit
40% OOB reproduced; 40% length validation added; 20% valid records still parse
R03.05.01.02-T06 Remove a format-string vulnerability
Original instruction
fixtures/vuln/logger passes user text as the format argument to printf-family calls. Add test/poc_fmt showing attacker-controlled conversions, patch to use a constant format with the value as an argument, and commit across all affected call sites.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Pre-patch PoC shows uncontrolled format output; post-patch prints the literal input and compiles with -Wformat-security clean; grep finds no remaining user-as-format call; tests pass.
Proposed partial credit
50% all unsafe call sites fixed; 30% -Wformat-security clean; 20% PoC neutralized
R03.05.01.02-T07 Close a path traversal in a file server
Original instruction
fixtures/vuln/fileserv (Python) serves /files/<name> and allows ../ escape outside the root. Add tests/test_poc_traversal.py reproducing reads of /etc/hostname, patch to canonicalize and confine paths to the root, and commit.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch PoC returns a file outside the root; post-patch returns 403/404 for traversal attempts and still serves legitimate files; pytest passes; the canonicalization is on the resolved real path.
Proposed partial credit
40% traversal reproduced; 40% path confinement added; 20% legit files still served
R03.05.01.02-T08 Eliminate an OS command injection
Original instruction
fixtures/vuln/pinger (Python) builds a shell string from user input for ping. Add a PoC test that injects a second command, patch to pass an argument list without a shell and validate the host, and commit.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch PoC runs the injected command (marker file created); post-patch no marker is created and invalid hostnames are rejected; the subprocess call uses a list with shell=False; pytest passes.
Proposed partial credit
40% injection reproduced; 40% shell-free call + validation; 20% valid hosts still work
R03.05.01.02-T09 Parameterize a SQL injection
Original instruction
fixtures/vuln/usersvc (Python+SQLite) concatenates user input into a query. Add a PoC test extracting another row via injection, patch to use parameterized queries throughout the module, and commit. Keep the query results identical for valid inputs.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch PoC returns unauthorized rows; post-patch the same input returns no rows and valid lookups match the golden output; static check finds no string-built SQL; pytest passes.
Proposed partial credit
40% injection reproduced; 40% all queries parameterized; 20% golden results unchanged
R03.05.01.02-T10 Disable external entities to stop XXE
Original instruction
fixtures/vuln/xmlimport parses uploaded XML with external entities enabled. Add a PoC test that reads a local file via an external entity, patch the parser to forbid DTD/external entities, and commit. Valid documents must still import.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Pre-patch PoC exfiltrates file contents into parsed output; post-patch raises a parse error on external entities and imports valid XML unchanged; pytest passes.
Proposed partial credit
40% XXE reproduced; 40% entity resolution disabled; 20% valid docs import
R03.05.01.02-T11 Replace insecure deserialization
Original instruction
fixtures/vuln/session (Python) loads client cookies with an unsafe loader. Add a PoC proving code execution on crafted input, patch to a safe signed/structured format (e.g. JSON + HMAC), and migrate the readers. Commit with a short MIGRATION note.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Pre-patch PoC triggers the planted marker on load; post-patch rejects tampered/forged tokens (HMAC check) and no loader executes payloads; round-trip tests for valid sessions pass.
Proposed partial credit
30% RCE reproduced; 40% safe format + signature; 30% session round-trip tests pass
R03.05.01.02-T12 Add an SSRF allowlist to a URL fetcher
Original instruction
fixtures/vuln/fetch (Python) retrieves user-supplied URLs with no restriction. Add a PoC reaching the lab metadata service at 169.254.169.254, patch to resolve-and-validate against an allowlist blocking internal ranges, and commit. External allowed hosts still work.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Pre-patch PoC fetches the lab internal endpoint; post-patch blocks link-local/private/loopback targets after DNS resolution and still fetches allowlisted hosts; redirects re-validated; pytest passes.
Proposed partial credit
30% SSRF reproduced; 40% resolved-IP allowlist (incl. redirects); 30% allowed hosts work
R03.05.01.02-T13 Fix a TOCTOU race in temp-file handling
Original instruction
fixtures/vuln/spool (C) checks then opens a predictable temp path, racing a symlink swap. Build a PoC harness that wins the race in the sandbox, patch to O_CREAT|O_EXCL with a private dir, and commit. Keep normal spooling behavior.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Pre-patch PoC redirects the write via symlink (marker outside spool dir); post-patch open fails closed on an existing path and uses a 0700 dir; the race harness no longer succeeds over N trials; tests pass.
Proposed partial credit
30% race reproduced; 40% atomic O_EXCL open + private dir; 30% race harness fails + tests pass
R03.05.01.02-T14 Stop a memory leak under load
Original instruction
fixtures/vuln/conn (C) leaks a buffer on each handled request. Add test/poc_leak that drives many requests under LeakSanitizer, patch the missing free on all paths (including error paths), and commit.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Pre-patch LSan reports growing leaks across the PoC; post-patch LSan reports zero leaks after the same load and on error-path tests; functional tests pass.
Proposed partial credit
50% leak reproduced via LSan; 50% all paths freed (LSan clean)
R03.05.01.02-T15 Bound uncontrolled recursion (stack exhaustion)
Original instruction
fixtures/vuln/jsonparse (Python) recurses on nested input until it crashes. Add a PoC with deeply nested JSON, patch to enforce a configurable max depth returning a clean error, and commit. Normal-depth documents must parse.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch PoC crashes with RecursionError/stack overflow; post-patch raises DepthExceeded at the configured limit and parses normal inputs; pytest passes.
Proposed partial credit
40% crash reproduced; 40% depth limit added; 20% normal docs parse
R03.05.01.02-T16 Prevent CRLF/header injection
Original instruction
fixtures/vuln/redirect (Python web handler) copies user input into a response header, allowing CRLF injection. Add a PoC splitting the response, patch to reject/strip CR and LF in header values, and commit. Legitimate values pass through.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Pre-patch PoC injects an extra header/body; post-patch rejects values containing CR/LF (400) and passes clean values; response-splitting test fails against the patched build; pytest passes.
Proposed partial credit
40% injection reproduced; 40% CR/LF rejected; 20% valid headers pass
R03.05.01.02-T17 Fix catastrophic regex backtracking (ReDoS)
Original instruction
fixtures/vuln/validate (Python) uses a regex that backtracks exponentially on crafted input. Add a PoC that times out the matcher, replace with a linear-time pattern or a bounded matcher preserving the accepted language, and commit.
- Platforms
lin· Linux- Applications
- git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Pre-patch PoC exceeds a 2s budget on a short input; post-patch matches the same accept/reject set on the test corpus in <50ms for the PoC input; equivalence tests pass.
Proposed partial credit
40% ReDoS reproduced; 40% linear pattern with same language; 20% equivalence tests pass
R03.05.01.02-T18 Correct an off-by-one bounds error
Original instruction
fixtures/vuln/ring (C) writes one element past its ring buffer at wrap. Add test/poc_offbyone under ASan, patch the index arithmetic, and commit. The buffer's documented capacity semantics must be preserved.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Pre-patch ASan flags the 1-byte/element OOB write at wrap; post-patch ASan clean with capacity N honored (N-1 usable per docs); fill/drain tests pass.
Proposed partial credit
40% off-by-one reproduced; 40% index fixed (ASan clean); 20% capacity semantics preserved
R03.05.01.02-T19 Resolve a double-free on an error path
Original instruction
fixtures/vuln/decode (C) frees the same buffer twice when decoding fails midway. Reproduce with ASan (test/poc_dfree), patch the cleanup so each resource is freed exactly once on every path, and commit.
- Platforms
lin· Linux- Applications
- sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Pre-patch ASan reports double-free on the PoC error path; post-patch ASan+LSan clean on success and all error paths (fault-injection tests); functional tests pass.
Proposed partial credit
40% double-free reproduced; 40% single-free ownership on all paths; 20% fault-injection tests pass
R03.05.01.02-T20 Build a fuzz harness, triage a crash, and patch
Original instruction
For fixtures/vuln/mediademux (C), write an AFL++/libFuzzer harness over its parse entrypoint, run until the provided seed corpus yields a crash, minimize the crashing input, root-cause it, patch, and commit the harness + minimized repro + fix.
- Platforms
lin· Linux- Applications
- fuzzers · sanitizers · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandstype· Text & forms- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
The harness builds and finds the crash on pre-patch under ASan; the minimized testcase reproduces it; post-patch the testcase runs clean under ASan and the fuzzer finds no crash in a bounded run; regression tests pass.
Proposed partial credit
25% working fuzz harness; 25% crash found+minimized; 30% root cause patched (ASan clean); 20% bounded re-fuzz clean
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.