Debugging & runtime inspection
Set breakpoints, step and inspect runtime state (variables, arrays, call stacks, data viewers) to explain behavior or find a root cause, then fix it when there is a defect.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- VS Code · Xcode · Visual Studio · gdb
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R02.03.01.01-T01 Use a conditional breakpoint in VS Code to find the row that corrupts a total
Original instruction
`/work/ledger-import` (Python) imports `fixtures/march.csv` (48k rows) and the final balance is off by 1,204.17. Using the VS Code Python debugger with a conditional breakpoint in `apply_row()`, find the first row after which the running balance diverges from `expected_running.csv`, write its row id to `ROW.txt`, and fix the parsing bug.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- VS Code · debugpy
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
ROW.txt equals the seeded first-divergent row; hidden tests on 3 other CSVs pass after the fix; debugpy adapter log shows a conditional breakpoint set in apply_row and hit (logged by the instrumented adapter).
Proposed partial credit
40% correct row; 40% fix passes hidden tests; 20% debugger evidence
R02.03.01.01-T02 Diagnose a release-only C++ crash from static initialization order with gdb
Original instruction
`/work/allocsvc` (C++) crashes at startup only in `-O2` release builds. Use gdb on the release binary (symbols in a separate .debug file) to find which static object is used before construction, write the two translation units involved to `ROOT.txt`, and fix it without disabling optimizations.
- Platforms
lin· Linux- Applications
- gdb · CMake · gcc
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
ROOT.txt names the seeded TU pair; release build starts and passes the smoke test 100/100 runs; fix uses construct-on-first-use or equivalent (no -O0, no link-order hack only); gdb history file shows bt in the crashing frame.
Proposed partial credit
40% root cause identified; 40% fixed in release; 20% fix approach valid
R02.03.01.01-T03 Find a Swift use-after-free with Xcode's Zombies and Address Sanitizer
Original instruction
The macOS app `PhotoStack` crashes with EXC_BAD_ACCESS after deleting an album while thumbnails load. In Xcode, enable Address Sanitizer and Zombie Objects in the scheme, reproduce the crash in the running app, identify the over-released object and fix the ownership bug. Note the class in `ROOT.txt`.
- Platforms
mac· macOS- Applications
- Xcode · LLDB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
ROOT.txt names the seeded class; scheme file has ASan enabled; hidden UI test (XCUITest) deleting albums during load 50 times passes without crash or ASan report; fix keeps behavior tests green.
Proposed partial credit
40% correct object; 40% UI stress test passes; 20% scheme diagnostics configured
R02.03.01.01-T04 Break a WPF deadlock using Visual Studio's Parallel Stacks
Original instruction
The WPF app `C:\work\Dispatcher` freezes when clicking `Sync now` twice quickly. Run it under the Visual Studio debugger, Break All while frozen, use Parallel Stacks and Threads windows to identify the lock-order inversion, record the two lock objects in `C:\work\ROOT.txt`, and fix it.
- Platforms
win· Windows- Applications
- Visual Studio · WPF
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
ROOT.txt names both seeded lock fields; hidden UI automation (FlaUI) double-clicks Sync 200 times with no hang (watchdog 5 s); unit tests green; no lock removed without replacement (static check that shared state is still synchronized).
Proposed partial credit
40% locks identified; 40% no hang; 20% synchronization preserved
R02.03.01.01-T05 Use a DOM breakpoint in Chrome DevTools to find who hides a form field
Original instruction
On the sandbox signup page served from `/work/signup-web`, the `VAT number` field disappears 2 s after selecting country `Portugal`. Use Chrome DevTools DOM change breakpoints (subtree/attribute) to find the script and line that hides it, write `file:line` to `ROOT.txt`, and fix the bug in source.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Chrome DevTools
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.txt equals the seeded location (minified bundle source-mapped; the culprit is a third-party-looking module, not the obvious form code); hidden Playwright test confirms field remains visible for PT and hidden for US.
Proposed partial credit
40% correct location; 50% fix verified by tests; 10% no regressions for other countries
R02.03.01.01-T06 Fix an edit-resistant Python bug by stepping through it with pdb
Original instruction
`/work/nightmare/solver.py` returns wrong answers for some inputs because of an aliasing bug hidden behind dynamic attribute access and a decorator cache. Using pdb (or `breakpoint()`), step through a failing case from `cases.json`, find the mutation that corrupts state, and fix it.
- Platforms
lin· Linuxmac· macOS- Applications
- pdb · pytest
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden cases (200) pass; fix located in the seeded function (diff location check); pdb session transcript saved to `pdb.log` by the provided wrapper shows stepping into the decorator.
Proposed partial credit
70% hidden cases; 20% correct fix location; 10% debugger transcript
R02.03.01.01-T07 Catch memory corruption with an LLDB watchpoint
Original instruction
`/work/ringbuf` (C) occasionally corrupts the `head` field of a struct after many operations. Reproduce with the provided seed, set an LLDB (or gdb) hardware watchpoint on `rb->head`, find the out-of-bounds write that changes it, record the function in `ROOT.txt`, and fix it.
- Platforms
mac· macOSlin· Linux- Applications
- LLDB · gdb · clang
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.txt equals the seeded writer function; hidden randomized test (1e6 ops × 20 seeds) passes with ASan; debugger history shows a watchpoint command.
Proposed partial credit
40% correct function; 50% randomized tests pass; 10% watchpoint evidence
R02.03.01.01-T08 Trace an unhandled promise rejection with async stack traces in VS Code
Original instruction
The Node service `/work/orders-node` logs 'UnhandledPromiseRejection: undefined' under load and occasionally loses orders. Use the VS Code JavaScript debugger with 'Caught exceptions' and async stack traces to find where the rejection originates and which promise is not awaited; fix so no order is lost.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- VS Code · Node
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Hidden load test (5,000 orders with injected DB timeouts) persists all orders or returns 503 for retried ones; zero unhandled rejections with --unhandled-rejections=strict; root-cause function named in ROOT.txt.
Proposed partial credit
30% root cause named; 50% no lost orders; 20% no unhandled rejections
R02.03.01.01-T09 Debug an Android crash on screen rotation in Android Studio
Original instruction
`FieldNotes` (Android) crashes when the device rotates during photo capture. In Android Studio, run on the emulator with the debugger attached, break on the exception, inspect the fragment state, and fix the lifecycle bug so state survives rotation.
- Platforms
mac· macOSwin· Windowslin· Linuxand· Android- Applications
- Android Studio · Android Emulator
- Interfaces
desk· Desktop GUImob· Mobile touch- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden Espresso test rotates 20 times during capture flow: no crash, captured photo path preserved; unit tests green; ROOT.txt names the seeded field/lifecycle method.
Proposed partial credit
30% root cause; 50% rotation test passes; 20% state preserved
R02.03.01.01-T10 Find a crash's root cause from a core dump with gdb
Original instruction
`/work/imgd` crashed in production; you have the binary with debug symbols and `core.imgd.4412`. Using gdb post-mortem, determine the crashing thread, the corrupted variable and the input that triggered it (it is in a heap buffer), write them to `ROOT.json`, then fix the bug.
- Platforms
lin· Linux- Applications
- gdb
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.json thread id, variable name and triggering filename (recovered from memory) equal seeded values; fix passes hidden tests including the recovered input; no crash under ASan.
Proposed partial credit
20% thread; 30% variable; 20% input recovered; 30% fix
R02.03.01.01-T11 Locate native heap corruption with page heap and the Visual Studio debugger
Original instruction
`C:\work\Tiler` (C++ Win32) crashes randomly far from the bug. Enable full page heap for Tiler.exe with gflags, run under the Visual Studio debugger to break at the first invalid access, identify the buffer overrun, record function and buffer in `C:\work\ROOT.txt`, and fix it.
- Platforms
win· Windows- Applications
- Visual Studio · gflags · Application Verifier
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
ROOT.txt matches the seeded function/buffer; hidden stress run with page heap enabled completes 500 tiling jobs without access violation; output images identical to reference; gflags setting removed afterwards.
Proposed partial credit
40% root cause; 40% stress run clean; 10% outputs unchanged; 10% gflags cleaned up
R02.03.01.01-T12 Find a goroutine leak with Delve
Original instruction
`/work/streamer` (Go) grows goroutines without bound under reconnects. Attach Delve to the running process, use `goroutines` grouping and stack inspection to find where goroutines block forever, record the blocking call site in `ROOT.txt`, and fix the leak.
- Platforms
lin· Linuxmac· macOS- Applications
- Delve · go
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.txt equals the seeded file:line; hidden test performs 10k reconnects and asserts runtime.NumGoroutine returns to baseline ±5; goleak passes; dlv history shows goroutines listing.
Proposed partial credit
40% call site; 50% leak fixed; 10% debugger evidence
R02.03.01.01-T13 Pin down a Java stream NullPointerException with IntelliJ's evaluate expression
Original instruction
`/work/catalog-java` throws an NPE deep inside a stream pipeline for one supplier's feed. In IntelliJ IDEA, set an exception breakpoint for NullPointerException, use Evaluate Expression and the Stream Trace debugger to find which element and which mapping step produce null, write both to `ROOT.txt`, and fix the mapping.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- IntelliJ IDEA · Maven
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrolltype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.txt names the seeded SKU and mapping lambda; hidden feed tests pass; fix handles nulls per spec (skips with warning log) rather than catching NPE broadly (static check for catch NullPointerException).
Proposed partial credit
40% element + step; 40% tests pass; 20% fix approach
R02.03.01.01-T14 Use rr reverse execution to find who wrote a bad value
Original instruction
`/work/simcore` (C++) produces NaN in `state.energy` after ~2M steps, nondeterministically. Record a failing run with `rr record` using the provided seed script, then use reverse-continue with a watchpoint to find the instruction that first wrote NaN, write its function and source line to `ROOT.txt`, and fix the numerical bug.
- Platforms
lin· Linux- Applications
- rr · gdb
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
ROOT.txt equals the seeded source location; hidden runs (20 seeds × 3M steps) produce no NaN and energy drift < 1e-6; an rr trace directory exists from the session.
Proposed partial credit
40% source location; 50% fixed across seeds; 10% rr trace evidence
R02.03.01.01-T15 Analyze a user-mode crash dump in WinDbg and fix the bug
Original instruction
`C:\dumps\Invoicer.exe.1188.dmp` is a full user-mode dump from a .NET 8 app crash. In WinDbg with SOS, find the exception, the managed stack and the object state that caused it (`!pe`, `!clrstack`, `!do`), write the exception type and the offending field value to `C:\work\ROOT.json`, and fix the source in `C:\work\Invoicer`.
- Platforms
win· Windows- Applications
- WinDbg · SOS · Visual Studio
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
type· Text & formsnav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
ROOT.json values equal the seeded exception type and field value (only recoverable from the dump heap); hidden tests pass with the fix; WinDbg log file contains !clrstack output.
Proposed partial credit
40% dump findings; 50% fix; 10% debugger log
R02.03.01.01-T16 Debug a race between the main thread and a web worker in DevTools
Original instruction
The sandbox spreadsheet web app at `/work/gridcalc` sometimes shows stale totals after fast edits. Using Chrome DevTools (breakpoints in the worker thread, async stack traces, logpoints), find the message-ordering bug between main thread and worker, record the function in `ROOT.txt`, and fix it.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Chrome DevTools
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Hidden Playwright test performs 500 rapid edits: displayed totals always equal recomputed totals after settle; ROOT.txt names the seeded handler; no logpoints/console noise left in source.
Proposed partial credit
35% root cause; 55% race fixed; 10% cleanup
R02.03.01.01-T17 Remote-debug a Python service inside a Docker container
Original instruction
`/work/pricing-svc` runs in Docker Compose and returns wrong prices only inside the container (works locally). Attach VS Code's debugger to the container via debugpy (port mapping), inspect the environment-dependent value, write it to `ROOT.txt`, and fix the code so results are environment-independent.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- VS Code · Docker · debugpy
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickcode· Code & commandsmulti· Cross-app- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ROOT.txt names the seeded env-dependent setting (container locale + decimal context); hidden tests in container and host pass; debugpy listen config added to compose dev override only.
Proposed partial credit
40% root cause; 40% fix passes in both envs; 20% dev-only debug config
R02.03.01.01-T18 Debug firmware in QEMU through a gdb stub
Original instruction
`/work/fw-stm32` firmware runs on QEMU's `netduinoplus2` machine. The UART prints garbage after the scheduler starts. Start QEMU with `-s -S`, attach `gdb-multiarch`, step through clock and UART init, find the misconfigured register value, record register and value in `ROOT.txt`, and fix the firmware.
- Platforms
lin· Linux- Applications
- QEMU · gdb-multiarch · arm-none-eabi-gcc
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
ROOT.txt names the seeded register (USART BRR) and wrong value; rebuilt firmware prints the expected banner and 100 task messages on QEMU serial; no change to the linker script.
Proposed partial credit
40% register identified; 50% correct UART output; 10% minimal change
R02.03.01.01-T19 Prove a root cause with a breakpoint hit log before fixing
Original instruction
`/work/cartcalc` (TypeScript) computes wrong shipping for multi-warehouse carts. Using the VS Code debugger, set a logpoint that prints the warehouse split at the decision point for `fixtures/cart_17.json`, save the debug console output to `debug.log`, identify the faulty branch, and fix it.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- VS Code · Node
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
debug.log contains logpoint output with the split for cart_17 matching the runtime values (values only available at runtime, e.g., computed hash ids); hidden tests pass; fix in seeded function.
Proposed partial credit
30% logpoint evidence; 50% hidden tests; 20% fix location
R02.03.01.01-T20 Fix an Excel VBA macro using the VBA editor's debugger
Original instruction
`C:\work\Payroll.xlsm` macro `RunPayroll` gives wrong overtime for night shifts. In the VBA editor, use breakpoints, the Locals window and Watches to find the faulty time arithmetic, fix the macro, run it, and save the workbook so sheet `Output` matches `expected_output.csv`.
- Platforms
win· Windows- Applications
- Excel · VBA editor
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrolltype· Text & forms- Work product
WP06· Code/engineering- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Output sheet equals expected_output.csv for 120 employees; VBA module diff limited to the seeded procedure (olevba extract); macro still runs on a hidden second month input producing correct values. Trust access to the VBA project object model is disabled by policy and Sysmon detects Python/PowerShell processes opening Payroll.xlsm, so the macro must be fixed in the VBA editor.
Proposed partial credit
30% fault located (diff location); 50% output correct; 20% generalizes to hidden month
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.