Outbreak & epidemic modeling
Fit and project epidemic models and report uncertainty.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- R · SaTScan · Python
- Class GUI-need label
CL· CLI-sufficient
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R09.04.01.02-T01 Fit an SIR model to early outbreak incidence and estimate R0
Original instruction
input/school_flu.csv has daily new cases for the first 25 days of a synthetic school influenza outbreak (N=1,200, recovery period 3 days). Fit a deterministic SIR model by least squares on daily incidence in R or Python, and write output/sir.json with beta, gamma (fixed 1/3), R0 and predicted peak day.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
R0 within 5% and peak day within +-1 of the reference fit; checker re-runs the script and confirms parameters reproduce the observed curve (RMSE below threshold).
Proposed partial credit
20% model implemented; 50% R0; 30% peak day
R09.04.01.02-T02 Estimate time-varying Rt with EpiEstim
Original instruction
Using R and EpiEstim, estimate the instantaneous reproduction number from input/covid_county_incidence.csv with parametric serial interval mean 4.7 days, SD 2.9, sliding weekly windows. Write output/rt.csv (window end date, mean, 2.5%, 97.5% quantiles) and output/rt.png.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Rt means within 0.01 and quantiles within 0.02 of reference estimate_R output for every window; window convention (t_start/t_end) matches EpiEstim default weekly windows.
Proposed partial credit
20% correct SI config; 50% Rt means; 30% credible intervals
R09.04.01.02-T03 Run a space-time permutation scan in the SaTScan GUI
Original instruction
In the SaTScan desktop GUI create a session with the daily synthetic gastrointestinal case file input/gi_cases.cas (ZIP, date, count) and coordinates input/zip.geo; run a retrospective space-time permutation scan (max spatial 50% of cases, temporal 1-14 days, 999 replications, seed default). Save output/gi.prm and the results, and report the most likely cluster to output/cluster.json.
- Platforms
win· Windowsmac· macOS- Applications
- SaTScan
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
.prm settings parsed (model, scan, windows, replications); cluster centroid ZIP, radius, start/end dates exact vs reference run, p-value within 0.01 (Monte Carlo).
Proposed partial credit
35% session parameters; 45% cluster location and dates; 20% p-value
R09.04.01.02-T04 Project SEIR scenarios with vaccination for 12 weeks
Original instruction
Implement an SEIR model in Python with parameters from input/params.yaml (latent 3 d, infectious 5 d, R0 2.2, N=500,000, initial state from input/state.json) and three vaccination scenarios (0, 5,000 and 15,000 doses/day starting day 7, 70% efficacy). Write output/scenarios.csv with daily incidence and output/summary.json with peak day, peak incidence and attack rate per scenario.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Checker integrates the reference model (RK45, rtol 1e-8): peak day within +-1, peak incidence and attack rate within 1% for all 3 scenarios; vaccination applied to susceptibles only.
Proposed partial credit
25% base model; 25% vaccination mechanism; 50% scenario summaries
R09.04.01.02-T05 Nowcast recent cases adjusting for reporting delays
Original instruction
input/reports.csv contains, for a synthetic outbreak, each case's onset date and report date up to the data cutoff 2026-10-10. Estimate the reporting-delay distribution from complete periods and nowcast total cases by onset date for the last 14 days with 90% intervals in R (e.g. a negative binomial or Bayesian nowcast). Write output/nowcast.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Scored against the eventually-reported counts held back by the checker: mean absolute percentage error over the 14 days <=15% and 90% interval coverage >=70%; delay distribution median within 1 day of truth.
Proposed partial credit
25% delay distribution; 45% nowcast accuracy; 30% interval coverage
R09.04.01.02-T06 Estimate R and dispersion k from transmission cluster sizes
Original instruction
input/clusters.csv lists final sizes of 180 transmission chains of a synthetic zoonotic pathogen. Fit a negative-binomial branching process by maximum likelihood to estimate R and dispersion k with profile-likelihood 95% CIs, and report the share of cases generated by the top 20% of spreaders. Write output/branching.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python · R
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
R and k within 2% of the reference MLE; CI bounds within 5%; top-20% share within 2 percentage points of the analytic value under the fitted model.
Proposed partial credit
40% R; 30% k; 15% CIs; 15% superspreading share
R09.04.01.02-T07 Produce quantile hospitalization forecasts in FluSight format
Original instruction
Using weekly influenza hospitalization data through 2026-01-03 in input/hosp_admissions.csv for 5 synthetic states, produce 1-4 week ahead forecasts with the 23 FluSight quantiles in the hub submission format (output/2026-01-03-team-model.csv), using any model you build.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Submission passes the hub validation schema; scored against held-out truth: mean WIS relative to a seasonal-naive baseline <=0.9 and 95% interval coverage >=80%.
Proposed partial credit
20% valid format; 50% relative WIS; 30% coverage
R09.04.01.02-T08 Estimate the serial interval from interval-censored transmission pairs
Original instruction
input/pairs.csv lists 140 infector-infectee pairs with onset dates recorded as date windows (interval-censored) and some negative serial intervals. Fit a normal and a gamma-with-shift serial interval distribution accounting for censoring, choose by AIC and write output/si.json (distribution, mean, SD, 95% CIs).
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Chosen family equals reference; mean within 0.1 days and SD within 0.15 days of reference doubly-censored MLE; CIs within 10%.
Proposed partial credit
20% censoring handled; 20% family selection; 40% mean and SD; 20% CIs
R09.04.01.02-T09 Run a purely spatial Bernoulli scan for a cancer cluster in SaTScan
Original instruction
Using SaTScan (GUI or batch), run a purely spatial Bernoulli scan on synthetic case file input/cases.cas, control file input/controls.ctl and coordinates input/blocks.geo, scanning for high rates with max 25% of population and 9,999 replications. Report the most likely cluster and any secondary clusters with p<0.05 in output/clusters.json and save the .prm.
- Platforms
win· Windowsmac· macOSlin· Linux- Applications
- SaTScan
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandsclick· Point & click- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Parameters parsed from .prm; cluster centres, radii, observed/expected and relative risk exact vs reference run; p-values within 0.005.
Proposed partial credit
30% parameters; 50% most likely cluster; 20% secondary clusters
R09.04.01.02-T10 Estimate epidemic growth rate and doubling time
Original instruction
From input/daily_cases.csv (synthetic early-phase epidemic) estimate the exponential growth rate over 2026-09-01..2026-09-21 by Poisson regression of counts on time with a day-of-week effect, and report growth rate, doubling time and 95% CIs in output/growth.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Growth rate within 0.001/day and doubling time within 0.1 day of reference GLM; CI bounds within 2%.
Proposed partial credit
50% growth rate; 30% doubling time; 20% CIs
R09.04.01.02-T11 Compute R0 of an age-structured model via the next-generation matrix
Original instruction
Build an age-structured SEIR model (4 age groups) in Python with contact matrix input/contacts.csv, susceptibility vector and infectious period from input/params.yaml. Compute R0 as the spectral radius of the next-generation matrix, calibrate the transmission probability so R0=1.8, and report final attack rate per age group in output/age_seir.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Calibrated transmission probability within 0.5% of reference; per-age attack rates within 0.5 percentage points (final-size equations or long simulation).
Proposed partial credit
30% NGM construction; 30% calibration; 40% attack rates
R09.04.01.02-T12 Fit a Bayesian SIR model to hospitalization data in Stan
Original instruction
Fit an SIR model with an ascertainment fraction to synthetic daily hospitalizations input/hosp.csv using Stan (cmdstanr or cmdstanpy) with priors in input/priors.md and a negative-binomial observation model. Run 4 chains, check R-hat<1.01, and write posterior medians and 90% intervals for beta, gamma, R0 and ascertainment to output/posterior.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Checker reruns with fixed seed: all R-hat<1.01, ESS>400; posterior medians within 5% and interval endpoints within 10% of reference posterior; data-generating values inside 90% intervals.
Proposed partial credit
20% model compiles and samples; 20% convergence; 40% posterior medians; 20% intervals
R09.04.01.02-T13 Build a median ensemble and score it with WIS and coverage
Original instruction
input/forecasts/ holds quantile forecasts from 3 models for 10 forecast dates and 4 horizons; input/truth.csv has observations. Build an equally weighted quantile median ensemble, score all 4 models with WIS (and its components) and 50%/95% coverage, and write output/scores.csv and output/ensemble.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Ensemble quantiles exact; WIS within 1e-6 of scoringutils reference; coverage exact; table sorted by mean WIS.
Proposed partial credit
35% ensemble; 45% WIS; 20% coverage
R09.04.01.02-T14 Estimate major-outbreak probability by stochastic simulation
Original instruction
Write a Gillespie stochastic SIR simulation in Python (N=1,000, one initial case, R0=1.5, gamma=1/5) and estimate by 20,000 runs the probability that the outbreak exceeds 50 cases, compare with 1 - 1/R0, and report mean final size of major outbreaks in output/stoch.json. Seed 2026.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Major-outbreak probability within 0.015 of 1/3; mean final size of major outbreaks within 2% of the deterministic final-size solution (~583); simulation algorithm checked by reproducing a known 2-event trace.
Proposed partial credit
30% correct Gillespie events; 40% probability; 30% final size
R09.04.01.02-T15 Reproduce a published model's projection figure from its parameters
Original instruction
input/paper.pdf describes a two-strain transmission model with all parameters in its Table 1 and shows projections in Figure 3. Implement the model in R or Python and reproduce Figure 3's curves; write output/fig3_data.csv (day, strain1, strain2) and output/fig3.png.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
tutorial· Follow docs/tutorial- Proposed difficulty
- 4 / 5
Proposed verifier
Curves compared with digitized Figure 3 (supplied to the checker): normalized RMSE <5% per strain, peak days within +-2; equations audited for the cross-immunity term.
Proposed partial credit
30% model structure; 50% curve agreement; 20% peak timing
R09.04.01.02-T16 Configure a prospective weekly space-time surveillance scan in SaTScan
Original instruction
In the SaTScan GUI set up prospective space-time surveillance for synthetic weekly ILI data (input/ili_weekly.cas, input/sites.geo): Poisson model with population file, study period 2024-2026, scanning only clusters still active at the end, recurrence-interval reporting, adjusting for earlier analyses using input/history.txt. Save output/ili.prm and report alarms with recurrence intervals >1 year in output/alarms.json.
- Platforms
win· Windowsmac· macOS- Applications
- SaTScan
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
.prm prospective settings exact (analysis type, end date, adjust for earlier analyses); alarm list matches reference run with recurrence intervals within 10%.
Proposed partial credit
45% prospective settings; 40% alarms; 15% recurrence intervals
R09.04.01.02-T17 Project arrival times with a metapopulation mobility model
Original instruction
Implement a stochastic metapopulation SEIR model in Python for 30 synthetic cities with population input/cities.csv and daily mobility matrix input/mobility.npy, seeded with 10 cases in city 0. Run 500 simulations (seed 1) and report the median and 90% interval of the arrival day (first local case) for each city in output/arrival.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 5 / 5
Proposed verifier
Median arrival days within +-2 days and interval bounds within +-4 days of the reference ensemble for >=27 of 30 cities; rank correlation with effective-distance prediction >=0.8.
Proposed partial credit
25% model and mobility coupling; 45% median arrival days; 20% intervals; 10% rank correlation
R09.04.01.02-T18 Estimate vaccine effectiveness from a test-negative design
Original instruction
input/tnd.csv contains synthetic test-negative data (test result, vaccination status, days since vaccination, age, week, region). Estimate VE = 1 - adjusted OR by logistic regression adjusting for age (spline), calendar week and region, overall and by time since vaccination (14-90, 91-180, >180 days). Write output/ve.csv with 95% CIs.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
VE estimates within 1 percentage point and CI bounds within 2 points of the reference model with the specified covariates; exclusions (tested <14 days post-dose) applied.
Proposed partial credit
20% exclusions; 40% overall VE; 40% waning strata
R09.04.01.02-T19 Estimate a delay-adjusted case fatality ratio
Original instruction
From synthetic daily cases and deaths in input/outbreak_cfr.csv and the onset-to-death distribution (lognormal, mean 13 days, SD 12.7) in input/delay.json, compute the naive CFR and the delay-adjusted CFR (Nishiura et al. method) with 95% CIs at the last date, and the adjusted CFR time series. Write output/cfr.json and output/cfr_series.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- R · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Naive CFR exact; adjusted CFR within 0.2 percentage points of reference; CI bounds within 0.4 points; time series correlation >0.99 with reference.
Proposed partial credit
15% naive CFR; 50% adjusted CFR; 20% CIs; 15% series
R09.04.01.02-T20 Calibrate an epidemic model to cases and deaths with ABC
Original instruction
Calibrate the stochastic SEIRD model in input/model.py (do not edit it) to synthetic weekly cases and deaths input/targets.csv using ABC-SMC (pyabc or own implementation) over priors in input/priors.yaml for R0, IFR and reporting rate. Report posterior medians and 90% intervals in output/abc.json and posterior predictive 90% bands in output/ppc.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Data-generating parameters fall inside 90% intervals; posterior medians within 10% of truth; interval widths not >2x reference ABC posterior; posterior predictive covers >=85% of target points; model.py hash unchanged.
Proposed partial credit
15% model untouched; 40% parameter recovery; 20% interval width; 25% predictive coverage
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.