Segmentation pipelines & visual QC
Configure segmentation and correct failures by inspection.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- CellProfiler · Cellpose · ilastik
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R08.07.01.02-T01 Fix an over-segmenting CellProfiler pipeline so nuclei counts match
Original instruction
Open input/pipeline.cppipe in CellProfiler with the images in input/images/. The nuclei module over-segments: using Test Mode overlays, adjust typical diameter and the thresholding settings until test image A01 yields 120±5 objects without clipping clumps. Save output/pipeline_fixed.cppipe.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Headless rerun of the saved pipeline on A01 gives 115–125 objects and per-object IoU ≥0.7 to the hidden masks for ≥85% of objects; other modules unchanged (pipeline diff limited to the nuclei module).
Proposed partial credit
40% object count in band; 35% segmentation quality; 25% no collateral edits.
R08.07.01.02-T02 Build a nuclei-and-cytoplasm CellProfiler pipeline and report the effective dose
Original instruction
In CellProfiler build a pipeline over input/plate/ (DNA and GFP channels, 6 doses × 3 wells): IdentifyPrimaryObjects on DNA, IdentifySecondaryObjects on GFP, MeasureObjectIntensity, and export per-well nuclear/cytoplasmic GFP ratios. Save output/translocation.cppipe and output/per_well.csv, and write the lowest dose with a ratio change above 20% to output/answer.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Headless rerun reproduces per_well.csv ratios within 5% of reference; the reported dose equals gold; pipeline must contain the four module types with secondary objects derived from the primary ones.
Proposed partial credit
25% pipeline structure; 30% per-well ratios; 30% correct dose; 15% reproducible rerun.
R08.07.01.02-T03 Tune Cellpose GUI parameters for touching cells and save the model settings
Original instruction
In the Cellpose GUI open input/confluent.tif, run the cyto3 model, and tune diameter, flow threshold and cellprob threshold by inspecting the overlay until the masks separate the touching cells. Save output/masks.npy, export output/masks.png, and record the final parameters in output/params.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Cellpose
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Average precision at IoU 0.5 ≥0.80 against hidden masks and merged-object rate <5%; rerunning Cellpose headless with params.json must reproduce the submitted masks (AP within 0.02).
Proposed partial credit
45% AP ≥0.8 (scaled from 0.5); 25% merge rate; 30% parameters reproduce the result.
R08.07.01.02-T04 Train an ilastik pixel classifier for membranes, cytoplasm and background
Original instruction
In ilastik create a Pixel Classification project on input/membranes.tif, choose a feature set, and paint brush labels for membrane, cytoplasm and background until the live preview is clean. Save output/project.ilp and export the probability maps to output/probabilities.h5.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- ilastik
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
path· Continuous pathclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Membrane-channel probability map AUC ≥0.95 against hidden membrane annotations on a held-out crop; project must contain brush labels for all three classes (≥200 labelled pixels each) and the chosen features. ilastik's --headless mode and Python API are disabled, h5py is absent, and a file-system audit flags .ilp/.h5 files not written by ilastik.
Proposed partial credit
25% three label classes painted; 20% feature selection; 45% held-out AUC; 10% export.
R08.07.01.02-T05 Flag the segmentation overlays that leak into background
Original instruction
input/overlays/ holds 10 segmentation overlay PNGs from a nuclei pipeline. Inspect each and write output/qc.csv listing, for every image, pass or fail and the failure mode for the three whose masks leak into background or merge objects.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Fiji/ImageJ
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Pass/fail set must equal gold exactly (3 failures); failure modes must match the seeded defect (background leak, merge, halo inclusion). Two distractors have unusual but correct morphology. Fiji's macro runner, Script Editor and --headless mode are disabled and Python imaging libraries are absent, so overlays must be judged by viewing.
Proposed partial credit
70% exact pass/fail set; 30% failure modes correct.
R08.07.01.02-T06 Build an ilastik object-classification pipeline to separate live and dead cells
Original instruction
Using input/cells.tif with the probability maps in input/probs.h5, create an ilastik Object Classification project: threshold objects, then label at least 15 live and 15 dead cells by appearance. Save output/objclass.ilp and export the object predictions table to output/objects.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- ilastik
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Held-out object accuracy ≥0.85 and balanced accuracy ≥0.80 against hidden labels; project must hold ≥15 labels per class; object count within 10% of reference thresholding. ilastik's --headless mode and Python API are disabled, h5py is absent, and a file-system audit flags .ilp/.h5 files not written by ilastik.
Proposed partial credit
25% objects thresholded; 25% labels per class; 40% held-out accuracy; 10% export.
R08.07.01.02-T07 Add illumination correction to a CellProfiler pipeline and fix edge-well bias
Original instruction
input/plate_pipeline.cppipe measures intensities that drift across each field. Add CorrectIlluminationCalculate and Apply (per-plate, regular background, median filter) before segmentation, verify in Test Mode that the field gradient is gone, and save output/pipeline_illum.cppipe plus output/per_image.csv from a headless run.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
In per_image.csv the correlation between mean intensity and field x/y position must drop below |0.1| (input >0.6); object counts stay within 10% of the input pipeline; the two modules present with the stated settings.
Proposed partial credit
30% modules added correctly; 40% gradient removed; 20% counts preserved; 10% outputs.
R08.07.01.02-T08 Correct Cellpose masks by hand in the GUI and retrain a model
Original instruction
In the Cellpose GUI open input/hard_field.tif with the predicted masks, fix them by drawing missing cells and splitting merges (about 25 corrections), save as output/hard_field_seg.npy, then retrain from the cyto3 model on the corrected image plus input/train/ and save output/model/.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Cellpose
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
path· Continuous pathclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Corrected masks AP@0.5 ≥0.85 vs hidden masks; the retrained model must beat cyto3 on a held-out field by ≥0.05 AP; training log must list the corrected image. The agent has no Python shell or cellpose CLI (only the GUI launcher), and a file-system audit flags _seg.npy or model files not written by the Cellpose GUI process.
Proposed partial credit
45% corrected masks; 35% retrained model improvement; 20% training records.
R08.07.01.02-T09 Set up a StarDist 2D segmentation in Fiji and tune its thresholds
Original instruction
In Fiji run StarDist 2D (Versatile fluorescent nuclei) on input/nuclei.tif, tune probability and overlap thresholds against the overlay until clumped nuclei are separated without false splits, export label image output/labels.tif, and record the thresholds in output/params.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Fiji/ImageJ · StarDist
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
AP@0.5 ≥0.85 vs hidden masks with false-split rate <5%; rerunning StarDist headless with params.json reproduces the labels within 0.02 AP.
Proposed partial credit
45% AP; 25% false-split rate; 30% reproducible parameters.
R08.07.01.02-T10 Configure CellProfiler to count and measure cytoplasmic puncta per cell
Original instruction
In CellProfiler build a pipeline on input/puncta/ : nuclei from DNA, cells by propagation, then EnhanceOrSuppressFeatures (speckles, 5 px) and IdentifyPrimaryObjects for puncta, RelateObjects to assign puncta to cells. Export per-cell puncta counts to output/puncta.csv and save output/puncta.cppipe.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Headless rerun: per-cell puncta counts within ±1 of gold for ≥85% of cells; mean count within 10%; relation module must link puncta to parent cells (parent column populated).
Proposed partial credit
25% enhancement and detection; 25% cell objects; 35% per-cell counts; 15% relation output.
R08.07.01.02-T11 Diagnose why a Cellpose model fails on a new magnification
Original instruction
A pipeline tuned at 20× produces almost no masks on the 60× images in input/images60/. In the Cellpose GUI determine the cause, adjust the settings so segmentation works at 60×, save output/masks60.npy, and write the diagnosis and the corrected parameters to output/diagnosis.md.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Cellpose
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmenttype· Text & forms- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
AP@0.5 ≥0.8 on the 60× field; diagnosis.md must identify the diameter/scale mismatch (keyword check) and its parameters must reproduce the masks when rerun headless.
Proposed partial credit
30% diagnosis; 50% AP; 20% reproducible parameters.
R08.07.01.02-T12 Build a two-stage ilastik workflow: pixel then boundary-based segmentation
Original instruction
On input/em_slice.h5 (electron microscopy), train an ilastik Pixel Classification project for membrane probability, then use Multicut (or Boundary-based Segmentation) to produce cell segments, adjusting the beta until over- and under-segmentation balance. Save output/pixel.ilp, output/multicut.ilp and output/segments.h5.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- ilastik
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
path· Continuous pathclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizontutorial· Follow docs/tutorial- Proposed difficulty
- 5 / 5
Proposed verifier
Variation of Information vs the hidden ground truth below the reference threshold (≤0.45) and adapted Rand error ≤0.12 on a held-out crop; both projects saved with labels and the chosen beta. ilastik's --headless mode and Python API are disabled, h5py is absent, and a file-system audit flags .ilp/.h5 files not written by ilastik.
Proposed partial credit
25% membrane classifier; 25% multicut configured; 40% segmentation metrics; 10% artifacts saved.
R08.07.01.02-T13 Fix a QuPath cell-detection setup that misses faint cells in one region
Original instruction
In QuPath open input/slide.ome.tif with the project's saved cell-detection settings: cells in the faintly stained region marked in input/weak_region.png are missed. Adjust detection parameters (or add a region-specific run) so ≥90% of cells there are found without doubling false positives elsewhere, and save the project plus output/detections.geojson.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- QuPath
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
In the weak region recall ≥0.90 against expert annotations; in control regions false-positive rate ≤1.2× the original; parameter changes recorded in the project. QuPath's script editor and command-line script mode are disabled, and a file-system audit flags exports not written by QuPath.
Proposed partial credit
45% recall in weak region; 35% no regression elsewhere; 20% saved project and export.
R08.07.01.02-T14 Export CellProfiler measurements to a database and verify the schema
Original instruction
Add ExportToDatabase (SQLite) to input/pipeline.cppipe with per-image and per-object tables and an experiment name, run the pipeline headless on input/images/, and verify in output/results.db that object counts per image match the CSV output. Save output/pipeline_db.cppipe and output/verify.sql.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler · SQLite
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
results.db holds Per_Image and Per_Object tables with matching ImageNumber keys; per-image object counts equal the CSV counts exactly; verify.sql returns the comparison when rerun.
Proposed partial credit
30% export module configured; 40% database contents match; 20% schema correct; 10% verification query.
R08.07.01.02-T15 Build a 3D nuclei segmentation pipeline for a confocal stack in CellProfiler
Original instruction
In CellProfiler configure a 3D pipeline on input/stack.tif (voxel size in input/meta.txt): set volumetric processing, median filter, Watershed-based 3D nuclei identification, and MeasureObjectSizeShape. Export output/objects3d.csv and save output/pipeline3d.cppipe, with the object count in output/answer.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
tutorial· Follow docs/tutorial- Proposed difficulty
- 4 / 5
Proposed verifier
Headless rerun: object count within 10% of gold; mean volume within 15% in µm³ (voxel anisotropy must be set); per-object IoU ≥0.6 for ≥80% of objects.
Proposed partial credit
25% 3D mode and voxel sizes; 30% segmentation; 30% volume measurements; 15% outputs.
R08.07.01.02-T16 Normalize a plate for well-position bias and recompute Z-factor
Original instruction
input/plate_measurements.csv (from a CellProfiler run) shows edge-well bias. In CellProfiler Analyst or a script, apply a per-plate normalization appropriate to the layout in input/platemap.csv, recompute the Z-factor for the positive and negative controls, and write output/normalized.csv and output/answer.json (Z before and after).
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler Analyst · Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickcode· Code & commandsvis· Visual judgment- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Z-factor after normalization ≥0.5 and within 0.05 of the reference method; edge-vs-centre control difference reduced by ≥70%; control wells correctly identified from the platemap.
Proposed partial credit
25% platemap parsed; 35% normalization effect; 30% Z-factors; 10% outputs.
R08.07.01.02-T17 Control: segment nuclei headless with Cellpose and report AP
Original instruction
Control pair of T03. Without the GUI, run Cellpose from the terminal on input/confluent.tif with cyto3, choosing diameter and thresholds by a scripted sweep against input/val_masks.tif (a small validation crop), write output/masks.npy and the chosen parameters to output/params.json.
- Platforms
lin· Linuxmac· macOS- Applications
- Cellpose
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same checker as T03 (AP@0.5 ≥0.80 on the hidden full-field masks, merge rate <5%, params reproduce masks). Scored as the CLI control of T03.
Proposed partial credit
50% AP; 25% merge rate; 25% parameters.
R08.07.01.02-T18 Review 60 segmentation thumbnails and build a QC gate rule
Original instruction
input/qc_montage/ holds 60 thumbnails with mask outlines, plus input/measurements.csv. Inspect them, label each pass/fail in output/qc_labels.csv, then propose in output/gate.json a threshold rule on measured features (e.g., form factor, area, intensity SD) that reproduces your labels, and report its accuracy.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Fiji/ImageJ
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Labels agree with expert labels on ≥54 of 60; the gate rule applied to measurements.csv reproduces the expert labels with accuracy ≥0.85 on a hidden 40-image set and uses at most 3 features. Fiji's macro runner, Script Editor and --headless mode are disabled and Python imaging libraries are absent, so thumbnails must be labelled by viewing.
Proposed partial credit
40% labels; 40% gate generalizes; 20% rule is simple and documented.
R08.07.01.02-T19 Rescue a failing CellProfiler run whose images have mixed channel order
Original instruction
A headless CellProfiler run over input/images/ fails midway: some files have DNA and GFP swapped relative to the metadata in input/filelist.csv. In CellProfiler fix the NamesAndTypes/Metadata configuration so channels are assigned by image content or metadata rules, rerun, and save output/pipeline_fixed.cppipe and output/per_image.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP06· Code/engineering- Conditions
hidden· Hidden/stale state- Proposed difficulty
- 5 / 5
Proposed verifier
Rerun completes on all images; per-image nuclei counts within 10% of gold for every well, including the 8 swapped files (wrong assignment gives counts off by >50% there); configuration must handle them by rule, not by deleting them.
Proposed partial credit
30% run completes; 40% swapped files handled correctly; 20% counts match; 10% no images dropped.
R08.07.01.02-T20 Compare three segmentation tools on the same field and justify a choice
Original instruction
Segment input/hard_field.tif with Cellpose (GUI), StarDist in Fiji and a CellProfiler watershed pipeline, each tuned by visual inspection. Export the three label images to output/, write output/compare.csv (tool, object count, visual failure notes) and recommend one tool with reasons in output/recommendation.md.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Cellpose · StarDist · CellProfiler
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmenttype· Text & formsmulti· Cross-app- Work product
WP12· Answer/decision- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Three label images present; AP@0.5 computed per tool against hidden masks, and the recommended tool must be the one with the highest AP (tolerance 0.02); reported counts within 10% of each tool's own output; failure notes must match the seeded weaknesses.
Proposed partial credit
40% three segmentations produced; 30% correct recommendation; 20% metrics table; 10% notes.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.