Bioinformatics pipelines
Run and debug pipelines to correct outputs.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Nextflow · GATK · BEDTools
- Class GUI-need label
CL· CLI-sufficient
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R08.06.01.03-T01 Repair a failing nf-core/sarek germline run on HG002 chr22 and report F1
Original instruction
The staged Nextflow run in work/ fails during the variant-calling step. Fix the configuration and sample sheet (input/samplesheet.csv, input/custom.config) so nf-core/sarek completes germline calling with DeepVariant on chr22 of input/HG002.cram, then benchmark against input/giab_chr22.vcf.gz with hap.py and write precision, recall and F1 to output/metrics.json.
- Platforms
lin· Linux- Applications
- Nextflow · nf-core/sarek · hap.py
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Pipeline must reach a successful completion trace; SNV F1 ≥0.99 and INDEL F1 ≥0.95 inside the GIAB high-confidence regions, matching output/metrics.json within 0.001. Hard-coding metrics without an execution trace scores 0.
Proposed partial credit
35% pipeline completes; 20% correct reference/intervals; 30% F1 thresholds; 15% metrics file.
R08.06.01.03-T02 Merge three CTCF narrowPeak files into non-overlapping peaks with BEDTools
Original instruction
Using BEDTools on input/ctcf_rep1.narrowPeak, rep2 and rep3, produce output/union_peaks.bed: the union of all peaks merged into non-overlapping intervals, sorted by coordinate, keeping only intervals supported by at least two replicates, with a fourth column giving the support count. Save the exact commands to output/cmds.sh.
- Platforms
lin· Linuxmac· macOS- Applications
- BEDTools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Output must be sorted, have no self-overlaps, and match gold intervals exactly (bedtools jaccard = 1.0) with correct support counts; cmds.sh must reproduce the file when rerun.
Proposed partial credit
40% intervals exact; 25% support counts; 20% sorted/non-overlapping; 15% reproducible commands.
R08.06.01.03-T03 Run a GATK best-practices germline pipeline on a WES subset
Original instruction
From input/sample_R1.fastq.gz and R2, run bwa-mem2 alignment to input/chr20.fa, mark duplicates, BQSR with input/known_sites.vcf.gz, HaplotypeCaller in GVCF mode over input/exome.bed, GenotypeGVCFs, then hard filters (QD<2, FS>60, MQ<40 for SNPs). Write output/filtered.vcf.gz plus output/metrics.txt with the dedup rate and mean target coverage.
- Platforms
lin· Linux- Applications
- GATK · bwa-mem2 · samtools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
VCF must be bgzipped, indexed, filters applied as named, and reach ≥0.98 concordance with the reference callset over the target BED; dedup rate and coverage within 1% of reference tools' values.
Proposed partial credit
25% alignment/dedup; 20% BQSR; 25% calling; 30% filters, concordance and metrics.
R08.06.01.03-T04 Build a Nextflow DSL2 RNA-seq pipeline with a resume-safe cache
Original instruction
Write output/main.nf and output/nextflow.config implementing an RNA-seq quantification pipeline (fastp → salmon quant → tximport summary) over input/samples.csv (6 samples), with per-process containers, a 'test' profile capping resources, and publishDir for results. Run it, then rerun with -resume so no process re-executes, and save both traces to output/.
- Platforms
lin· Linux- Applications
- Nextflow
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
First trace: all processes succeed for 6 samples; second trace: 100% cached. Salmon TPMs for 3 spiked genes within 5% of reference; the gene-level summary has the expected 6 columns.
Proposed partial credit
30% pipeline runs; 25% resume fully cached; 25% quantification values; 20% config structure.
R08.06.01.03-T05 Debug a Nextflow channel bug that drops paired-end samples
Original instruction
output/main.nf (copied from input/buggy.nf) processes only 4 of 8 samples in input/samplesheet.csv because of a channel/operator mistake. Find and fix it so all 8 run, without changing process scripts, and write a two-sentence root cause to output/cause.txt.
- Platforms
lin· Linux- Applications
- Nextflow
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Trace shows 8 successful executions of each process with distinct sample IDs; process script blocks byte-identical to the input; cause.txt keyword-matches the seeded bug (groupTuple/size or first() misuse).
Proposed partial credit
50% all 8 samples processed; 30% scripts unchanged; 20% root cause stated.
R08.06.01.03-T06 Run differential expression with DESeq2 and export significant genes
Original instruction
Using input/counts.tsv and input/metadata.csv (treated vs control, 2 batches), run DESeq2 with design ~ batch + condition, apply apeglm shrinkage, and write output/de_results.csv (all genes, ordered by padj) and output/sig_genes.txt (padj < 0.05 and |log2FC| > 1). Save the script to output/de.R.
- Platforms
lin· Linuxmac· macOS- Applications
- R · DESeq2
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Significant gene set Jaccard ≥0.98 with the reference run using the same design and shrinkage; log2FC values within 0.05; omitting the batch term changes the set beyond tolerance.
Proposed partial credit
35% design correct; 25% shrinkage; 30% gene set; 10% files and script.
R08.06.01.03-T07 Annotate variants with VEP and filter to high-impact coding changes
Original instruction
Annotate input/calls.vcf.gz with VEP using the offline cache in input/vep_cache (GRCh38, --everything), then write output/high_impact.tsv with one row per variant whose canonical-transcript consequence is HIGH impact, listing gene, HGVSc, HGVSp and gnomAD AF. Save commands to output/cmds.sh.
- Platforms
lin· Linux- Applications
- VEP · bcftools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Row set must equal gold exactly (canonical-only, HIGH impact); HGVS strings and AF values must match the VEP output; using all transcripts yields extra rows and fails.
Proposed partial credit
30% annotation run; 40% row set; 20% HGVS/AF fields; 10% reproducible commands.
R08.06.01.03-T08 Fix a bacterial assembly pipeline failing on low-coverage contamination
Original instruction
input/reads_R1.fastq.gz and R2 fail to assemble into a usable genome with the staged script run_assembly.sh (SPAdes). Diagnose with FastQC/Kraken2, remove the contaminating species and adapter content, reassemble, and produce output/assembly.fasta with output/quast_report.txt. Record the contaminant species in output/answer.json.
- Platforms
lin· Linux- Applications
- SPAdes · Kraken2 · fastp · QUAST
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Assembly N50 ≥120 kb, total length within 5% of the reference genome, ≤80 contigs ≥500 bp; BUSCO complete ≥97%; contaminant species equals gold.
Proposed partial credit
20% contaminant identified; 20% cleaning; 40% assembly metrics; 20% QUAST/BUSCO reports.
R08.06.01.03-T09 Build a Snakemake ChIP-seq peak-calling workflow with a conda environment
Original instruction
Write output/Snakefile and output/envs/*.yaml for a ChIP-seq workflow (bowtie2 → filter MAPQ≥30 and dedup → MACS2 narrow peaks vs input control → multiqc) over input/units.tsv (2 ChIP, 1 control). Run with --use-conda and produce output/peaks/ plus output/multiqc_report.html, and confirm a dry run is clean afterwards.
- Platforms
lin· Linux- Applications
- Snakemake · bowtie2 · MACS2
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
All targets built; peak count within 15% of reference and ≥90% of reference peaks overlapped by a submitted peak; dry run reports nothing to do; conda envs pinned with versions.
Proposed partial credit
25% workflow structure; 25% run completes; 30% peak agreement; 20% reproducibility (dry run, pinned envs).
R08.06.01.03-T10 Make a single-cell QC and clustering analysis reproducible with scanpy
Original instruction
Using input/filtered_feature_bc_matrix.h5, write output/analysis.py that filters cells (≥200 genes, <15% mito), normalizes, selects 2,000 HVGs, runs PCA 50, neighbours, Leiden at resolution 0.5 with a fixed seed, and saves output/adata.h5ad plus output/cluster_counts.csv. Rerunning must reproduce identical cluster assignments.
- Platforms
lin· Linuxmac· macOS- Applications
- scanpy · Python
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Cell count after QC equals gold; cluster count within ±1 and ARI ≥0.9 vs reference labels; a second run of analysis.py gives byte-identical cluster_counts.csv.
Proposed partial credit
25% QC thresholds; 25% pipeline steps; 30% clustering agreement; 20% deterministic rerun.
R08.06.01.03-T11 Lift over a variant set to GRCh38 and reconcile rejected records
Original instruction
Lift input/calls_hg19.vcf.gz to GRCh38 with Picard LiftoverVcf and input/hg19ToHg38.chain, writing output/calls_hg38.vcf.gz and output/rejected.vcf.gz. Then classify every rejected record by reason in output/rejects_summary.csv (counts per reason) and confirm the lifted file is correctly sorted and indexed.
- Platforms
lin· Linuxmac· macOS- Applications
- Picard · bcftools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Lifted record count and per-reason reject counts equal gold; the output is coordinate-sorted against the GRCh38 dictionary with a valid index; REF alleles match the GRCh38 reference at every lifted site.
Proposed partial credit
35% liftover output correct; 25% REF consistency; 30% reject summary; 10% sorted and indexed.
R08.06.01.03-T12 Call somatic variants with Mutect2 and a panel of normals
Original instruction
Run GATK Mutect2 on input/tumor.bam with input/normal.bam, input/pon.vcf.gz and input/af_only_gnomad.vcf.gz over input/targets.bed, then GetPileupSummaries, CalculateContamination, LearnReadOrientationModel and FilterMutectCalls. Write output/somatic_filtered.vcf.gz and the contamination estimate to output/answer.json.
- Platforms
lin· Linux- Applications
- GATK
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
PASS call set F1 ≥0.90 against the reference truth in the targets; contamination within 0.01 of reference; orientation-bias and contamination filters present in the VCF header.
Proposed partial credit
25% Mutect2 run with PoN and germline resource; 25% filtering steps; 35% F1; 15% contamination value.
R08.06.01.03-T13 Benchmark two aligners on simulated reads and summarize accuracy
Original instruction
Align input/sim_R1.fastq.gz and R2 (read names encode true positions) with bwa-mem2 and minimap2 to input/ref.fa. Write output/benchmark.csv with, per aligner, the fraction mapped, fraction mapped within 50 bp of truth, MAPQ≥30 precision, and wall-clock seconds; name the better aligner in output/answer.json.
- Platforms
lin· Linux- Applications
- bwa-mem2 · minimap2 · samtools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Each metric within 1% of the reference evaluation parsing the same truth encoding; the recommendation must match the reference ranking on correct-position precision.
Proposed partial credit
25% both alignments complete; 45% metrics correct; 20% timing recorded; 10% recommendation.
R08.06.01.03-T14 Containerize a pipeline step and pin it in the Nextflow config
Original instruction
Write output/Dockerfile for the alignment step of input/main.nf pinning bwa-mem2 2.2.1 and samtools 1.19, running as a non-root user with no build tools in the final image. Build it, point the process at it in output/nextflow.config via a container directive, and run the step on input/test_reads/ to produce output/test.bam.
- Platforms
lin· Linux- Applications
- Docker · Nextflow
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Image: both tools at the exact versions, default user non-root, no gcc/make present, size under 500 MB. The Nextflow run completes using that image (trace container field) and test.bam passes samtools quickcheck with ≥99% reads mapped.
Proposed partial credit
30% image builds with pinned versions; 20% hardening; 30% pipeline uses the image; 20% output BAM valid.
R08.06.01.03-T15 Trace and fix a reference-mismatch error in a joint-genotyping run
Original instruction
GenotypeGVCFs on input/gvcfs/ fails with a sequence-dictionary error. Diagnose which GVCF was produced against the wrong reference build, regenerate only that sample's GVCF from its BAM in input/bams/ with the correct reference, rerun joint genotyping, and write output/joint.vcf.gz plus the offending sample in output/answer.json.
- Platforms
lin· Linux- Applications
- GATK · samtools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Joint VCF contains all samples with a consistent dictionary; the named sample equals gold; its per-sample genotype concordance with the reference callset ≥0.98; other samples' GVCFs unmodified (checksums).
Proposed partial credit
30% correct sample identified; 30% regenerated GVCF; 30% joint call set; 10% others untouched.
R08.06.01.03-T16 Compute per-exon coverage statistics and flag dropouts with mosdepth
Original instruction
With mosdepth and BEDTools on input/sample.bam and input/exons.bed, write output/exon_coverage.tsv (exon, mean depth, fraction ≥20×) and output/dropouts.bed listing exons with mean depth <20× or ≥10% of bases below 20×. Save commands to output/cmds.sh.
- Platforms
lin· Linuxmac· macOS- Applications
- mosdepth · BEDTools
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP02· Records/table- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Mean depths within 1% and fractions within 0.01 of the reference computation; dropout set exactly equal to gold; commands rerun reproduce both files.
Proposed partial credit
40% coverage table; 40% dropout set; 20% reproducible commands.
R08.06.01.03-T17 Run a Galaxy history for RNA-seq QC and trimming in the browser
Original instruction
In the local Galaxy instance (test account), upload input/reads_R1.fastq.gz and R2, run FastQC on both, then Trimmomatic (SLIDINGWINDOW:4:20, MINLEN:36), rerun FastQC on the trimmed reads, and build this as a reusable workflow named 'FC_QC_Trim'. Export the workflow to output/workflow.ga and the history to output/history.tar.gz.
- Platforms
web· Any OS (web)- Applications
- Galaxy
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & droptype· Text & forms- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Workflow .ga must contain the four steps correctly wired with the stated Trimmomatic parameters; the history must show completed jobs whose trimmed-read count and post-trim mean quality match the reference run within 1%. The test user is issued no API key, so BioBlend/planemo cannot reach Galaxy and jobs and the workflow must be built in a browser session.
Proposed partial credit
25% jobs completed; 30% parameters; 30% workflow wiring; 15% exports.
R08.06.01.03-T18 Control: run the same QC and trimming from the command line
Original instruction
Control pair of T17. Using fastqc and trimmomatic from the terminal on input/reads_R1.fastq.gz and R2 with SLIDINGWINDOW:4:20 and MINLEN:36, produce the trimmed FASTQs, pre- and post-trim FastQC reports, and a short output/summary.tsv with read counts and mean qualities.
- Platforms
lin· Linuxmac· macOS- Applications
- FastQC · Trimmomatic
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Trimmed read counts and mean qualities within 1% of the T17 reference values; four FastQC reports present. Scored as the CLI control of T17.
Proposed partial credit
40% trimming parameters and outputs; 40% metrics match; 20% reports present.
R08.06.01.03-T19 Reproduce a published differential-abundance result from a methods section
Original instruction
input/paper_methods.pdf describes a 16S analysis (DADA2 defaults stated, SILVA 138 taxonomy, rarefaction depth, ANCOM-BC2 with covariates). Reproduce it on input/fastq/ and input/metadata.csv, and write output/results.csv (taxa, log-fold change, q-value) plus output/deviations.md noting any step you could not reproduce exactly and why.
- Platforms
lin· Linux- Applications
- QIIME 2 · DADA2 · R
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizontutorial· Follow docs/tutorial- Proposed difficulty
- 5 / 5
Proposed verifier
Significant taxa set Jaccard ≥0.85 with the paper's reported table and log-fold-change signs all matching; ASV count within 10% of reference; deviations.md must name the one under-specified parameter (seeded) rather than silently guessing.
Proposed partial credit
25% ASV table; 20% taxonomy; 35% differential results; 20% deviations documented.
R08.06.01.03-T20 Package a pipeline for a Slurm cluster with retry and resource tuning
Original instruction
Adapt input/main.nf for a Slurm cluster: write output/nextflow.config with an executor profile, per-process labels (small/medium/large) mapped to CPU, memory and time, retry-with-backoff on exit codes 137 and 140 escalating memory, and a work directory on scratch. Run it on the local Slurm test queue over input/samples.csv and save the trace and report.
- Platforms
lin· Linux- Applications
- Nextflow · Slurm
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
long· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
Trace shows all tasks submitted through Slurm and completing, at least one retried task whose second attempt requested more memory, and resource requests matching the label table; the report lists no failed processes.
Proposed partial credit
30% Slurm executor profile; 25% resource labels; 30% retry escalation observed; 15% run completes.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.