EEG/MEG & electrophysiology analysis
Clean, epoch and analyze recordings to reported effects.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- EEGLAB · MNE · Brainstorm
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R08.09.01.04-T01 Reject eye-blink ICA components in EEGLAB by topography and time course
Original instruction
In EEGLAB load input/subj01_ica.set. Inspect the first 20 independent components' topographies, time courses and spectra, select the blink and lateral-eye-movement components, remove them, and save output/subj01_clean.set plus the rejected indices in output/rejected.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Rejected set must match the expert list exactly (2 components); the cleaned data must have the frontal blink-band (1–3 Hz at Fp1/Fp2) power reduced by ≥70% while occipital alpha power changes <10%. ICLabel is removed from the installation. The MATLAB command window and -batch/-r script execution are disabled, so EEGLAB must be driven from its menus.
Proposed partial credit
50% correct components; 30% frontal artifact reduction; 20% alpha preserved.
R08.09.01.04-T02 Preprocess, epoch and average an auditory oddball ERP in EEGLAB
Original instruction
In EEGLAB load input/oddball.set: high-pass 0.1 Hz, low-pass 30 Hz, re-reference to average mastoids, epoch −200 to 800 ms around the standard and deviant markers, baseline-correct, reject epochs exceeding ±100 µV, and save output/erp.set plus output/mmn.csv with the mean deviant-minus-standard amplitude at Fz in 150–250 ms.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Processing history in the .set must show the filters, reference and epoching as specified; rejected-epoch count within 10% of reference; MMN amplitude within 0.3 µV of the reference pipeline value.
Proposed partial credit
30% filtering and reference; 25% epoching and baseline; 20% rejection; 25% MMN amplitude.
R08.09.01.04-T03 Mark bad channels and interpolate them in EEGLAB
Original instruction
In EEGLAB load input/noisy.set and inspect the continuous data and channel spectra. Identify the bad channels (flat, drifting or high-frequency noisy), mark them, interpolate spherically, and save output/interp.set with the list in output/badchans.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
licensed· Licensed pro app- Proposed difficulty
- 2 / 5
Proposed verifier
Bad-channel set F1 ≥0.9 against the expert list (5 bad of 64); interpolated channels' correlation with neighbours >0.7; non-bad channels unchanged (identical samples). clean_rawdata is disabled. The MATLAB command window and -batch/-r script execution are disabled, so EEGLAB must be driven from its menus.
Proposed partial credit
50% bad channels; 30% interpolation quality; 20% other channels untouched.
R08.09.01.04-T04 Compute a time-frequency ERSP for one condition in EEGLAB
Original instruction
In EEGLAB open input/epoched.set and compute the ERSP at channel Cz for the 'target' condition with newtimef wavelets, cycles [3 0.5] (3 cycles at 3 Hz, increasing at half the FFT rate), 3–45 Hz, baseline −400 to −100 ms, 200 permutations with FDR correction. Export the ERSP matrix to output/ersp.mat and report the peak alpha-band suppression (time, frequency, dB) in output/answer.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
ERSP matrix correlates ≥0.98 with the reference newtimef run at those settings; reported peak within 50 ms, 1 Hz and 0.5 dB of reference; significance mask present. The EEGLAB GUI history (EEG.history / STUDY log) must show the time-frequency dialog call.
Proposed partial credit
30% settings; 35% ERSP matches; 25% peak values; 10% export.
R08.09.01.04-T05 Build a reproducible MNE-Python preprocessing script for one subject
Original instruction
Preprocess input/sub-01_raw.fif: in MNE's interactive raw browser mark bad channels and bad segments by eye (annotations 'BAD_muscle'), then write output/preproc.py applying notch 50 Hz, band-pass 1–40 Hz, ICA (20 components, picard) with EOG-correlated components (r>0.8) removed, and epoching −0.2 to 0.5 s on event id 2. Save output/sub-01_epo.fif, output/annotations.csv and output/report.html.
- Platforms
lin· Linuxmac· macOS- Applications
- MNE-Python
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
vis· Visual judgmentclick· Point & clickcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Annotations must cover ≥80% of the seeded muscle segments with ≤20% false-positive duration; epochs file has the right event selection (count within 2 of reference), filter parameters in info, EOG components removed (frontal artifact power down ≥70%); rerunning preproc.py with the saved annotations reproduces the epochs with a fixed random state.
Proposed partial credit
25% annotations by eye; 25% filtering; 25% ICA artifact removal; 25% epoching and deterministic rerun.
R08.09.01.04-T06 Reject EEG epochs by visual inspection and report rejection reasons
Original instruction
In EEGLAB open input/epochs_60.set and scroll through the 60 epochs. Mark for rejection those with muscle bursts, electrode pops or sweat drift (not blinks, which ICA already removed), and save output/pruned.set plus output/rejections.csv listing epoch index and reason.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Rejected set F1 ≥0.85 against expert marks (11 bad epochs); reasons match the seeded artifact per epoch for ≥80% of hits; kept epochs' data unchanged. The MATLAB command window and -batch/-r script execution are disabled, so EEGLAB must be driven from its menus.
Proposed partial credit
55% rejected set; 30% reasons; 15% data integrity.
R08.09.01.04-T07 Run a cluster-based permutation test on two conditions in Brainstorm
Original instruction
In Brainstorm load the protocol in input/protocol/ with averaged trials for conditions A and B across 18 subjects. Run a two-tailed paired cluster-based permutation test on sensor data, 0–600 ms, 1,000 permutations, cluster alpha 0.05, and export the significant cluster summary (time window, channels, p) to output/clusters.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Brainstorm · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 3 / 5
Proposed verifier
Cluster count, cluster p-values within 0.01, time windows within 20 ms and channel sets Jaccard ≥0.9 versus the reference analysis with the same settings and seed.
Proposed partial credit
25% test configured; 45% clusters match; 20% statistics; 10% export.
R08.09.01.04-T08 Localize an ERP component with dSPM source estimation in MNE
Original instruction
Using input/sub-01-ave.fif, input/sub-01-cov.fif and input/sub-01-fwd.fif, write output/source.py computing a dSPM inverse solution (loose 0.2, depth 0.8, SNR 3), then open the result in MNE's 3D time-viewer, confirm the 80–120 ms peak is on the lateral surface (not a depth artifact), and save output/answer.json (peak vertex, latency), output/stc/ and a viewer screenshot output/peak.png.
- Platforms
lin· Linuxmac· macOS- Applications
- MNE-Python
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
code· Code & commandsvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP07· Simulation result- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Peak vertex within 15 mm of the reference peak and latency within 5 ms; inverse parameters as specified in the STC metadata; peak.png must show the inflated surface with the hotspot within 20 px of the reference projection of that vertex.
Proposed partial credit
25% inverse parameters; 35% peak location; 20% latency; 20% visual confirmation screenshot.
R08.09.01.04-T09 Measure sleep-spindle density by marking events in EEGLAB
Original instruction
In EEGLAB open input/n2_sleep.set (10 min of N2 sleep at C3/C4). Visually mark sleep spindles as events named 'spindle' with onset and duration, then compute density (spindles per minute per channel) and save output/marked.set plus output/density.csv.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Event onsets F1 ≥0.75 against expert scoring with a 0.5 s tolerance; durations within 0.3 s for matched events; density within 15% per channel. Automatic spindle detectors are not installed. The MATLAB command window and -batch/-r script execution are disabled, so EEGLAB must be driven from its menus.
Proposed partial credit
55% spindle detection F1; 25% durations; 20% density values.
R08.09.01.04-T10 Spot and fix an event-code misalignment in an EEG recording
Original instruction
In EEGLAB load input/shifted.set: stimulus markers are delayed relative to the photodiode channel. Measure the offset from the photodiode, correct every event latency accordingly, re-epoch, and save output/aligned.set plus the offset in milliseconds in output/answer.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scrollclick· Point & clicktype· Text & forms- Work product
WP10· Annotations- Conditions
licensed· Licensed pro apphidden· Hidden/stale state- Proposed difficulty
- 3 / 5
Proposed verifier
Offset within 3 ms of the seeded value; after correction, the mean photodiode onset relative to events is within 3 ms of zero and the ERP P1 latency falls within the expected 90–120 ms window.
Proposed partial credit
40% offset measured; 35% events corrected; 25% ERP latency plausible.
R08.09.01.04-T11 Run and QC an ICA decomposition in Brainstorm and remove cardiac components
Original instruction
In the Brainstorm protocol input/protocol/ run ICA (infomax, 40 components) on subject 02's continuous MEG data, review the topographies and time series to find the two cardiac components, remove them, and save the cleaned link plus output/components.json (removed indices and justification).
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Brainstorm · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Removed components must match the expert pair (allowing one substitution if its ECG correlation exceeds 0.7); after cleaning, correlation of sensor data with the ECG channel drops below 0.2 (from >0.6); brain-band power 8–12 Hz changes <10%. The MATLAB command window and -batch execution are disabled, so Brainstorm must be driven from its GUI rather than bst_process scripts.
Proposed partial credit
45% correct components; 35% cardiac contamination reduced; 20% brain signal preserved.
R08.09.01.04-T12 Control: compute the Cz target-condition ERSP with headless EEGLAB newtimef
Original instruction
Control pair of T04. Without the EEGLAB GUI, run newtimef headless (matlab -batch) on input/epoched.set at Cz for 'target' with cycles [3 0.5], 3–45 Hz, baseline [−400 −100] ms, 200 permutations with FDR. Save output/ersp.mat and the peak alpha-band suppression (time, frequency, dB) in output/answer.json.
- Platforms
lin· Linuxmac· macOS- Applications
- EEGLAB · MATLAB
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro app- Proposed difficulty
- 2 / 5
Proposed verifier
Same checker and reference as T04: ERSP correlation ≥0.98 with the reference newtimef run, peak within 50 ms, 1 Hz and 0.5 dB, significance mask present. Scored as the CLI control of T04.
Proposed partial credit
30% newtimef settings; 40% ERSP matches; 20% peak values; 10% significance mask.
R08.09.01.04-T13 Spike-sort an extracellular recording in Phy and curate clusters
Original instruction
Open input/ks_output/ in Phy (Kilosort results, 12 clusters). Using waveform shape, autocorrelograms, amplitude drift and cross-correlograms, label clusters as good, MUA or noise, merge the two clusters that are the same unit, split the one with a bimodal amplitude distribution, and save the curation to output/cluster_group.tsv plus notes in output/curation.md.
- Platforms
lin· Linuxmac· macOS- Applications
- Phy
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Labels match the expert curation for ≥10 of 12 clusters; the correct pair merged and the correct cluster split (checked against gold spike-train assignment, ≥90% spike agreement); refractory-period violations of 'good' units <1%. Phy's IPython view is disabled, the agent has no Python shell, and a file-system audit flags a cluster_group.tsv not written by Phy.
Proposed partial credit
40% labels; 25% merge; 25% split; 10% notes.
R08.09.01.04-T14 Measure action-potential properties from current-clamp traces in Clampfit
Original instruction
Open input/current_clamp.abf in Clampfit. For the first action potential at each of the 10 current steps, measure threshold, peak amplitude, half-width and after-hyperpolarization, then export output/ap_props.csv and report rheobase and input resistance in output/answer.json.
- Platforms
win· Windows- Applications
- Clampfit
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
Per-step measurements within 5% (amplitude, half-width) and 2 mV (threshold, AHP) of the expert analysis; rheobase exact to the step and input resistance within 10%. pyABF, Python and other ABF readers are not installed on the VM, and a file-system audit log flags exports not written by Clampfit.
Proposed partial credit
50% AP properties; 25% rheobase; 25% input resistance.
R08.09.01.04-T15 Build a group ERP analysis in EEGLAB STUDY and report a condition effect
Original instruction
In EEGLAB create a STUDY from the 16 datasets in input/study/ with condition and group variables, precompute ERP measures, plot the condition comparison at Pz, run the built-in statistics (permutation, FDR), and export output/study_stats.csv with the significant time windows plus save output/study.study.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 4 / 5
Proposed verifier
STUDY design variables match the specification; significant windows within 20 ms of reference boundaries and p-values within 0.01; precomputed measures present for all 16 datasets.
Proposed partial credit
25% STUDY design; 25% measures precomputed; 35% statistics match; 15% exports.
R08.09.01.04-T16 Control: reject blink components by ICLabel on the command line
Original instruction
Control pair of T01. From the terminal with MNE-Python (ICLabel available here), load input/subj01_raw.fif and input/subj01-ica.fif (the T01 recording and decomposition exported to FIF), classify components automatically, remove eye components above 0.8 probability, and save output/subj01_clean.fif plus output/rejected.json.
- Platforms
lin· Linuxmac· macOS- Applications
- MNE-Python · ICLabel
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same checks as T01 (rejected set equals expert list, frontal artifact power down ≥70%, occipital alpha preserved). Scored as the CLI control of T01; the classifier is provided here because the GUI task must judge by eye.
Proposed partial credit
50% components; 30% artifact reduction; 20% alpha preserved.
R08.09.01.04-T17 Analyse miniature synaptic currents in Clampfit with template detection
Original instruction
Open input/minis.abf in Clampfit. Build an event template from five clean mEPSCs, run template-based event detection over the 120 s recording, reject false positives by inspection, and export output/events.csv (time, amplitude, rise time, decay tau) plus frequency and mean amplitude in output/answer.json.
- Platforms
win· Windows- Applications
- Clampfit
- Interfaces
desk· Desktop GUI- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickdrag· Drag & dropvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP02· Records/table- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Event F1 ≥0.85 against expert detection (10 ms tolerance); mean amplitude within 10% and frequency within 15%; decay tau within 20% for matched events. pyABF, Python and other ABF readers are absent from the VM, and a file-system audit flags exports not written by Clampfit.
Proposed partial credit
25% template built; 40% event detection F1; 20% amplitude/frequency; 15% kinetics.
R08.09.01.04-T18 Diagnose and remove 50 Hz line noise without distorting the beta band
Original instruction
input/linenoise.set has strong 50 Hz contamination with harmonics. In EEGLAB inspect the spectra, apply an appropriate approach (CleanLine or spectrum interpolation; a narrow notch is acceptable only if justified), and save output/clean.set plus output/notes.md explaining the choice and showing before/after spectra as output/spectra.png.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clickvis· Visual judgmenttype· Text & forms- Work product
WP10· Annotations- Conditions
licensed· Licensed pro app- Proposed difficulty
- 3 / 5
Proposed verifier
50, 100 and 150 Hz peak power reduced by ≥20 dB; power in 15–25 and 30–45 Hz changed <1 dB; notes.md names the method; spectra.png shows both curves (two traces detected).
Proposed partial credit
40% line-noise suppression; 35% neighbouring bands preserved; 15% figure; 10% justification.
R08.09.01.04-T19 Score five minutes of sleep EEG into stages in EEGLAB
Original instruction
In EEGLAB open input/sleep_10epochs.set (ten 30 s epochs with EEG, EOG and EMG). Score each epoch as W, N1, N2, N3 or REM following AASM criteria using the displayed channels, and write output/hypnogram.csv (epoch, stage) plus the deciding feature per epoch.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- EEGLAB · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
vis· Visual judgmentnav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
licensed· Licensed pro app- Proposed difficulty
- 4 / 5
Proposed verifier
Stages must match the consensus scoring for ≥8 of 10 epochs, with both REM epochs correct; deciding features must match the expected marker (spindles/K-complexes, slow waves, REMs with atonia) for ≥7 epochs. The MATLAB command window and -batch/-r script execution are disabled, so EEGLAB must be driven from its menus.
Proposed partial credit
60% stage agreement; 25% REM epochs correct; 15% deciding features.
R08.09.01.04-T20 Build a full MEG analysis from raw to source-level statistics in Brainstorm
Original instruction
In the Brainstorm protocol input/protocol_raw/ for 10 subjects: review and mark bad segments, run SSP for cardiac artifacts, epoch and average both conditions, compute head models (overlapping spheres) and minimum-norm sources, project to the default anatomy, and run a source-level paired permutation test 0–400 ms. Export output/source_stats.csv and the protocol to output/protocol.zip.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- Brainstorm · MATLAB
- Interfaces
desk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scrolltype· Text & forms- Work product
WP07· Simulation result- Conditions
licensed· Licensed pro applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
All 10 subjects have head models and source maps on the default anatomy; significant source clusters overlap the reference clusters with Dice ≥0.7 and p within 0.01; bad-segment marking recall ≥0.8 against expert annotation.
Proposed partial credit
20% artifact handling; 25% averaging and head models; 25% source estimation; 30% statistics match.
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.