Video tracking annotation
Track objects across frames with keyframes and interpolation.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- CVAT · Label Studio
- Class GUI-need label
G3· GUI-only
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R03.08.01.02-T01 Track the red car across 300 frames
Original instruction
In CVAT track mode, open task 'street_seq' and track the red car from frames 0-300 using a 'car' track: set keyframes where its motion changes and let interpolation fill the rest. Export the track.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Exported track mean IoU >= 0.6 vs the gold track across frames 0-300. Track built in CVAT track mode; a pasted interpolation JSON without keyframe metadata is rejected. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% track present across range; 30% keyframes at motion changes; 30% mean IoU >= 0.6
R03.08.01.02-T02 Mark when a person leaves frame
Original instruction
In CVAT, continue the existing 'person' track in task 'exit_seq' and set the 'outside' flag on the frame where the person fully leaves the view, so the box stops being shown afterward. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
The 'outside' keyframe is within +/-5 frames of the gold exit frame and the track has no visible box afterward. Graded on the exported track's outside flag. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
60% outside flag within +/-5 frames; 40% no box shown after exit
R03.08.01.02-T03 Fix tracker drift in a window
Original instruction
In CVAT task 'drift_seq', an auto-track drifts off the object between frames 120 and 150. Add corrective keyframes in that window so the box re-aligns to the object, leaving frames outside the window unchanged. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Mean IoU over frames 120-150 rises to >= 0.6 vs gold while IoU outside the window is unchanged. Graded on the per-frame IoU in and out of the window. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
50% drift window corrected (IoU>=0.6); 30% re-aligned smoothly; 20% rest untouched
R03.08.01.02-T04 Split a track at an occlusion
Original instruction
In CVAT task 'split_seq', one track mistakenly spans two different objects before and after a full occlusion. Split it into two separate track IDs at the occlusion boundary so each ID follows one object. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
The result has two tracks split at the correct frame (+/-3) and each track's boxes match the respective object (mean IoU >= 0.6). Graded on track-ID structure vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
50% split at correct frame; 50% both IDs follow correct objects
R03.08.01.02-T05 Interpolate a slow-moving object
Original instruction
In CVAT task 'slow_seq', place two keyframes (first and last frame) on the slowly drifting box for the 'sign' track and rely on linear interpolation between them. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Two keyframes set; interpolated track mean IoU >= 0.7 vs gold across the clip. Graded on the exported interpolated track. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
50% both keyframes correct; 50% interpolation IoU >= 0.7
R03.08.01.02-T06 Track multiple pedestrians maintaining IDs
Original instruction
In CVAT task 'ped_seq' (200 frames, 6 pedestrians), create one track per pedestrian and keep each person's ID consistent throughout, including through brief partial occlusions. Export all tracks.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
MOTA >= 0.7 vs gold with ID switches <= 2 across the 6 pedestrians and mean IoU >= 0.6. Graded on the exported multi-track MOT metrics. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
30% all 6 tracked; 30% IoU threshold; 40% MOTA + few ID switches
R03.08.01.02-T07 Track a ball through fast motion
Original instruction
In CVAT task 'ball_seq', track the fast-moving ball with a 'ball' track, adding dense keyframes where it changes direction (bounces) so interpolation stays on it. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Track mean IoU >= 0.55 vs gold with keyframes within +/-3 frames of each bounce. Graded on the exported track. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% track spans motion; 30% keyframes at bounces; 30% IoU threshold
R03.08.01.02-T08 Merge two tracks of one object
Original instruction
In CVAT task 'merge_seq', the same car was split into two tracks (IDs 3 and 7) across a tunnel. Merge them into a single track so one ID spans the whole sequence. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
The export has a single track for the car spanning both segments with consistent boxes (mean IoU >= 0.6) and track count reduced by one. Graded on track-ID structure vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
60% tracks merged into one ID; 40% continuous coverage/IoU
R03.08.01.02-T09 Annotate occlusion over time
Original instruction
In CVAT task 'pole_seq', track the car passing behind a pole and set the 'occluded' attribute true only on the frames where the pole covers it, false otherwise. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
The occluded attribute's true-interval matches the gold interval within +/-3 frames at each edge and the track IoU >= 0.6. Graded on the attribute timeline vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% track present; 60% occluded interval correct
R03.08.01.02-T10 Rotated-box track for a turning object
Original instruction
In CVAT task 'turn_seq', track the forklift with a rotated bounding box, adjusting the angle at keyframes as it turns so the box stays aligned to its body. Export rotated-box track.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Mean IoU >= 0.6 and mean angle error <= 10 degrees vs the gold rotated track. Graded on the exported rotated boxes. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% track coverage; 30% IoU threshold; 30% angle error <= 10 deg
R03.08.01.02-T11 Keyframe a box through a zoom
Original instruction
In CVAT task 'zoom_seq', the camera zooms so the tracked building grows; keyframe the box size at the start, middle, and end so the interpolated box scales with it. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Track mean IoU >= 0.65 vs gold with box area scaling consistent with the zoom (size error below threshold at mid frame). Graded on the exported track. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% three scale keyframes; 30% interpolation tracks size; 30% IoU >= 0.65
R03.08.01.02-T12 Track an object entering mid-sequence
Original instruction
In CVAT task 'enter_seq', a deer enters at around frame 80. Start an 'animal' track at its first appearance (not frame 0) and track it to the clip's end, leaving earlier frames empty. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
The track's first keyframe is within +/-5 frames of the gold entry, no boxes before it, and mean IoU >= 0.6 afterward. Graded on the exported track. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% correct entry frame; 60% track IoU after entry
R03.08.01.02-T13 Correct an injected ID swap
Original instruction
In CVAT task 'swap_seq', two tracked cyclists have their IDs swapped at one crossing frame. Fix the assignment so each ID follows one cyclist for the whole clip, without changing any box geometry. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
ID switches vs gold drop to 0 and boxes are unchanged. Graded on track-ID assignment vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
60% swap corrected (0 ID switches); 40% geometry unchanged
R03.08.01.02-T14 Keyframe around scene cuts
Original instruction
In CVAT task 'cut_seq', the clip has two hard cuts; the tracked object jumps position across them. Add keyframes immediately before and after each cut so interpolation does not slide the box across the cut. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Keyframes exist on both sides of each cut (within 1 frame) and no interpolated box spans a cut; mean IoU >= 0.6. Graded on the exported track + keyframe placement. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% keyframes bracket both cuts; 30% no cross-cut interpolation; 30% IoU threshold
R03.08.01.02-T15 Keep IDs through a path crossing
Original instruction
In CVAT task 'cross_seq', two similar-looking runners cross paths. Track both and ensure their IDs do not swap at the crossing, using appearance cues across adjacent frames. Export both tracks.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Zero ID switches at the crossing vs gold and both tracks' mean IoU >= 0.6. Graded on MOT ID metrics vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% both tracked; 30% IoU threshold; 30% no ID swap at crossing
R03.08.01.02-T16 Box one object across 50 frames
Original instruction
In CVAT task 'short_seq' (50 frames), track one pedestrian using 5 well-placed keyframes and interpolation for the rest. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scroll- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Track mean IoU >= 0.7 vs gold over the 50 frames with about 5 keyframes (interpolation covers the rest). Graded on the exported track. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
50% full-range track; 50% mean IoU >= 0.7
R03.08.01.02-T17 Add a parked-state attribute to a track
Original instruction
In CVAT task 'park_seq', track the car and set a boolean 'moving' attribute true while it drives and false during the interval it is stopped at the curb. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
The 'moving' attribute's false-interval matches the gold stopped interval within +/-5 frames and the track IoU >= 0.6. Graded on the attribute timeline vs gold. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
40% track present; 60% stopped interval correct
R03.08.01.02-T18 Track with class attribute over segments
Original instruction
In Label Studio video, track the vehicle in task 'class_seq' and keep its label 'truck' consistent across the clip, adding keyframes so the box stays on it through a sharp turn and a brief partial occlusion. Submit.
- Platforms
web· Any OS (web)- Applications
- Label Studio
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Track mean IoU >= 0.6 vs gold with the label consistent throughout and keyframes bracketing the turn and occlusion. Graded on the exported track. Label Studio API tokens are disabled for the agent and import is turned off, so annotations can be submitted only from the labeling UI; each must carry a non-zero lead_time.
Proposed partial credit
40% consistent label+track; 30% keyframes at turn/occlusion; 30% IoU threshold
R03.08.01.02-T19 Dense multi-object tracking to MOTA
Original instruction
In CVAT task 'mot_dense' (200 frames, ~12 vehicles), track every vehicle maintaining consistent IDs through merges at an intersection. Export all tracks for MOT evaluation.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
MOTA >= 0.65 and IDF1 >= 0.7 vs gold across all vehicles. Graded on the exported MOT tracks. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
30% all vehicles tracked; 30% IoU/coverage; 40% MOTA + IDF1 thresholds
R03.08.01.02-T20 Full MOT with occlusion handling
Original instruction
In CVAT task 'mot_full' (300 frames), track all vehicles and all pedestrians with separate labels, maintaining IDs through several full occlusions (buses passing). Correctly use outside flags during total occlusions. Export.
- Platforms
web· Any OS (web)- Applications
- CVAT
- Interfaces
brow· Browser- GUI need · task setting
G3· GUI-only- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
drag· Drag & dropgeom· Precise geometrynav· Pan/zoom/scrollvis· Visual judgment- Work product
WP10· Annotations- Conditions
long· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Per-class MOTA >= 0.65, IDF1 >= 0.7, and ID switches at occlusions below the stated cap vs gold; outside flags used during total occlusions. Graded on the exported MOT tracks. CVAT accepts only browser sessions (CSRF + session cookie, no API token), annotation import and AI trackers are disabled for the agent role, and the client event log must show the keyframes edited in the UI.
Proposed partial credit
25% all objects tracked; 25% IoU/coverage; 25% occlusion outside flags; 25% MOTA/IDF1
Mapped research references
No benchmark reference is assigned to this class in the current mapping.