R02.02.01.04 · TASK CLASS

Repository hosting & merge-request administration

Configure repositories, branch protection, reviews and merge requests in GitLab/GitHub web UIs.

20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.

Typical applications: GitLab · GitHub · Gitea
Class GUI-need label: HY · Hybrid

Task proposals · 20

Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.

R02.02.01.04-T01 Protect main and release/* branches in GitLab

Original instruction

Self-hosted GitLab EE (test license), project platform/api. Protect 'main': allowed to merge Maintainers, allowed to push and merge No one, force push off, code owner approval required. Protect wildcard 'release/*': merge Developers + Maintainers, push No one. Remove the stale protection on 'develop'.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Grader GET /projects/:id/protected_branches: main merge=40, push=0, allow_force_push=false, code_owner_approval_required=true; release/* merge=30, push=0; develop absent. PAT creation is disabled for the run and the agent has no token; /api/v4 writes by the agent's user are detected in production_json.log -> 0.

Proposed partial credit

45% main rule exact; 35% release/* rule exact; 20% develop unprotected

R02.02.01.04-T02 GitHub repository ruleset with status checks and bypass team

Original instruction

In the sandbox GitHub org 'acme-sbx', repo 'billing', create an active ruleset 'protect-main' targeting the default branch and release/**: require a PR with 2 approvals, dismiss stale approvals, require status checks 'ci/test' and 'lint' (strict), require linear history, block force pushes and deletions. Only team 'release-managers' may bypass.

Platforms: web · Any OS (web)
Applications: GitHub
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 4 / 5

Proposed verifier

Grader GET /repos/acme-sbx/billing/rulesets/{id}: enforcement active, include ~DEFAULT_BRANCH and refs/heads/release/**, rules pull_request(2, dismiss_stale), required_status_checks strict with both contexts, required_linear_history, non_fast_forward, deletion; bypass_actors = that team only. gh is unauthenticated; org audit log entries with programmatic_access_type detected -> 0.

Proposed partial credit

25% targets correct; 35% PR and status-check rules; 20% history, force-push and deletion rules; 20% bypass list exact

R02.02.01.04-T03 Push mirror from Gitea to a second Gitea instance

Original instruction

On git.test (Gitea 1.22) configure repo infra/terraform to push-mirror to https://backup.test/mirror/terraform.git using the token in fixtures/backup_token.txt, syncing on commit and every 8 hours. Trigger a sync and confirm the backup has all branches and tags.

Platforms: web · Any OS (web)
Applications: Gitea · git
Interfaces: brow · Browser term · Terminal
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: AP · API-sufficient
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Gitea DB push_mirror row: remote URL, sync_on_commit=true, interval 8h; backup.test repo refs (7 branches, 12 tags) equal source SHAs. Grader pushes a new commit to source and checks it appears on backup within 60 s.

Proposed partial credit

40% mirror configured with interval and on-commit; 35% initial sync complete; 25% on-commit sync works

R02.02.01.04-T04 Resolve MR conflicts in the GitLab web editor and squash-merge

Original instruction

MR !214 in platform/api conflicts with main in config/settings.yaml (3 hunks) and src/routes.py (1 hunk). Resolve them in GitLab's conflict editor per fixtures/resolution.md (which side to take per hunk, one manual merge), then merge with squash and the message 'feat(routes): add v2 health endpoints (!214)'.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms nav · Pan/zoom/scroll
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 4 / 5

Proposed verifier

main HEAD tree: both files byte-equal the expected resolutions; squash commit message exact; MR state merged, squash=true. The agent's user lacks push access over git (protected branch, no SSH/HTTPS creds), so resolution must use the web editor; commit 'Resolve conflicts' authored via the UI session is checked.

Proposed partial credit

35% settings.yaml resolved exactly; 25% routes.py manual merge exact; 25% squash-merged; 15% message exact

R02.02.01.04-T05 Path-scoped approval rules for a GitLab project

Original instruction

In GitLab EE project platform/api create approval rules: 'Security' requiring 1 approval from group security-team, 'Backend' requiring 2 approvals from users ana, ben, chen; set 'Prevent approval by author' and 'Remove all approvals when commits are added'. Add a CODEOWNERS file via the web IDE assigning /src/auth/ to @security-team.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 4 / 5

Proposed verifier

Grader API: approval_rules has Security(1, group) and Backend(2, 3 users); approvals settings merge_requests_author_approval=false, reset_approvals_on_push=true; CODEOWNERS in main has the /src/auth/ rule (commit via Web IDE). Agent holds no token; /api/v4 writes detected in production_json.log -> 0.

Proposed partial credit

40% two approval rules exact; 25% approval settings; 35% CODEOWNERS committed and parsed

R02.02.01.04-T06 Review a PR with line comments and a suggestion

Original instruction

In sandbox repo acme-sbx/billing, review PR #42: leave the three line comments in fixtures/review.md on the exact files and lines given, add a suggested change replacing line 88 of src/invoice.py with the code in the fixture, and submit the review as 'Request changes' with summary 'Blocking: rounding + nulls'.

Platforms: web · Any OS (web)
Applications: GitHub
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms nav · Pan/zoom/scroll
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Grader GET /pulls/42/reviews and /comments: one review state CHANGES_REQUESTED with exact body; 4 comments with correct path/line on the RIGHT side; one body contains a ```suggestion block equal to the fixture. gh unauthenticated; audit log detects API-token reviews -> 0.

Proposed partial credit

45% three comments on exact lines; 30% suggestion block exact; 25% review state and summary

R02.02.01.04-T07 Triage 15 Gitea issues by a rules file

Original instruction

In Gitea repo app/web apply fixtures/triage_rules.md to the 15 open issues created this week: add type and area labels, set milestone v2.4 for P1/P2, assign per area owner, and close duplicates with a comment 'Duplicate of #N' referencing the older issue.

Platforms: web · Any OS (web)
Applications: Gitea
Interfaces: brow · Browser api · API/SDK
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Gitea DB issue rows compared to the gold triage: label sets, milestone, assignee, closed state; duplicate comments name the right issue numbers. UI or API both allowed (hybrid).

Proposed partial credit

35% labels; 20% milestones; 20% assignees; 25% duplicates closed with correct reference

R02.02.01.04-T08 GitLab merge-request settings to team policy

Original instruction

Project platform/web: set merge method 'Fast-forward merge', squash 'Encourage', pipelines must succeed, all threads must be resolved, delete source branch by default, and the merge commit template '%{title} (%{reference})'. Leave other settings unchanged.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 2 / 5

Proposed verifier

Grader API project: merge_method=ff, squash_option=default_on, only_allow_merge_if_pipeline_succeeds=true, only_allow_merge_if_all_discussions_are_resolved=true, remove_source_branch_after_merge=true, merge_commit_template exact; settings snapshot diff shows no other change. No token; /api/v4 PUT detected -> 0.

Proposed partial credit

30% merge method; 40% four checkboxes and squash; 15% template exact; 15% no collateral changes

R02.02.01.04-T09 CODEOWNERS and required code-owner review on GitHub

Original instruction

In acme-sbx/billing add .github/CODEOWNERS from fixtures/owners.txt (teams for /api/, /infra/, *.sql) via the web editor on a branch, open and merge the PR, then enable 'Require review from Code Owners' in the existing branch protection for main.

Platforms: web · Any OS (web)
Applications: GitHub
Interfaces: brow · Browser
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

main contains CODEOWNERS equal to the fixture with no errors in GET /codeowners/errors; branch protection require_code_owner_reviews=true; a grader PR touching /infra/ requests review from the infra team automatically.

Proposed partial credit

40% CODEOWNERS valid on main; 30% code-owner review required; 30% auto-request verified

R02.02.01.04-T10 Transfer and rename a project between GitLab groups

Original instruction

Move project legacy/old-api into group platform, rename its path to 'api-v1' and name to 'API v1 (legacy)', archive it, and confirm the old clone URL still redirects. Do not change its visibility.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: risk · Irreversible/risky
Proposed difficulty: 3 / 5

Proposed verifier

Grader API: project path_with_namespace platform/api-v1, name exact, archived=true, visibility unchanged; git ls-remote on the old URL succeeds via redirect_routes. Agent has no token; transfer via API detected in production_json.log -> 0.

Proposed partial credit

40% transferred to platform; 25% path and name; 20% archived; 15% redirect works and visibility kept

R02.02.01.04-T11 Migrate a GitLab project into Gitea with issues and PRs

Original instruction

Use Gitea's 'New Migration' wizard to import gitlab.test/platform/sdk into org 'mirror' on git.test with issues, labels, milestones, merge requests, releases and wiki, using the read-only token in fixtures/gl_token.txt. Then make it a regular (non-mirror) repo.

Platforms: web · Any OS (web)
Applications: Gitea · GitLab
Interfaces: brow · Browser
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: multiapp · Multi-app
Proposed difficulty: 4 / 5

Proposed verifier

Gitea repo mirror/sdk: is_mirror=false; counts equal source: 58 issues, 14 labels, 4 milestones, 23 PRs, 6 releases, wiki pages 11; 5 sampled issue bodies and comments match; all branch SHAs equal.

Proposed partial credit

25% git data and branches; 35% issues, labels and milestones; 25% PRs and releases; 15% wiki

R02.02.01.04-T12 Open a draft PR from a fork with template and reviewers

Original instruction

As user 'sam' in the GitHub sandbox, fork acme-sbx/docs, edit docs/install.md in the web editor per fixtures/change.md on a new branch, and open a draft PR to upstream main that fills every section of the PR template, links issue #19 with 'Closes', and requests review from team 'docs-team'.

Platforms: web · Any OS (web)
Applications: GitHub
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 2 / 5

Proposed verifier

Grader API: PR from sam:<branch> draft=true; diff equals fixture; body keeps all template headings with non-empty sections; closingIssuesReferences contains #19; requested_teams includes docs-team. Sam has no PAT; audit log detects programmatic access -> 0.

Proposed partial credit

30% fork branch with correct change; 30% draft PR body complete; 20% issue linked; 20% team review requested

R02.02.01.04-T13 Cherry-pick and revert merged MRs with GitLab buttons

Original instruction

In platform/api, cherry-pick merged MR !198 into release/2.3 using the MR page's Cherry-pick action (start a new MR, title kept) and merge it; then revert MR !205 on main with the Revert action through a new MR and merge it.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

release/2.3 contains a commit with message 'cherry picked from commit <sha of !198>' and tree diff equal to !198; main contains 'Revert' commit of !205 merge; both via merged MRs whose source branches start cherry-pick-/revert-. Agent has no push creds or token, so CLI cherry-picks are blocked.

Proposed partial credit

50% cherry-pick on release/2.3 via MR; 50% revert on main via MR

R02.02.01.04-T14 Branch protection via API and glab (control pair of T01)

Original instruction

Control pair of T01: same project and the same three requirements, but you have a maintainer PAT in fixtures/pat.txt and glab is installed. Apply the protections with the REST API or glab only, and write the commands you ran to ~/protect.sh.

Platforms: lin · Linux
Applications: glab · curl · GitLab
Interfaces: term · Terminal api · API/SDK
GUI need · task setting: AP · API-sufficient
GUI need · ordinary access: AP · API-sufficient
Interaction primitives: code · Code & commands
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 2 / 5

Proposed verifier

Same protected_branches checks as T01; protect.sh re-runs idempotently against a fresh project copy and yields the same state.

Proposed partial credit

45% main rule exact; 35% release/* rule exact; 20% develop unprotected and script idempotent

R02.02.01.04-T15 GitLab push rules for commit hygiene

Original instruction

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Grader API push_rule fields exact (reject_unsigned_commits, commit_message_regex, branch_name_regex, max_file_size=25, prevent_secrets, member_check); grader test pushes: 4 bad pushes rejected, 1 good push accepted. No token; API PUT on push_rule detected -> 0.

Proposed partial credit

35% regex fields exact; 30% other four rules; 35% test pushes behave correctly

R02.02.01.04-T16 Deployment environments with reviewers and wait timer

Original instruction

In acme-sbx/billing create environments 'staging' (deploy from any branch) and 'production' (required reviewers team 'ops', 15-minute wait timer, deployment branches: main and tags v*). Add environment secret DEPLOY_KEY (dummy value in fixtures) to production only, then run the workflow 'deploy.yml' to production and approve it as 'ops-lead'.

Platforms: web · Any OS (web)
Applications: GitHub · GitHub Actions
Interfaces: brow · Browser
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms
Work product: WP08 · App/system state
Conditions: multiapp · Multi-app
Proposed difficulty: 4 / 5

Proposed verifier

Grader API: environment protection_rules required_reviewers [ops], wait_timer 15; deployment_branch_policy custom with main and v*; production secret DEPLOY_KEY exists, staging has none; a deployment run shows approval by ops-lead and succeeded after the wait.

Proposed partial credit

35% production protection rules; 20% branch policy; 15% secret scoped; 30% approved deployment run

R02.02.01.04-T17 Deploy key and scoped project access token rollout

Original instruction

In GitLab, enable the existing deploy key 'ci-readonly' (read-only) on projects platform/api, platform/web and platform/sdk; create a project access token 'renovate' on platform/web with role Developer, scopes api and write_repository, expiring 2026-12-31, and store its value in the CI/CD variable RENOVATE_TOKEN (masked, protected) of group platform.

Platforms: web · Any OS (web)
Applications: GitLab
Interfaces: brow · Browser
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms multi · Cross-app
Work product: WP08 · App/system state
Conditions: None declared
Proposed difficulty: 3 / 5

Proposed verifier

Grader API: deploy_keys_projects rows for 3 projects with can_push=false; project token renovate access_level 30, scopes exact, expires_at 2026-12-31; group variable masked+protected whose value authenticates as that token.

Proposed partial credit

30% deploy key on 3 projects read-only; 40% token with exact role, scopes and expiry; 30% variable stored masked and valid

R02.02.01.04-T18 Roll a branch policy across 12 projects in a group

Original instruction

Group 'payments' has 12 projects with inconsistent settings. Apply fixtures/policy.yaml to every project: protected main and release/*, MR approvals 2, pipelines must succeed, squash 'Require', and default branch 'main' (rename 'master' where needed, retargeting open MRs). Produce ~/policy_report.csv of before/after per project.

Platforms: web · Any OS (web) lin · Linux
Applications: GitLab · glab · Python
Interfaces: brow · Browser api · API/SDK term · Terminal
GUI need · task setting: HY · Hybrid
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click code · Code & commands
Work product: WP08 · App/system state
Conditions: long · Long-horizon
Proposed difficulty: 5 / 5

Proposed verifier

Grader API checks all 12 projects against the policy (protections, approvals_before_merge, pipeline rule, squash_option=always, default_branch=main); 9 open MRs retargeted to main; report rows match the grader's before/after diff.

Proposed partial credit

35% protections on all projects; 25% MR settings; 25% default branch renamed with MRs retargeted; 15% report correct

R02.02.01.04-T19 Enable a merge queue and land three PRs through it

Original instruction

On acme-sbx/billing enable a merge queue for main (squash, build concurrency 3, minimum group size 1, wait 2 minutes, require 'ci/test'). Then add PRs #51, #52 and #53 to the queue in that order; #52 will fail CI in the queue: remove it, fix the failing test in the web editor, and requeue it last.

Platforms: web · Any OS (web)
Applications: GitHub · GitHub Actions
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms nav · Pan/zoom/scroll
Work product: WP06 · Code/engineering
Conditions: interrupt · Interruptions
Proposed difficulty: 5 / 5

Proposed verifier

Grader API: ruleset merge_queue params exact; main history shows squash commits of #51, #53, #52 in that order, each via merge_group events; #52's fix commit authored via web edit. gh unauthenticated; org audit log detects programmatic queue operations -> 0.

Proposed partial credit

30% merge queue configured; 30% #51 and #53 landed via queue; 40% #52 fixed and landed last

R02.02.01.04-T20 Address review threads on a Gitea PR and merge with rebase

Original instruction

Gitea PR #31 in app/web has 4 open review threads. Apply the reviewer's suggestions for threads 1-3 in the web file editor on the PR branch, reply 'Won't fix: tracked in #40' on thread 4, resolve all four, re-request review from 'lee', and once approved (grader approves) merge with 'Rebase then create merge commit'.

Platforms: web · Any OS (web)
Applications: Gitea
Interfaces: brow · Browser
GUI need · task setting: G2 · GUI-essential
GUI need · ordinary access: G2 · GUI-essential
Interaction primitives: click · Point & click type · Text & forms nav · Pan/zoom/scroll
Work product: WP06 · Code/engineering
Conditions: None declared
Proposed difficulty: 4 / 5

Proposed verifier

PR branch diff equals gold after 3 fixes; thread 4 reply text exact; all review comments resolved; review request to lee recorded before approval; merge style rebase-merge visible in main history. Agent has no git push creds or token; edits must come through the web editor session.

Proposed partial credit

40% three suggestions applied exactly; 20% thread 4 reply and all resolved; 15% review re-requested; 25% merged with right style

Mapped research references

Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.

Repository hosting & merge-request administration

Task proposals · 20

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Original instruction

Proposed verifier

Proposed partial credit

Mapped research references

In this workflow family