Packaging, signing & release
Produce signed installers or packages and a tagged release.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- Xcode · signtool · GitHub Releases
- Class GUI-need label
HY· Hybrid
Digital-employee context
Project class assignments, not demonstrated employee capability or imported benchmark coverage. Acceptance criteria and value metrics are proposals; no ROI has been measured.
Related roles
- IT and software delivery · Separate deliverables cover service tickets, restored configuration, identity records, repository fixes, passing CI and release packages.
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R02.02.01.03-T01 Sign, notarize and staple a macOS app for distribution outside the App Store
Original instruction
On the Mac, `~/build/Clipper.app` is unsigned. Sign it and all nested frameworks/helpers with the `Developer ID Application` identity of the sandbox team in the keychain, enabling Hardened Runtime with only the entitlements in `entitlements.plist`, submit it with notarytool using keychain profile `ci-sandbox`, staple the ticket, and zip it as `Clipper-2.3.zip`.
- Platforms
mac· macOS- Applications
- codesign · notarytool · Xcode
- Interfaces
term· Terminaldesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commandsclick· Point & click- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
`codesign --verify --deep --strict` OK; every Mach-O signed with runtime flag and same Team ID; entitlements equal the file; `spctl -a -vv` accepted 'Notarized Developer ID'; `stapler validate` OK on the app inside the zip.
Proposed partial credit
30% nested code signed correctly; 20% hardened runtime + entitlements; 30% notarized; 20% stapled zip
R02.02.01.03-T02 Authenticode-sign an MSI and verify it
Original instruction
On Windows, sign `C:\out\FieldSync-1.8.0.msi` with the test code-signing certificate `Lab Code Signing` in the CurrentUser store using SHA-256 and the sandbox RFC 3161 timestamp server `http://ts.lab.test`. Verify the signature.
- Platforms
win· Windows- Applications
- signtool
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
`signtool verify /pa /v` succeeds (lab root trusted on grader); digest SHA256; signer thumbprint equals the lab cert; RFC 3161 timestamp present from ts.lab.test; MSI content (streams other than signature) unchanged.
Proposed partial credit
40% signed with right cert; 20% SHA-256; 30% timestamp; 10% verified
R02.02.01.03-T03 Build a per-machine MSI installer with WiX including upgrade logic
Original instruction
Create a WiX v4 installer for `C:\work\FieldSync\publish\` (app + 40 files): per-machine install to Program Files, Start menu shortcut, file association for `.fsync`, major-upgrade handling with a fixed UpgradeCode, and an uninstall that leaves user data in `%APPDATA%` intact.
- Platforms
win· Windows- Applications
- WiX · msiexec
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Grader VM: msiexec /i silent install succeeds; files and shortcut present; .fsync opens the app (assoc); installing v1.8.1 built from the same project upgrades in place (one ARP entry); uninstall removes files, keeps %APPDATA% data.
Proposed partial credit
30% install layout; 20% shortcut + association; 30% major upgrade; 20% clean uninstall
R02.02.01.03-T04 Archive and export a Mac app with Developer ID through Xcode Organizer
Original instruction
In Xcode, archive the `Notely` scheme (Release), then use Organizer > Distribute App > Developer ID to export with the sandbox team's signing identity and automatic notarization upload, and save the exported app to `~/Exports/Notely/`. Fix the archive's version to 3.1 (build 310) first in the target's General tab.
- Platforms
mac· macOS- Applications
- Xcode
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsnav· Pan/zoom/scroll- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
~/Library/Developer/Xcode/Archives has a Notely archive with CFBundleShortVersionString 3.1 and CFBundleVersion 310; exported app signed Developer ID with hardened runtime; Organizer's distribution log (ExportOptions.plist method developer-id) present. eslogger exec events detect xcodebuild, notarytool or codesign runs not spawned by Xcode.app (e.g. from Terminal), which void the archive/export milestones.
Proposed partial credit
25% version set; 30% archive created; 30% Developer ID export; 15% notarization submitted
R02.02.01.03-T05 Build a wheel and sdist and publish to a sandbox package index
Original instruction
Build sdist and wheel for `/work/tinyorm` and upload them to the sandbox devpi index `http://devpi.lab.test/team/dev` using the token in `~/.config/devpi_token`, then verify installation from that index in a fresh venv.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- build · twine · devpi
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
devpi index lists both files for the project version with metadata matching pyproject; `twine check` passes; `pip install -i ... tinyorm==<ver>` in a clean venv imports successfully.
Proposed partial credit
40% builds pass twine check; 40% uploaded; 20% installable
R02.02.01.03-T06 Package a Linux daemon as a .deb with a systemd unit and maintainer scripts
Original instruction
Create a Debian package `metricsd_1.4.0_amd64.deb` from `/work/metricsd/dist/` that installs the binary to `/usr/bin`, config to `/etc/metricsd/` as a conffile, a systemd unit enabled on install, a system user created in postinst, and clean removal on purge. Lintian must be clean of errors.
- Platforms
lin· Linux- Applications
- dpkg-deb · debhelper · lintian
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
On a clean Debian VM: dpkg -i works; service active as user metricsd; edited config preserved on upgrade to 1.4.1 test package; purge removes user and config; lintian reports 0 errors.
Proposed partial credit
30% install + service; 25% conffile behavior; 25% purge; 20% lintian
R02.02.01.03-T07 Publish a CLI through a Homebrew tap with bottles
Original instruction
Create a Homebrew tap repo on the sandbox Git server for `/work/logq` (Go CLI): a formula that builds from the release tarball with checksum, installs shell completions and a man page, has a test block, and bottles for arm64_sonoma built in CI. `brew install lab/tap/logq` must work on the Mac.
- Platforms
mac· macOS- Applications
- Homebrew · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
`brew audit --strict --new lab/tap/logq` clean; `brew install` pours the bottle (no build from source) on arm64; completions and man page installed; `brew test` passes.
Proposed partial credit
35% formula installs; 20% audit clean; 25% bottle poured; 20% completions/man/test
R02.02.01.03-T08 Draft a GitHub release with changelog, artifacts and checksums
Original instruction
For `/work/cli-tool` on the sandbox GitHub-compatible forge, create tag `v2.4.0` on `main`, generate release notes from Conventional Commits since `v2.3.0` grouped by type, attach the 6 built artifacts from `dist/` plus `SHA256SUMS`, and leave the release as a draft for maintainer approval.
- Platforms
web· Any OS (web)lin· Linux- Applications
- GitHub Releases · git
- Interfaces
brow· Browserterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formscode· Code & commands- Work product
WP06· Code/engineering- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Forge API: tag v2.4.0 at main HEAD; release draft=true (not published); body has Features/Fixes sections listing exactly the commits in range; 7 assets; SHA256SUMS lines match asset hashes.
Proposed partial credit
15% tag; 35% notes accurate; 30% assets + checksums; 20% left as draft
R02.02.01.03-T09 Repackage a desktop installer as MSIX with the MSIX Packaging Tool
Original instruction
Use the MSIX Packaging Tool on the Windows VM to convert `C:\installers\LabelDesigner_5.2.exe` (an EXE installer with a silent switch) into `LabelDesigner.msix`, signed with the lab test certificate, with the publisher name matching the cert subject, the package display name `Label Designer`, and the first-launch tasks cleaned of the installer's update checker.
- Platforms
win· Windows- Applications
- MSIX Packaging Tool · signtool
- Interfaces
desk· Desktop GUI- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicktype· Text & formsnav· Pan/zoom/scroll- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Add-AppxPackage on a clean VM succeeds; AppxManifest Publisher equals cert subject, DisplayName exact; app launches; no updater executable entry point in manifest; signature verifies. Sysmon detects MakeAppx.exe or MsixPackagingTool CLI (create-package) runs, which void the packaging milestones; signing with signtool is allowed.
Proposed partial credit
35% package installs; 25% manifest fields; 20% updater removed; 20% signature
R02.02.01.03-T10 Build a signed macOS .pkg installer with a distribution file
Original instruction
Produce `MetricsAgent-1.4.0.pkg` for macOS: component package installing the app to /Applications and a LaunchDaemon plist, preinstall/postinstall scripts that stop and load the daemon, a distribution file with a license and min OS 13 check, signed with the sandbox `Developer ID Installer` identity.
- Platforms
mac· macOS- Applications
- pkgbuild · productbuild · productsign
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
`pkgutil --check-signature` shows Developer ID Installer chain; installer -pkg on a test VM succeeds; daemon loaded (launchctl print system/...); installation on macOS 12 VM refused by the volume check; license shown in distribution XML.
Proposed partial credit
30% payload layout; 25% scripts load daemon; 20% distribution checks; 25% signed
R02.02.01.03-T11 Ship an Electron app for macOS and Windows with an auto-update feed
Original instruction
Configure electron-builder for `/work/notes-desktop` to produce a signed and notarized dmg (sandbox Developer ID) and a signed NSIS installer (lab cert), plus `latest.yml`/`latest-mac.yml` update feeds on the sandbox update server, so v1.0.0 auto-updates to v1.0.1.
- Platforms
mac· macOSwin· Windows- Applications
- electron-builder · codesign · signtool
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Both installers verify (spctl / signtool); feeds contain correct sha512 and version; on test VMs, installed 1.0.0 detects and applies 1.0.1 (version file after restart).
Proposed partial credit
25% mac build signed+notarized; 25% Windows signed; 20% feeds valid; 30% update works on both
R02.02.01.03-T12 Sign a container image and attach an SBOM attestation
Original instruction
Build `registry.lab.test/team/api:1.9.0` from `/work/api`, generate an SPDX SBOM with syft, sign the image with cosign using the key pair in `~/keys/` (password in env), attach the SBOM as an in-toto attestation, and write a policy check script that verifies both before deploy.
- Platforms
lin· Linux- Applications
- Docker · cosign · syft
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
`cosign verify --key` and `cosign verify-attestation --type spdxjson` succeed for the digest; SBOM lists the 3 known base packages; the policy script exits 0 for this image and 1 for an unsigned hidden image.
Proposed partial credit
25% image pushed; 30% signature; 30% SBOM attestation; 15% policy script
R02.02.01.03-T13 Prepare an npm package release with provenance and dry-run publish
Original instruction
Prepare `/work/tinyvalidate` for release 3.0.0: update version and changelog, ensure `files` excludes tests and fixtures, build, and run a publish to the sandbox Verdaccio registry with provenance metadata from the sandbox CI. Stop before tagging `latest` on the public registry.
- Platforms
lin· Linuxmac· macOSwin· Windows- Applications
- npm · Verdaccio
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Verdaccio has 3.0.0 with tarball containing only dist/, README, LICENSE, package.json; package.json version 3.0.0; CHANGELOG entry; no publish attempt to registry.npmjs.org (proxy log).
Proposed partial credit
30% version/changelog; 30% tarball contents; 30% sandbox publish; 10% no public publish
R02.02.01.03-T14 Sign every PE file in a .NET app's output and produce a catalog
Original instruction
The publish folder `C:\work\Scheduler\publish` contains 37 PE files (exe and DLLs, some third-party already signed). Sign only unsigned first-party binaries with the lab certificate and timestamp, leave valid third-party signatures untouched, and create and sign a catalog file covering the rest of the content.
- Platforms
win· Windows- Applications
- signtool · MakeCat
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
All first-party PE files verify with the lab cert and timestamp; third-party files keep their original signer and hashes; catalog signed and `signtool verify /c` succeeds for non-PE files listed; no double-signed files.
Proposed partial credit
40% first-party signed; 25% third-party untouched; 25% catalog; 10% timestamps
R02.02.01.03-T15 Automate semantic versioning and changelog generation for releases
Original instruction
Configure semantic-release (or release-please) for `/work/sdk-js` so merges to `main` compute the next version from Conventional Commits, update CHANGELOG.md, tag, and create a draft release on the sandbox forge. Dry-run it on the provided commit history and save the output.
- Platforms
lin· Linux- Applications
- semantic-release · git
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Dry-run output computes 4.0.0 (history contains a BREAKING CHANGE footer) not 3.x; generated notes include the expected 7 entries; config uses the sandbox forge URL and draft releases; commitlint config present.
Proposed partial credit
40% correct version computed; 30% notes; 20% config correct; 10% commit lint
R02.02.01.03-T16 Sign an Android App Bundle and upload it to an internal test track draft
Original instruction
Bump `/work/fieldnotes-android` to versionCode 42/versionName 2.6.0, build a release AAB signed with the upload key in `keystore.jks` (password in env), and upload it in the sandbox Play Console-like web app to the Internal testing track as a draft with release notes from `notes.txt`. Do not roll out.
- Platforms
web· Any OS (web)lin· Linux- Applications
- Gradle · jarsigner · Play Console sandbox
- Interfaces
term· Terminalbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
code· Code & commandsclick· Point & clicktype· Text & forms- Work product
WP06· Code/engineering- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
AAB signed by the upload cert (apksigner/jarsigner verify), versionCode 42; sandbox console DB: internal track draft release containing that bundle hash with notes text; status draft (not rolled out).
Proposed partial credit
25% version bump; 30% signed AAB; 30% uploaded draft with notes; 15% not rolled out
R02.02.01.03-T17 Fix provisioning-profile mismatches to export an ad-hoc iOS build
Original instruction
Exporting `FieldCam` for ad-hoc distribution in Xcode fails: the app extension uses a different bundle ID prefix, the profile lacks the Push entitlement the target requests, and automatic signing is half-enabled. Using Xcode's Signing & Capabilities and the sandbox team's profiles, produce `FieldCam.ipa` for ad-hoc devices.
- Platforms
mac· macOS- Applications
- Xcode · xcodebuild
- Interfaces
desk· Desktop GUIterm· Terminal- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollcode· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
IPA exists; app and extension embedded.mobileprovision are ad-hoc profiles of the same team; entitlements match profiles (codesign -d --entitlements); bundle IDs consistent prefix; installs on the test simulator-equivalent device check.
Proposed partial credit
30% bundle IDs fixed; 30% entitlements/profiles consistent; 30% IPA exported; 10% install check
R02.02.01.03-T18 Package a desktop app as a Flatpak with a strict sandbox
Original instruction
Write a Flatpak manifest for `/work/inkboard` (GTK4 app, Meson build) using the GNOME 46 runtime, with finish-args limited to Wayland/X11 fallback, GPU, and the portal for file access (no `--filesystem=home`). Build it, export to a local repo, install and run.
- Platforms
lin· Linux- Applications
- flatpak-builder · Flatpak
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
flatpak-builder succeeds; `flatpak info --show-permissions` lists only allowed permissions (no filesystem=home/host); app launches headless smoke test (xvfb) and opens a file via the portal test.
Proposed partial credit
35% builds; 35% permissions minimal; 30% runs + portal works
R02.02.01.03-T19 Run a full cross-platform release: bump, build, sign, checksum and draft
Original instruction
Release `/work/syncer` 5.0.0 end to end: bump the version in 5 files, update the changelog, build macOS (universal), Windows and Linux artifacts, sign/notarize mac with sandbox Developer ID, sign Windows with the lab cert, produce SHA256SUMS and a detached GPG signature, and create a draft release on the sandbox forge with all assets.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- codesign · signtool · gpg · GitHub Releases
- Interfaces
term· Terminalbrow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
HY· Hybrid- Interaction primitives
code· Code & commandsclick· Point & clickmulti· Cross-app- Work product
WP06· Code/engineering- Conditions
long· Long-horizonmultiapp· Multi-apprisk· Irreversible/risky- Proposed difficulty
- 5 / 5
Proposed verifier
All 5 version strings 5.0.0; mac binary universal (lipo) and notarized; Windows signed+timestamped; SHA256SUMS correct and GPG signature verifies with the release key; forge draft has all assets; nothing published.
Proposed partial credit
15% version bump; 30% platform artifacts; 30% signatures; 25% checksums and draft-only release
R02.02.01.03-T20 Rotate an APT repository signing key without breaking existing clients
Original instruction
The sandbox APT repo at `/srv/apt` is signed with an expiring GPG key. Generate a new signing key, publish it via a keyring package and the repo, re-sign Release/InRelease with both keys during the transition, and verify that a client trusting only the old key and one trusting only the new key can both `apt update` cleanly.
- Platforms
lin· Linux- Applications
- gpg · reprepro · apt
- Interfaces
term· Terminal- GUI need · task setting
CL· CLI-sufficient- GUI need · ordinary access
CL· CLI-sufficient- Interaction primitives
code· Code & commands- Work product
WP06· Code/engineering- Conditions
risk· Irreversible/risky- Proposed difficulty
- 5 / 5
Proposed verifier
InRelease has two valid signatures (old and new fingerprints); keyring package installs the new key to /usr/share/keyrings; two test clients (old-only, new-only) apt update with no GPG errors; old key not deleted from the signer.
Proposed partial credit
25% new key; 35% dual-signed metadata; 25% both clients succeed; 15% keyring package
Mapped research references
No benchmark reference is assigned to this class in the current mapping.