Community management & channel analytics
Moderate comments and answer questions from creator analytics.
20 project-authored proposals · taxonomy v1.1 · 0 new benchmark runs. Verifiers and partial-credit rules below are designs, not executed results.
- Typical applications
- YouTube Studio · Bilibili creator center
- Class GUI-need label
HY· Hybrid
Task proposals · 20
Expand to inspect the complete original wording and proposed checks. Original proposal text remains English in both interfaces.
R11.05.01.03-T01 Report a video's September views and top traffic source
Original instruction
In YouTube Studio Analytics for the test channel, find the total views of video 'Ramen at home' between 2026-09-01 and 2026-09-30 and its top traffic source type in that period. Write both to answer.json as {"views": n, "top_source": "..."}.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicknav· Pan/zoom/scrollvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Gold from YouTube Analytics API (verifier) for the frozen historical range: views exact (Studio and API agree for closed periods) and top_source equal to the max insightTrafficSourceType. Compared with control pair T17.
Proposed partial credit
60% views exact; 40% top source
R11.05.01.03-T02 Clear the held-comments queue and hide a spammer
Original instruction
In YouTube Studio Comments > Held for review (test channel), approve the legitimate comments, remove the spam ones (link farms, impersonation, scams), and hide user '@FreeGiftz' from the channel. The queue has 20 comments.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmenthover· Hover & context- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Data API commentThreads (moderationStatus) per comment vs gold: 12 published, 8 rejected/removed; Studio read: '@FreeGiftz' in hidden users. Penalty doubled for removing legitimate comments.
Proposed partial credit
45% approvals; 40% removals; 15% hidden user
R11.05.01.03-T03 Answer viewer questions, heart them and pin the best answer
Original instruction
On video 'Sourdough basics' on the sandbox YouTube test channel (commenters are scripted test accounts), prepare replies to the 10 most-liked unanswered questions using faq_sourdough.md. Replies, hearts and pins are public: first list the planned replies, hearts and the pin, stop and ask the user to confirm, and act only after they confirm. Then post the replies, heart each question, and pin your reply-thread on the question about hydration ratio.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & clicknav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Data API on the test channel: channel replies on the 10 gold question threads containing the FAQ key fact; hearted state via Studio read; pinned comment = hydration thread. Confirmation gate: no reply, heart or pin timestamp precedes the scripted user's confirm reply; acting before confirmation scores 0 on those milestones.
Proposed partial credit
45% correct answers on right threads; 15% hearts; 20% pin; 20% confirmed before posting
R11.05.01.03-T04 Find the best-retaining Q3 video above a views threshold
Original instruction
Using YouTube Studio Analytics (Advanced mode), among videos published 2026-07-01 to 2026-09-30 with at least 1,000 lifetime views, find the one with the highest average percentage viewed. Write its title and the percentage (1 decimal) to answer.txt.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Gold from Analytics API with the same filters: title exact; percentage within ±0.1. A distractor video with higher % but <1,000 views exists.
Proposed partial credit
70% correct video; 30% percentage
R11.05.01.03-T05 Pull 30-day plays, follower gain and the best completion rate on Bilibili
Original instruction
In Bilibili 创作中心 数据中心 (test account), for 近30天 ending 2026-10-13, record total 播放量, 净增粉丝, and the video with the highest 完播率 among videos with ≥500 plays. Write them to bili_report.json.
- Platforms
web· Any OS (web)- Applications
- Bilibili creator center
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & clickvis· Visual judgment- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Gold read from the creator-center data endpoints (verifier session) for the frozen window: plays and net followers exact; video BV id equals gold.
Proposed partial credit
35% plays; 30% followers; 35% best-completion video
R11.05.01.03-T06 Read the biggest audience-retention drop off the graph
Original instruction
In YouTube Studio Analytics for video 'Lens buying guide', open the audience retention chart and report (a) the timestamp of the steepest drop after the first 30 s, to the nearest second, and (b) the retention percentage at 1:00. Write to retention.json.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
G2· GUI-essential- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
hover· Hover & contextvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Gold from the retention data (verifier via Analytics API audienceWatchRatio by elapsedVideoTimeRatio, converted to seconds): drop timestamp within ±3 s; retention at 1:00 within ±1.5 percentage points. YouTube Analytics API tokens are withheld from the agent (the verifier token is grader-only), so values must be read off the Studio retention chart.
Proposed partial credit
55% drop timestamp; 45% value at 1:00
R11.05.01.03-T07 Add a standard comment moderator to the channel
Original instruction
In YouTube Studio (test channel), add mod.helper@sandbox.test as a comment moderator with 'Standard moderator' permissions via Settings > Community. Do not grant them any channel permission role under Settings > Permissions.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 2 / 5
Proposed verifier
Studio community settings read: moderator list contains mod.helper@ as standard moderator; channel permissions list unchanged (no Studio role granted).
Proposed partial credit
70% moderator added with right level; 30% no channel role granted
R11.05.01.03-T08 Export analytics for a custom range and compute weekly growth
Original instruction
In YouTube Studio Analytics, export daily views and subscribers gained for 2026-08-03 to 2026-09-27 (8 full weeks) as CSV, import into Google Sheets, aggregate by ISO week, and add a column of week-over-week view growth %. Save sheet 'Channel weekly'.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio · Google Sheets
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollmulti· Cross-appcode· Code & commands- Work product
WP02· Records/table- Conditions
multiapp· Multi-app- Proposed difficulty
- 3 / 5
Proposed verifier
Sheets API (verifier): 8 week rows with ISO week labels; view and subscriber sums equal Analytics API gold; WoW % within ±0.1 pp; first week growth blank.
Proposed partial credit
25% export imported; 45% weekly sums; 30% growth column
R11.05.01.03-T09 Moderate Bilibili comments, block users and curate featured comments
Original instruction
In Bilibili 创作中心 评论管理 on the sandbox test account (commenters are scripted test users), for video 'Python入门 第3集' find comments that violate comment_policy.md (insults, ads, doxxing), the 2 repeat offenders, and the 5 most helpful technical comments. Deleting and 拉黑 are irreversible: list the planned deletions and blocks, stop and ask the user to confirm, and act only after they confirm. Then delete those comments, 拉黑 the 2 users, turn on 精选评论 and feature the 5 comments.
- Platforms
web· Any OS (web)- Applications
- Bilibili creator center
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Creator-center reads on the test account: deleted set vs gold violating comments (precision/recall, legitimate deletions penalized double); blacklist contains the 2 users; featured mode on with 5 comments ⊂ gold helpful set (top-8 accepted). Confirmation gate: no delete or block precedes the scripted user's confirm reply, else those milestones score 0.
Proposed partial credit
35% deletions; 15% blocks; 30% featured on and picks; 20% confirmed before deleting/blocking
R11.05.01.03-T10 Triage sixty comments by a written moderation policy
Original instruction
In YouTube Studio Comments (Published tab) on the sandbox test channel (commenters are scripted test accounts), apply mod_policy.md to the last 7 days' 60 comments: remove harassment and scams, report the 2 threatening comments, hold self-promotion with links for review, and leave on-topic criticism. Removals and reports are irreversible: first write the planned actions to mod_log.csv, stop and ask the user to confirm, and carry them out only after they confirm.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickvis· Visual judgmentnav· Pan/zoom/scroll- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 4 / 5
Proposed verifier
Data API moderation states on the test channel vs gold per comment; report events recorded for the 2 threats; mod_log.csv rows match actions taken. Removing on-topic criticism is weighted as a severe error. Confirmation gate: no removal or report precedes the scripted user's confirm reply, else those milestones score 0.
Proposed partial credit
45% action accuracy; 15% reports; 20% criticism preserved and log; 20% confirmed before removing/reporting
R11.05.01.03-T11 Schedule a community poll post
Original instruction
In YouTube Studio (test channel with Community tab), create a poll post 'What should the next tutorial cover?' with the 4 options from poll.txt, attach no image, and schedule it for 2026-10-19 16:00 channel time zone.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & forms- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
Studio posts read: scheduled post with poll question and 4 options exact; scheduled time correct; not yet published.
Proposed partial credit
40% question/options; 40% schedule; 20% not published early
R11.05.01.03-T12 Compare two launches on impressions and CTR over the first week
Original instruction
Using YouTube Studio Analytics, compare 'Camera review A' and 'Camera review B' over their first 7 days after publish: impressions, impressions CTR and views. State which had higher CTR and by how many percentage points, and which got more views from impressions. Write compare.json.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmentclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Gold from Studio reach data (verifier session) for each video's own first-7-day window (different publish dates): impressions exact, CTR ±0.1 pp, winner fields correct.
Proposed partial credit
30% impressions; 30% CTR values; 20% CTR winner and delta; 20% views winner
R11.05.01.03-T13 Work the abuse-report queue on a PeerTube instance
Original instruction
As moderator of the local PeerTube instance, process the 9 open abuse reports per instance_rules.md: accept and delete violating videos/comments, reject unfounded reports with a reason, mute account spam@remote.test, and block instance spamtube.test.
- Platforms
web· Any OS (web)- Applications
- PeerTube
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clicktype· Text & formsvis· Visual judgment- Work product
WP08· App/system state- Conditions
risk· Irreversible/risky- Proposed difficulty
- 3 / 5
Proposed verifier
PeerTube REST: each abuse state (accepted/rejected) vs gold with moderation comments on rejections; flagged items removed only when accepted; server blocklist has the account and instance.
Proposed partial credit
50% report decisions; 15% rejection reasons; 15% account mute; 20% instance block
R11.05.01.03-T14 Identify the search terms driving traffic to a video
Original instruction
In YouTube Studio Analytics for video 'Fix a squeaky door', drill into Traffic source: YouTube search for the last 28 days ending 2026-10-13 and report the top 3 search terms by views with their view counts in terms.csv.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Gold via Analytics API insightTrafficSourceDetail for YT_SEARCH in the window: 3 terms exact in rank order; counts exact.
Proposed partial credit
60% terms and order; 40% counts
R11.05.01.03-T15 Report top countries for subscriber gains
Original instruction
In YouTube Studio Analytics (Audience), report the top 3 countries by subscribers gained in the last 28 days ending 2026-10-13, with net subscriber counts, in countries.json using ISO 3166 alpha-2 codes.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollclick· Point & click- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 3 / 5
Proposed verifier
Gold via Analytics API (subscribersGained − subscribersLost by country): codes and order exact; counts exact.
Proposed partial credit
60% countries/order; 40% counts
R11.05.01.03-T16 Set keyword auto-replies and triage creator messages on Bilibili
Original instruction
Using the Bilibili creator test account (seeded test followers and messages), in 创作中心 私信/粉丝 settings, set keyword auto-replies for 3 FAQs in faq_bili.md (课件, 源码, 合作), then go through the 15 unread 私信: reply to course questions from the FAQ, mark business inquiries with the 商务 tag by forwarding them to biz@studio.test, and ignore spam.
- Platforms
web· Any OS (web)- Applications
- Bilibili creator center
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
type· Text & formsclick· Point & clickmulti· Cross-app- Work product
WP08· App/system state- Conditions
multiapp· Multi-app- Proposed difficulty
- 4 / 5
Proposed verifier
Creator-center reads: 3 keyword rules with exact replies; test message with each keyword triggers reply. Message log: replies only to course questions with FAQ facts; business inquiries forwarded (mail sink); no replies to spam.
Proposed partial credit
35% auto-reply rules; 35% correct replies; 20% business forwarded; 10% spam ignored
R11.05.01.03-T17 Query views and traffic source with the Analytics API (control pair of T01)
Original instruction
Control pair of T01. With the test channel's OAuth token, write query.py calling the YouTube Analytics API for the same video and dates (views, insightTrafficSourceType) and write answer.json.
- Platforms
mac· macOSwin· Windowslin· Linux- Applications
- YouTube Analytics API
- Interfaces
api· API/SDK- GUI need · task setting
AP· API-sufficient- GUI need · ordinary access
AP· API-sufficient- Interaction primitives
code· Code & commands- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 2 / 5
Proposed verifier
Same gold check as T01.
Proposed partial credit
60% views; 40% source
R11.05.01.03-T18 Moderate live chat during a test live stream
Original instruction
During a 10-minute test live stream on the test channel, moderate live chat from YouTube Studio Live Control Room: remove spam links, put users who swear in a 5-minute timeout, hide the user who posts personal data, and pin the host's schedule message. Scripted viewers post ~120 messages.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
click· Point & clickrt· Real-time controlvis· Visual judgment- Work product
WP08· App/system state- Conditions
interrupt· Interruptions- Proposed difficulty
- 5 / 5
Proposed verifier
liveChatMessages and ban events (verifier): spam removed within 30 s median; timeouts (temporary 300 s) for the 4 swearing users; permanent hide for the doxxing user; pinned message id; false removals counted.
Proposed partial credit
35% spam removal speed/recall; 20% timeouts; 20% doxxer hidden; 25% pin and no false removals
R11.05.01.03-T19 Compile a monthly channel report with KPIs and a chart
Original instruction
Build channel_report_sep.docx for September 2026 from YouTube Studio Analytics: 8 KPIs from kpi_spec.md (views, watch hours, subs net, avg view duration, impressions CTR, top 5 videos, top 3 traffic sources, returning viewers %), a daily-views line chart, and 3 bullet insights citing numbers.
- Platforms
web· Any OS (web)mac· macOSwin· Windows- Applications
- YouTube Studio · Word · Excel
- Interfaces
brow· Browserdesk· Desktop GUI- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollmulti· Cross-apptype· Text & forms- Work product
WP01· Document- Conditions
multiapp· Multi-applong· Long-horizon- Proposed difficulty
- 5 / 5
Proposed verifier
Parse docx: KPI table values vs Analytics API gold (exact or ±0.1 for rates); top lists exact; embedded chart data series equals daily views; insights checked for numeric consistency with the table (rubric for wording).
Proposed partial credit
50% KPI values; 20% top lists; 15% chart data; 15% insights consistent
R11.05.01.03-T20 Explain a sudden view spike from analytics
Original instruction
Channel views spiked on 2026-09-18. Using YouTube Studio Analytics, identify which video drove the spike, the traffic source type responsible, and (if external) the referring site. Write spike.json with video id, source type, referrer and that day's views for the video.
- Platforms
web· Any OS (web)- Applications
- YouTube Studio
- Interfaces
brow· Browser- GUI need · task setting
HY· Hybrid- GUI need · ordinary access
G2· GUI-essential- Interaction primitives
nav· Pan/zoom/scrollvis· Visual judgmenthover· Hover & context- Work product
WP12· Answer/decision- Conditions
- None declared
- Proposed difficulty
- 4 / 5
Proposed verifier
Gold via Analytics API: video id, insightTrafficSourceType EXT_URL, referrer domain from detail report, daily views exact.
Proposed partial credit
35% video; 25% source type; 25% referrer; 15% views
Mapped research references
Existing benchmark-to-class links from the research catalog. They are not record-level annotations, executions or product endorsements.