R15.01.01.03 · 任务小类

量规评分与行内反馈

Rubric grading with inline feedback

Grade submissions against rubrics with annotations.

20 条本项目编写的提案 · 分类 v1.1 · 新增基准运行 0 次。下方验证方式与部分得分规则是设计，不是执行结果。

历史小类地址：R10.05.01.03 · 提案 ID 保留原值。

典型应用: SpeedGrader · Gradescope · Turnitin
小类 GUI 需求标签: G2 · GUI 必要

任务提案 · 20

展开查看完整原文与建议检查方式；两种界面均保留提案英文原文。

R10.05.01.03-T01 Score three short essays on a three-criterion rubric in SpeedGrader

原始指令

Canvas SpeedGrader, course 'ENGL 101 (sandbox)', assignment 'Reflection 1' (3 submissions, synthetic students). Score each on the attached rubric (Thesis, Evidence, Mechanics; 4 levels each) using input/grading_notes.pdf, which gives the instructor's judgments, and add a one-sentence summary comment per student.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 2 / 5

建议验证方式

rubric_assessment per submission: rating per criterion equals the notes; total equals sum; one comment each. Agent has no API token; submissions show grader_id of the UI session.

建议部分得分

75% criterion ratings; 25% comments.

R10.05.01.03-T02 Grade five essays with inline comments anchored on each thesis sentence

原始指令

Canvas SpeedGrader, assignment 'Argument Essay' (5 PDF submissions, synthetic). Score each on the 4-criterion rubric in input/rubric_guide.pdf, then in the DocViewer highlight each essay's thesis sentence and attach a comment that names the strength or weakness of the thesis, and add a summary comment.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: path · 连续路径 click · 指向与点击 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Per-criterion MAE vs expert gold <= 0.5; DocViewer annotations: one highlight per essay overlapping the gold thesis sentence bbox >= 70% with a comment; summary comment present. Canvas API token creation is disabled for the agent's role, and DocViewer annotations are not exposed by the Canvas API, so they can be made only in the DocViewer UI.

建议部分得分

45% rubric MAE; 40% thesis highlights with comments; 15% summaries.

R10.05.01.03-T03 Mark three induction proofs with negative rubric items in Gradescope

原始指令

Gradescope (sandbox course), assignment 'HW4 Induction', question 2 (3 scanned proofs). Create rubric items 'Missing base case (-3)', 'Inductive hypothesis not used (-4)', 'Algebra error (-1)' and 'Correct (0)' under negative scoring, apply them per input/key.pdf, and leave a point comment on the line with each algebra error.

平台: web · 任意系统（网页）
应用: Gradescope
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 vis · 视觉判断 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Evaluations export: rubric items defined with exact points; applied item sets per submission equal gold; algebra-error comments anchored on the correct page region (within 40 px of gold line). The Gradescope sandbox issues no API token and accepts grading only from browser sessions (CSRF + session cookie), so grading must be done in the web UI.

建议部分得分

25% rubric items; 55% applied sets; 20% anchored comments.

R10.05.01.03-T04 Grade one exam question across twenty scanned submissions with answer groups

原始指令

Gradescope sandbox, 'Midterm 1', question 2 (short derivation; 20 scanned exams). Build the rubric from input/answer_key.pdf, group identical final answers into answer groups, grade each group consistently while checking the work, adjust two rubric items' points after noticing a key error (input/key_erratum.txt), and confirm the change propagated.

平台: web · 任意系统（网页）
应用: Gradescope
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 vis · 视觉判断 nav · 平移 / 缩放 / 滚动
工作产物: WP10 · 标注
条件: 未声明
提案难度: 4 / 5

建议验证方式

Per-submission score equals gold (after erratum) for >= 19 of 20; rubric item points equal corrected key; answer groups present; scores for all submissions reflect the adjusted points. The Gradescope sandbox issues no API token and accepts grading only from browser sessions (CSRF + session cookie), so grading must be done in the web UI.

建议部分得分

60% scores; 20% rubric correction propagated; 20% answer groups.

R10.05.01.03-T05 Review a similarity report and mark unattributed passages in Turnitin

原始指令

Turnitin Feedback Studio (sandbox, via Canvas LTI), assignment 'Policy Brief'. For student submission S-1188, exclude quotes and bibliography as the course policy in input/integrity_policy.pdf allows, then mark each remaining match that is unattributed copying with the QuickMark 'Missing citation' and a comment naming the source, and enter the rubric grade from input/notes.pdf.

平台: web · 任意系统（网页）
应用: Turnitin Feedback Studio
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 path · 连续路径 vis · 视觉判断
工作产物: WP10 · 标注
条件: licensed · 授权专业软件
提案难度: 3 / 5

建议验证方式

Report settings: quotes/bibliography excluded; QuickMarks anchored on the 3 gold unattributed passages (char overlap >= 60%) with source named; no QuickMark on properly cited matches; grade equals notes. Turnitin is reachable only through the Canvas LTI launch and no Turnitin API key is issued, so QuickMarks must be placed in Feedback Studio.

建议部分得分

15% filters; 55% passages marked; 15% no false marks; 15% grade.

R10.05.01.03-T06 Grade a forty-student four-question exam with consistent partial credit

原始指令

Gradescope sandbox, 'Final Exam' (40 scanned exams, 4 questions). Using input/solutions.pdf and the department partial-credit guide, build rubrics for each question, grade every submission, apply consistent point adjustments for the 2 common alternative methods, then publish grades and export the evaluations to output/final_evals.csv.

平台: web · 任意系统（网页）
应用: Gradescope
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 vis · 视觉判断 nav · 平移 / 缩放 / 滚动
工作产物: WP10 · 标注
条件: long · 长流程
提案难度: 5 / 5

建议验证方式

Per-question scores vs expert gold: MAE <= 0.5 points and >= 90% exact; same rubric item applied to equivalent answers (consistency check on seeded duplicates); grades published; export matches the platform state. The Gradescope sandbox issues no API token and accepts grading only from browser sessions (CSRF + session cookie).

建议部分得分

55% score accuracy; 20% consistency; 15% published; 10% export.

R10.05.01.03-T07 Annotate a PDF submission and grade with a marking guide in Moodle

原始指令

Moodle sandbox, assignment 'Lab Report 1', student Ana Ruiz (test). In the PDF annotation tool, add a comment at the mislabeled axis on page 2 and a 'tick' stamp next to the correct conclusion on page 4, then score the 3-criterion marking guide per input/notes.txt and save.

平台: web · 任意系统（网页）
应用: Moodle assignment
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 geom · 精确几何 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 1 / 5

建议验证方式

assignfeedback_editpdf_cmnt/annot: comment on page 2 within 60 px of gold axis label, stamp on page 4 near conclusion; gradingform_guide_fillings scores equal notes. Web services are disabled for the role and the agent has no server shell; only web-origin log events are accepted.

建议部分得分

35% comment placement; 25% stamp; 40% marking-guide scores.

R10.05.01.03-T08 Grade with a Moodle marking guide using frequently used comments

原始指令

Moodle sandbox, assignment 'Case Analysis' (6 submissions). The marking guide has 4 criteria with a frequently-used-comments bank. Grade each submission per input/grading_decisions.xlsx (scores and which bank comments apply), inserting bank comments from the guide rather than typing new text, and add one custom remark where the decisions sheet says so.

平台: web · 任意系统（网页）
应用: Moodle assignment · Excel
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单 multi · 跨应用
工作产物: WP10 · 标注
条件: multiapp · 多应用
提案难度: 3 / 5

建议验证方式

Fillings: scores exact; criterion remarks equal the bank comment texts specified (exact string) plus the custom remarks; 6 grades saved. Web services are disabled for the role and the agent has no server shell; only web-origin log events are accepted.

建议部分得分

50% scores; 35% bank comments; 15% custom remarks.

R10.05.01.03-T09 Grade a group project individually with peer-evaluation adjustments

原始指令

Canvas SpeedGrader, group assignment 'Capstone Report' (graded individually, 4 groups). Score each group's report on the rubric from input/rubric_scores.csv, then adjust individual members' scores using the peer-evaluation multipliers in input/peer_evals.xlsx (cap at 100%), and leave a private comment for each adjusted student explaining the factor.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader · Excel
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单 multi · 跨应用
工作产物: WP10 · 标注
条件: multiapp · 多应用
提案难度: 4 / 5

建议验证方式

16 student scores equal gold (group rubric x multiplier, capped); rubric assessments present for each; private comments on the 5 adjusted students with the factor value. Personal access token creation is disabled for the agent's role, so Canvas API calls are blocked and grading must be done in SpeedGrader.

建议部分得分

30% group rubric scores; 45% adjusted individual scores; 25% comments.

R10.05.01.03-T10 Circle errors on an image submission with the drawing tool

原始指令

Canvas SpeedGrader, assignment 'Circuit Diagram' (student submissions are PNG images). For the 3 submissions listed in input/errors.txt, use the DocViewer freehand/shape tools to circle each wiring error described, add a point comment explaining it, and enter the score.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: path · 连续路径 vis · 视觉判断 click · 指向与点击
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Annotations: each of 7 gold error regions enclosed by an ink/shape annotation (bbox IoU >= 0.4) with a nearby point comment; no circles on correct regions; scores exact. Canvas API token creation is disabled for the agent's role, and DocViewer annotations are not exposed by the Canvas API, so they can be drawn only in DocViewer.

建议部分得分

55% error regions circled; 25% comments; 20% scores.

R10.05.01.03-T11 Override flaky autograder results and add a style rubric in Gradescope

原始指令

Gradescope sandbox programming assignment 'PA3'. Review autograder logs for 12 submissions; for the 4 where a test failed due to the known timeout flake documented in input/flake_note.md, override the test score to full credit with an explanation. Then add a manual 'Code style' question with rubric items from input/style_rubric.pdf and grade it for all 12 using the code viewer.

平台: web · 任意系统（网页）
应用: Gradescope
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 nav · 平移 / 缩放 / 滚动 vis · 视觉判断
工作产物: WP10 · 标注
条件: 未声明
提案难度: 4 / 5

建议验证方式

Overrides exactly on the 4 flaky submissions (other failures untouched) with explanation; style question exists with rubric items exact; style item sets vs gold F1 >= 0.85. The Gradescope sandbox issues no API token and accepts grading only from browser sessions (CSRF + session cookie), so overrides and grading must be done in the web UI.

建议部分得分

40% correct overrides; 15% style rubric; 45% style grading.

R10.05.01.03-T12 Grade twelve lab reports to calibration with anchored data-table comments

原始指令

Canvas SpeedGrader, 'Lab Report 3' (12 PDFs). Grade with the department rubric (5 criteria) following the calibration examples in input/calibration/, and for every incorrect value in a student's data table place a point annotation on that cell with the correct value. Your scores will be compared with the department's expert grades.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: vis · 视觉判断 click · 指向与点击 nav · 平移 / 缩放 / 滚动
工作产物: WP10 · 标注
条件: long · 长流程
提案难度: 5 / 5

建议验证方式

Quadratic weighted kappa vs expert criterion scores >= 0.7; point annotations: recall of the 31 seeded wrong cells >= 0.85 (within the cell bbox) with the correct value in the comment; precision >= 0.8. Canvas API token creation is disabled for the agent's role, and DocViewer annotations are not exposed by the Canvas API, so they can be placed only in DocViewer.

建议部分得分

50% rubric agreement (QWK); 35% wrong-cell annotations; 15% precision.

R10.05.01.03-T13 Annotate and rubric-grade submissions in Brightspace and publish feedback

原始指令

D2L Brightspace sandbox, course 'NURS 310 (sandbox)', assignment 'Care Plan'. For 4 submissions, use the annotation tool to highlight the nursing diagnosis statement and add a comment, score the analytic rubric per input/notes.xlsx, and publish the feedback so students can see it.

平台: web · 任意系统（网页）
应用: Brightspace
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: path · 连续路径 click · 指向与点击 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Annotation export: highlight overlaps gold diagnosis statement >= 60% with comment; rubric levels exact; feedback published flag true for all 4. Valence API app keys are withheld from the agent, so annotations and rubric scores must be entered in the Brightspace UI from the browser session.

建议部分得分

35% annotations; 45% rubric levels; 20% published.

R10.05.01.03-T14 Fill a Turnitin rubric scorecard and leave summary feedback

原始指令

Turnitin Feedback Studio sandbox, assignment 'Policy Brief', submission S-1203. Open the rubric scorecard, select levels for its 4 criteria as in input/notes.txt, and add the summary text comment provided.

平台: web · 任意系统（网页）
应用: Turnitin Feedback Studio
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单
工作产物: WP10 · 标注
条件: licensed · 授权专业软件
提案难度: 2 / 5

建议验证方式

Scorecard levels exact; computed grade equals rubric; summary comment text exact. Turnitin is reachable only through the LMS LTI launch and no Turnitin API key is issued, so the scorecard must be filled in Feedback Studio.

建议部分得分

70% rubric levels; 30% summary comment.

R10.05.01.03-T15 Box incorrect steps in scanned handwritten math homework

原始指令

Canvas SpeedGrader, 'Problem Set 6' (scanned handwritten PDFs). For 6 submissions, draw a box around each incorrect step (not just the final answer), attach a point comment naming the error type from input/error_types.md, and score each problem per the partial-credit rules in input/key.pdf.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: geom · 精确几何 vis · 视觉判断 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 4 / 5

建议验证方式

Box annotations vs 18 gold error-step regions (IoU >= 0.3, one-to-one); error-type labels in comments exact; per-problem scores equal gold for >= 90%. Canvas API token creation is disabled for the agent's role, and DocViewer annotations are not exposed by the Canvas API, so boxes can be drawn only in DocViewer.

建议部分得分

45% boxes on wrong steps; 20% error types; 35% scores.

R10.05.01.03-T16 Build a comment library and apply it consistently to ten submissions

原始指令

Canvas SpeedGrader, 'Lab Report 2' (10 submissions). Create the 6 reusable comments in input/comment_bank.txt in the Comment Library, then for each submission add exactly the library comments listed for it in input/assignments.csv and the score given there.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Comment library contains the 6 comments; each submission's comments equal the listed set (exact text); scores exact. Personal access token creation is disabled for the agent's role, so Canvas API calls are blocked and comments must be added in SpeedGrader.

建议部分得分

25% library; 50% comments applied; 25% scores.

R10.05.01.03-T17 Leave inline code-review feedback on a GitHub Classroom pull request

原始指令

GitHub Classroom (sandbox org), assignment 'linked-list'. For 3 student repositories, open the Feedback pull request, add review comments on the exact lines with the defects listed in input/defects.csv (off-by-one, leaked node, missing null check), fill the rubric score in the PR description template, and submit each review as 'Request changes'.

平台: web · 任意系统（网页）
应用: GitHub Classroom · GitHub
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 type · 文字与表单 nav · 平移 / 缩放 / 滚动
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Grader reads PR reviews: comments on the gold file/line for each defect (exact line, defect named); PR body rubric totals exact; review state CHANGES_REQUESTED. Agent's token lacks API scope; reviews created via web UI.

建议部分得分

55% comments on correct lines; 25% rubric; 20% review state.

R10.05.01.03-T18 Investigate five high-similarity submissions and score with integrity deductions

原始指令

Canvas with Turnitin Feedback Studio (sandbox), assignment 'Research Essay'. Five submissions exceed 35% similarity. For each, separate properly quoted and cited matches from copying, mark copied passages with QuickMarks naming the source, apply the rubric with the integrity deduction from input/policy.pdf, and file the integrity report form in the sandbox portal for the cases that meet the referral threshold.

平台: web · 任意系统（网页）
应用: Turnitin Feedback Studio · Canvas SpeedGrader · integrity portal (sandbox)
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: vis · 视觉判断 path · 连续路径 multi · 跨应用
工作产物: WP10 · 标注
条件: licensed · 授权专业软件 multiapp · 多应用 long · 长流程
提案难度: 5 / 5

建议验证方式

Per submission: QuickMarks on gold copied passages (F1 >= 0.8), none on cited ones; rubric score with deduction exact; integrity reports filed for exactly the 2 cases over threshold with source list. No Turnitin or portal API keys are issued and Canvas token creation is disabled, so marks, scores and reports must be entered in the web UIs.

建议部分得分

40% copied-passage marking; 25% scores with deductions; 25% correct referrals; 10% no false marks.

R10.05.01.03-T19 Submit the same five rubric assessments through the Canvas API (control)

原始指令

Control pair of T02. With a token in env CANVAS_TOKEN, post rubric assessments and summary comments for the 5 'Argument Essay' submissions via the submissions API, using the same rubric guide; inline thesis annotations are out of scope.

平台: lin · Linux
应用: Canvas API · Python
交互表面: api · API / SDK term · 终端
GUI 需求 · 任务设定: AP · API 足够
GUI 需求 · 普通权限: AP · API 足够
交互原语: code · 代码与命令
工作产物: WP10 · 标注
条件: 未声明
提案难度: 3 / 5

建议验证方式

Same rubric MAE metric as T02; comments present. Isolates scoring judgment from DocViewer annotation work.

建议部分得分

75% rubric MAE; 25% comments.

R10.05.01.03-T20 Finalize moderated grades from two TAs' provisional marks

原始指令

Canvas sandbox, moderated assignment 'Thesis Proposal' (8 submissions, two TA provisional graders). As final grader, compare each pair of provisional rubric assessments; where they differ by more than one level on any criterion, read the submission and enter your own final rubric; otherwise select the provisional grade from the TA named in input/policy.txt. Then post grades.

平台: web · 任意系统（网页）
应用: Canvas SpeedGrader
交互表面: brow · 浏览器
GUI 需求 · 任务设定: G2 · GUI 必要
GUI 需求 · 普通权限: G2 · GUI 必要
交互原语: click · 指向与点击 vis · 视觉判断 nav · 平移 / 缩放 / 滚动
工作产物: WP10 · 标注
条件: risk · 高风险 / 不可逆
提案难度: 4 / 5

建议验证方式

Moderation export: selected provisional grade source per submission equals gold rule; the 3 high-disagreement submissions have final-grader rubrics with criterion MAE <= 0.5 vs expert; grades posted. Personal access token creation is disabled for the agent's role, so Canvas API calls are blocked and moderation must be done in the web UI.

建议部分得分

40% correct selections; 40% own rubrics on disputed items; 20% posted.

已有研究映射

以下沿用研究目录中的基准与小类映射，不是逐题标注、执行记录或产品背书。

Agents' Last Exam (ALE)

量规评分与行内反馈

任务提案 · 20

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

原始指令

建议验证方式

建议部分得分

已有研究映射

同工作流族的小类