Teacher QC (Quality Control)

Teacher QC is a human-standardisation layer for Criteria-Based Assessment. Teachers record their own per-criterion verdicts and written feedback, and the system compares and scores them against the AI's evaluation.

The AI verdict always stays the official grade. Teacher QC never changes a student's result — it exists for standardisation (IQA/IV) and for coaching the quality of teacher feedback.

Two modes

Each batch runs in one mode; you can switch it at any time from the QC results page.

How feedback is scored (advisory)

Each criterion's teacher feedback is scored 0–100 on a set of axes and mapped to a headline band. By default the axes are Factual correctness, Specificity & actionability, and Alignment with the AI's feedback.

The scoring is deliberately fair and AI-anchored:

• Correctness is judged against the AI's own verdict, not a short excerpt. If the AI found a criterion met, a teacher who says the same thing scores high — the scorer never calls accurate feedback "wrong" just because the supporting passage was outside its reading window.
• Accurate-but-general feedback is fine. Quoting the work is a bonus, not a requirement; only vague boilerplate ("good work") scores low.
• Deductions are pinpoint. Every rationale quotes the teacher's actual words, names the specific gap for that criterion, and suggests a stronger version. Scores are reproducible across re-runs.

Custom rubrics

You can use the built-in rubric or define your own. Open Teacher QC → QC Rubrics.

• See and customise the default. The built-in default card shows its exact axes and scoring instructions; "Customise our default" copies them into an editable rubric for your organization.
• Build axes from scratch. Each axis has a name, scoring instructions (the text the AI follows), a weight, whether it counts toward the headline score, and whether it can be N/A.
• Per-axis performance levels. Give each axis named levels (e.g. Excellent / Good / Needs improvement), each with a descriptor and a 0–100 range. Levels appear as badges on the results and drive the level distribution in the report.
• Band thresholds. Move the Specific / Generic / Weak cut-offs.
• Where it applies. A rubric can be your organization default and/or chosen per batch. Resolution order: per-batch rubric → org default → built-in default.
• Who can edit. Organization admins and examiners.

A fixed safeguard always wraps every rubric (custom or default), so no rubric can reintroduce unfair scoring.

Results and reporting

• Results grid / comparison detail — each axis shows its level badge, descriptor, score bar, and pinpoint rationale, plus an aggregated "what went well / what to fix".
• Batch QC report (Report button on the results page) — band distribution, per-axis averages with level distribution, a per-teacher rollup, and per-criterion averages. Exportable to PDF and Word.
• Calibration dashboard — each teacher's agreement %, strictness bias, and top mismatches (comparative batches only).

Capturing teacher verdicts

Teacher verdicts and feedback can be entered:

• Inline on a submission,
• by CSV / Excel upload,
• or as free-form narrative (a single student's feedback letter or bulk files) that the AI parses into structured per-criterion verdicts.

After uploading verdicts, click Run comparison to compute the scores. Re-running on the same set of verdicts is free.

Quick start

1. Open Teacher QC and pick a batch that already has AI evaluations.
2. (Optional) QC Rubrics → Customise our default (or New rubric) to define your axes and performance levels; set it as the org default or pick it on the batch.
3. Upload teacher verdicts/feedback (inline, CSV/Excel, or narrative).
4. Choose Advisory or Comparative mode.
5. Click Run comparison.
6. Review the results grid, open a submission for detail, and download the Report (PDF/Word).