Assessment integrity
What AI does to the validity of assessment.
-
Automation bias is a code problem
Automation bias isn't solved by better UI, training, or disclaimers — those are nudges that fail under load. In AI-validated credentialing, the only thing that holds is code that structurally forces human engagement at the exact points where a human would otherwise defer. Automation bias is a code problem, not a training problem.
-
Assessment validity is the new security perimeter
Generative AI collapsed take-home assessment validity, and proctoring is a failed patch — bypassable and biased. The right frame is security engineering: assessment integrity is a custody problem, not a policing problem, and the perimeter moved to the evidence.
-
Council design for assessment: quorum, agreement metrics, and the cost of conviction
A multi-model AI council that grades work is a measurement instrument, not a voting trick. Its validity depends on design: quorum that makes disagreement visible, agreement metrics that catch correlated rubber-stamping, conviction-weighted dissent, and drift monitoring for closed models that update silently.
-
The explainability gap: what can a learner appeal when a model council fails them?
When an AI council grades a learner's work and the learner disagrees, what can they appeal? A chain-of-thought rationale is a story, not a transcript. A contestable verdict needs an appeal-grade audit trail — and that reshapes council design from the start.
-
The proctoring bias tax — and why it falls hardest on neurodivergent learners
Automated proctoring doesn't just fail at integrity — it manufactures false-positive flags against the exact learners credentialing claims to serve, and a credential earned under a biased proctor is a lower-trust credential.
-
Human review is advisory, not a blocker
A common AI-validation design mistake: making human review a deterministic gate. Generalized from the Mneurix Lattice G-AIP-1 design decision — why disagreement should resolve deterministically, and what human auditors are actually for.
-
The credentialing trust stack
Why most micro-credentials die at the employer-trust checkpoint — and the five layers a credential must stack to actually be trusted: identity, assessment integrity, evidence, issuance, and portability.