The accreditation conversation has shifted. Regional accreditors, professional program accreditors, and program review committees have moved from aggregate-level outcome metrics — "82% of graduates pass the licensure exam" — toward individual mastery evidence. The question is no longer just whether the program produced good average outcomes. It's whether the assessment infrastructure can demonstrate that specific competencies were measured at the individual level, with instruments that have documented psychometric properties, across multiple cohorts.
Most university assessment programs are not equipped to answer this question. Not because of lack of effort, but because the tools they've been using weren't built to produce individual-level psychometric evidence. An end-of-course quiz in Canvas records a percentage score and a completion date. An in-house standardized exam has p-values and Cronbach's alpha. Neither gives a curriculum committee what accreditors are now asking for: an ability estimate with a confidence interval, on a stable scale, that can be compared across cohorts and connected to a credential that an employer can independently verify.
What accreditors are actually asking for now
The language varies by accrediting body, but the operational requirements have converged on a consistent set of expectations. SACSCOC (Southern Association of Colleges and Schools Commission on Colleges), HLC (Higher Learning Commission), and WASC now routinely ask for evidence of "direct assessment" of learning outcomes — meaning scored artifacts that demonstrate individual competency, not satisfaction surveys or grade distributions. Professional accreditors (ABET, ACEN, CCNE, AACSB) have pushed further: specific competency domains must be demonstrably achieved at the individual student level before graduation or certification.
The practical implication is that a program can no longer show an accreditor a graph of average exam scores and call it done. The questions that follow are: What was the reliability of this instrument? Were items calibrated for this population? How do you know a student who scored at the cut point is genuinely competent versus borderline-guessing? Can you show individual student data with uncertainty quantification?
These are not unreasonable questions. They are exactly the questions that the psychometrics literature has been answering for 40 years. The gap is between the state of the science and the state of the typical institutional measurement tool.
The practical infrastructure modernization
Modernizing university assessment infrastructure involves three separable decisions that are often confused as a single project.
Measurement model upgrade: CTT to IRT. This is the scoring engine question. Moving from raw-score-based CTT scoring to an IRT-based ability estimate requires item calibration (field testing with sufficient sample sizes for stable parameter estimation) and a platform that maintains calibrated parameters and reports theta scores with standard error values. This is the hardest part technically and the most impactful for measurement quality. It's also the part that most "modern" EdTech platforms don't actually do — they have slicker interfaces over CTT backends.
Delivery modernization: fixed-form to adaptive. This is the assessment delivery question. Computerized adaptive testing reduces test length by 40–60% without sacrificing measurement precision, because items are selected to maximize Fisher Information at the learner's current ability estimate rather than administered in a fixed order to everyone. This requires a sufficient item bank (a minimum of 80–100 calibrated items per domain for a 20-item adaptive assessment) and a delivery engine that implements item selection correctly. Many platforms advertise "adaptive" features that are actually difficulty-stratified random selection — not genuine CAT.
Credentialing modernization: transcript to verifiable digital credential. This is the output question. A course grade or a transcript notation is difficult for employers to verify and carries no machine-readable metadata about what competency was measured or at what level. Open Badges 3.0 compliant credentials solve this — they are publicly verifiable, machine-readable, and can carry the ability estimate and assessment metadata as claims. This is the part that directly benefits learners and is most visible to employers and graduate school admissions offices.
These three decisions are related but independent. An institution can modernize its credentialing output without changing its measurement model (Accredible and similar platforms issue digital badges against any assessment). It can upgrade to IRT scoring without adopting adaptive delivery. But the most defensible combination — the one that produces evidence that satisfies both accreditors and employers — requires all three: IRT-based measurement, adaptive delivery efficiency, and credential output that carries the score evidence in a verifiable format.
A realistic implementation timeline
The most common mistake institutions make when beginning an assessment modernization project is treating it as a technology procurement decision rather than a measurement redesign decision. The platform selection is the easy part. The hard parts are item bank development and pilot calibration.
A realistic timeline for a single program shifting to an IRT-based adaptive assessment for one learning outcome domain looks like this. Months 1–3: domain specification and item writing. Months 4–5: pilot administration (minimum 150–200 respondents per item for stable parameter estimates). Month 6: item parameter review — items with poor discrimination (a < 0.5), high guessing (c > 0.35), or unexpected differential item functioning are revised or removed. Month 7: first live adaptive administration. Months 8–12: concurrent calibration from live data, cut score validation, and first accreditation reporting cycle.
That timeline assumes the item writing is competent and that the pilot administration can be completed efficiently. For programs that already have item banks from prior static assessments, retrospective calibration of existing response data can bootstrap parameter estimates and compress the timeline — though cold-start items without prior response data still require the pilot phase.
What to say to the accreditor
The framing that works with accreditation review committees is not technical. It's evidentiary. The argument is: "We measure competency X using an adaptive assessment calibrated using 3-Parameter Logistic IRT. Each assessment produces an individual ability estimate (θ) on a standardized scale, with a 95% confidence interval. Students earning our program credential have theta estimates above the cut score established through [modified Angoff / bookmark] standard-setting, with a standard error of measurement below 0.30. We can show you the distribution of ability estimates for this year's graduating cohort, and the same distribution for the prior two cohorts, on the same scale."
That statement is defensible because every element of it is verifiable. It answers the accreditor's actual concern, which is not "what technology platform are you using?" but "do you have evidence that your graduates are individually competent, measured against a stable standard, in a way you can demonstrate longitudinally?"
Getting to that statement requires the infrastructure decisions described above. But for programs beginning the conversation now, the good news is that the technology to make this affordable for non-consortium institutions has arrived in the last few years. The window to build a defensible modernized assessment program before accreditor expectations formalize further is open. It won't stay open indefinitely.
