Worked example | AI Strategic Compass

Example scores. Unlike the Compass in action examples, which stop at the decision, this one continues through tracking and reflection. The scores and targets are representative. Your institution may score the same system differently based on its own criteria, data practices, and context.

1 · Score

Score profile: total 17, HighPeaks at strategic fit and operations, where the value is. The three 3s (ethics, cost, and trust) are where the safeguards go.

Dimension	Score	Reasoning

Total 17: High Strong alignment and clear operational potential, with notable risks around data governance, financial sustainability, and stakeholder trust. Recommended only with clear ethical oversight, a well-defined budget and support infrastructure, transparent communication with stakeholders, and ongoing monitoring.

2 · Plan

Mission outcomes

Problem: Students at risk of failing courses often go unnoticed, leading to higher dropout rates.
Outcome: Improved student retention and academic success, supporting equitable education.
Objective: Enhance student retention through early AI interventions.
Key result: Increase course pass rates for at-risk students by 15% within one academic year.
Key result: Reduce dropout rates by 10% by the end of the academic year.

Project outputs

Contribution: Analyzes student data (e.g., grades, attendance) to predict risk and recommend interventions.
Deliverable: An AI early warning system integrated into the learning management system.
KPI: Identifies 90% of at-risk students within the first four weeks of the semester.
KPI: Delivers intervention recommendations to 95% of identified students within 48 hours.
Efficacy: At least 80% of flagged students are confirmed at risk by advisors (precision).
Efficacy: Flag accuracy stays within 5 points across student demographic groups.
Efficacy: Retention gain is measured against a matched comparison group, not the prior year alone.

3 · Track & reflect

After the pilot, compare actuals to targets and record what the institution learned. Illustrative results:

KPI · project output

At-risk students identified within four weeks

85% / 90%

94% of target. Close; advisor workload slowed response.

OKR · mission outcome

Reduction in dropout rate

7% / 10%

70% of target. The output nearly landed, the outcome lagged. That gap is why the decision was Iterate.

Efficacy · AI performance

Flagged students confirmed at risk (precision)

68% / 80%

85% of target. The model over-flagged. This is the false-positive problem advisors reported.

Measure	Target	Actual	Notes
KPI	Identifies 90% of at-risk students within four weeks	85%	Close, but advisor workload slowed response.
OKR	Reduce dropout rates by 10%	7%	Improved, but needs a longer pilot.
Efficacy	80% of flagged students confirmed at risk	68%	False positives confused advisors; threshold needs tuning.

What worked

The tool accurately flagged at-risk students; advisors liked the interface.

What didn't

False positives confused advisors; student trust was low due to lack of communication.

Next step: iterate

Add opt-in consent and retrain advisors on interpreting flags; pilot again in the fall.

Try it with this example

Open the scorer pre-filled with the early warning system, then adjust the scores and see how the band changes.

Open in the scorer