Designing an Operational Analytics Framework for Literacy Interventions

1. Business Problem & Context

In large-scale educational environments, intervention programmes generate vast amounts of data; however, the core challenge is a lack of actionable operational insight. Existing reporting purely focused on activity, such as total time spent, rather than actual efficiency. This contextual deficit made it incredibly difficult to spot disengagement or productivity issues early, leading to inconsistent operational visibility.

2. Stakeholders & Intended Users

• Primary Audience: Educational stakeholders and intervention managers.
• Core Need: Clear decision-support allowing for the early identification of operational issues and inconsistent outcomes.

3. Requirements & Success Measures

• Requirements: Move beyond raw activity tracking to design an operational ecosystem that accurately measures the effort-to-output ratio.
• Success Measures: Create a scalable, standardised framework capable of identifying inefficient usage patterns at scale across different cohorts.

4. Data Understanding, Assumptions & Limitations

• Identifying the Gaps: The “AS-IS” reporting suffered from misleading metrics where completion counts failed to account for the effort required, and raw minutes lacked actual production context.
• Data Standardisation: Implemented data dictionaries and structured metadata to guarantee reporting logic remained consistent across the entire organisation.

5. Analytical Approach & Tools

• Tools: Built as a scalable, tool-agnostic ecosystem to handle increasing data volumes while ensuring operational visibility.
• Skills Demonstrated: KPI Design & Metric Engineering, Threshold & Classification Modelling, Root Cause Analysis (RCA), and Process Optimisation.

6. Validation & Responsible Analytical Practice

• Objective Segmentation: Used Exception Reporting and Root Cause Analysis to objectively separate poor performance into distinct categories: “Needs Usage” (low engagement) versus “Needs Instruction” (engaging, but doing so inefficiently).
• Longitudinal Tracking: Designed the tracking ecosystem to compare termly progress, validating seasonal trends instead of relying on isolated snapshots.

7. Dashboard, Solution or Business Output

The core solution was a custom KPI suite and classification model designed to quantify operational efficiency.

Engineered Metrics:

• MSPU (Minutes Spent Per Unit): Assessed how much time investment was required to achieve a single unit of progress.
• UGPM (Units Gained Per Minute): Measured raw productivity to enable direct cohort comparisons.

Operational Classification Thresholds:

Efficiency Band	MSPU	UGPM	Operational Interpretation
High Concern	>7.9	<0.13	Low efficiency requiring immediate investigation.
Standard Range	4.4 – 7.9	0.13 – 0.23	Expected operational pacing and progress.
High Efficiency	<4.4	>0.23	Accelerated progress and highly productive engagement.

8. Findings & Recommendations

• Activity Does Not Equal Success: The framework proved that stakeholders logging high minutes were sometimes actually showing high MSPU (low efficiency), requiring a shift from engagement barriers to instructional barriers.
• Targeted Strategy: By segmenting performance, a highly targeted intervention strategy was developed to accurately address specific barriers to success.

9. Impact, Expected Value & Validation

• Process Redesign: Through targeted process redesigns, such as altering intervention pathways and seating-plan structures, major inefficiencies were eliminated.
• Quantifiable Results: The implementation of this framework drove a dramatic reduction in average MSPU from ~45 down to ~4.3.

10. Reflection & Next Steps

This project firmly reinforced the absolute necessity of translating raw data into measurable systems. Ultimately, engineering KPIs and building structured classification models shifts the analytical conversation from simply asking “what happened?” to proactively answering “how can we make the system work better?”.