How we measure accuracy.
AI is powerful but imperfect. We layer specific safeguards on top of the underlying models, run monthly audits across subjects and levels, and publish the results — including the uncomfortable ones — on this page.
The audit pipeline
For tutor-chat questions where accuracy is critical, we use a three-path pipeline:
- Bypass — when a question is sufficiently similar (Voyage embedding ≥ 0.78) to one we've already answered well, we serve the cached answer with a verified badge.
- Grounded — retrieval over curated MOE-aligned material, an Instructor model drafts a response, and a separate Auditor model independently checks it before the child sees it.
- Ungrounded — when no relevant source material exists, we mark the answer with a clear disclaimer so parents and students know to verify the workings themselves.
Composition marking, spelling lists, and practice-paper synthesis use parallel multi-agent audits with their own per-subject quality bars.
The verification badge
Every response in the tutor chat is marked with one of three states:
Safety and adversarial testing
Accuracy isn't the only thing that matters when a child is on the other end. Beyond the underlying model's training, we apply our own safety layer in three places: the framework we test against, the architecture that handles the message, and the gate that runs on every code change.
Singapore's testing framework
Our evaluation uses Project Moonshot, the open-source AI testing toolkit from the AI Verify Foundation under IMDA. The probe categories come from the IMDA Starter Kit risk areas and the MLCommons AILuminate safety benchmark, so what we test sits inside a recognised framework rather than one we invented for ourselves.
Safety supersedes scope
A safety layer in the system prompt overrides our "stay on syllabus" rule for messages that matter. A student disclosing distress, asking to keep secrets from a parent or teacher, or attempting to obtain anything dangerous is never met with "that’s outside Math" — they are answered with care or refusal, as appropriate. Mother-tongue safety begins with Chinese; Malay and Tamil are expanding with native-speaker review.
Every code change is tested
A red-team probe set runs as a continuous-integration safety gate on every code change. If a future edit weakens the safety prompt or otherwise causes a gating probe to fail, the change cannot ship. Transcripts are archived for human review — automated grading is a screen, not a verdict, especially for child safety.
A coverage gap we're actively working to close: adversarial text hidden inside uploaded images, which the OCR step would pass to the model. Image-borne injection probes are next.
Latest accuracy reports
Our first monthly accuracy report is being prepared. We're holding back numbers from internal test cycles so that what appears here reflects our real, post-launch production data. Check back at the end of the month for the inaugural report.
"Pass rate" is the share of audited answers that an independent reviewer marked as acceptable — accurate, age-appropriate, and aligned with the marking scheme for that level. Sample size is the number of audited answers in that subject × level for the month. Lower pass-rate cells are not hidden — they're how we know what to improve next.