When the Rollout Itself Goes Dark

A panel at last month's ASU+GSV Summit in San Diego did something unusual for an industry conference. Three district leaders sat on stage and said, in plain language, what they had gotten wrong.¹

The session, hosted by MagicSchool AI during the April 12 to 15 summit, brought together Tamra Collins, superintendent of New York City Public Schools District 19; Sherri Wilson, director of innovative learning at Broward County Public Schools in Florida; and Curtis Finch, superintendent of Deer Valley Unified School District near Phoenix.¹² Each of them is in the middle of an AI rollout that touches tens of thousands of teachers, administrators, and students. Each of them said, in essence, that their first attempt had not worked the way they thought it would.

Collins admitted that her district's professional development had assumed a level of adult readiness it did not actually have. Leaders needed more time, she said, before they could lead the work in their own buildings. So District 19 slowed down. It picked the schools that were ready and let them go first.¹

Wilson said Broward had spent so much energy building governance and policy at the district level that the dissemination piece thinned out. The work of ethics, cybersecurity, and procurement is real work. But the version of it that lives only in a binder does not, by itself, change a classroom.¹

Finch said his early mistake as an administrator was not pressing vendors hard enough on integration with the district's existing systems. With 4,000 employees serving 31,000 students in Deer Valley, every adopted tool becomes a multi-year training event. A poor fit is not a procurement problem. It is a calendar problem that absorbs years.¹

Different districts. Different mistakes. One pattern.

The Hidden Variable Was the Rollout Itself

What is striking about all three confessions is that none of them are really about AI. Collins did not say her tools were bad. Wilson did not say her policy was wrong. Finch did not say his vendors were dishonest. They said they could not see what mattered until it was already too late to act on it cheaply.

They could not see which adults were ready and which were not. They could not see whether a policy written in a conference room was reaching the teachers who had to live by it. They could not see whether a product would actually fit until it was in their stack and entangled with everything else.

This is the same problem teachers have inside a single classroom. They can see who shows up, who turns work in, and who scores well. They cannot, with the tools most schools currently use, easily see who paused, who revised, who tried a sentence and erased it, who quietly moved from confusion to clarity over three drafts. The work happens. The record of how it happened evaporates.

Districts and classrooms are running the same opacity at different scales.

What Visible Learning Would Mean for Adults, Too

The version of school technology worth building is the one that does not pretend to know things it cannot. It surfaces what is actually happening on the ground, in real time, so that the people responsible for children can make good calls earlier and cheaper.

For students, that looks like Aidan, the AI tutor we work on at Koan. Not a black box that returns a percentage and asks to be trusted, but a record of every revision, every long pause, every breakthrough. A teacher can scroll back and watch the thinking move. So can the parent. So can the student.

The same logic applies to a rollout. A district that wants to know whether its principals are ready does not need a survey six months in. It needs visibility into what those principals are doing, asking, and avoiding now. A district that wants to know whether its AI policy is reaching the classroom does not need a compliance memo. It needs a reading of how the policy is being interpreted by the teachers who are actually deciding, tonight, whether to let a student draft an essay with a chatbot.

This is unsexy work. It is also the work that distinguishes a real implementation from a press release.

The Honesty Behind the Panel

It takes courage for a sitting superintendent to walk onto a conference stage and say their rollout was uneven. The temptation under those lights, especially while flanked by vendors, is to declare success. Collins, Wilson, and Finch declined that temptation, and the field is better for it.

The lesson is not that AI in schools is failing. The lesson is that the parts of an AI rollout most likely to fail are the ones that were never made visible. Adult readiness was a guess. Policy reach was a guess. Vendor fit was a guess. When the guesses were wrong, the cost showed up months later in stalled timelines and frustrated teachers.

The future of school technology will favor the districts that make the rollout itself legible. Not just the tools, the rollout. Not just the policy, the practice. Not just the metrics, the moment a teacher decides to push a button.

If three superintendents could finally see what they had missed, what would change tomorrow if every teacher in their districts could see the same thing about every child?