The Skills the Bubble Sheet Couldn't See

A piece in EdWeek MarketBrief this week walked through a pilot now running in classrooms across North Carolina, Indiana, Wisconsin, Nevada, and Rhode Island. It is called Skills for the Future, a joint initiative between ETS, the country's largest testing provider, and the Carnegie Foundation for the Advancement of Teaching.¹ Thousands of students in grades 9 through 12 are working through a new kind of assessment, designed to surface the durable skills employers and colleges have asked schools to develop for forty years and that no standardized test has ever quite been able to see.¹

Tim Knowles, who runs the Carnegie Foundation, described the project in language that is both ambitious and revealing. "In some ways," he said, "we are building a human skills genome."² The first three traits being mapped are collaboration, communication, and critical thinking.²

The pilot is interesting. The question underneath it is more interesting.

The Forty-Year Complaint

Anyone who has worked in a school in the last four decades has heard the complaint. Employers and college admissions officers say students arrive without the skills that actually matter at work: the ability to think under uncertainty, the ability to disagree productively, the ability to write a paragraph that says what it means. Schools have responded with portrait-of-a-graduate documents, capstone projects, performance tasks, and rubrics. The skills have always been on the wall.

The problem is they have rarely been on the transcript.

The reason is older than the technology. Standardized testing measures a moment. A student fills in bubbles for ninety minutes and the score lands at the bottom of the page. The thing that has always been hardest to measure is process. How a student starts a problem. How they recover after the first wrong turn. Whether they revise because a peer pointed something out or because they noticed it themselves. The artifact at the end is often the same. The path that produced it is not.

What AI Actually Adds Here

The Skills for the Future pilot is, in a careful sense, less a story about AI than a story about what AI makes affordable. Performance-based assessments have existed for decades. Capable teachers have been able to evaluate critical thinking on a real piece of work for as long as there has been student work. What has never scaled is the labor. Reading a thousand essays carefully takes a thousand hours. Watching how a thousand students made revision decisions across a quarter takes more.

Danielle Eisenberg at ETS describes the goal in a way that resists the most familiar caricature of edtech. "It's become increasingly clear," she has said, "that the knowledge, and the skills, and the dispositions that are needed to succeed in the 21st century aren't singularly demonstrated through time in the classroom, and yet, we're stuck in that system now that equates time with learning."³ The test, in other words, is supposed to become a side effect of the work, not the work itself.

That is the part worth holding. AI in this pilot is not asked to replace teacher judgment. It is asked to make teacher judgment visible at the scale of a district. North Carolina alone won nearly four million dollars in federal funding last fall to begin its piece of the work, with four districts piloting first.⁴

Where Critical Thinking Actually Lives

If you ask a teacher where, in a finished essay, you can see critical thinking, she will pause. The finished essay is the wrong place to look. Critical thinking lives in the place a student crossed out a sentence and tried it again. It lives in the question the student asked at minute fourteen, when she realized her opening claim could not survive the counterexample her partner had raised. It lives in the silence before the breakthrough, and in the small reorganization of the second paragraph that nobody asked for.

These traces have always existed. Until very recently they were unrecoverable. A teacher who lived through the class might catch a handful. A teacher reading the final draft alone, at her dining table at nine at night, would catch almost none.

What a system like Koan tries to do, in the quietest possible way, is keep those traces. The drafts a student kept. The questions she asked Aidan when she got stuck. The pauses that mean something and the pauses that do not. None of it is in itself an assessment. It is closer to the raw material that has always been required for one, and was never quite at hand.

The Question Underneath

The Skills for the Future pilot is ambitious because it is admitting that the things we say we value have not been the things we have measured.⁵ There are reasons for the gap, mostly practical. Multiple-choice items scale and rubric-scored performance tasks do not. The shape of a school year, the shape of a transcript, the shape of a college application, all bend toward what is easy to count.

A generation of testing reformers tried to fix this from the testing side. The newer attempt is to fix it from the evidence side. To collect, throughout the year, the records of work that show what a student can actually do, and only then to summarize.

The risk worth naming is that the same architecture that lets you see critical thinking lets you surveil it. The promise depends on whose hands the record sits in, and on whether students and teachers feel like authors of the record rather than subjects of it. Schools that get this right will hold the data lightly. Schools that get this wrong will turn evidence into a dossier.

If the durable skills you most want to teach can only really be seen in the process, what would change about your classroom tomorrow if you decided to keep the process?