The Question That Quietly Leaves the Meeting

A technology renewal comes up in a meeting. The tool was purchased a few years earlier because it was supposed to improve student learning. By the time it comes up for renewal, the conversation is about how many teachers use it, whether people would be frustrated if it disappeared, and whether the money is there. Those are fair questions and somebody has to ask them. They are just not the question that justified buying the thing in the first place. Somewhere between implementation and renewal, the original question quietly left the meeting.

We have all sat in that meeting, and most of us have been the person asking about usage. It is rarely because anyone is avoiding accountability. What actually happens is more ordinary than that. Implementation takes over. People need training, problems need solving, staff turn over, budgets reset, and eventually the thing we bought becomes part of how people work. A couple of years later, reconstructing what we expected it to accomplish is genuinely hard, and the questions that are easy to answer move in to fill the space.

We Have Done This Before

Here is the part that should concern us more than any single purchase. We often adopt something during one swing of the pendulum and evaluate it during another, and the pendulum in education never stops moving.

Think about how much has turned over in a single career. Reading instruction went from phonics to whole language to balanced literacy and back toward phonics, and districts reorganized around each turn. Math had its own version of the same argument, more than once. We opened up classrooms and then closed them again. Class size, then standards, then accountability, then social emotional learning, then grading practices, then a device for every student, and now devices are the problem. Some of those swings were well supported by research. Some of them were supported by a persuasive keynote and a room full of people nodding. From inside a district, the two can be difficult to tell apart, because both arrive the same way. Somebody credible says this is what good schools are doing now, and it is genuinely hard to argue for standing still.

I am not making the case that these movements are baseless or that districts should ignore them. Often the research is real and the change is an improvement. The problem is what we do with it. We tend to adopt the swing wholesale, implement it thoroughly, and never write down what we expected it to change in our own schools. So when the field turns again, and it always turns again, we have nothing of our own to consult except the new mood. We cannot say whether the last thing worked here, because we never established what working would have looked like. We just move.

That is how a profession ends up reversing itself every several years while feeling responsible the entire time.

The Pendulum Is Moving Again

For most of the last decade, schools were encouraged to add technology. More devices, more platforms, more integration. Some of it helped. Some of it was purchased because it was available and because standing still felt like falling behind.

The mood has changed, and Jared Cooney Horvath's The Digital Delusion is one of the clearest voices in that change. School leaders are reading it, boards are hearing about it, and it is worth understanding why it lands. Horvath does something districts rarely have the time or capacity to do. He goes past the research summaries and looks at the studies underneath them, and much of what he finds does not hold up well. He also makes a distinction education should sit with for a while. Producing better work with a tool is not the same as getting better at the work. A student can turn in a stronger essay with AI and be no better at writing an essay without it. Those are two different outcomes, and they look identical when the finished product is sitting on a teacher's desk.

His conclusion is that AI is "almost certainly harming learning" and that schools should avoid it for learning purposes. That is a strong claim, and it is worth noticing that it runs somewhat ahead of what he establishes. When he sets aside the studies he considers unreliable, what is left is a smaller benefit than was advertised, not evidence of harm. He gets to harm by way of his expectation that the research will continue to look worse and his argument about cognitive offloading. Both deserve consideration. Neither is settled.

I point that out not to argue with him. I point it out because this is how swings arrive. Not with bad arguments, but with good ones that are a little more certain than the evidence underneath them. The enthusiasm that preceded him worked the same way. Somebody serious makes a persuasive case, the case travels faster than the research does, and districts start moving before anyone has decided what moving is supposed to accomplish.

He may well turn out to be right. I do not think anyone knows yet. And here is the thing that should shape what a district does next: it does not much matter which way this one resolves.

"Don't Buy It" Is Not a Governance Plan

Suppose Horvath is completely right about AI and student learning. A district still has decisions to make on Monday.

Teachers already have access to these tools and students certainly do. Products we purchased years ago are adding AI features through routine updates, sometimes with a note in a release summary and sometimes without one. Staff are using free tools that never went through a purchasing process. Boards want to know what the district allows, and families want to know what protections are in place.

Don't buy it may be a perfectly reasonable answer when a new product comes up for purchase. It does not answer what employees may enter into an AI system, what students may use AI to produce, when AI use should be disclosed, whether a feature that appeared inside an existing platform stays turned on, or what happens when people use something the district never bought. Privacy, data protection, instruction, assessment, and employee use are all still on the table whether or not a district ever signs an AI contract.

Waiting is a decision. Prohibiting is a decision. Permitting is a decision. Each one creates consequences the district will own, and each one requires some explanation of what we are trying to protect or accomplish. The broad research can help us decide how cautious to be. It cannot make those decisions for us.

Ask a Smaller Question

Part of the trouble may be that does AI work is not a question anyone can answer. An AI tutor working directly with a student is not the same thing as a teacher using AI to draft a lesson. A writing tool is not translation software. Something bought to save staff time should not be judged the same way as something bought to improve student learning. We are asking one question about a dozen different things and then acting surprised when the research disagrees with itself.

The question gets much more useful when it gets smaller. What is this particular use supposed to accomplish? If the purpose is to give teachers back planning time, then measure planning time. If the purpose is better feedback, decide what students should be doing differently because the feedback improved. If the purpose is student learning, then at some point students have to demonstrate that learning without the tool supplying the performance. The metric should follow the claim.

The problem is that we usually know far more about the implementation than we do about the claim. Take an AI writing tool bought because students were supposed to become better writers. A year in, most teachers are using it, training went well, students are engaged, and support tickets are manageable. All of that tells us the rollout worked. None of it tells us whether students became better writers. And there is a harder question underneath that one. Did they become better writers, or did they get better at producing writing while the tool was open? If nobody decided in advance how we would tell the difference, the renewal conversation drifts toward the things that are easy to see, which is exactly where this article started.

Write It Down While the Initiative Can Still Fail

One practical way to keep the original question from disappearing is to write it down before implementation begins. This does not need to become a district framework or another evaluation system, and it should not. It needs to be specific enough that people can come back to it later and honestly decide whether the thing did what we said it would.

Say a district is considering an AI feedback tool for ninth-grade writing.

Claim: Students will make more substantive revisions when they get faster feedback on early drafts.

If the claim is true: Second drafts show more changes to ideas, structure, evidence, and argument, not just cleaner sentences.

If the claim is false: Students revise more without revising better, or the improvement disappears when the tool is not available.

Independent check: At a few points during the year, students draft and revise comparable work without the tool.

Decision date: Look at the evidence before the renewal recommendation is written, not after.

There is nothing sophisticated about those five lines. Anybody in the building could write them in ten minutes. The hard part is agreeing to them while the initiative can still fail. Once money has been spent, staff have been trained, routines have formed, and people have gotten used to a tool, the question changes without anyone deciding to change it. What began as did this work slowly becomes are people willing to give it up.

That is not unique to AI and it is not unique to technology. It happens with programs, instructional practices, professional learning, and budget lines that renew year after year because they are already there. The longer something exists, the easier it is for its existence to become the argument for keeping it.

Deciding in a Way That Lets You Find Out

The pendulum is going to keep moving. There will be another study, another book, another product, another worry, and another year when the responsible-sounding position is the opposite of last year's responsible-sounding position. Some of those swings will be right. We will not know which ones for a while, and by then the field will be arguing about something else.

Here is what I keep coming back to. A district that wrote down what it expected, what it would accept as evidence, and what would change its mind has something of its own to consult when the mood shifts. It can say this worked here, or this did not, and it can hold that up against whatever is arriving next. A district that wrote nothing down has only the arriving thing. It ends up defending decisions it cannot explain and reversing decisions it never really made, on somebody else's schedule.

That is the real cost of the pendulum. Not that we change our minds, which we should, but that we keep changing them without ever finding out whether the last change accomplished anything.

When the evidence is unsettled, the standard for deciding should go up, not down. Be clearer about what is actually being decided. Say what you expect to happen because of it. Decide what you would have to see to conclude you were right, and be just as specific about what you would have to see to conclude you were wrong.

You still have to decide, even when the research does not give you a clean answer. The uncertainty does not take the decision away from you. It raises the price of deciding badly.

And the goal is not to be right the first time. The goal is to decide in a way that lets you find out.