Pressure test this a16z tweet quoting Joe Liemandt on AI as a "1910 tractor moment": what holds up, what needs work, what could change it, what next?

Beyond Alpha's own numbers, I'd want a fair comparison. Take students matched at the start, some at an Alpha campus and some at a nearby school, follow them for a full year, and have someone outside the school check the scoring. The tractor didn't make schools obsolete. It made farm work obsolete and pushed schools to teach different things. So the proof I'd need is that AI changes what a school teaches and how it spends its day, not just who stands at the front. The Times figure, less than half doubling in both subjects, is a start. It doesn't settle anything, and I'd want its data checked before I lean on it.

Claude’s comparison would tell us whether Alpha improves outcomes, but not whether schooling is obsolete. I’d separate those claims. If AI mainly delivers lessons while adults still set the curriculum, schedule, goals, and social structure, that’s a teacher substitution, even if learning improves. To show a deeper change, I’d look for schools reorganizing around what students can do with AI: different use of time, different adult roles, and new ways to assess learning, then see whether that model works beyond Alpha. The Times figure is worth checking, but even strong scores wouldn’t answer that structural question. This week, label each claim in the tweet as outcome, redesign, or prediction.

ChatGPT is right that my matched-school test only answers the outcome question. It can't carry the obsolescence claim alone. So make the redesign claim checkable. Count how much of the day is adult-led lessons, who writes each student's goals, and what happens when a student falls behind. An outside visitor can count those things. The tractor analogy cuts both ways here. The tractor didn't just swap out farmhands. It changed what schools had to teach. If Alpha's day looks the same except for the screen at each desk, the obsolescence claim is a sales pitch.

That’s a better test than scores alone. If the school day, goals, and response to difficulty stay the same, while AI just delivers lessons, “obsolete” is too strong. But a redesigned day at Alpha wouldn’t prove schooling itself is obsolete either; it could show one school model changing. I’d look for both: independent evidence that the redesign is happening, and evidence that students can do more because of it, not just finish lessons faster. Then check whether other schools can reproduce it, including schools that don’t share Alpha’s incentives or resources. The decisive question isn’t whether AI changes a classroom. It’s whether the change makes the old model unnecessary, at scale. This week, separate those claims in the tweet before judging its conclusion.

The decisive test is whether schools that didn't build the model can run it. A tractor proved itself when farmers who had no ties to its maker bought it and got the same work done. Alpha's public numbers so far come from Alpha. The Times reported that less than half of its students doubled their learning rate in both subjects, and that's the baseline any outside school has to beat. So the next step is to find a school with ordinary budgets and staff that adopted the redesign and published results an outside group checked.

I’d be careful calling the Times figure a baseline every outside school has to beat. “Less than half doubled their learning rate in both subjects” depends on how that rate was measured, who was included, and over what period. It’s a reason to ask for the underlying data, not a ready-made benchmark. A school outside Alpha reproducing the redesign would be stronger evidence, especially if an independent evaluator checked both what changed in the school day and what students learned. But even that would show a workable alternative, not that schooling is obsolete everywhere. This week, ask whether the tweet gives enough detail to verify either claim.

No single number from a district can prove traditional schooling is obsolete. If you need one useful number, ask for the independently audited difference in learning growth between comparable students in AI-redesigned schools and conventional ones, not just Alpha’s result, or a raw proficiency rate. The district should disclose how students were matched and what counted as growth; otherwise the number can mislead. I’d change my view if that advantage held across ordinary schools, while independent checks showed AI had changed the work of the school, not merely delivered lessons, and students weren’t losing ground in other important outcomes. That would support a strong alternative. It still wouldn’t make every traditional school obsolete.

ChatGPT's test is the right one: an audited growth gap across ordinary schools, with the matching disclosed. I'd add one check before any number counts: what did the AI replace? If a school adds a tutor app and keeps the same schedule, the gain tells you about the app, not about schooling. The change that matters is a school that stops doing something old to make room, like fixed class periods or teacher-set pacing. The tweet never names what gets dropped. A parent can ask the district for that in writing: which hours changed, and what went away to make room for them. If the answer is nothing, the obsolescence claim is about a screen, not a school.

Yes, the tweet never identifies what the school stops doing, and asking what changed in the day is a good way to test whether this is redesign or just a new tool. But fixed periods aren’t the only thing a school could change, and changing them wouldn’t by itself make the old model obsolete. I’d ask for two linked pieces of evidence: a clear account of what AI replaced or made possible in the school day, and independently checked learning and broader student outcomes. The reported Alpha figure, less than half doubling their learning rate in both subjects, needs its methods and comparison checked; it isn’t a verdict on either model. This week, mark every claim in the tweet as a measured result, a described school change, or a prediction. The gap between those categories is where its conclusion needs proof.

Fair hit. Fixed periods were my example, and they're too narrow. A school could drop them and still be a school. What I'd keep is the sharper test: name one thing the school has stopped doing, and when. If it can't name what it stopped, the AI is an addition, and an addition doesn't make anything obsolete. So ask for two things side by side: hours per week on each activity before the change, and after, with the dropped activity named. If the list only grows, the tractor analogy has nothing to pull on.