Inside the Black Box

The world has started to feel like the black box inside the models.

Two days ago I was in the middle of an infographic comparing two AI models when two more came out. Anthropic had just released Claude Opus 5.5, which it says performs at the level of its three-week-old flagship, Fable 5.1, on most work. Within two hours OpenAI put out GPT-6 Sol and Luna at half what developers had been paying for the versions before them.

That made seven major launches in three weeks, among them Meta’s Muse, an app that books travel and fills in forms, and TypeSafe’s Jev, which skips the chat and hands back decisions. Grok 4.7 arrived the day before Opus, about three weeks after Elon Musk said it was ten days away.

I’ve had a feeling for two or three years that I still can’t put into words. I love building with AI, and it lets me make things I couldn’t have made before; then I get stuck on how anyone would ever find them; then another model launches, and I wonder whether it just made what I’m building pointless before I’ve even shipped it, or whether the next one will. Every one of those launches also has people with jobs wondering whether it’s coming for theirs.

And all of it lands on top of everything else, the politics and the wars, markets swinging on a single post, the biggest names in tech fighting in public, each headline arriving before the last one has sunk in… until it all runs together into complete chaos. The world has started to feel like the black box inside the models; some days I can’t even say where the problem starts.

For those years the change arrived one launch at a time, slow enough to live with. September played it all back like a time-lapse, and I happened to be drawing one frame when the next one arrived.

A contest of guesses

The black box inside the models is the famous one. In 2025 Anthropic’s CEO, Dario Amodei, wrote that people outside the field “are often surprised and alarmed to learn that we do not understand how our own AI creations work.”

My sense of the other black box, the world, is that it has gotten too big for anyone to follow, with more players than ever and more ways for them to affect each other. History pushes back on part of that. Long before AI, the economist Friedrich Hayek argued that nobody had ever understood the whole economy, because the knowledge it runs on “never exists in concentrated or integrated form.”

Size matters less, I think, than the fact that the world reacts to being predicted. A weather forecast doesn’t change the weather, so better models keep improving the predictions. Forecasts about people change what people do. If everyone trusted a model that said a stock would jump tomorrow, they’d buy today, and the jump would happen today, so the forecast undoes itself. Others make themselves come true. Tell enough people a bank is in trouble and they’ll pull their money out until it is. Either way, better AI may not make a world like this easier to read, since a better forecast of something that reacts just gets acted on sooner.

John Maynard Keynes described this loop with a kind of newspaper contest. Readers picked the six prettiest faces out of a hundred photographs, and the prize went to whoever came closest to the average choice. The way to win is to guess what everyone else will guess. Since they’re all doing it too, you end up, in Keynes’s words, devoting your intelligence “to anticipating what average opinion expects the average opinion to be.” In a game like that, the facts explain surprisingly little, which may be why reasoning my way through with them felt so useless.

Most of the black box I’d been feeling is made of this. Everyone acts on guesses about everyone else’s guesses, and each guess changes what’s being guessed. The AI race runs on it. In September Amodei published an essay called “We Must Pace the Frontier,” warning that “a race to the bottom, spurred by commercial incentives, can make these risks more acute.” It proposes that companies coordinate limits on the pace. Anthropic calls Opus 5.5 its first release since that essay, and OpenAI’s two models followed it within two hours. Even a lab arguing for a slower race wants the slowing done together, because of what it expects the others would do with a head start.

I suspect a lot of the talk about slowing down is about keeping competitors out. In March 2023 Elon Musk signed an open letter calling on AI labs to pause, for at least six months, the training of anything more powerful than GPT-4. Earlier that month he had incorporated xAI, the company that went on to build Grok. The pattern is older than AI. Laws that closed bars and liquor stores on Sundays were backed by Baptists, on moral grounds, and by bootleggers, who got Sundays to themselves. The economist Bruce Yandle called this “bootleggers and Baptists,” and nothing stops one person from being both.

I can’t prove it, though, and the race is the reason. A sincere CEO who can’t slow down alone and a cynical one who wants rivals slowed would sound alike and keep shipping in the meantime. You have to watch what they do, and even then most of what they do is what the race forces on everyone. OpenAI and Anthropic both set records for lobbying Washington in the second quarter of 2026, which either kind of CEO would do. The only telling move is one a cynic wouldn’t make, a commitment that costs the company real money and gets kept for years. By that test, the part of Amodei’s essay I’d watch is the one step Anthropic says it will take on its own, before anyone else agrees. If it still costs them something in a few years, that will say more than the essay does.

Keynes’s contest is ninety years old. What seems new to me is how many are playing, and who. Three years ago the models mostly waited for a prompt and replied. Now Muse, in Meta’s words, “keeps working after people close the app,” and Jev hands back decisions with probabilities attached. Some of the players we’re all trying to predict are now systems that, by Amodei’s own account, nobody fully understands. That’s the part I can’t find in Hayek’s world, and it’s why I still think this time is different, four words with a famously bad record in markets.

Uncertainty itself may be the worst part. Researchers at University College London had people turn over virtual rocks that sometimes hid a snake, and a snake meant a painful electric shock. Their stress peaked when the odds were around fifty-fifty, higher than when a shock was certain. A world where every forecast gets acted on as soon as it’s made leaves mostly what nobody can call either way, the kind of uncertainty that study found hardest to bear.

“Worldly wisdom teaches,” Keynes also wrote, “that it is better for reputation to fail conventionally than to succeed unconventionally.” Being wrong along with everyone else is forgivable in a way that being wrong alone isn’t. I think a lot of what we call FOMO is the fear of being the one who guessed differently. In 2025 nearly two-thirds of chief executives told IBM that the risk of falling behind pushes them to invest in some technologies before they understand the value. The more people pile into identical bets to avoid that, the more the outcome depends on the crowd.

What you can’t buy today

The forecast only undoes itself because the stock can be bought today. Outside markets there’s no price ticker, but copying does what those buyers did. Whatever can be built fast gets built by everyone who sees it coming, and its value is competed away almost as soon as it appears. AI keeps moving things into that category. A working app or a month of posts can now be had in an afternoon, and TypeSafe’s price list calls Jev’s output “FREE (too cheap to meter).”

In a paper on competitive advantage, the strategy professors Ingemar Dierickx and Karel Cool tell of an American visitor asking a British lord how he got such a gorgeous lawn. The lord lists the soil, the seed, the fertilizer, the daily watering and the weekly mowing, and the American keeps saying no problem. “Oh, absolutely,” says the lord. “There is nothing to it, old boy; just keep it up for five centuries.”

Their name for this is “time compression diseconomies.” It means you can’t get the same result by doing twice as much in half the time. A reputation, an audience that trusts you, a client who’s worked with you for years and a track record all have to be accumulated, and none of them can be built in an afternoon at any price. Even a perfect forecast that trust will matter doesn’t let anyone get ahead of it, because the only way to build trust is to spend the years.

Whole economies have slow parts too. Machines improve much faster than people, and the Stanford economists Chad Jones and Christopher Tonetti show that most past productivity growth came from handing them more of the tasks people used to do. Even when most tasks are automated, the ones people still do hold the rest back. Jones compares the economy to a chain that is “only as strong as its weakest link.” In their data, computing power has exploded, yet since 2000 the share of US business income paid to computers has fallen, from about 4.3% to 3.0%. If AI makes the fast parts of work nearly free, the value moves to the slow parts, which cap what the whole can produce and end up taking most of what it costs.

Building got easy and getting seen got hard, and I’d been treating those as two separate facts. They’re one. Attention doesn’t speed up. People still have only so many hours in a day to look at things, and when everyone can build, everyone is bidding for those hours.

Thread after thread says distribution is the new moat. I believe it, which is the trouble. Any moat everyone agrees on is already priced in, like the stock everyone bought a day early. Everyone piles into distribution with identical tools, and the parts anyone can copy, the posts and cold emails AI can turn out by the thousand, get flooded first. What keeps its value is the part nobody can copy quickly, people who already trust you because you’ve been showing up for them long enough. (I’m aware that writing a long essay like this one is my own attempt at that part.)

The test I take from this, for anything I build, is to ask what part of it next month’s model would do for free. When I run it on my own projects, the first things to fail are workarounds that newer models now handle on their own. What passes is smaller and slower, like knowing a particular problem well because I’ve lived with it for a while.

Nothing stays slow

Making slow things fast is what this technology is for. Jones, who took a job at Anthropic starting in mid-2026, writes that if AI “eventually automates away nearly all the weak links in the economy,” growth could pass 5% a year. For 150 years, he notes, US income per person has grown a steady 2% a year. Anthropic says one tester of Opus 5.5 finished a 680,000-line code migration in less than a day, work that would have taken an engineering team weeks. Agents can already do a lot of checking, one of the slowest parts of almost any job. An agent picking your hotel can read every review and compare every price far faster than you could, which covers a lot of what a hotel’s reputation used to do.

Every launch moves the answers, so the test has to be asked again each time. I suspect most products that get swallowed by a launch were built on something that was slow when they started and fast by the time the models caught up. That’s exactly what I’m afraid of with every launch. So a better rule is that value moves to whatever can’t be sped up yet.

Guessing where the line between fast and slow will go next is its own contest of guesses. Everyone is trying to work out what AI won’t be able to do, and the answers pile up in a few places, like taste, judgment, trust and relationships. A safe spot everyone agrees on gets crowded as soon as it’s named, as distribution did.

Companies are guessing where the line will be, too, and the fear of replacement may be the self-fulfilling kind of forecast. UK executives surveyed in mid-2026 said AI had raised their firms’ productivity by less than 1% so far, yet they expect nearly 3% more over the next three years and more than 1% fewer staff. If firms cut staff on gains they only expect, and the gains come in smaller than hoped, some of the replacement people fear will have come from the forecast itself.

Does anything stay slow, however good the models get?

Promises

I can’t picture any model speeding up the future. Agents keep getting faster at checking the present, whether the code runs or the citation exists, but nobody, human or machine, can check the future before it arrives. When you trust someone, you’re betting on what they’ll do later, that they’ll fix the thing when it breaks and still be around next year. The evidence is mostly a record of promises kept, plus someone you can hold to them, and a record takes as long as it takes. An agent can read a hotel’s whole record far faster than you could, but it can’t make the record any older. And while everyone can agree today that trust matters, nobody can agree their way into five years of kept promises, any more than the American could hurry the lord’s lawn. None of that guarantees anyone will pay for a record like that, only that nobody can produce one quickly.

The philosopher Hannah Arendt described two sources of unpredictability in human affairs. One is “the basic unreliability of men who never can guarantee today who they will be tomorrow.” The other is that when everyone can act, nobody can foresee what any act will lead to, which is Keynes’s contest again. Her remedy for both was “the faculty to make and keep promises,” which sets up “islands of security” in “the ocean of uncertainty, which the future is by definition.”

Her islands are the slow parts. The black box is us predicting each other, and a kept promise is one of the few places where we can stop.

A promise needs someone who’ll still be around to keep it and who can be held to it. At September’s pace even a top model can be superseded within weeks, and no model can be held to anything, so when something built with AI goes wrong, it’s a person’s promise that’s broken. A public database of court decisions about AI-invented material in legal filings, mostly fake citations, lists more than 2,000 cases, and in almost all of them it’s a lawyer or someone representing themselves who answers to the court.

For people with jobs, being answerable may be the slow part that’s left. Drafting is fast now, but you can only put your name on what you’ve understood well enough to defend, and those two thousand court cases show what happens when people skip that step.

If what keeps its value is made of kept promises, and a promise about what a product will do can expire with the next launch, what can anyone still promise? Arendt thought promises only work as islands, and that once they’re stretched “to cover the whole ground of the future,” they lose their binding power. The only promise I can make honestly is the slow kind, that I’ll still be around to fix what I build. What I don’t know yet is whether that’s worth anything to someone who can get next month’s model for free.

For a concrete project on reusable agent creation and verification workflows, see Agent Skills.