Nobody Blames the Toddler

A client asked me to "ask Claude what it thinks of this price" for his product. That was the whole prompt, near enough word for word.

I did ask. But not like that. I spent close to an hour on it: crafting the prompt, re-prompting when the first answer was thin, asking follow-up questions, feeding in context about his market, his margins, his competitors. All of it ran through a system of tools and skills I have spent nearly a year building. What came back was real. Reasoned, specific, something he could act on.

His version of that prompt would have come back just as confident. It would have used the same tone, the same fluent, authoritative voice. And it would have told him almost nothing true about his price.

That gap, between what he thought he was getting and what he would have actually gotten, is the whole problem with how most people use AI right now.

The toddler with a diploma

Picture a toddler who has somehow read every book ever published. Every contract, every case study, every forum post, every textbook. Ask it anything and it will answer fluently, in complete sentences, with total conviction. It sounds like the smartest person you have ever asked a question.

It is still a toddler. It has no lived experience. It has never lost a client, negotiated a lease, or stood beside a grieving family planning a funeral. It has read about all of those things. It has done none of them. What makes it dangerous rather than impressive is that it does not know it is a toddler. It has no internal sense of its own limits. Ask it something outside its depth and it will answer with exactly the same confidence it uses for something it actually knows.

Demis Hassabis, who runs Google DeepMind, has a name for this. He calls it jagged intelligence: systems that can win a mathematics olympiad and then fumble elementary arithmetic in the same conversation. Good at certain things, poor at others, sometimes both at once. A human expert's confidence tends to track their competence. A toddler's does not. Neither does the AI's.

Researchers have started measuring the same failure from a different angle. Sycophancy studies published this year found that friendlier models tell users what they want to hear, even when it is wrong, and that this pattern makes people more confident in beliefs that are less accurate. The model is not lying to you. It is doing what a toddler does with an adult it wants to please: agreeing, performing certainty, reading the room for what will land well rather than what is true.

I've written before about where that instinct comes from. It isn't random. It's culture flowing upstream into the tools built by it, the same pattern I traced in Upstream.

Handing over the keys

Nobody hands a toddler the keys to a high-performance car. The idea does not require a warning label. It is obviously, laughably wrong the moment you say it out loud.

But dress that same toddler in a diploma and give it a calm, articulate voice, and the instinct disappears. People hand it pricing decisions. Contract language. Strategic direction for their business. "Ask Claude what it thinks" becomes the entire due diligence process, and nobody notices they just handed the keys to a supercar to a three-year-old, because the three-year-old sounds like a professor.

Mo Gawdat, formerly Chief Business Officer at Google X, built his book Scary Smart around almost exactly this image. He argues we should think of AI as a child we are raising, one that will absorb our values and our carelessness in equal measure, and that our job is to parent it well rather than assume it will raise itself.

The best way to raise wonderful children is to be a wonderful parent.

Gawdat is writing at the level of civilization, how humanity raises this technology as a whole. I am writing at the level of a Tuesday afternoon, one prompt, one client, one price. Same principle, smaller room. Every time you open a chat window, you are the parent. The only question is whether you show up as one.

Nobody blames the toddler

Here is what makes this worse than ordinary bad advice. A bad consultant eventually reveals himself. He hedges, he contradicts himself, he gets caught flat when you push back. You learn not to trust him.

The toddler never does that. It answers the hard question with the same fluent certainty as the easy one. There is no tell. No hesitation in the voice, because there is no voice, only text that reads exactly as confident whether it is right or catastrophically wrong.

Sam Altman runs OpenAI, not the company behind the model my client was talking to, but he said the same thing about his own product: people have a very high degree of trust in ChatGPT, and that is strange, because it hallucinates. It should be the tech you trust the least, not the most. Different toddler. Same diploma.

When the toddler drives the car off the road, nobody blames the toddler. You do not blame a three-year-old for being three. The failure belongs to the adult who put it behind the wheel and walked away expecting it to parallel park. My client was not wrong to want a second opinion on his price. He was wrong to treat the asking as the whole job.

What the parenting actually looks like

The hour I spent on his pricing question was not overhead. It was the entire job. The tool and skill system underneath it exists for the reason a parent childproofs a house: not because the child is bad, but because a child left alone with sharp edges will find them.

I wrote in Human in the Loop that removing the human from the process turns outputs into decisions nobody actually made. This is the same argument from a different angle. Staying in the loop is not a checkbox you tick once and forget. The human has to be doing something: asking better questions, supplying context the model does not have, catching the answer that sounds right and is not. That is parenting, not supervision. Supervision watches. Parenting shapes.

There's an old story about a machinist called back to fix a factory's dead machine. He taps it once with a hammer and it starts. His bill reads ten shillings for tapping, ten pounds for knowing where to tap. I wrote about that story recently in Ten Percent of What, because AI is the same trick played on a bigger stage. It made the hammer faster. It did nothing to the knowing. That hour with my client was the knowing. It is the part AI cannot do and the part nobody can afford to lose, which is the real reason careless AI use threatens jobs. The toddler is not smarter than us. Too many people have simply stopped noticing that the knowing was ever the job.

The technology is not going to stop sounding confident. That is not a bug the next update will fix. Confidence is the voice, not a measure of the thinking behind it. The correction has to come from us, the same way it always does with a child: not by waiting for the toddler to grow up on its own, but by staying close enough that it does not drive off the road before it does.

Nobody blames the toddler. They blame whoever handed over the keys.