How Google Knows If Its AI Search Is Working

Sundar Pichai spent a good chunk of his recent Decoder interview batting away the idea that AI is about to strangle web traffic. Buried in that defense was a more useful admission, one about how Google actually decides whether its AI search is working. The measure he pointed to has nothing to do with clicks or the traffic Google sends out. Google grades its AI search on user satisfaction, tracked the same way it has for 25 years.

The signal Google trusts

In Pichai’s telling, the great thing about search is that user satisfaction is easy to measure. Google has spent 25 years learning to read it, correlating what users do with whether the product is actually getting better. He was careful to say this runs on the long term, not a quick snapshot, which is why Google leans on extended studies rather than day-to-day swings. If an experience is bad, he said, it shows up in the metrics, and Google course-corrects.

The signals underneath are the familiar ones. How long people stay engaged, whether they come back, and whether they bounce straight back to the results page looking for something better. Google has judged regular search this way for years, and Pichai’s point was that AI search gets held to the same bar. AI Mode and AI Overviews live or die by whether people are satisfied with what they got, measured in behavior over time.

There’s no dial for it

For anyone used to optimizing toward a target, this one is slippery, because there’s nothing to optimize toward. You can’t tune a page for an aggregate, long-term satisfaction score. There’s no field for it, no schema, no checklist. The only input you control is whether the person who landed on your page is glad they did. Did you answer the thing they came for, clearly enough that they didn’t need to go hunting again? Nothing else moves the needle, and that’s deliberately hard to fake.

Pichai added a wrinkle that cuts the other way for publishers. Because AI answers can be shaped by the user, through follow-ups and how they phrase things, the traffic that comes out the other side is less predictable than the old model. Two people asking about the same topic can be steered to different sources. So even doing everything right, the referral you earn from AI search is a looser, less repeatable thing than a fixed ranking used to be.

The blind spot in the metric

There’s a hole in this that Pichai skated past. Engagement and return behavior show how people act, not how they feel. Someone can read an AI answer, accept it, and move on without ever realizing it was shallow or subtly wrong. To Google’s metrics, that looks like a satisfied user. Nothing bounced, nobody complained, the numbers stay green. Whether the answer was actually good is a separate question the behavior can’t fully see.

That gap gets riskier as AI search changes fast. Behavioral signals are slow and correlational by design, built to smooth out noise over long studies. Smoothing over long studies is a reasonable way to tune a mature product like ten blue links. It’s a shakier way to govern a system rewriting itself every few months, where a confidently wrong answer can satisfy the metric and mislead the user at the same time. Google’s grading system is real, but it’s measuring a proxy, and the proxy has limits.

The work that satisfies either way

For everyone doing the actual work, the takeaway lands in a reassuring place. There’s no trick that satisfies Google’s satisfaction signal, because the signal is only people being glad they found you. You get there by answering real questions well, with enough depth and honesty that a reader doesn’t need a second opinion. And the blind spot is an argument for the same thing, not against it. In a system that can’t fully tell a good answer from a confident one, being reliably, checkably right is what earns the return visits and the references from credible sites that outlast any single algorithm. The metric may be a proxy. Useful, trustworthy work is how you win it anyway.