The Answer Is Not the Product, the Explanation Is
GeoGenius can look at a Street View shot and tell you where it is. That turned out to be the least useful thing it does.
GeoGenius drops you into Street View and helps you work out where in the world you are. It is a trainer for people who play GeoGuessr and want to stop guessing badly.
The first version just answered. You looked at a scene, asked, and a vision model told you the country. It was accurate and it was genuinely impressive for about four minutes.
Then I noticed I was getting worse.
An oracle removes the reason you came
If the model names the country, the loop is: look, ask, get told, move on. Nothing is retained, because nothing was worked out. You have automated the exact cognitive step the product exists to develop.
This is a specific trap for AI training tools, and it is easy to walk into because the demo is so good. A model that answers instantly looks like the strongest possible version of the product. For anything whose purpose is to make the user more capable, it is the weakest, because capability transfer happens in the reasoning and you just deleted the reasoning.
Any product where the user’s goal is to get better at something has to resist giving out answers, even though answers are the easiest thing it can produce.
Grade the evidence instead
So the model’s job changed. It no longer leads with a location. It reads the frame and names what is actually diagnostic in it:
- Which side of the road traffic is on
- Bollard and guardrail design, which is close to a regional fingerprint
- Road markings, line colour, and dash spacing
- Licence plate shape and proportions
- Script and language on any signage
- Vegetation, soil colour, and sun angle relative to the horizon
Those are the things a strong player reads in about two seconds. They are transferable, which the answer never was. Learn what Nordic bollards look like once and you carry it into every future round.
The location still comes, but it comes last, as the conclusion of a chain you can check rather than a fact you have to accept.
What I took from it
The output of a teaching tool is reasoning, not conclusions. If the user could have looked it up, you have built a lookup.
Impressive and useful are different axes. Instant answers demo better and teach worse.
Make the model show its evidence. It improves the product and it also makes wrong answers debuggable, because you can see which clue it misread.