Do AI idea validators ever say no? We checked ours

What we checked

ShipFit puts an idea through nine decisions. The first, "is it worth building", returns a market score out of 100 and one of six verdicts, the harshest of which tells the founder to drop the idea. We pulled the latest first-stage result for every idea submitted between October 2025 and August 2026 and counted them. The same data, with the rest of the corpus and the full method, is in what 637 startup ideas show.

The verdict that never fires

Two thirds of all ideas got the same verdict, "promising, needs focus". The kill verdict was never assigned. Not rarely: never.

The scores cluster at the top

If the verdicts were cautious but the scores spread out, the score would still be useful. They do not. The median idea scored 82. Seven ideas in 520 scored below 50.

It is not one score, it is the pattern

The same generosity shows up at every stage that asks the engine for a judgement. It rated one proposed solution in 238 as weakly differentiated. It called reaching the buyer hard for fewer than one idea in sixteen. And when it scoped a first version, it recommended the same mid-sized package 208 times out of 208.

Why an AI validator says yes

This is our reading of the data, not something the data proves. Three things push in the same direction.

• The model is trained to be helpful. Large language models are tuned on human ratings, and people rate encouraging answers more highly. Telling someone their idea is dead is the least helpful-feeling thing a model can say.
• It only hears the founder's side. The engine judges the idea as the founder describes it. Founders describe their own ideas well. The buyer who would never pay is not in the room.
• There is always a softer option. Given six verdicts, a model that is unsure picks one of the hedged ones. "Let us find out" is a no that does not feel like one.

Founders heard the soft no anyway

Here is the part that rescues the verdict. Founders who got "let us find out" almost never carried on: 2% of them finished all nine stages. Founders told their idea had a strong signal finished 78% of the time. Founders read the hedge for what it was, so the verdict that never said no still did the job of one.

How to read any AI verdict on your idea

None of this is specific to ShipFit. Ask any general chatbot whether your startup idea is good and you will get a version of the same answer. If you use an AI to judge an idea, these rules make it useful.

1. Ignore the number. When almost everything scores above 80, a score of 84 tells you nothing. Compare ideas against each other, never against the scale.
2. Read the named flaws. The engine's diagnoses were far more varied than its scores. Half of all ideas were flagged as two products at once, which is specific and fixable.
3. Treat a hedge as a no. "Needs focus", "let us find out" and "worth exploring" are what a polite system says instead of no.
4. Ask for the case against. "Who would never buy this, and why?" gets a more honest answer than "is this a good idea?".
5. Test with behaviour, not opinion. A pre-sale or a priced waitlist answers the question no model can. The Mom Test covers how to ask people without getting polite lies back.

What this data does not show

It covers one engine: ours. We have not measured other tools, and the sample is people who chose to run an AI validation tool, not founders in general. It also says nothing about whether the ideas were good. It is possible, if unlikely, that 520 people in a row submitted viable ideas. What it does show is that the engine's verdict could not have told you if they had not.

If you want to compare tools on what they actually return, the idea validator comparison lists the main options, and the free idea score calculator shows the inputs behind a score instead of just the score.