Blog

Thoughtbubble Blog

The Riskiest Assumption Framework for AI Product Validation

Product managers can de-risk a new AI feature by finding its single riskiest assumption and testing it with a tiny experiment before production code.

Before an AI feature reaches a sprint board, it carries assumptions about whether the model can do the work, whether people want the outcome, and what each request costs to serve. AI product validation treats those assumptions as testable questions instead of beliefs. The method below helps you find the one assumption that, if wrong, makes the feature pointless, then test it with an experiment small enough to run in days.

Why AI features break MVP habits

Standard MVP thinking assumes you can ship a rough version and learn from usage. Generative AI adds a second layer of risk: the model itself may not reliably produce the output the feature depends on, and each call carries a variable cost. A prototype that demos well can still fail on consistency, latency, or the cost of serving real traffic. Before assuming a model belongs in the flow at all, weigh whether AI or traditional automation is the better fit for the task. So the cheapest experiment is rarely the full feature. It is the smallest test of the assumption the feature rests on.

Step 1: Isolate the one assumption that can sink the feature

Sort your assumptions into three constraints: can it work, can we afford it, will they use it. Can it work covers whether the model reliably produces the output the feature needs, such as structured summaries that hold their shape across edge cases. Can we afford it covers the cost of serving the output, especially when each user action triggers model calls. Will they use it covers whether people want the outcome enough to change their current workflow.

Then apply a kill test. Ask which single assumption, if proven false, makes the whole feature not worth shipping. If no output is usable, cost and demand are irrelevant. Write that assumption as one falsifiable sentence, because that sentence is what your experiment will test.

Step 2: Map impact against uncertainty

Rank the remaining assumptions on two axes: impact if wrong, and your current uncertainty. High-impact and highly uncertain items deserve the experiment first. Well-understood assumptions, or ones with a cheap fallback, can wait. The goal is to isolate the variable that decides the feature's fate, not to average risk across a backlog. If two assumptions look equally dangerous, ask which failure would rule out more of the design, and test that one first.

Step 3: Build the smallest honest test

A tiny experiment simulates the feature without production code, and the discipline is the same as when you test an AI workflow before building the full app. The pattern you choose depends on the assumption you wrote in step 1. Run a prompt set: take the exact model you expect to use and feed it 20 real inputs, including awkward ones, and inspect the outputs by hand. Run a Wizard of Oz: build the interface, but have a person generate the outputs behind the curtain, so you learn whether people value the result before the model work begins. Run a cost probe: wire one API call into a spreadsheet and multiply usage by the price per call to see whether serving the feature at real volume breaks the budget.

Each pattern tests one assumption. Keep the test small enough to finish in a few days.

Step 4: Set the pass mark before you run it

Decide what success looks like before seeing results, because thresholds chosen afterwards bend toward whatever the data happened to say. Write your thresholds down. For a prompt set, that might be the share of outputs you would ship with only light edits, with a minimum count of usable outputs per run. For a Wizard of Oz, it might be how many participants describe the outcome as solving their problem, or would use it again. For a cost probe, it is a ceiling on cost per user action that the business can actually absorb. Name the sample size too, even a modest one, so the result is a decision rather than a debate.

Step 5: Decide, pivot, or kill

Read the result against the threshold you set in advance. When the test passes, the assumption holds for now and the feature can move forward with that risk reduced. When it fails clearly, kill or reshape the feature early, before engineering spend compounds. Between those outcomes sits the useful middle: a failure that points at the prompt, the interface, or the scope rather than the core idea. That is a pivot, and it deserves a narrow retest of the changed component only. Record the decision and the evidence so the next team does not re-litigate it.

If your team wants help running this kind of experiment before committing to a build, explore AI prototyping with Thoughtbubble.