The Product Manager's Guide to AI Features
AI features break almost every rule that product managers have learned to rely on. You can't fully specify them upfront because their behaviour is probabilistic. You can't unit-test them to a pass/fail threshold. Their performance drifts over time without any code changes.
Redefining Done
For conventional software features, "done" is binary: the feature does what the spec says. For AI features, "done" is a moving target. Model performance changes as the underlying model is updated. Data distributions shift as user behaviour evolves.
This means the definition of "done" for an AI feature has to include an ongoing monitoring commitment, not just a QA pass before launch. Before you ship an AI feature, you need to know: what metric tells us it's working? Who is responsible for monitoring it? What's the threshold at which we pull the feature?
Setting Expectations That Don't Backfire
The most effective framing we've seen in user research consistently positions AI features as tools that assist, not tools that decide. "Here are three suggested responses — choose the one that fits" dramatically outperforms "The AI will write your response automatically" — even when the underlying AI output is identical.
"Frame your AI feature as a tool that assists, not a tool that decides. The user's sense of control determines whether they trust it — not the model's accuracy rate."
The Metrics That Actually Matter
Task completion rate. Does the user actually accomplish their goal faster with the AI feature than without it?
Override rate. How often do users reject the AI's output and do the task manually instead?
Trust calibration over time. Do users' trust levels become more accurate as they develop intuition for when the AI is reliable?