dQuality insight
AI is a systems question
Many AI initiatives begin with a model comparison. Teams evaluate providers, discuss retrieval, and sketch an assistant into an existing interface. Those decisions matter, but they rarely determine whether the resulting feature creates durable business value.
A useful AI product depends on the system surrounding the model: the quality of available data, the clarity of the user's task, the product's ability to capture feedback, and the organisation's capacity to operate something probabilistic. If those foundations are weak, a more capable model usually makes the weakness more expensive rather than less visible.
AI readiness is therefore not a question of whether a company can connect to an API. It is a question of whether the business can define, measure, and safely improve an AI-assisted workflow.
Start with a specific decision or task
"Add AI" is not a product requirement. A better starting point is a recurring task in which users currently spend time interpreting information, producing a draft, identifying an exception, or deciding what to do next.
The task should be concrete enough that a team can describe a useful output and recognise an unacceptable one. "Help customers understand their account activity" is still broad. "Explain why this month's invoice differs from the previous one, using verified billing data" is testable.
Before implementation, document:
- who performs the task and how often;
- what information they need;
- what a correct result enables them to do;
- what happens when the result is incomplete or wrong;
- which parts require human judgement;
- how the business will know the workflow improved.
This reframing keeps the model in its proper role: one component inside a product experience designed around a real job.
Audit the data path
An AI feature cannot reliably use knowledge the product cannot locate, permission, or interpret. Teams should map the complete path from source data to generated output.
That includes identifying the systems of record, the owners of each dataset, the frequency of updates, and any access rules that vary by user or organisation. It also means finding ambiguous fields, duplicated records, unstructured documents, and historical content that should no longer influence an answer.
Data readiness does not require every source to be perfect. It does require the product to know which sources are authoritative and to preserve that authority through retrieval and presentation.
For knowledge-heavy features, provenance should be part of the interface. Users need a practical way to inspect the material behind an answer. For transactional features, the system must separate explanatory suggestions from actions that change data or trigger external processes.
Design for uncertainty
Traditional software is expected to behave predictably for a defined input. Generative systems operate differently. The same prompt can produce outputs with varying wording, emphasis, and quality.
The interface should communicate that uncertainty without transferring all responsibility to the user. Useful patterns include constrained inputs, structured outputs, source links, editable drafts, confirmation steps, and clear boundaries around what the system can access or execute.
A confident paragraph is not evidence of a correct answer. Product design should make verification easier than blind acceptance.
The appropriate degree of autonomy depends on consequence. A low-risk internal summary may be produced automatically. A customer-facing recommendation may require review. A financial, legal, or account-changing action should have explicit controls and an auditable trail.
Build an evaluation set before launch
Teams often rely on informal testing: a few people try prompts and decide whether the output feels impressive. That is useful during exploration, but it is not a release standard.
Create a representative evaluation set from real task patterns, with sensitive data removed or handled appropriately. Include routine cases, incomplete inputs, conflicting sources, adversarial requests, and examples where the correct response is to decline or ask for clarification.
For each case, define what matters. Depending on the workflow, criteria may include factual grounding, completeness, correct use of permissions, tone, format, latency, or the absence of unsupported claims.
Evaluation should continue after launch. Model behaviour, source data, prompts, and user expectations all change. An AI feature is an operated capability, not a one-time integration.
Prepare the operating model
AI products introduce costs and responsibilities that sit across product, engineering, design, support, security, and legal teams. Ownership must be explicit.
Someone needs to monitor failures, review user feedback, maintain prompts and retrieval logic, manage provider changes, and decide when quality is sufficient for wider release. Support teams need language for explaining limitations. Product teams need visibility into usage and abandonment. Engineering teams need safeguards for cost, latency, and dependency failure.
A staged rollout is usually more useful than a broad launch. Begin with a narrow audience and a well-bounded task. Observe where users hesitate, edit, retry, or leave the workflow. Those behaviours often reveal more than a simple satisfaction score.
A practical readiness test
A company is ready to build an AI feature when it can answer six questions clearly:
- What specific user task will improve?
- Which data sources are authoritative?
- How will users verify or correct the output?
- What is the consequence of an incorrect result?
- How will quality be evaluated over time?
- Who owns the feature after launch?
If several answers are missing, the next step is not necessarily more AI research. It may be better data access, a clearer workflow, or a smaller product scope.
The strongest AI products rarely begin with the broadest ambition. They begin with a disciplined understanding of where intelligence can remove friction without removing accountability.
