Definition
Testing and Testability Testing checks whether software behaves as intended under known conditions, while testability is the degree to which a system can be controlled, observed and isolated during those checks. AI systems need conventional unit, integration and end-to-end tests for deterministic code, plus evals and error analysis for probabilistic model behaviour, retrieval quality and tool-use workflows.
The testing pyramid combines unit, integration, contract, system and acceptance tests. Traditional tests verify deterministic behaviour; evals verify probabilistic behaviour. Both are needed in AI applications.
Testability means designing components so dependencies, data and model calls can be controlled or substituted. It makes failures reproducible and lowers evaluation cost.
Engineering Mode
Testability means the AI system can prove behavior repeatedly enough to change safely. Architecture should expose deterministic seams, traces, fixtures and isolated model boundaries. Design must define observable outcomes and recovery states. Build work combines unit tests, integration tests, evals and manual review. Operations track regressions after model, prompt or data changes. The risk is accepting impressive examples that cannot be reproduced.
Evaluation check
Ask what observable evidence would show that this concept has been applied well in a real system, repository or learning exercise.
Real Example
A support-ticket classifier can be made testable with labelled historical tickets, boundary examples and regression suites. When the prompt or model changes, the team reruns the set and inspects routing errors. This catches silent degradation before customers wait in the wrong queue.
Related Concepts
Further study: Kent Beck, Test Driven Development , and Martin Fowler, Testing .
