Testing activity is not the same as evidence that a service is fit to accept. Test counts, pass rates and defect totals can look reassuring while critical coverage, production-like conditions, traceability or operational acceptance remain weak.
Enigma sits on the client’s side of the table. We independently assess whether quality and test evidence is sufficient for the decision an SRO, programme board or accountable owner must make. We do not provide a second version of the supplier’s test report, and nobody is permitted to assure controls or evidence they directly designed, managed or executed.
The problem: programmes measure activity more easily than confidence
Quality reporting often concentrates on the number of tests planned, executed and passed. Those figures are useful, but they do not answer whether the right things were tested, whether results are trustworthy or whether residual exposure is acceptable. A high pass rate can coexist with incomplete requirements, untested integrations, unrealistic data, unstable environments, deferred defects or weak business acceptance.
The reporting gap becomes more serious near a contractual milestone, migration rehearsal, go-live decision or operational handover. Time pressure can turn exit criteria into negotiable aspirations. Known defects are reclassified without a clear impact assessment. Evidence from lower environments is treated as proof of production readiness. The board is then asked to accept risk without a complete account of what is known, what is assumed and what has not been demonstrated.
The control gap
Delivery teams should own quality planning, testing, defect resolution and acceptance evidence. Their governance should track progress and intervene when controls fail. Independent assurance has a different purpose: it tests whether those controls and their outputs are sufficiently designed, consistently applied and evidenced to support the client’s decision.
Without that independent layer, a supplier or internal delivery function can effectively mark its own homework. The client may receive detailed reports but still lack an unbiased conclusion on coverage, execution integrity, defect exposure and readiness.
When independent quality assurance is needed
- Exit or acceptance criteria are unclear, changing or routinely waived.
- Supplier pass-rate reporting cannot be traced to requirements, risks and business processes.
- Defect severity, priority, ageing or workarounds are contested.
- System integration, non-functional, security, migration or operational testing is incomplete.
- Environments and test data do not represent the intended service.
- Business acceptance is being compressed into the final delivery stage.
- A board must decide whether to proceed, proceed with conditions, hold or reject.
The Enigma quality and test assurance method
Our review follows the evidence-to-decision discipline set out in How We Assure. Scope is based on the decision and exposure, not on reproducing every delivery activity.
- Frame the decision. We identify the acceptance, release, migration, cutover or contractual decision that the evidence must support.
- Establish the test basis. We test whether requirements, business processes, risks, architecture and acceptance conditions form a controlled basis for coverage.
- Assess control design. We examine strategy, plans, environments, data, traceability, defect management, reporting and governance.
- Test execution integrity. We sample source evidence to confirm that reported results are reproducible, attributable and consistent with the stated conditions.
- Evaluate residual exposure. We connect gaps, defects, untested conditions and workarounds to business and operational consequences.
- Conclude for the decision. We state the confidence supported by evidence and any conditions that must be met before proceeding.
What evidence is required
The evidence set is tailored to the stage and decision. It commonly includes the quality or test strategy, phase plans, requirements and acceptance criteria, risk-based coverage model, traceability records, environment and test-data controls, execution results, defect records, non-functional evidence, migration reconciliation, business acceptance, operational readiness, waivers and governance decisions.
We look beyond document presence. Evidence must show that the control operated, exceptions were understood, departures were authorised and residual risk has a named client owner. Where evidence is missing or inconsistent, that uncertainty is reported rather than converted into an artificial green status.
A multidisciplinary view of quality
Quality is not owned by the test team alone. Programme direction, programme management, agile delivery, technical architecture, data migration, cutover and operations all shape whether the delivered service is usable and supportable. Enigma brings those disciplines together as one client-side assurance capability.
Where overall delivery credibility is in question, quality findings connect to Programme Health Checks and Recovery. Where evidence depends on supplier claims, we coordinate with Supplier Delivery Assurance. Migration reconciliation and go-live controls are addressed through Data Migration and Cutover Assurance.
Outputs
- An executive conclusion aligned to the acceptance or release decision.
- A confidence assessment covering test basis, coverage, environments, data, execution, defects and acceptance.
- Evidence-linked findings with consequence, owner and required action.
- A clear statement of residual exposure and untested conditions.
- Conditions for proceeding and the evidence required to close them.
- A focused improvement plan where controls need strengthening before the next decision point.
Use the Independent Assurance Readiness Checklist to prepare the evidence set. The Sample Independent Assurance Report demonstrates how findings are connected to decisions. Further practical material is available in Resources.
Decision value
The purpose is not to declare that testing is complete. It is to help the client decide whether the evidence justifies acceptance, whether conditions are needed, whether delivery should pause, or whether risk must be explicitly retained by an accountable owner. The board receives a view of confidence and uncertainty that can withstand challenge.
Commission quality and test assurance
An engagement can focus on a quality strategy, a critical test phase, supplier evidence, defect exposure, acceptance readiness or a specific go-live decision. We begin by defining the decision, timeline, delivery ownership and available evidence, then set a proportionate assurance scope.
Discuss an independent quality and test assurance review or explore the wider Independent Delivery Assurance service.