Your CAF Assessment Is Complete. But Can You Trust the Results?

A recovery exercise is completed successfully. The systems are restored, the test report is signed off and the evidence is filed.

This is one of the challenges at the heart of the NCSC Cyber Assessment Framework (CAF). An assessment can be completed, evidence can be collected and judgements can be recorded without necessarily providing a reliable picture of the organisation's resilience. The difference lies in what has genuinely been demonstrated.

For organisations responsible for essential services and critical systems, that distinction matters. At E2E Security Consulting (E2E SC), our experienced cyber security consultants help organisations look beyond assessment completion. We examine whether the correct requirements have been applied, whether the evidence supports the conclusions being reached and whether the resulting assessment provides a meaningful basis for security and resilience decisions.

The First Risk: Assessing Against the Wrong Requirements

Before an organisation can establish whether it meets a CAF outcome, it needs to establish which requirements apply. That sounds straightforward. In practice, it can be surprisingly complex.

CAF v4.0 provides a common foundation of four objectives, fourteen principles and 41 contributing outcomes. However, different regulatory and assurance programmes can apply their own profiles, sector-specific requirements and reporting arrangements. Three examples illustrate why this matters.

An assessment must therefore do more than demonstrate that security measures exist. It must establish whether the evidence supports the achievement levels required by the applicable Government CAF Profile. An organisation assessing only the generic CAF outcomes could overlook requirements that are relevant to its regulatory obligations and operational risk. Although both the Ofgem and NHS approaches introduce an Objective E, they address different risks and contain different expectations.

Experienced CAF consultants help organisations establish the governing requirements before the assessment begins, reducing the risk of overlooked outcomes, inappropriate assumptions and costly rework. But applying the correct requirements is only the first challenge.

A Successful Recovery Test Does Not Always Prove Service Resilience

The organisation has evidence that part of its recovery capability works. It does not yet have sufficient evidence from that exercise to conclude that the complete essential service can be recovered under the scenario being assessed.

That distinction is particularly relevant to CAF Principle D1 - Response and Recovery Planning, including the expectations around response and recovery capability and testing and exercising. The NCSC's framework focuses on the organisation's ability to minimise the impact of cyber incidents on essential functions, including restoring those functions where necessary.

The missing identity service does not automatically mean the organisation's recovery arrangements are ineffective. There may be alternative arrangements, existing assurance evidence or other exercises demonstrating that the dependency can be recovered. But those possibilities need to be examined rather than assumed.

An experienced CAF consultant can help determine whether the available evidence is sufficient, whether additional testing is necessary and how any remaining uncertainty should be reflected in the assessment. If the investigation identifies a genuine resilience weakness, the organisation can take proportionate action:

• updating its recovery arrangements;
• including the dependency in future exercises;
• validating alternative access mechanisms;
• addressing the underlying technical limitation.

That is the difference between recording activity and establishing meaningful assurance.

Evidence Can Be Genuine Without Being Sufficient

One of the most common assessment difficulties is not the absence of evidence. It is uncertainty about what the evidence actually proves.

For example, vulnerability management reports may demonstrate that identified vulnerabilities are being tracked and remediated. But if the underlying scanning coverage excludes critical systems, the reports cannot by themselves establish that vulnerability management is effective across the entire assessed environment. Similarly, a monitoring dashboard may demonstrate that alerts are being generated. It does not necessarily establish that the organisation can detect relevant attacks, investigate them effectively and respond within an appropriate timeframe.

This is where professional assessment experience becomes particularly important. The NCSC makes clear that Indicators of Good Practice (IGPs) inform expert judgement rather than functioning as an inflexible checklist. The formal CAF judgement is made at contributing-outcome level, taking account of the relevant indicators, organisational circumstances, supporting evidence and any appropriate compensating measures.

The challenge for an assessor is therefore not simply to determine whether documentation has been supplied. It is to establish whether the available evidence genuinely supports the security or resilience outcome being claimed.

The Danger of Unsupported Positive Judgements

Assessment results inevitably attract management attention. A report showing a substantial proportion of outcomes assessed as Achieved can create confidence that the organisation has a strong security position. That confidence may be justified. But it depends on the quality of the underlying assessment.

Equally, an assessment should not automatically treat missing evidence as proof that a control has failed. There is a meaningful distinction between a control that has been tested and found ineffective, a control whose operation has not been sufficiently demonstrated and an assessment that has not yet examined the complete set of relevant indicators.

These situations may require different further work and different treatment under the applicable assessment methodology. An experienced assessor needs to understand that distinction.

That may mean confirming that an outcome is achieved, identifying a genuine weakness or explaining why further evidence is needed before a conclusion can be reached. What matters is that the judgement follows from the evidence, not the other way around.

Would Your Assessment Stand Up to Independent Scrutiny?

An effective assessment should be understandable to someone who was not involved in preparing it. That person should be able to see which essential functions and systems were assessed, which requirements applied and why the available evidence supports each material conclusion. They should also be able to identify the assessment's limitations.

When that traceability is missing, even a technically sound security arrangement can become difficult to demonstrate. Independent scrutiny may expose inconsistencies between evidence and judgements, uncertainty about ownership or assumptions about systems and dependencies that have never been validated. Addressing those issues before formal review is generally more valuable than explaining them after they have been challenged.

At E2E Security Consulting, we help organisations examine the strength of their assessment results through practical, evidence-focused challenge. This may involve reviewing assessment boundaries, questioning whether technical and operational evidence covers the intended scope, examining the rationale behind proposed contributing-outcome judgements and identifying where further investigation is necessary.

We also help distinguish weaknesses in the underlying security arrangements from weaknesses in the evidence available to demonstrate them. That distinction matters when determining what should happen next.

The objective is not to challenge conclusions for the sake of finding deficiencies. It is to help the organisation develop an assessment position that reflects its actual security and resilience capabilities.

Turning Assessment Confidence into Resilience Improvement

A CAF assessment should do more than describe the organisation's position at a single point in time. Its findings should support informed decisions about where improvement is needed and which actions matter most.

Returning to the recovery example, extending the exercise to include the omitted identity service may provide more than better assessment evidence. It could reveal a dependency that would otherwise delay recovery during a real cyber incident. Addressing that dependency may improve the organisation's operational resilience directly.

The same principle applies to vulnerabilities outside scanning coverage, inadequate supplier assurance, gaps in security monitoring or controls whose effectiveness has never been properly tested. When assessment findings are understood in the context of essential services and operational consequences, they become a practical input to risk management and improvement planning. They also provide a stronger basis for management decisions.

Specialist CAF Expertise Where It Matters Most

Not every organisation needs a consultant to manage its entire CAF assessment.

E2E Security Consulting can provide targeted support in these areas or work alongside internal teams throughout the wider assessment. Our consultants bring practical cyber security and assurance experience to the assessment process, helping organisations identify unsupported assumptions, understand what their evidence genuinely demonstrates and develop clear, defensible conclusions.

A Completed Assessment Is Not the Final Measure of Success

Completing a CAF assessment is an important milestone. But completion alone does not establish that the relevant cyber security outcomes are being achieved, or that the organisation fully understands the resilience of its essential services. That requires the right assessment scope, appropriate evidence, informed professional judgement and a willingness to examine conclusions that may initially appear satisfactory.

The purpose of CAF assurance is not simply to demonstrate that an assessment has been completed. It is to provide confidence that the organisation understands its cyber resilience and knows where improvement is needed.

You can also explore our GovAssure and CAF consultancy services.