ESP-IA: the new CCN methodology for evaluating the security of AI-enabled products

An assistant disclosing another user’s data or an agent taking an unauthorised action is a product security issue, not just a poor-quality response. Evaluating it requires understanding how models, data, tools and permissions interact.

To address this need, Spain’s National Cryptologic Centre (CCN) has published the ESP-IA methodology (CCN-STIC 2011), the AI Product Security Evaluation Methodology (Spanish-language source). If your product includes models, agents or AI-driven capabilities, it is worth understanding the required documentation, access and tests before planning an evaluation.

The full breakdown is available in our CCN-STIC 2011 guide.

What ESP-IA evaluates (and what it does not)

ESP-IA focuses on vulnerabilities whose materialisation depends on AI-specific mechanisms: inference, learning, adaptation, context or state, learned representation, or actions mediated by an AI capability.

The boundary is clear: if replacing the AI component with conventional software would leave essentially the same vulnerability and the same testing procedure, that vulnerability is generally outside the specific scope of ESP-IA. Scope is defined for the declared product version and configuration, not a guarantee against every AI-related risk.

How it decides which requirements apply to your product

This is the core of the methodology, and what sets it apart from a fixed list of requirements:

1. The manufacturer describes the product architecture: interfaces, AI models, datasets, services and tools, and how they relate to each other.

2. It answers the characterisation questions (C01 to C21) about its AI capabilities: whether an external actor can control inputs, whether the AI interprets instructions, whether it can trigger actions with relevant consequences, whether it uses external tools or APIs, whether it learns after deployment, and so on.

3. The applicability matrix turns those answers into the threats, policies and Security Functional Requirements (SFRs) that apply. For example, YES answers to C01 and C03 make direct-injection protection applicable only if the controllable input reaches the model interpreting instructions. All rules whose conditions are satisfied apply cumulatively.

   For example, an AI assistant that queries internal documentation and uses external tools may be subject to requirements related to instruction injection, information disclosure and the execution of unauthorized actions. Actual applicability will depend on its capabilities, interfaces and configuration, not simply on including an AI model.

4. The laboratory verifies each SFR through testing and additionally runs AI-specific penetration tests.

In total, ESP-IA defines 27 SFRs: 22 derived from threats (evasion, injection, extraction, membership, inversion, poisoning, disclosure, agency and resource exhaustion) and 5 derived from policies (human oversight of critical actions and traceability).

That said, the answers describe facts about the product, not vulnerabilities. An applicable SFR does not mean the product is vulnerable. It means there is a property to verify through testing. Not every product must meet all 27 requirements. Existing controls do not remove applicable requirements: they are documented and tested.

How the evaluation runs

The process relies on three actors: the applicant or manufacturer, an evaluation laboratory accredited and authorised by the CCN and the CCN itself, which validates the Security Declaration and issues the final result.

The evaluation has five stages:

• Analysis of the Security Declaration (DS), prepared per Annex A and previously validated by CPSTIC.
• Access to and configuration of the test environment for the declared product version.
• Verification of the applicable SFRs through functional, adversarial or penetration testing.
• Additional penetration testing, with a minimum of three person-days and the OWASP AI Testing Guide as the minimum reference.
• Technical Evaluation Report (ITE) per Annex B.

The three person-days are the minimum effort for that additional stage, not the duration of the whole evaluation. Applicable requirements must also be verified.

The verdict is PASS only if every applicable SFR passes, no non-conformity is recorded, and the penetration testing does not identify AI-specific vulnerabilities that compromise the product’s security. Otherwise, FAIL.

What manufacturers should start preparing

Four areas are worth considering when preparing an evaluation. The guide does not itself establish a general requirement to evaluate every AI-enabled product before selling to the public sector:

1. An AI capability map: what the product’s AI does, with what data, through which interfaces, and with which tools or external systems.
2. Characterisation answers worked through internally: they determine the set of SFRs that will ultimately apply.
3. Observability: logs of AI activity, protection of persistent state, and the ability to intervene in critical actions. These requirements apply when the relevant matrix rules are satisfied.
4. Evaluation environment: product version and configuration, technical documentation, user and configuration guides, and the access and means needed to run the tests.

How CYBSER can help

At CYBSER we handle the whole process: product characterisation, preparation of the Security Declaration, determining the applicable SFRs, performing the tests and producing the Technical Evaluation Report for review by the CCN. Your team provides the information, access and product corrections that only the manufacturer can supply.

Does your product embed AI and do you need an ESP-IA evaluation? At CYBSER we carry out security evaluations of AI-enabled products under the CCN-STIC 2011 methodology. Tell us about your case and we will help you determine the scope, the applicable requirements and the steps needed to carry out the evaluation.