Skip to content

03_Secure

Jump to bottom
Phillip Bailey edited this page Jun 24, 2025 · 5 revisions

Secure

The Secure function focuses on applying safeguards that protect AI systems from threats to confidentiality, integrity, availability, and trustworthiness. This includes both technical controls and organizational safeguards across the AI lifecycle.

This section aligns with:

  • NIST CSF 2.0: PROTECT
  • NIST AI RMF 1.0: MEASURE
  • EU AI Act: Title IV (Obligations of Providers of High-Risk AI Systems)

Objectives

  • Secure training data, models, and outputs across the AI lifecycle
  • Implement controls to ensure explainability, fairness, and robustness
  • Prevent adversarial manipulation (e.g., prompt injection, data poisoning)
  • Protect privacy and enforce access control for sensitive AI components
  • Embed responsible AI principles into development and deployment workflows

Outcomes

  • AI systems are protected against known vulnerabilities and adversarial threats
  • Technical safeguards support compliance with legal and ethical expectations
  • AI behavior aligns with trustworthiness attributes (e.g., explainability, fairness, resilience)
  • High-risk AI systems meet EU AI Act obligations for risk controls and documentation

Key Security Elements

Element Description
Secure Development Practices Apply secure coding and CI/CD controls to AI models, pipelines, and APIs
Model Robustness Evaluate models against adversarial inputs, overfitting, and misuse
Privacy and Data Protection Implement differential privacy, anonymization, and access controls
Access Management Restrict and audit access to models, APIs, datasets, and infrastructure
Explainability Controls Integrate explainability techniques to surface model logic and limitations
Fairness Controls Apply debiasing tools and testing to minimize harmful or discriminatory outcomes
Third-Party Controls Vet and secure pre-trained models, datasets, APIs, and AI vendors
Guardrails and Filters Deploy prompt filters, output validators, and context restrictions for LLMs and generative models

Example Practices

  • Use model cards and datasheets for transparency of training data and limitations
  • Apply OWASP Top 10 for LLMs to evaluate and mitigate model exposure
  • Enforce SAST, SCA, and IaC security scans in ML pipelines
  • Run adversarial robustness tests (e.g., FGSM, text attacks) as part of QA
  • Implement least privilege access to AI pipelines and secrets
  • Integrate model explainability dashboards for critical systems

Related NIST Categories

CSF 2.0 PROTECT Categories

  • PR.AC – Identity Management and Access Control
  • PR.DS – Data Security
  • PR.IP – Information Protection Processes and Procedures
  • PR.MA – Maintenance
  • PR.PT – Protective Technologies

AI RMF MEASURE Focus Areas

  • Effectiveness of implemented risk controls
  • Verification of trustworthy characteristics (e.g., robustness, privacy)
  • Mitigation of system vulnerabilities
  • Post-deployment security validation

Artefacts

  • AI Threat Model & Mitigation Plan
  • OWASP LLM Top 10 Mapping Sheet
  • Security & Privacy Checklist for AI Pipelines
  • Model Card Template (Robustness, Bias, Explainability)
  • LLM Prompt Filtering Policy

Clone this wiki locally