Quick Navigation
AI Types & FoundationsAI Governance PrinciplesAI Lifecycle StagesEU AI Act — Risk ClassificationEU AI Act — Obligations by RoleEU AI Act — GPAI & Systemic RiskEU AI Act — PenaltiesGDPR Applied to AINIST AI Risk Management FrameworkISO/IEC 42001 — AI Management SystemOther Standards & FrameworksPrivacy-Enhancing TechnologiesBias & FairnessExplainability & InterpretabilityData Governance for AIAI Security ThreatsTesting & ValidationThird-Party & Vendor RiskHuman Oversight ModelsAI Roles & ResponsibilitiesPost-Deployment MonitoringAdditional AI Laws & RegulationsKey Formulas & MetricsAgentic AI GovernanceAI Documentation Requirements
AI Types & Foundations
- Narrow AI (Weak AI)
- AI designed for a specific task or domain (e.g., image classification, spam filtering). Cannot generalize beyond its trained purpose. All current AI systems are narrow AI.
- General AI (Strong AI / AGI)
- Hypothetical AI with human-level cognitive abilities across all domains. Can reason, learn, and adapt to any task. Does not yet exist.
- Supervised Learning
- Model trains on labeled data (input-output pairs). Used for classification and regression tasks. Requires ground-truth labels. Examples: spam detection, credit scoring.
- Unsupervised Learning
- Model finds patterns in unlabeled data without ground-truth labels. Used for clustering, dimensionality reduction, and anomaly detection. Examples: customer segmentation.
- Reinforcement Learning
- Agent learns by interacting with an environment, receiving rewards or penalties. Used for game playing, robotics, and RLHF (Reinforcement Learning from Human Feedback) for LLMs.
- Deep Learning
- ML using multi-layered neural networks to learn hierarchical representations. Excels at image recognition, NLP, and complex pattern detection. Subset of machine learning.
- Transformers
- Neural network architecture using self-attention mechanisms. Foundation for modern LLMs and generative AI. Processes entire input sequences in parallel rather than sequentially.
- Large Language Models (LLMs)
- Large-scale transformer models trained on massive text corpora. Capable of text generation, reasoning, summarization, and tool use. Outputs are probabilistic, not deterministic.
- Foundation Models
- Large pre-trained models adapted for many downstream tasks via fine-tuning or prompting. Examples: GPT, Claude, Llama. Basis for transfer learning in AI applications.
- Generative AI
- AI that creates new content (text, images, code, audio) based on learned patterns. Raises unique governance challenges around IP, hallucinations, and content provenance.
AI Governance Principles
- Ethical AI
- Focus on moral principles and values guiding AI development. Emphasizes doing what is right and avoiding harm. Theory-oriented dimension of responsible AI.
- Responsible AI
- Broader operational framework including accountability, governance processes, and organizational commitment. Encompasses ethical AI but adds implementation and oversight structures.
- Trustworthy AI
- Systems meeting technical and governance standards for reliability, safety, fairness, and transparency. NIST defines 7 characteristics. Encompasses both ethical and responsible AI.
- Fairness
- AI systems must treat all people equitably, avoid discrimination, and not create or reinforce unfair bias based on protected characteristics. Multiple mathematical definitions exist.
- Transparency
- Open access to model structure, data, and decision-making logic. How the model was created. Distinct from explainability (how it decided) and interpretability (how it works).
- Accountability
- Clear ownership and responsibility for AI system outcomes. Organizations and individuals must be answerable for AI decisions and consequences. Requires defined governance structures.
- Human Oversight
- Continuous human supervision of AI systems to ensure they operate within defined boundaries. Mandated by EU AI Act Article 14 for high-risk systems. Multiple oversight models exist.
- AI Literacy (Article 4 EU AI Act)
- Shared obligation for ALL providers and deployers to ensure staff operating/using AI systems have sufficient AI literacy. Takes into account technical knowledge, experience, and context of use.
- AI Value Creation vs. Risk
- AI creates organizational value (efficiency, insights, automation) while introducing risks (bias, errors, legal liability). Governance balances value realization with risk management.
AI Lifecycle Stages
- Business Case & Problem Definition
- First lifecycle stage: identifying the problem AI will solve, expected outcomes, success criteria, ROI justification, and whether AI is the appropriate solution.
- Data Collection & Preparation
- Gathering, cleaning, labeling, and transforming raw data. Most time-consuming lifecycle phase. Includes data quality validation, provenance tracking, and bias assessment.
- Model Development & Training
- Selecting algorithms, engineering features, training models, and tuning hyperparameters. Includes architecture decisions, training data splits, and cross-validation.
- Model Testing & Validation
- Evaluating model performance, fairness, safety, and robustness. Includes red teaming, adversarial testing, bias detection, and A/B testing before deployment.
- Release Readiness & Go/No-Go
- Governance gate before deployment: documentation completeness, risk assessment, approval sign-offs, halt conditions identified, and all governance conditions satisfied.
- Deployment
- Putting models into production. Options include on-premise, cloud, edge, RAG architectures, and agentic systems (NEW in v2.1). Vendor assessment required for third-party AI.
- Post-Deployment Monitoring
- Continuous performance tracking, drift detection, fairness monitoring, and incident response. Most under-prepared domain for AIGP candidates.
- Update, Retraining & Decommissioning
- Processes for updating, retraining, and retiring AI systems. Includes retraining triggers (threshold-based, time-based, data volume-based) and deactivation controls.
- v2.1 Key Change: Models to Systems
- AIGP v2.1 replaced 'AI models' with 'AI systems' throughout, reflecting that governance extends beyond the model itself to include entire systems, supply chains, and downstream uses.
EU AI Act — Risk Classification
- Unacceptable Risk (Prohibited)
- Banned since February 2, 2025: social scoring, subliminal/manipulative techniques, exploiting vulnerabilities, biometric categorization inferring sensitive attributes, untargeted facial scraping, real-time remote biometric ID in public (narrow exceptions), emotion recognition in workplaces/schools.
- High-Risk Systems (Annex II)
- Safety components of regulated products covered by existing EU harmonization legislation (e.g., medical devices, machinery, toys, aviation). Subject to extensive requirements.
- High-Risk Systems (Annex III)
- Standalone systems in specific areas: biometric categorization, critical infrastructure, education access, employment/hiring, credit scoring, law enforcement, migration/border, justice/democratic processes.
- High-Risk Requirements
- Risk management system, data governance, technical documentation, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity. Conformity assessment and CE marking required.
- High-Risk Timeline
- Standalone high-risk obligations (Annex III): August 2, 2026. Embedded in regulated products (Annex II): August 2, 2028 (per AI Omnibus). Registration in EU AI database required.
- Limited Risk
- Transparency obligations only. Chatbots must disclose AI interaction. Deepfakes must be labeled. AI-generated content must be marked as such.
- Minimal/No Risk
- No specific regulatory requirements. Examples: AI-enabled video games, spam filters. Voluntary codes of conduct encouraged.
EU AI Act — Obligations by Role
- Provider
- Organization developing or placing AI on market. Most extensive obligations: system design, technical documentation, conformity assessment, registration, CE marking, post-market monitoring, incident reporting.
- Deployer
- Organization using AI under own authority. Responsibilities: operate as intended, ensure human oversight, monitor performance, conduct FRIAs where required, notify users of high-risk AI use.
- Deployer Becoming Provider
- Deployers can BECOME providers if they: put their name/trademark on a high-risk AI system, make substantial modifications, or change the intended purpose. They then assume full provider obligations.
- Importer
- Entity placing non-EU provider AI on the EU market. Must verify conformity assessment completed, CE marking applied, technical documentation available, and provider meets obligations.
- Distributor
- Entity in the supply chain (other than provider/importer) making AI available on EU market. Must verify CE marking, required documentation, and that storage/transport conditions do not compromise compliance.
- Conformity Assessment (Provider)
- Provider obligation to verify the AI system meets all legal requirements BEFORE market placement. May be self-assessment or third-party (notified body) depending on system type.
- FRIA — Fundamental Rights Impact Assessment
- EU AI Act Article 27. Mandatory for deployers of high-risk AI in employment, education, essential services. Must be completed BEFORE deployment. Notify market surveillance authority. Effective August 2, 2026.
- Post-Market Monitoring
- Provider obligation to actively collect and review data on AI system performance after deployment. Must establish a post-market monitoring plan proportionate to the nature and risks of the system.
EU AI Act — GPAI & Systemic Risk
- General Purpose AI (GPAI)
- AI models trained on vast datasets enabling broad applications across many tasks. Not designed for a single specific purpose. Rules applicable since August 2, 2025.
- GPAI Provider Obligations
- Technical documentation, transparency to downstream providers, compliance with EU copyright laws, publishing a sufficiently detailed training content summary.
- GPAI with Systemic Risk
- GPAI models with high-impact capabilities posing systemic risk. Threshold: trained using total compute >10^25 FLOPs, or designated by the AI Office based on other criteria.
- Systemic Risk Obligations
- In addition to standard GPAI obligations: model evaluation against state-of-the-art benchmarks, assess and mitigate systemic risks, track and report serious incidents, ensure adequate cybersecurity.
- AI Office
- EU body within the European Commission responsible for overseeing GPAI rules. Has power to designate models as systemic risk, request documentation, and enforce compliance.
- Copyright Compliance for GPAI
- GPAI providers must comply with EU copyright law, respect opt-out mechanisms for text and data mining, and publish a sufficiently detailed summary of training content.
- Open-Source GPAI Exception
- Open-source GPAI models have reduced obligations (mainly publishing training data summary and copyright compliance) UNLESS they pose systemic risk, in which case full obligations apply.
EU AI Act — Penalties
- Prohibited AI Violations
- Up to EUR 35 million or 7% of worldwide annual turnover (whichever is higher). Highest penalty tier for deploying banned AI practices.
- High-Risk & GPAI Violations
- Up to EUR 15 million or 3% of worldwide annual turnover (whichever is higher). For non-compliance with high-risk system requirements or GPAI obligations.
- Incorrect Information to Authorities
- Up to EUR 7.5 million or 1% of worldwide annual turnover (whichever is higher). For supplying incorrect, incomplete, or misleading information to notified bodies or authorities.
- SME/Startup Adjustments
- Fines are capped at the lower of the percentage or the fixed amount for SMEs and startups, ensuring proportionality. Administrative fines must be effective, proportionate, and dissuasive.
- Enforcement Timeline
- Prohibited AI bans: February 2, 2025. GPAI rules: August 2, 2025. High-risk (Annex III): August 2, 2026. High-risk (Annex II/embedded): August 2, 2028.
GDPR Applied to AI
- Article 5 — Processing Principles
- Purpose limitation (data used only for specified purposes), data minimization (only necessary data), accuracy (keep data correct), storage limitation. All apply to AI training data.
- Article 6 — Lawful Basis
- Six lawful bases: consent, contract, legal obligation, vital interests, public interest, legitimate interests. AI systems processing personal data must establish a valid lawful basis.
- Article 9 — Special Categories
- Heightened protection for sensitive data: racial/ethnic origin, political opinions, religious beliefs, health, biometrics, sexual orientation. Generally prohibited unless explicit exception applies.
- Article 22 — Automated Decision-Making
- Right not to be subject to solely automated decisions with legal/significant effects. Requires explicit consent, contractual necessity, or legal authorization. Must provide meaningful info about logic involved and clear path for human intervention.
- Articles 13/14 — Transparency
- Controllers must provide information about automated decision-making including meaningful information about the logic involved, significance, and envisaged consequences. Applies at data collection (Art 13) or when data obtained indirectly (Art 14).
- Article 35 — DPIA
- Data Protection Impact Assessment required BEFORE processing likely to result in high risk to rights and freedoms. AI systems using profiling, automated decisions, or large-scale processing typically trigger DPIA requirement.
- AI-Specific DPIA Factors
- Must assess: model error rates across demographic groups, evidence of disparate impact, whether human oversight is genuinely meaningful, provenance/quality/representativeness of training data.
- Controller vs. Processor in AI
- Controller determines purposes and means of processing (greater obligations). Processor acts on controller's behalf. AI providers may be either depending on the arrangement. Role confusion is a common exam trap.
- DPIA vs. FRIA Distinction
- DPIA (GDPR Art 35): focuses on data protection risks. FRIA (EU AI Act Art 27): focuses on fundamental rights impacts of high-risk AI. Different legal bases, different scopes, but may overlap.
NIST AI Risk Management Framework
- GOVERN Function
- Cross-cutting function that infuses throughout all other functions. Establishes organizational policies, accountability structures, risk culture, and governance processes across the AI lifecycle.
- MAP Function
- Context identification: stakeholders, system boundaries, potential harms, intended use, lifecycle impact analysis. Establishes the context for risk assessment and categorizes AI risks.
- MEASURE Function
- Quantitative, qualitative, or mixed-method tools to analyze, assess, benchmark, and monitor AI risk. Uses metrics and evidence to evaluate identified risks from MAP function.
- MANAGE Function
- Risk resource allocation, treatment prioritization, risk acceptance decisions, incident response, and system decommissioning. Acts on risks identified and measured in prior functions.
- NIST Trustworthy AI — 7 Characteristics
- 1) Valid and Reliable, 2) Safe, 3) Secure and Resilient, 4) Accountable and Transparent, 5) Explainable and Interpretable, 6) Privacy-Enhanced, 7) Fair with Harmful Bias Managed.
- NOT NIST Characteristics
- Common exam traps: 'Tested and Effective' and 'Commercially Viable' are NOT NIST Trustworthy AI characteristics. Do not confuse with the actual 7 characteristics.
- NIST AI 600-1 (GenAI Profile)
- Companion to AI RMF for generative AI. 12 risk categories including: CBRN, confabulation, data privacy, environmental impacts, harmful bias, information integrity, IP, value chain integration. Four considerations: governance, content provenance, pre-deployment testing, incident disclosure.
- NIST ARIA — REMOVED in v2.1
- NIST ARIA (AI Risk and Impact Assessment) step-by-step procedures were REMOVED from the v2.1 Body of Knowledge. No longer testable content.
ISO/IEC 42001 — AI Management System
- ISO 42001 Overview
- Certifiable AI management system standard. 10 clauses following the Harmonized Structure (context, leadership, planning, support, operation, performance evaluation, improvement). Tested at level of purpose and structure.
- Annex A — 38 Controls, 9 Domains
- A.2 AI Policies (2), A.3 Internal Organization (2), A.4 Resources for AI Systems (5), A.5 Assessing Impacts (4), A.6 AI System Lifecycle (8), A.7 Data for AI Systems (5), A.8 Information for Interested Parties (4), A.9 Use of AI Systems (4), A.10 Third-Party and Customer Relationships (4).
- Statement of Applicability
- Required document that records which Annex A controls apply to the organization and provides justifications for any exclusions. Critical for certification audits.
- Annex B
- Implementation guidance for Annex A controls. Provides practical direction on how to implement each control domain effectively.
- ISO 42001 vs. NIST AI RMF
- ISO 42001 creates a certifiable evidence structure. NIST AI RMF provides an operational risk management model. Organizations often use both: NIST for methodology, ISO for certification.
- ISO 42001 vs. ISO 27001
- ISO 42001 (AI management) follows the same Harmonized Structure as ISO 27001 (information security). Organizations certified to ISO 27001 will find familiar patterns in 42001.
Other Standards & Frameworks
- ISO/IEC 23894 — AI Risk Management
- Published February 2023, companion to ISO 31000 and ISO 42001. Covers: communication/consultation, scope/context/criteria, risk assessment (identification, analysis, evaluation), risk treatment, monitoring/review, recording/reporting.
- ISO/IEC 42005 — AI Impact Assessment (NEW v2.1)
- Not certifiable. Provides methodology for determining AI system impacts on people, society, and environment. Feeds into overall AI risk management lifecycle. Added to AIGP Body of Knowledge in v2.1.
- OECD AI Principles
- Global governance norms adopted by 40+ countries. Core principles: inclusive growth, human-centered values, transparency/explainability, robustness/security/safety, accountability. Updated 2023-2024 for generative AI.
- UNESCO Recommendation on Ethics of AI
- First global standard-setting instrument for ethical AI (November 2021). Core values and principles. 11 policy action areas: ethical impact assessment, data governance, environment, gender equality, education, health, culture, labor.
- Cross-Framework Mapping
- Risk management: EU AI Act Art 9 / NIST MAP+MEASURE / ISO 42001 A.5. Data governance: Art 10 / MAP 2.x / A.7. Transparency: Art 50 / GOVERN+MAP / A.8. Human oversight: Art 14 / GOVERN+MANAGE / A.9.
- Framework Integration Approach
- Real-world organizations apply all three simultaneously: EU AI Act establishes legal requirements, NIST AI RMF provides the operational model, ISO 42001 creates certifiable evidence structure. Exam tests translation between them.
Privacy-Enhancing Technologies
- Differential Privacy (DP)
- Adds calibrated noise (Gaussian/Laplacian) to gradients or outputs so information learned from any individual record is bounded by a privacy budget (epsilon). Mathematically provable privacy guarantee.
- Federated Learning (FL)
- Collaborative model training without sharing raw data. Edge devices train locally and share only model updates (gradients). Data never leaves the local device. Used by Google Keyboard, Apple.
- Homomorphic Encryption (HE)
- Computations performed on encrypted data without decryption. Results, when decrypted, match results of operations on plaintext. Very computationally expensive. Enables privacy-preserving inference.
- Secure Multi-Party Computation (SMPC)
- Multiple parties jointly compute a function over their inputs while keeping individual inputs private. No single party sees the others' data. Used for collaborative analytics.
- Trusted Execution Environments (TEEs)
- Hardware-based isolated computation environments (enclaves) that protect data during processing. Examples: Intel SGX, ARM TrustZone. Provides confidential computing for AI inference.
- Synthetic Data Generation
- Creating artificial data that preserves statistical properties of real data without containing actual personal data. Reduces privacy risk while maintaining model utility. Must validate against re-identification.
- Anonymization vs. Pseudonymization
- Anonymization: irreversible removal of identifiers, data no longer personal data under GDPR. Pseudonymization: replacing identifiers with tokens, reversible with additional info, still personal data under GDPR.
- FL + DP Combination
- Federated learning provides decentralized architecture; differential privacy adds noise to prevent inference attacks on model updates. Combined approach addresses both data locality and gradient leakage risks.
Bias & Fairness
- Demographic Parity (Statistical Parity)
- Probability of positive outcome must be similar across sensitive groups. P(Y=1|A=0) = P(Y=1|A=1). Does not account for base rates or qualifications — can conflict with accuracy.
- Equalized Odds
- True positive rate AND false positive rate must be equal across groups. Conditions on actual outcome. Stricter than equal opportunity. P(Y_hat=1|Y=y,A=a) is same for all a.
- Equal Opportunity
- True positive rates must be equal across groups (relaxed version of equalized odds). Only requires equal TPR, not equal FPR. Focuses on not disadvantaging qualified individuals.
- Disparate Impact (80% / Four-Fifths Rule)
- Ratio of positive prediction rates between groups must be at least 80%. Selection rate of protected group / selection rate of majority group >= 0.8. Below this threshold indicates potential discrimination.
- Individual Fairness
- Similar individuals should be treated similarly. Requires defining a meaningful similarity metric. Focuses on individual treatment rather than group-level statistics.
- Calibration
- Among those predicted positive at a given probability, the actual positive rate should be the same across groups. A model that says '70% likely' should be right 70% of the time for all groups.
- Impossibility Theorem
- Critical exam concept: fairness definitions can be mathematically incompatible — you cannot satisfy demographic parity, equalized odds, and calibration simultaneously (except in trivial cases). Must choose which fairness criterion is appropriate for context.
- Types of Bias in AI
- Historical bias (societal inequities in data), representation bias (underrepresentation of groups), measurement bias (flawed proxy variables), aggregation bias (single model for diverse populations), evaluation bias (unrepresentative test sets).
- Bias Mitigation Strategies
- Pre-processing: rebalancing data, removing proxies. In-processing: fairness constraints during training. Post-processing: adjusting outputs to satisfy fairness criteria. Each stage has trade-offs with accuracy.
Explainability & Interpretability
- Explainability
- How the model arrived at a SPECIFIC result. Providing reasoning behind individual decisions. Post-hoc explanations for complex models. Answers 'why did the model decide X for this input?'
- Interpretability
- Making the overall AI process understandable. How the model makes decisions IN GENERAL. Inherent property of simpler models (decision trees, linear regression). Answers 'how does the model work?'
- Explainability vs. Interpretability vs. Transparency
- Three distinct concepts: Explainability = WHY a specific decision was made (post-hoc). Interpretability = HOW the model works in general (inherent). Transparency = HOW the model was created (documentation). The exam tests scenario-based distinction between all three.
- SHAP (SHapley Additive exPlanations)
- Game-theory based approach assigning each feature an importance value for a particular prediction. Based on Shapley values from cooperative game theory. Provides consistent, locally accurate explanations.
- LIME (Local Interpretable Model-agnostic Explanations)
- Creates a simple interpretable model (e.g., linear) that approximates the complex model locally around a specific prediction. Model-agnostic: works with any black-box model.
- Attention Visualization
- Technique for understanding transformer model behavior by visualizing which parts of the input the model focuses on. Useful for debugging and understanding LLM reasoning patterns.
- Feature Importance
- Ranking input features by their contribution to model predictions. Global importance shows overall model behavior. Local importance shows feature contribution for a specific prediction. Methods include permutation importance and SHAP values.
- Black Box vs. Interpretable Models
- Black box: complex models (deep neural networks) whose internal reasoning is opaque. Interpretable: simpler models (decision trees, linear regression) whose logic is directly understandable. Trade-off with performance.
Data Governance for AI
- Data Quality Standards
- Training data must meet quality standards: accuracy, completeness, consistency, timeliness, and relevance. EU AI Act Article 10 requires high-risk systems to use high-quality training data.
- Data Provenance
- Recording the complete history of data — origin, custody chain, and all transformations applied. Essential for AI governance, reproducibility, and regulatory compliance. More comprehensive than lineage.
- Data Lineage
- Tracking data's origin, movement, and transformation through the pipeline. Documents where data came from and how it was modified. Subset of provenance focused on transformation chain.
- Data Minimization
- Collecting and retaining only the minimum data necessary for the AI task. GDPR principle (Article 5) that directly applies to AI training data collection and retention.
- Ground Truth
- The verified correct labels or outcomes used to train and evaluate supervised learning models. Quality of ground truth directly impacts model accuracy and fairness.
- Train/Test Split
- Dividing data into training set (model learns from) and test set (model evaluated on). Prevents overfitting. Common splits: 80/20 or 70/15/15 (train/validation/test).
- Cross-Validation
- Technique to assess model generalization by splitting data into k folds, training on k-1 folds, testing on remaining fold, and rotating. Reduces evaluation variance from a single split.
- Overfitting vs. Underfitting
- Overfitting: model memorizes training data, performs poorly on new data (high variance). Underfitting: model too simple to capture patterns (high bias). Both indicate poor generalization.
- Feature Engineering
- Process of creating, selecting, and transforming input variables (features) to improve model performance. Governance concern: features may be proxies for protected characteristics.
AI Security Threats
- Adversarial Attacks (Evasion)
- Crafting inputs with subtle perturbations imperceptible to humans that cause model misclassification. Applied at inference time. Example: modifying pixels to make a stop sign classified as speed limit.
- Data Poisoning
- Injecting malicious or manipulated data into training datasets. Causes model to learn incorrect patterns, produce biased outputs, or create hidden backdoors that activate on trigger inputs.
- Model Poisoning
- Directly tampering with model weights, architecture, or training process. Can introduce trojans/backdoors. Especially dangerous in federated learning where malicious participants can corrupt shared model.
- Prompt Injection
- Direct: malicious user input overrides system instructions. Indirect: hidden instructions in retrieved data (web pages, documents, emails) manipulate model behavior without user's knowledge.
- Jailbreaking
- Techniques to bypass model safety guardrails and content policies. Methods: role-playing, encoding tricks, hypothetical framing, multi-turn manipulation. Distinct from prompt injection.
- Membership Inference
- Determining whether a specific data record was used in model training. Privacy violation revealing individuals' presence in sensitive datasets. Mitigated by differential privacy.
- Model Extraction (Theft)
- Stealing a proprietary model by systematically querying it and using responses to train a clone. Violates IP, enables offline adversarial attacks. Mitigated by rate limiting and watermarking.
- Model Inversion
- Reconstructing training data from model outputs. Attacker queries model repeatedly to reverse-engineer sensitive data. Privacy threat especially for models trained on personal data.
- Supply Chain Attacks
- Compromising AI components (pre-trained models, libraries, datasets, dependencies) before they reach the target organization. v2.1 emphasizes third-party risk management for AI supply chain.
Testing & Validation
- Red Teaming
- Adversarial testing where a team actively tries to make the AI system fail, produce harmful outputs, or exhibit unintended behavior. Tests safety boundaries and robustness before deployment.
- A/B Testing
- Comparing two versions of an AI system with real users to measure differences in performance, fairness, or user experience. Used for deployment decisions and continuous improvement.
- Shadow Deployment
- Running new AI model in parallel with production model without affecting end users. New model receives real inputs but outputs are only logged and compared, not served.
- Stress Testing
- Testing AI systems under extreme conditions: edge cases, adversarial inputs, high load, unusual data distributions. Identifies failure modes and robustness limits.
- Fairness Testing
- Evaluating model outputs across demographic groups for discriminatory patterns. Tests disparate impact, equal opportunity, and calibration metrics before deployment.
- Pre-Deployment Testing (NIST 600-1)
- One of four primary considerations in NIST GenAI Profile. Structured testing before release including safety evaluations, bias assessments, and robustness checks.
- Accuracy Metrics
- Precision (correct positive predictions / total positive predictions), Recall (correct positive predictions / actual positives), F1 Score (harmonic mean of precision and recall), AUC-ROC.
- Confusion Matrix
- Table showing True Positives, True Negatives, False Positives (Type I error), False Negatives (Type II error). Foundation for calculating accuracy, precision, recall, and fairness metrics.
Third-Party & Vendor Risk
- Third-Party AI Due Diligence
- v2.1 expanded: evaluating vendor AI systems for compliance, security, fairness, and governance before adoption. Includes reviewing documentation, testing, and assessing vendor governance maturity.
- Vendor AI Assessment
- Structured evaluation of AI vendor capabilities: data handling practices, model documentation, bias testing results, security controls, incident response processes, and regulatory compliance.
- Contract Requirements for AI
- v2.1 added contract management: must include AI-specific clauses for data usage, model updates, performance SLAs, audit rights, liability allocation, IP ownership, and incident notification.
- Audit Rights
- Contractual right to inspect vendor AI systems, documentation, and processes. Essential for verifying compliance claims and maintaining governance over third-party AI components.
- AI Supply Chain Risk
- Risks from third-party models, datasets, libraries, and infrastructure. EU AI Act creates obligations throughout the AI supply chain. Includes foundation model dependencies and data provider risks.
- Model Licensing
- Legal terms governing use of third-party AI models. Includes usage restrictions, modification rights, liability/indemnification, data processing terms, and output ownership.
- IP Policy for AI (v2.1 Update)
- I.C.2 modified: organizations must evaluate and update intellectual property policies to address AI-specific concerns including training data rights, generated output ownership, and model IP.
Human Oversight Models
- Human-in-the-Loop (HITL)
- Active human involvement at EACH decision point. No action taken without human approval. Fits medium-to-high risk situations. Most restrictive model. Timing: BEFORE each action.
- Human-on-the-Loop (HOTL)
- AI runs independently while human supervises via dashboards and alerts. Human intervenes for exceptions and anomalies. System continues when human not actively engaged. Timing: MONITORING during operation.
- Human-in-Command (HIC)
- Humans retain final strategic authority over AI operations. AI provides recommendations but humans are ultimate decision makers. Timing: STRATEGIC oversight and command authority.
- Key Distinction: Timing
- HITL = before each action (approval gate). HOTL = monitoring during operation (exception-based). HIC = strategic authority over system (command level). Exam tests WHEN each model is appropriate.
- EU AI Act Article 14
- Requires human oversight measures for high-risk AI systems. Must be designed to enable effective oversight. Individuals assigned oversight must understand system capabilities and limitations.
- Meaningful Human Oversight
- Oversight must be genuinely meaningful, not merely performative. DPIAs must assess whether human oversight is truly effective. Rubber-stamping automated decisions does not satisfy legal requirements.
AI Roles & Responsibilities
- AI Governance Officer
- Develops and manages ethical AI policies. Oversees the AI governance program. Ensures organizational compliance with AI regulations and internal standards.
- Responsible AI Program Manager
- Leads responsible AI deployment across the organization. Coordinates cross-functional teams. Manages AI ethics initiatives and governance program implementation.
- AI Risk Analyst
- Identifies and mitigates AI-related risks including technical, ethical, compliance, and business risks. Conducts risk assessments and monitors risk indicators.
- AI Ethics Board/Committee
- Cross-functional advisory body that reviews AI use cases, provides ethical guidance, resolves ethical dilemmas, and establishes organizational AI ethics policies.
- Data Protection Officer (DPO)
- GDPR-mandated role for organizations processing personal data at scale. Advises on data protection obligations, monitors compliance, and serves as contact for supervisory authorities. Key role when AI processes personal data.
- Provider (EU AI Act Role)
- Organization that develops AI or has AI developed and places it on the market or puts it into service under its own name/trademark. Bears most extensive regulatory obligations.
- Deployer (EU AI Act Role)
- Organization using an AI system under its own authority (except for personal non-professional use). Operational obligations: use as intended, ensure oversight, monitor, conduct FRIAs.
- Cross-Functional Collaboration
- AI governance requires collaboration across legal, compliance, engineering, data science, business, and ethics functions. No single role can govern AI in isolation.
Post-Deployment Monitoring
- Data Drift
- Shifts in feature or label distributions over time compared to training data. Input data changes but the underlying relationship may remain the same. Detectable by monitoring input statistics.
- Concept Drift
- Changes in the input-output relationship due to external influences (e.g., changing consumer behavior, new regulations). Harder to detect because input distribution may look unchanged while the mapping shifts.
- Model Drift
- Overall degradation in model performance over time. Umbrella term encompassing effects of data drift, concept drift, and other factors causing declining accuracy or fairness.
- Retraining Triggers
- Threshold-based (performance drops below KPI), time-based (scheduled periodic retraining), data volume-based (sufficient new data accumulated). Must define triggers before deployment.
- KPI & Fairness Metric Monitoring
- Continuous tracking of accuracy, precision, recall, and fairness metrics (demographic parity, equalized odds) in production. Alerts when metrics deviate beyond defined thresholds.
- Incident Response
- Protocols for AI system failures, biased outputs, security breaches, or safety incidents. Includes stakeholder notification requirements, regulatory reporting, escalation paths, and root cause analysis.
- Serious Incident Reporting (EU AI Act)
- Providers of high-risk AI must report serious incidents (death, serious harm, fundamental rights violations, environmental damage) to market surveillance authorities. Prompt notification required.
- Continuous Improvement
- Ongoing cycle of monitoring, evaluating, updating, and improving AI systems. Cross-functional collaboration during live operation. Feedback loops from deployment inform future development.
Additional AI Laws & Regulations
- South Korea AI Basic Law (NEW v2.1)
- Effective January 2026. Second comprehensive AI law globally (after EU AI Act). Transparency/disclosure obligations, human-in-the-loop for high-impact AI, fundamental rights impact assessments, fines up to KRW 30 million. Applies to foreign companies.
- Colorado AI Act — Repealed
- Original SB 24-205 targeted high-risk AI algorithmic discrimination. REPEALED May 2026. Replaced with narrower transparency law SB 26-189, effective January 2027. Exam may test the transition.
- Texas TRAIGA
- Signed June 2025. Narrowed scope to mostly government AI use. Prohibits AI for: self-harm encouragement, unlawful discrimination, constitutional rights infringement, CSAM generation.
- US State AI Law Landscape
- Growing patchwork of state laws addressing algorithmic discrimination, transparency, and sector-specific AI use. No comprehensive federal AI law. Creates compliance complexity for multi-state organizations.
- AI-Generated Content & Copyright (US)
- Purely AI-generated content generally NOT eligible for copyright protection in US. Human authorship requirement. AI-assisted works may qualify if human creative control is sufficient.
- Fair Use & Training Data
- Whether using copyrighted works to train AI models constitutes fair use remains legally contested. Factors: purpose, nature of work, amount used, market effect. Active litigation ongoing.
- Nondiscrimination Law Applied to AI
- Existing anti-discrimination laws (Title VII, ECOA, FHA in US; Equality Act in UK) apply to AI-driven decisions in employment, lending, housing. AI does not create exemption from existing civil rights obligations.
Key Formulas & Metrics
- Disparate Impact Ratio
- Selection rate of protected group / selection rate of majority group. Must be >= 0.8 (80%) to avoid presumption of discrimination. Also called the four-fifths rule. Below 0.8 triggers further investigation.
- Demographic Parity Formula
- P(Y_hat=1 | A=0) = P(Y_hat=1 | A=1). The probability of a positive prediction should be equal across sensitive attribute groups A. Does not condition on actual outcomes.
- Equalized Odds Formula
- P(Y_hat=1 | Y=y, A=0) = P(Y_hat=1 | Y=y, A=1) for all y in {0,1}. Both true positive rate AND false positive rate must be equal across groups.
- Precision
- True Positives / (True Positives + False Positives). Of all positive predictions, what fraction was correct? Important when cost of false positives is high.
- Recall (Sensitivity / TPR)
- True Positives / (True Positives + False Negatives). Of all actual positives, what fraction did the model catch? Important when cost of false negatives is high.
- F1 Score
- 2 * (Precision * Recall) / (Precision + Recall). Harmonic mean of precision and recall. Balanced metric when both false positives and false negatives matter.
- GPAI Systemic Risk Threshold
- GPAI models trained with total compute >10^25 FLOPs are presumed to have systemic risk. AI Office can also designate models based on other criteria (capabilities, reach, impact).
- EU AI Act Fine Tiers (Summary)
- Prohibited practices: EUR 35M or 7% turnover. High-risk/GPAI violations: EUR 15M or 3% turnover. False information: EUR 7.5M or 1% turnover. Always whichever is higher.
Agentic AI Governance
- Agentic AI (NEW v2.1)
- AI systems that can autonomously plan, use tools, and execute multi-step tasks. v2.1 added agentic architectures as a deployment option (IV.A.3). Raises unique governance challenges around autonomy and control.
- Autonomy Risk
- AI agents making decisions and taking actions with reduced human oversight. Risk increases with scope of permissions, tool access, and multi-step reasoning chains. Governance must define boundaries.
- Excessive Agency
- AI systems granted too many permissions, capabilities, or autonomy. Agent performs unintended actions through tools or APIs with overly broad access. Mitigated by least privilege principle.
- Privilege Escalation in AI Agents
- AI agent exploiting tool integrations or permissions to gain access beyond intended scope. Risk when agents interact with external systems, APIs, or other agents.
- Multi-Agent Systems
- Multiple AI agents interacting and collaborating. Governance challenges: accountability attribution, emergent behaviors, cascading failures, and ensuring consistent governance across agent interactions.
- RAG Architecture Governance
- Retrieval-Augmented Generation introduces governance requirements for: retrieval source quality, data freshness, relevance filtering, and preventing indirect prompt injection through retrieved documents.
- Least Privilege for AI Agents
- Grant AI agents only minimum permissions needed for their function. Limit tool access, API scope, and autonomous action authority. Prevents excessive agency and limits blast radius of failures.
AI Documentation Requirements
- Model Cards
- Document: model purpose, training data, evaluation metrics, performance across demographics, limitations, intended/out-of-scope uses, ethical considerations. Created during development, updated throughout lifecycle.
- Datasheets for Datasets
- Document: motivation, composition, collection process, preprocessing, intended uses, distribution, maintenance plan, biases, limitations. Created when dataset is finalized.
- AI Impact Assessment
- Structured methodology for identifying consequences of AI deployment on individuals, groups, and society. Microsoft RAI Template is a key reference. Required before deployment of high-risk systems.
- Technical Documentation (EU AI Act Art 11)
- Provider obligation for high-risk AI: system description, design specifications, development methodology, data governance practices, testing procedures, risk management measures. Must be maintained and kept up to date.
- Audit Trails
- Comprehensive logs of AI system decisions, inputs, outputs, and governance actions. Supports accountability, incident investigation, regulatory compliance, and continuous improvement.
- Go/No-Go Documentation
- Records of deployment readiness decisions: governance conditions satisfied, risk assessment completed, documentation complete, approval sign-offs obtained, halt conditions identified.
- Record-Keeping (EU AI Act Art 12)
- High-risk AI systems must include automatic logging capabilities. Logs must be retained for appropriate period, enable monitoring of system operation, and facilitate post-market surveillance.
- Content Provenance (NIST 600-1)
- One of four primary considerations in GenAI Profile. Tracking origin and history of AI-generated content. Includes watermarking, metadata, and provenance attestations for generated outputs.