Categories of AI risk, and risk assessment and calculation
An AI system can go wrong in ways that have nothing to do with each other. A cost problem, a discrimination problem and a security breach need different responses and land on different teams, so the first job is telling them apart. Sorting a problem into
Why this matters for the exam
An AI system can go wrong in ways that have nothing to do with each other. A cost problem, a discrimination problem and a security breach need different responses and land on different teams, so the first job is telling them apart. Sorting a problem into the wrong category sends it to the wrong team, and it stays unfixed while everyone assumes someone else has it.
What you need to know
Risk management is the second of the nine policy areas from two topics back, and it is the one with the most machinery behind it. This topic covers that machinery.
Five categories of AI risk
The course names five categories. They are not degrees of the same risk. Each has a different owner and a different remedy.
| Category | What it covers |
|---|---|
| Operational | The cost of actually running the algorithm: specialized hardware such as CPUs and GPUs, storage for large training sets, high-speed networking, and skilled staff, since data scientists command high salaries and need ongoing training. Environmental costs sit here too, both AI's carbon footprint and the cost of running greener operations. So do data corruption and poisoning, which usually trace back to insecure data or weak identity and access management. |
| Legal | Noncompliance with the applicable law, covering IP disputes, human rights violations and reputational damage. Mitigated through governance frameworks, regular legal review, and close collaboration with legal counsel. |
| Security | Internal and external attacks, including malicious manipulation of input data. The AI-specific blind spots are hallucinations, deepfakes, data-training poisoning, data leakage, a false sense of security, and misuse of AI itself. Mitigated by human oversight, regular audits and continual updates. |
| Privacy | Risks to an individual's privacy specifically. Four are named below. |
| Business | Bias and discrimination from bad training data or a poorly tuned algorithm. Job displacement. Vendor dependence, including lock-in, vendor failure or acquisition, and unclear liability. A black-box lack of transparency. IP infringement, since a model trained on scraped internet content can reproduce someone else's protected work. |
Cause separates operational risk from security risk. Weak access management that lets data get corrupted is operational. An attacker exploiting a vulnerability is security. The same corrupted dataset can arrive either way.
Four named privacy risks
- Data persistence — personal data outliving the person who generated it. Good practice is deleting it once the subject is gone, unless there is a valid retention purpose or consent.
- Data repurposing — data used beyond the purpose it was collected for, whether deliberately or because staff were never trained on which purposes are compatible.
- Data spillover — data collected on people who were never the target of collection, such as bystanders captured by a surveillance camera.
- Data derived from the model itself — data the AI generates through its own processing rather than data anyone collected. It raises its own consent, opt-out and deletion problems, because there was no collection moment at which to ask.
Sizing the risk for one system
Assessing risk for a specific system is context-specific. It depends on who owns and operates the system, the industry and use case, the potential social impact, the timing of use, and the jurisdictional controls that apply. There is no universal score. An organization assigns consistent values and measures against accepted standards, which is what allows it to set reasonable controls and to compare one system with another.
A full risk calculation goes beyond a single score. It weighs the business purpose and planned uses, the potential harms including false positives and negatives, and the training data with any sensitive data in it. It also weighs how the system performs, how far it scales, how it benchmarks against established processes, and the third-party risk covered in the next topic.
Risks escape through the gaps between separate risk-management strategies. An organization can either build AI risk consistently into its existing security, privacy and business strategies, or adopt one holistic AI risk-management approach. Either way it has to determine what risk can be mitigated, and who is responsible for the AI's risk and for any failure after implementation.
Writing the consequences down: the impact assessment
An AI system impact assessment, or AIIA, identifies who bears the consequences of a risk, where a risk score only sizes it. An organization that has one still needs the other.
It covers the same privacy, bias, transparency, accountability and security ground as the risk categories above, and adds the broader impacts on economic structures, cultural norms, political stability and the environment. Done early, it can inform a go or no-go decision on the project rather than documenting one already made.
ISO/IEC 42005:2025 is the international standard giving structured guidance for these assessments, and Domain II covers it alongside the other AI standards. What makes it work is a documented, AI-specific assessment procedure, and the standard sets out how that procedure integrates with an organization's wider AI risk management. The failure mode is adding AI line items to an unchanged risk register, because a process built for something else never asks the AI-specific questions.
When risk becomes an incident
Risk management covers the period before anything goes wrong. Incident management covers what happens afterward, and it is a policy area in its own right. Containing the problem is the first step. A defensible process also requires the following:
- Document the root cause. Recording what actually failed, not only what was seen to fail.
- Notify where required. Regulators or affected individuals, within whatever timelines the applicable law sets.
- Feed the lessons back. Into policy, controls, or retraining the model, so the same failure does not recur.
The instinct this guards against is the quiet fix: noticing a model misbehaving, retraining it, and moving on. A quiet fix destroys the evidence of what went wrong and bypasses any notification the organization owed.
Next up: third-party risk, and what an organization can do about a system it cannot see inside.
Remember
- Five categories of AI risk: operational, legal, security, privacy and business. Operational and security are distinguished by cause — weak access management is operational, an active attack is security.
- Four named privacy risks: data persistence, data repurposing, data spillover, and data derived from the model itself.
- Risk assessment is context-specific and produces no universal score, so consistent values and accepted standards are what make scores comparable.
- An impact assessment identifies who bears the consequences of a risk, where a risk score only sizes it. An organization needs both.
- ISO/IEC 42005 calls for a documented, AI-specific assessment procedure that integrates with wider AI risk management, rather than AI line items added to an unchanged risk register.
- Beyond containing an incident, a defensible process documents the root cause, notifies where required, and feeds lessons back into policy, controls or retraining.
Practise this topic
Domain I is free in the app, including its practice questions and flashcards, with progress tracking and no card details.
Previous: Evaluating and updating existing policies for AI
Next: Managing third-party AI risk
Back to the AIGP study guide.
AI Governance Study is an independent study aid. It does not represent a government entity: it is not affiliated with, endorsed by or authorised by any government, government agency or regulatory authority, and it does not provide government services or legal advice. Laws and frameworks are described in our own words — the official texts are listed at official sources. It is also not affiliated with, endorsed by, or sponsored by the IAPP. The AIGP name is used only to identify the exam this material helps you prepare for.