How to Actually Assess AI Risk Before You Roll Out a New Tool

If your business has ended up with AI tools in more than one department, each chosen independently with no shared way to check them first, you are not managing this badly. Most small and mid-sized businesses arrive here the same way: someone found a tool that solved an immediate problem, nobody paused to ask what happens to the data behind it, and now there is no consistent answer to whether any of it is actually safe to use. This guide sets out a repeatable way to assess a new AI tool's real risk before it spreads further through your business, not a one-off exercise you run once and file away.

Why 'We'll Sort It Out Later' Stops Working Once You've Got Four Tools

Take an operations lead at a roughly 30-person business, responsible for keeping day-to-day operations running across several teams. Over the past year, four separate departments each adopted their own AI tool: one for customer support replies, one for scheduling, one for drafting proposals, and one for transcribing client calls. Nobody signed off on any of them in advance, because each felt like a small, low-stakes decision made by the team using it.

Then a client complained that a call transcript containing personal financial details had somehow become visible outside the business. It turned out the transcription tool stored recordings on a server with no real access controls, and nobody had ever checked. The tool itself did its actual job well. Nobody had asked the one question that mattered most: where does the data go, and who else can see it.

That is the outcome this operations lead now wants to avoid repeating. Before: four tools live in the business, no consistent screening, and risk discovered only once something has already gone wrong. After: any new tool, regardless of which department wants it, goes through the same five-question check before anyone relies on it for real client or business information. The point is not to slow every team down with paperwork. It is to catch the one tool in five that has a real problem, before a client has to point it out.

The Plain-English Answer: What a Real Risk Assessment Actually Checks

A genuine AI risk assessment is not a compliance form you fill in once and forget. It is five specific questions, asked about any tool before your team relies on it for real information, and asked again whenever how the tool gets used changes in a meaningful way.

Those five questions cover where the tool's data actually goes, how much you can trust what it produces, how seriously the vendor takes security, who is accountable when it gets something wrong, and whether it treats different types of people or cases consistently. None of them require a legal background or a dedicated security team to answer. They require asking the vendor directly, reading what they publish, and being honest about what you do not know yet.

Data Privacy: Where Does the Information Actually Go

Every AI tool that touches customer or staff information is moving that data somewhere, and the first job of a risk assessment is finding out exactly where. Ask the vendor directly: where is data stored, how long is it kept, is it used to train the vendor's models, and does it get shared with any third party.

A vendor that cannot answer these questions clearly, or answers with vague marketing language instead of specifics, is telling you something important on its own. The businesses that get caught out are rarely the ones that asked and got a bad answer. They are the ones that never asked at all.

Pay particular attention to any tool handling information that would be genuinely damaging if it became public: financial details, health information, or anything a client would consider private. That is exactly the kind of data involved in the transcription tool example above, and exactly why this question comes first.

Accuracy and Reliability: Can You Trust What It Produces

AI tools that generate or summarise content can produce confident, well-formatted answers that are simply wrong. This matters more in some tasks than others. A tool drafting an internal meeting summary carries low stakes if it gets a detail wrong. A tool drafting a client-facing quote, a medical note, or a legal summary carries much higher stakes.

Before relying on a tool's output, test it against situations where you already know the correct answer, and see how often it gets close and how often it gets things confidently wrong. Build in a human review step for anything that reaches a client or affects a real decision, and be honest with your team about which tasks still need that check and which do not.

Security: Who Else Can Get In

A tool can have excellent privacy terms on paper and still be insecure in practice, which is exactly what happened with the transcription tool in the earlier example. Ask about encryption, both in transit and at rest, who inside the vendor's own organisation can access customer data, and what access controls exist on their end.

Also ask what happens if something goes wrong: does the vendor have a documented incident response process, will they notify you if your data is affected by a breach, and how quickly. A vendor with no clear answer to this last question is a genuine warning sign, not a minor gap.

Accountability: Who Owns a Mistake

When an AI tool gets something wrong, whether that is a factual error in a client email or a scheduling mistake that costs a booking, someone in your business needs to already know whose job it is to catch and fix it. Deciding this after the mistake has happened is too late.

Check the vendor's own terms for what they take responsibility for, and what they explicitly do not. Most vendor contracts place the responsibility for reviewing and approving output squarely on the business using the tool, not on the vendor. Assign an internal owner for each tool in use, someone whose job includes noticing when it starts producing bad output, not just someone who set it up once.

Bias and Fairness: Does It Treat People Consistently

This dimension matters most for tools involved in decisions about people: screening job applicants, prioritising customer support tickets, or assessing credit or eligibility. An AI tool trained on historical data can quietly repeat old patterns of unequal treatment, even when nobody intended that outcome.

Ask the vendor what testing they have done across different groups of users or applicants, and whether they can show you the results, not just a general assurance. For any tool influencing a decision about a specific person, keep a human in the loop who can review and override the outcome, and periodically check a sample of the tool's decisions for patterns that would not hold up if a customer or employee asked you to explain them.

Where International Frameworks Fit In

You do not need to become a policy expert to run this assessment, but it helps to know that the five questions above map closely to frameworks regulators and standards bodies already publish. Treat these as reference points, not requirements to memorise.

The US National Institute of Standards and Technology has published a voluntary AI Risk Management Framework, organised around four functions: govern, map, measure, and manage. It is widely referenced by organisations building a formal AI risk process, even outside the United States.

ISO/IEC 42001, published in 2023, is the first international standard specifically for AI management systems. It sets out what a formal AI governance program looks like structurally, and businesses that want a recognised standard to build toward can review the ISO 42001 standard page directly.

In the European Union, the EU AI Act sets binding, risk-based obligations for AI systems, with the strictest requirements applying to tools classified as high-risk, such as those used in hiring, credit, or law enforcement contexts. The European Commission's own regulatory framework page explains which categories apply where. In the United States, the FTC has stated it will apply its existing consumer protection powers to AI-related claims and practices it considers deceptive or unfair, detailed in its own published guidance.

Turning Five Answers Into One Decision

Once you have answers across all five dimensions, the useful next step is turning them into a single decision rather than five separate impressions. A simple approach: score each dimension green, amber, or red, based on whether the vendor's answer was clear and reassuring, unclear, or genuinely concerning.

All green or mostly green with no red flags generally means proceed. One or two amber scores usually means proceed with a specific condition attached, such as a mandatory human review step, or restricting the tool to lower-stakes tasks until you can get a clearer answer. Any red score, particularly on security or data privacy, is worth pausing on until it is resolved, regardless of how useful the tool is for its actual task.

Build in a trigger to redo the assessment, not just run it once: a tool being used for a new type of data, a new use case, or a change to the vendor's own terms of service are all good reasons to run the five questions again.

Making This Repeatable, Not a One-Off Audit

The businesses that avoid the kind of surprise described earlier are not the ones with the most sophisticated process. They are the ones that actually run the same simple check every time, rather than relying on whoever happens to be paying attention that month.

A workable version of this needs three things: a short shared document or template covering the five questions above, one named person responsible for running it whenever a new tool is proposed, and an agreed rule that no new AI tool touches real client or business data until it has gone through the check, no matter how small or low-stakes it initially seems.

Before any new AI tool moves from a trial into regular use, confirm the following:

• You know exactly where the tool stores and processes data, and for how long
• You have tested its output against situations where you already know the correct answer
• You know what security measures the vendor actually has in place, not just what their marketing page claims
• Someone specific is responsible for catching and fixing its mistakes
• You have checked whether it could affect different people or cases unevenly, if it is involved in any kind of decision about people
• The assessment is dated, so you know when it is due for another look

Related reading: our AI governance by region.

Free tools: AI Privacy Risk Scorer to score your current AI tool setup against data-privacy best practice | AI Compliance Checker to check whether your AI tools meet your compliance obligations.

How long should a basic AI risk assessment actually take?

For most everyday business tools, working through the five questions properly takes somewhere between thirty minutes and an hour, mostly spent reading what the vendor publishes and sending a couple of direct questions if the answers are not already clear. Tools that will handle sensitive client data or influence decisions about people are worth spending more time on.

What if a vendor won't answer basic security or data questions?

Treat a vendor's refusal or vague non-answer as information in itself. A legitimate vendor serving business customers should be able to explain, in plain terms, where data goes and how it is secured. If they cannot or will not, that is a reasonable basis to hold off, regardless of how good the tool seems otherwise.

Do we need to reassess AI tools we are already using?

Yes, particularly if nobody ran this kind of check when the tool was first adopted. Start with whichever tools handle the most sensitive information or influence the most consequential decisions, and work through the rest over time rather than trying to reassess everything at once.

Is a shared document enough, or do we need dedicated risk assessment software?

A simple shared document or spreadsheet is enough for most small and mid-sized businesses, as long as someone is actually responsible for using it consistently. Dedicated software becomes worth considering once you are assessing AI tools regularly enough that tracking them manually starts to slip.

Does every new AI tool need the full five-part assessment, even small, low-stakes ones?

A genuinely low-stakes tool, one that never touches client data and does not influence any decision about a person, can usually get a lighter version focused mainly on data privacy and security. The full five-part check matters most once a tool touches real client information or plays a role in decisions that affect people.