Compliance Gap Analysis: Why Search Can't Find What's Missing

Key Takeaways

• If your gap analysis relies on search, it will find the policies you have and miss the ones you don't.
• Compliance gap analysis gets faster when it starts from each obligation and records a verdict for every one, including "not covered."
• A chat assistant can compare one rule with one policy; firm-wide work also needs approval status, effective dates, and a search log.
• Before you buy an AI gap analysis tool, ask to see a "not covered" verdict on your own documents.
  

Compliance Gap Analysis: Why Search Can't Find What's Missing

An overlooked compliance gap rarely stays a single finding. Once a supervisor spots one, you're no longer explaining one policy. Suddenly you're expected to explain your whole control framework: how the gap was missed, how long it stayed open, and what else the same process might have let through.

In the UK, the regulator can require an independent skilled-person review under section 166 of the Financial Services and Markets Act 2000, and the firm usually pays for it. Supervisors in other jurisdictions have similar powers. Insurers and reinsurers are in the same position, since Article 41 of the Solvency II Directive requires written governance policies that are reviewed at least once a year.

What makes it all the more frustrating is that the regulation was probably already on file. Finding the rules your firm is subject to is quick and easy. What takes weeks is proving where your own policies fall short. The reason is technical: search tools return what matches your query, and a compliance gap fails precisely because it has nothing to match.

What Does a Compliance Gap Analysis Have to Prove?

A compliance gap analysis compares what a regulation requires with what your approved policies actually say and records where they fall short. For each obligation, you need three things: the requirement, cited to its article; your firm's position, cited to a named policy; and the distance between the two. Sometimes that distance is total, because none of your policies address the obligation.

Supervisors care about how you got there, too. If your gap analysis ends in actions as vague as reviewing policies to make sure they reflect the new rules, it tells them you haven't found your gaps yet. What they expect to see is the work itself: which obligation, which policy, what's missing, and who's fixing it.

Why Is Compliance Gap Analysis So Slow?

Finding the regulation is quick. Proving coverage is slow, because every single obligation morphs into a research project. You locate each policy that might address it, check that the version in front of you is approved and current, read the relevant section, decide whether it covers the requirement fully, partly, or not at all, and then write that judgment down with a reference a colleague or a supervisor can follow.

Now multiply that by every article in every regulation in scope. Then by every jurisdiction you operate in.

And the ground keeps shifting. CUBE's Cost of Compliance Report 2025 surveyed more than 2,000 senior compliance and risk officers at 1,300 financial institutions and found that firms still take more than a year, on average, to fully implement a regulatory change. Sixty percent expected the cost of managing regulatory change to go up over the following 12 months.

It doesn’t stop there. Draft and approved policies often live in the same folders with near-identical titles, so it's easy to lose an afternoon on a version nobody signed off on. And rules come with effective dates, which means a policy that was fine last year can fall short the day a transition period ends.

So you end up with two options. Read everything yourselves, or pay outside advisors to read it for you.

Why Can't Search Find What's Missing?

Surely search can speed up compliance gap analysis, right? Only partly. Search ranks documents by how well they match what you asked, and it always hands you the best match it has. A policy that doesn't exist has no text, so it can't show up in the results. What shows up instead is the closest thing in the index, and a close match looks a lot like coverage.

It's the same whether you're using keyword search or AI. Keyword search scores documents by the words they share with your query. Semantic search, the kind behind most AI assistants, turns your question and every passage into vectors – numerical representations of meaning – and then returns the passages that sit closest. Different ways of measuring closeness, but they fail for the same reason: as long as the library holds anything at all, there's always a top result.

Say you ask which policy covers a new requirement on handling complaints from vulnerable customers. The search returns your complaints-handling policy, which mentions vulnerable customers exactly once, in the definitions section. It's a relevant result. It also says nothing about the actual requirement, and nothing on the screen warns you. You only find out by opening the document and reading it.

That's why search speeds up the first half of gap analysis and barely touches the second. A search engine has no way to return a document that was never written.

What Changes When You Start from the Obligation?

Now flip the direction of the search. Start with the regulation, break it into individual obligations, and go looking for evidence of each one in your policy library. Every obligation then ends up with a recorded verdict, including the ones nothing covers.

In practice, that's four steps:

1. Extract the obligations. Pull each obligation out of the source text and tie it to its article.
2. Look for evidence. For each one, search your approved policies for passages that address it.
3. Record a verdict. Covered, partly covered, or not covered, each citing the passages it relied on. If no internal policy has been mapped to a regulation yet, it reads "not assessed," which is a very different thing from zero gaps.
4. Turn gaps into actions. Each gap gets remediation actions with an owner and a status. When the regulation or the policy changes, the affected obligations get assessed again.

Step 3 is where the missing policy finally becomes visible. Search may not have had anything to show you, but a verdict list has one line per obligation, so the gap appears as a line that says "not covered." We saw the same thing with bid teams in our post on governed answer libraries: the questions nobody has answered yet often turn out to be the most useful output.

AI earns its keep in steps 1 and 2, where the work is close reading of long regulatory texts and pulling candidate passages from thousands of documents. In Squirro's approach, grounded retrieval over your indexed library is the foundation, so every verdict points back to a passage you can open and check. If your firm already maintains a taxonomy or knowledge graph, Squirro Graphite can import it in standard formats such as SKOS, OWL, and RDF and use it to classify obligations more precisely. Helpful if you have one. You don't need one to get started.

Can ChatGPT Do a Gap Analysis?

For one rule and one policy pasted into the chat, yes, a general-purpose assistant like ChatGPT can give you a decent first-pass comparison. For a firm-wide compliance gap analysis, it hits limits that have little to do with how smart the model is.

It only sees what you paste in. It can't tell which version of a policy is approved, and it doesn't track effective dates. And when a supervisor asks how you reached a conclusion, a chat history on one analyst's laptop isn't much of an answer.

To be fair, that first-pass use is perfectly legitimate, and for a small firm with a handful of policies it might be all you need. At the scale of a bank or an insurance group, though, the chat window is just the way in. What makes the answer defensible sits underneath: an indexed library of approved documents, retrieval that respects who's allowed to see what, and a record of every search.

Five Questions to Ask Any AI Tool That Claims to Do Gap Analysis

Copy these into your next vendor conversation, ours included. And ask to see every answer demonstrated on your own documents.

1. Can it tell you when no policy covers an obligation, or does it only return matches?
2. Does it tell draft policies apart from approved ones?
3. Does it track effective dates and changes to the rules?
4. Does every finding cite the exact source document?
5. Is every search logged so you can show a supervisor how you reached the conclusion?

Question 1 is the one that separates gap analysis from search. If a tool can't show you a "not covered" verdict on your own library, it's a search tool, however polished its answers read.

Questions 2 and 3 tell you whether a verdict will still be true next quarter. Questions 4 and 5 tell you whether you can defend it: a citation lets a reviewer check a finding in seconds, and a search log lets you walk a supervisor through the steps behind it.

Who Signs Off on a Gap?

You do. An AI agent can extract obligations, gather evidence, and propose verdicts far faster than a team reading by hand, but a proposed gap only becomes a finding once an analyst has reviewed it. The tool's job is to make that review quick and to leave a trace.

So every verdict cites the passages behind it, and the reviewer can check the source directly. Every gap turns into remediation actions with a named assignee and a status, and the register updates as those actions are resolved. Every completed search is stored with its answer, the documents it cited, and the retrieval steps it took.

That audit trail can respect privacy, too. Searches can be attributed to a role and a pseudonym, with no individual named and the attribution deleted on a set schedule, in line with the purpose limitation and data minimization principles in Article 5 of the GDPR. Our expense policy agent runs the same audit pattern for finance teams.

The Regulatory Document Search Agent in Practice

Squirro's Regulatory Document Search Agent is built on this method. Our demo runs it against the regulations and internal policies of a sample financial group, covering twelve regulatory domains and six jurisdictions.

An analyst asks the question search can't answer: does this regulation apply to us today, and where are we short? The agent comes back with a short answer and the group's current position, then lists each obligation with its article, footnoted to the source passages. A Gaps tab lists each documented limitation, quoted directly from the group's own policy. A third view shows every corpus search and reasoning step the agent took, with how long each one took. That's what a supervisor sees when they ask, "Why did it say that?"

Behind the answers sits a register. In the demo, one regulation yields thirteen extracted obligations and thirteen recorded verdicts. Each gap names the policy assessed, its severity, and how many of its remediation actions are done. The register is built from records written when documents are indexed, so everyone sees the same numbers, and running a new search doesn't change them.

When you upload a new regulation, the agent stages and indexes it, extracts its obligations, and assesses them against your existing policies. Upload a revised policy and it re-assesses the obligations that policy is evidence for. Documents where only a cover page survived text extraction are marked "reference only" and left out of the analysis, so a thin document can't quietly pass as coverage.

A couple of things it doesn't do. There's no side-by-side view of how different regulators treat the same obligation, since jurisdiction works as a filter on retrieval. Audit trail exports are CSV files. And every verdict still goes to an analyst for sign-off.

What You Can Show When the Supervisor Asks

Think back to the supervisor from the start of this piece, the one asking how a gap was missed, how long it stayed open, and what else the process might have let through. Those questions are hard to answer when the evidence lives in inboxes, shared drives, and one analyst's memory. With a verdict for every obligation, a named owner for every gap, and a log of every search, each of them has an answer you can put on the table: when the obligation was assessed, which policy it was checked against, and the steps behind the conclusion.

You'll still find gaps. With this method, you're far more likely to find them before a supervisor does, and you'll have the record to show how.

Stop reading every policy to find what isn't there. See how the Squirro Regulatory Document Search Agent checks your policies against each obligation and flags the gaps, with every answer cited and logged.