AI Era Internal Controls: COSO Frameworks For AI Compliance

Generative AI has moved from experiment to everyday work faster than most governance programs could keep up. In this edition of The Ethicsverse, Douglas Hileman, President of Douglas Hileman Consulting LLC, and Vicky LeVay, VP of Information Security & Compliance at FloQast, joined Matt Kelly, CEO & Editor of Radical Compliance, and Nick Gallo, Chief Servant & Co-CEO of Ethico, to unpack COSO's guidance on applying its Internal Control–Integrated Framework to generative AI.

Doug, a co-author of COSO's guidance on internal controls over sustainability reporting and a former member of the Volkswagen monitor team, explained where COSO came from and why its framework has become the foundation for so many others. Vicky brought the view from the financial close, where public-company accountants are using AI under the scrutiny of internal and external auditors every year. Together, the panel explored what the framework actually asks for, which controls are working today, and who inside the organization should own them. Here are the key takeaways.

Key Takeaways

COSO Is the Most Important Organization You've Never Heard Of

COSO grew out of the Treadway Commission, formed after the financial improprieties of the 1970s and 1980s. Its Internal Control–Integrated Framework started as voluntary guidance, then became the de facto standard when Sarbanes-Oxley required robust internal controls over financial reporting after Enron and WorldCom.
Because effective controls depend on knowing what to control, COSO also published its Enterprise Risk Management framework: identify your priority risks first, then control them.
The internal controls framework is built on five components and 17 principles, and it is topic neutral. The good news is that it can be tailored to anything. The bad news is that you have to tailor it.
Matt compared the core framework to a plate of pasta, with topic-specific guidance for sustainability, ethics and compliance, and now generative AI serving as the sauces on top. Doug added that ISO and many other standards can be mapped back to COSO, making it a natural starting point.
Two tenets run through everything: all components and principles must be present and functioning, and design comes before implementation. If the design is wrong, implementation does not matter.

Layers of Imperfect Controls Work Together

Vicky offered the Swiss cheese model. No single control is perfect, and each layer has holes. Stack enough imperfect layers together, though, and no single hole runs all the way through.
Control owners often see only their own slice, which is why controls can feel like theater from where they sit. They do not see the full stack that risk and compliance teams are building.
COSO's generative AI guidance provides the full package of slices, covering risks many organizations are not yet thinking about, such as inaccuracies and internal fraud. Each organization's job is to choose the controls that fit the processes it is building with AI.

Start With What You Are Using AI to Do

The guidance opens with a capability-first taxonomy of eight AI capability types, each with a definition and examples. Before asking what the risk of AI is, ask what you are using it to do. The answer leads logically to the right controls.
Not every organization will use all eight capabilities. It depends on risk tolerance, technology, and what the tools can actually do. Unauthorized use may also mean capabilities are in use that were never part of the design.
Shared vocabulary matters. Terms like drift, hallucination, and shadow AI give compliance, IT, finance, and the board a common shorthand, which helps when making the case for resources or reining in risky use.
The guidance is designed for audit-ready control mapping, so organizations collect monitoring evidence as they go rather than scrambling when auditors or regulators ask.
In the financial close, Vicky sees human-AI collaboration as the most common capability. AI spots recurring transaction-matching patterns and proposes rules, while accountants still review and create those rules the way they always have. More advanced teams are adding AI-powered continuous monitoring to flag anomalies.

Applying the Five Components to Generative AI

Data ingestion: If the data going in is wrong, everything downstream goes off the rails and is hard to remediate later. As Doug put it, you cannot unscramble the omelet, so make sure what goes into it is what you want.
Risk assessment: Set clear objectives for AI use rather than treating it as a toy.
Control activities: Design where and when humans come back into the loop to confirm the AI is doing what it should, is not hallucinating, and aligns with organizational objectives. Document AI-related risk assessments, control designs, and testing results, because errors can scale quickly.
Information and communication: Articulate the limits of generative AI and watch for the point where outputs are no longer fit for purpose.
Monitoring: Evaluate deficiencies for severity and root cause, then communicate them to those responsible for corrective action. That only works if roles and responsibilities were defined in the control environment, which is why every component has to work together.

Who Owns AI Governance?

Matt pressed on a practical question: who is the "you" in all of this? A root cause analysis of an AI failure may require technical skills compliance officers lack, while IT may not understand the business process the AI is now part of.
Vicky sees IT and IT security embracing the problem most often, provisioning tools and enabling responsible use. But when the business builds its own agents, fixing a broken agent becomes the business's responsibility. The line is not yet clearly drawn.
Nick argued that defaulting entirely to IT leaves latent risk on the table, because ethics and compliance often understands the business context better. Like fire, you do not need to fully understand AI to put guardrails around it so it does not burn the house down.
Doug noted that internal audit lacks the bandwidth to chase AI after the fact. Embedding the capabilities table into the first and second lines of defense puts organizations ahead of the problem instead of behind it.

The Controls That Are Working Today

An inventory of agents. Auditors are asking for it first, and business continuity teams cannot run an impact analysis on agents they do not know exist. For large organizations, start with scope: if an AI-assisted process touches payroll, background checks, or another in-scope control, it belongs in the inventory. Automated inventory tools are emerging, but most organizations are still doing this manually.
Human in the loop, with substantiation. This is the primary control today, but it is wobbly. AI is convincing, so reviewers start trusting it over their own judgment, and AI produces more volume than people can keep up with. Require a note, calculation, or other evidence that the reviewer actually agrees with the output instead of checking a box.
Contractual protections. When procuring AI tools, look for zero data retention and agreements that vendors will not train models on your data, protecting both security and intellectual property.
COSO describes roughly 30 more advanced controls, such as monthly testing for model drift and confidence scoring that routes low-confidence outputs to a human. Vicky expects most of these to become common over the next five years.

Rethinking Human Oversight

Matt shared a cautionary example. He asked AI to categorize roughly 3,000 consumer complaints and chart their distribution. When he later asked whether it had hallucinated anything, it admitted it had regraded the categories to produce a normal-looking curve because it assumed that was what he wanted.
The lesson is that AI may do things no human would think to do, so oversight has to include asking the right follow-up questions. Doug pointed to libraries of approved prompts that instruct models to stay factual and flag anomalies, while noting that prompts can also be gamed.
Nick emphasized that ownership does not change. If a colleague hands you an analysis, you either trust their track record or check it yourself. The same applies to AI. Pushing agency and tools like adversarial review agents down to individual users, who best understand the fallout of a wrong answer, may be the most effective guardrail.

What Auditors Are Looking For

When generative AI first took off, Big Four auditors required every control they could imagine. After significant pushback, the pendulum swung. Now clients disclose and attest to an inventory of AI used across the close, and auditors run a risk assessment on each item.
After all that work, it still comes down largely to human in the loop. That approach will strain as agents take on work that entire teams used to do.
A growing risk is "accountants as coders." Accountants can now build sophisticated tools without the IT general controls, such as testing, approvals, and segregation of duties, that software teams follow. The result is SOX findings.
Matt warned this extends well beyond finance. Marketers building automation that violates privacy rules or legal teams building investigative tools that mishandle data could make IT general controls an enterprise-wide headache.

Closing Thoughts

The panel agreed that organizations are still riding with training wheels on when it comes to AI controls. Regulators in New York, the EU, and elsewhere are adding expectations, and no one has all the answers yet. COSO, however, is first out of the gate with a structured approach, and as with financial and sustainability reporting, its thinking is likely to work its way into audit protocols and other frameworks.

The practical starting point is the same one COSO has always emphasized: understand what you are using AI to do, identify what could go wrong, and design layered controls that are present and functioning. Whoever builds or uses the tool still owns the output.