AI Risk Management: The Emerging AI Assurance Industry

I thought this Forrester blog was pretty interesting. It discusses how recent legislative initiatives in California and elsewhere have increased the importance of proper validation of claims made about AI systems, and how this in turn is likely to drive the development of a third-party AI assurance industry. Here’s an excerpt:

California is not alone in recognizing the importance of AI assurance. Singapore has developed AI Verify and the Global AI Assurance Pilot to support testing and evidence generation. The United Kingdom is building competency frameworks and assurance ecosystems. The European Union has established conformity assessment mechanisms for certain high-risk AI systems. The emergence of specialist credentials such as ISACA’s Advanced in AI Audit certification further suggests that AI assurance is beginning to develop the characteristics of a standalone profession.

What makes California different is that it is beginning to focus not only on AI assurance itself but also on the organizations providing assurance. SB 813 and AB 1405 establish expectations around the independence, transparency, integrity, and qualifications of AI auditors and verification organizations. In effect, California is beginning to govern not just AI systems but the institutions responsible for validating them. It may become the first major jurisdiction to explicitly regulate the organizations and professionals performing AI assurance.

California’s focus on assurance is not emerging in isolation. At the United Nations General Assembly this month, governments are spending less time debating broad AI principles and more time discussing evaluation, oversight, safeguards, and human intervention. In New York, policymakers are increasingly scrutinizing how AI controls are assessed in practice rather than simply whether organizations claim to have them.

These efforts have very different objectives, but they point in a similar direction: Stakeholders are becoming less interested in what organizations say their AI governance programs do and more interested in the evidence demonstrating that those controls actually work.

The blog says that California’s actions may provide an early indication of where the market may be heading. Increasingly, the key issues in AI governance won’t be limited to models, applications or controls, but will expand to encompass the entities providing evidence about the credibility of vendors and the claims made for their AI enterprise systems.