Australia’s financial regulator has sounded the alarm about the way banks and superannuation trustees are managing AI technology. While many institutions are adopting AI to boost productivity and improve customer service, their governance and risk management practices are still catching up. The regulator, the Australian Prudential Regulation Authority (APRA), recently reviewed some of the largest regulated entities to see how they’re handling AI risks and safety.

AI Use in Financial Institutions and Growing Risks

APRA found that all the entities they looked at were using AI in some way. Common applications included software engineering, claims processing, and loan applications. Some firms also used AI for fraud detection and customer interactions. While this shows AI’s increasing role, the maturity of risk management around these tools varied widely. Many boards are eager to leverage AI for better results, but they are still developing strategies to manage the associated risks.

The regulator expressed concern that many boards rely heavily on vendor presentations and summaries instead of scrutinizing the actual AI models. This can lead to gaps in understanding, especially regarding unpredictable behavior or biases in AI systems. APRA stressed that boards need to deepen their understanding of AI, align their strategies with their risk appetite, and establish clear procedures for handling errors or failures.

Gaps in Monitoring, Management, and Security

APRA identified several specific areas where AI governance needs improvement. Monitoring how AI models behave over time is crucial, yet many entities lack proper tools or processes for this. Change management and decommissioning procedures are often missing or weak, raising concerns about outdated or misused AI tools remaining in operation. The regulator recommends keeping detailed inventories of all AI tools and assigning clear ownership to individuals responsible for each AI instance.

Another major concern is the need for human oversight in high-risk decisions. AI is increasingly involved in critical processes, but human judgment should still play a role, especially when significant outcomes are at stake. Cybersecurity is also under threat as AI adoption introduces new attack pathways like prompt injections and insecure integrations. Some firms haven’t adjusted their identity and access controls to account for AI agents, creating vulnerabilities.

APRA pointed out that the surge in AI-assisted software development is putting pressure on change and release controls. Organizations should implement strict controls around privileged access, configuration, and updates for AI systems. Testing AI-generated code for security flaws is also essential. Additionally, some institutions depend heavily on a single AI provider, with few having plans to switch or exit if needed. This dependency poses risks that firms need to address proactively.

Finally, APRA noted that AI can be present in upstream dependencies without organizations even realizing it. This hidden layer of reliance underscores the importance of transparency and comprehensive inventories. The regulator emphasized that effective governance must include clear policies for managing AI risks, continuous monitoring, and ensuring human involvement in decisions that could impact customers or the stability of the financial system.

Meanwhile, efforts to improve identity and permission controls are advancing through new standards work by groups like the FIDO Alliance. They are forming specialized working groups to develop standards for authenticating AI agents and managing their access rights more securely. All these developments point to a growing recognition that AI governance must evolve quickly to keep pace with technology and emerging threats.

Inspired by

• https://www.artificialintelligence-news.com/news/ai-agent-governance-control-gaps/

Sources

• unsplash.com
• ai-expo.net
• techexevent.com
• cybersecuritycloudexpo.com
• techforge.pub