AI could play a significant role in the retirement phase for millions of members through on-demand advice and guidance, but funds need ironclad governance and operational guardrails to make sure it’s a benefit, not a hazard, according to former Sue O’Connor, a former director of Mercer Super and CDC Data Centres and non-executive chair of Indara. 

“The demands of the members, particularly as we go into retirement, are significant,” O’Connor tells Investment Magazine.

“The opportunity we’ve got is to service them better using AI and we have to have governance in place that means we are more likely to be able to deliver that in a way that members, our organisations and our regulators require.”

While AI has significant potential for improving customer outcomes and service efficiency, there is persistent and heavy scepticism about its transparency and benefits, and its rapidly evolving nature means its behaviour can be unpredictable. If it’s not used correctly, it could have a disastrous impact. O’Connor points to the robodebt scandal as evidence of what can happen when “ethical considerations are subordinated to efficiency targets”, and calls it a “warning for all financial services organisations”.

“We don’t put governance in place after the crisis,” O’ Connor says. “We put it in to avoid the crisis.”

To that end, O’Connor is set to release a paper titled Pragmatic and ethical AI governance for the Australian financial services sector which lays out seven AI principles divided into two categories: human oversight and accountability, and system structure and design.

In the first category, these principles include making sure board directors and executives are accountable through the establishment of specialised AI committees with responsibility for oversight of AI; that algorithms are clear and explainable; and that AI systems are auditable by authorised third parties. The second category deals with building privacy considerations into machine learning techniques, as well as pre-deployment bias prevention, monitoring and compliance.

“You can use things like chat bots for general advice,” O’Connor says.

“But as soon you have anything that is very specific to the member, you need to have an additional process for engagement with an AI agent, and both the organisation and the member need to feel confident that it’s got that human oversight.

“It doesn’t matter if the superannuation fund is using an outsourced service provider, AI or my auntie Jean – they are still accountable, and they still need to have the processes in place to say ‘Does this make sense?’.”

While it’s tempting to look outside the organisation for expertise, funds are better off turning their gaze inwards – at least initially.

“The thing that’s really important for superannuation funds, and for all organisations, is the work that’s being done to skill-up your senior executives, your board and your staff, because this is pretty new for everyone and you need to have an overt program of skilling people up in the same way.”

Still, there’s bound to be some hesitancy from funds as they contemplate taking a leap into the unknown with AI at a time when heavy regulatory scrutiny is being applied to operational risk.

“The regulator needs to decide whether they are in risk removal or risk management, and what their risk appetite is. If you are in risk removal you won’t do anything.”

“Having been involved in transformations that come in the categories of the good, the bad and the extremely ugly, I have never seen a transformation that was faultless. In the same way as learning to serve in tennis, it’s not faultless. But it’s about how you mitigate and respond when you do have a problem and how you make yourself ready is really important and do you have the right people and the right resources in place.”

But on the internal question of how soon funds want to take that leap, O’Connor says that’s “a judgement call that everyone needs to make”.

“Some want to be leaders, some want to be fast followers, and some want to be slow followers. The amount of money you need for this compared to a large systems build, which is millions of dollars – you’re not committing as much as you were in the past.

“I don’t think super funds need to be leaders, but they need to make a decision about whether they’re a fast or a medium follower. Software companies like Atlassian and Canva need to be leaders in their spaces because they’ve got so much commercially that is required.”