It was 7.30am on 1 May this year when UniSuper chief executive Peter Chun realised the $135 billion super fund had a major issue with its website. At virtually the same time, the fund’s head of IT was in contact with Chun confirming there was a problem.
Unbeknownst to Chun and his team, one of the world’s largest cloud service providers, Google Cloud, had – overnight, and without warning – deleted UniSuper’s accounts. When Chun arrived in his office that Wednesday morning, he was walking into a crisis unprecedented in both scope and cause.
The event to unfold over the next week would test the fund’s crisis management plans unlike any simulation had or even could; that would test relationships with its more than 647,000 members; and which would, inevitably, invite comparisons with companies that had recently suffered IT-related crises, most notably Optus.
In the IT world this story was massive, attracting global attention, not only for Google’s failure and its impact on a major customer, but also for its response to the crisis it provoked.
It was, Chun says, “an unprecedented, classic black-swan event” that was “not a feasible planning scenario”. But while UniSuper clearly was the victim of a very peculiar issue, it wasn’t wholly unprepared.
By lunchtime on day one the full extent of the issue was clear, and UniSuper had initiated its crisis response plan. Later that day it was in contact with the regulator, and the task of communicating with members was underway. In fact, it was an email sent to members that tipped off Investment Magazine, which broke the story on May 1.
As events unfolded, UniSuper’s response stood in contrast to other businesses that had suffered high-profile and debilitating IT failures.
Crisis management plans should be baked into super funds’ operational systems, specifically through Prudential Standard CPS 230: Operational Risk Management, necessitated by trustees’ fiduciary responsibilities, and driven by pragmatic commercial concerns, the best plans are often tested to the limit not by sophisticated external attacks or by complex system failures, but by pure human error and dumb mistakes.
UniSuper’s member admin system operated on Google Cloud, one of the three largest cloud computing services on earth. Chun says the installation had considerable redundancy built in.
“When we go into these arrangements, we’re very mindful of backups,” he says.
“We have two private clouds with Google and they’re spread, geographically separated. [If one goes] down, there’s another [that] would come up for us. You have some confidence there’s that resilience.”
Without human intervention
But for reasons that are still not completely clear, and which are the subject of ongoing discussions between UniSuper and Google, “they had this auto-deletion, which wiped both clouds”, Chun says.
“And it was all without human intervention.”
Chun says UniSuper’s account was “misconfigured” by Google when the service was set up. It’s an error that has been publicly acknowledged by Google Cloud chief executive officer Thomas Kurian.
A technical note issued by Google said: “This had the unintended and then unknown consequence of defaulting the customer’s [Google Cloud VMWare Engine] Private Cloud to a fixed term, with automatic deletion at the end of that period.”
“The incident trigger and the downstream system behaviour have both been corrected to ensure that this cannot happen again,” it said. (More details can be found in Google’s full analysis.)
The upshot was that one year and one day after signing up, UniSuper’s member services data simply evaporated from Google Cloud.
“Remember, we’re dealing with San Francisco,” Chun says. “So it was 30 April San Francisco time, or 1 May our time.
“When you [ask when] was the CEO brought in, I came into the office on that morning, 7.30 am. I had an issue with the systems. My head of IT rang me probably 15 minutes later said, look, we’ve got an outage at the moment. By lunchtime, we knew this was serious.
“We had stood up our crisis management team. Me, my executives and some other members of the organisation then began governance of this crisis management team for the next week. And we met, all-up, about 14 times.”
Not a cyber-attack
Chun says the crisis team’s priority was to determine the exact nature of what was going on. Top of the list of potential issues was a cyber-attack by a malicious actor; it was quickly determined that this was not the case, and that no member data had been compromised, just deleted.
“The important thing to begin with is to highlight that our IT architecture and technology really was based on multi-cloud [providers], we were not just relying on Google,” Chun says.
“Our data backups, our services, we spread it across multiple cloud providers. There are really only three major players: Google Cloud, Amazon AWS, and Microsoft Azure. And importantly, our member data [and] member transaction data was actually separately backed up.”
Chun says that UniSuper’s investment management systems were untouched. The fund manages about 75 per cent of its assets in-house, and its investment team “was still able to trade and was still able to manage our portfolios”.
“That was not impacted at all, because that was diversified – that’s with a different cloud provider,” he says.
Irrespective of the precautions and redundancies UniSuper had created, the impact on members was immediate and severe. Online member services and the fund’s mobile app were inaccessible. The fact that the outage was caused by an unprecedented event was of little comfort to members understandably concerned over the whereabouts and security of their retirement savings.
“You can only be judged by how you respond,” Chun says.
“Right up front, we established key principles about how would we respond: transparency, fairness, and timeliness. And it absolutely guided our communications, our remediation, and how we supported our members.
“We decided very early we will communicate daily and, in fact, over-communicate. I’m sure you’ve seen other organisations, whether it’s been cyber issues or other [issues], when there’s been a vacuum of communication. That’s when it can go horribly wrong.”
Trust in organisations
Chun said UniSuper’s objective in its response was to address the twin issue of competence and character.
“Something that I’ve been familiar with in my career around trust and organisations and how customers trust organisations, [is that] it’s a combination of competency and character,” Chun says.
“When we think about outages like this, naturally our competency would be questioned. It would go down, because members, customers, would go, ‘How could that have happened? How incompetent is UniSuper?’
“So we had to focus on competency. And then people judge your character, and that’s when values, it’s your behaviours, and it’s how we responded.”
The issue affecting UniSuper turned out not to be a cyber-attack, but the fund’s preoccupation with that possibility – like many other funds’ similar focus – ultimately helped it cope with what did happen.
“Like all organisations, we have been almost obsessed with any chance of a cyber-attack,” Chun says.
“Every organisation knows it’s not a matter of if, but when. Luckily, in the last year, we did a very detailed scenario exercise around when your core system is down, what do you need to do, how do you respond?
“Quite quickly you stand up our governance team, which governs decision making, but then you’ve got streams: you’ve got a system stream, so that’s all the technology team. How do you then restore the system, how do you rebuild the data? We’re already in touch with Google.”
Chun says the fund’s response was organised into communication and member support streams, and a governance stream.
“I was in touch with our chair; we then gave the board a daily update,” he said.
“And that afternoon [of day 1], we immediately gave APRA, the regulator, the heads up.”
Funds ‘own and manage’ risks of service providers
APRA declined to comment specifically on the issue that affected UniSuper, and the fund’s response. However, a spokesperson for the regulator told Investment Magazine it had “recently published prudential practice guidance to help superannuation funds – as well as banks and insurers – strengthen their management of operational risks and improve their business continuity planning”.
“The guidance, developed to assist with implementation of prudential standard CPS 230 Operational Risk Management, which comes into effect in July 2025, makes clear that APRA-regulated entities own and manage the risks of service providers they use, and are required to have a policy that sets out how they will identify and manage risks associated with material service providers in the delivery of critical operations,” the spokesperson said
“Operational risk management has become increasingly important in this digital financial age, and APRA commends the proactive efforts organisations are taking to strengthen their operational resilience for their own benefit and for that of their customers.”
Despite Google’s acknowledgement that this issue had never happened before, and its assurances it can “never” happen again, Chun says a lesson from the experience of being deleted from the cloud is that even a seemingly watertight contract is no protection. Tick-a-box compliance and assurances from service providers can’t guarantee things won’t go wrong.
“There is no doubt that when you go through a crisis like this you become very alert to what can we do, what can we do to mitigate any other types of events?” Chun says. He says the experience will influence how UniSuper interacts with all of its suppliers, particularly those involved in critical infrastructure.
“Another very important relationship for any super fund is our custodian. We use BNP, the French bank, and their custody. Just this week the global CEO was out here and then just coincidentally I had a meeting with him.
“We were talking about continuity and resilience and outages, and I needed to get comfort that they’ve got bloody good plans, because this could happen to a custodian. And we stepped through the resilience.
“We can’t just trust the service providers. You’ve got to step through it, work through it. So, we’ve planned a workshop with BNP around resilience if the custody arrangement goes down.”
Chun says a personal learning from managing the May crisis is that “you’ve got to make sure we are having those conversations”.
“It’s not just what’s in a contract,” he says.
“We had an amazing contract with Google. It’s not just the physical contract. It’s actually stepping through how might you respond if such an unthinkable thing would happen.
“That’s not to say there’s [not] more work to be done. But I feel a sense of confidence in us responding to where the regulator is coming from. It’s not just a tick-the-box exercise. We feel it’s really valuable, because we are very critical to members.”
