Four technology incidents caught our attention over the past few months. These were: the XZ Utils backdoor, the UniSuper outage, the Iress hack, and most recently the Crowdstrike incident. The last of these is ongoing at the time of writing. We use the vague term “technology incident” because the differences between these events make it difficult to group them together under an umbrella designation. XZ Utils and Iress were cyber security incidents, while the UniSuper outage and Crowdstrike incident were not the work of malicious actors (though it is still early days for Crowdstrike). XZ Utils and Crowdstrike had startingly universal reach; Iress and UniSuper were limited to individual companies and their customers. Despite their diversity, grouping these incidents together reveals something about trust and dependency in modern technology, and the sheer number and variety of actors involved in keeping our systems running.
The XZ Utils backdoor: open-source esoterica and benevolent nerds
In late March 2024, Andres Freund, a developer at Microsoft, happened to notice that SSH was failing after unusually high CPU usage. SSH is a network protocol commonly used to access a remote computer — IT folk talk about how they “SSH” into a computer they’re not sitting in front of. Further investigation revealed that the cause was a piece of malicious code that had been embedded in XZ Utils, a set of data compression programs used in the operating system Linux, which is in turn used by hundreds of millions of computers, including the vast majority of public web servers. The exploit would have allowed hackers to remotely access any of these machines.
Freund’s detective work, and the fact that the compromised version of XZ Utils had not yet been widely deployed, averted major catastrophe. The incident also highlighted something that many — including popular webcomic xkcd — had long known to be true: critical technology infrastructure is highly dependent on obscure, open source, and altruistically maintained software. XZ Utils was created and primarily maintained by one developer, Lasse Collin. Collin described it as an “unpaid hobby project”. Citing mental health challenges, he had slowly been relinquishing control to a developer identifying himself as Jia Tan, likely a state sponsored hacker, and the author of the backdoor exploit.
The lesson for organisations to take from this one is that cyber-attacks and technology incidents are inevitable. Modern computing is the accumulation of trillions of lines of code, interdependent and interconnected in incomprehensibly complex ways. Critical infrastructure relies on unpaid hobby projects that most people will never hear of unless they break. The best thing we can do in the face of this is be prepared. Your organisation should have a cyber incident response plan, test it regularly, and make sure it’s fit for purpose. And, as UniSuper proved, maintaining backups is essential.
The UniSuper outage: blank fields and backups
On 1 May 2024, Google Cloud accidentally deleted UniSuper’s accounts. Google Cloud identified a blank parameter and a now deprecated internal tool as the culprits. UniSuper’s website, member portal, and other member-facing services were offline for over a week.
UniSuper had georedundancy built into its Google Cloud deployment, such that if Google data centres in one location failed, they would be able to keep running. But this doesn’t account for your cloud provider proactively deleting your data. Fortunately, UniSuper had the foresight to maintain backups with other service providers and used these to restore service with Google Cloud.
The UniSuper outage illustrates the degree to which the modern enterprise is dependent on giant cloud service providers — namely Amazon, Microsoft, and Google. Failures like the one at the core of this incident are rare, but it does highlight the benefits of adopting a multi-cloud strategy. Cost and inefficiencies generally prohibit organisations from replicating their entire tech stack across multiple environments. UniSuper’s experience, however, shows that when it comes to critical components like backups, mitigating vendor risk can be just as important as mitigating geographical risk — even for vendors as large and sophisticated as Google. APRA-regulated entities like UniSuper should include these kinds of considerations in their preparation for CPS 230, within which business continuity planning is a key component.
The Iress hack: gits and tweets
As things returned to normal for UniSuper and its customers, Iress revealed that they had detected unauthorised access to their GitHub user space. It soon became apparent that hackers had stolen credentials from this GitHub repository and gained access to production environments containing Iress’ client’s data. As Iress tried to establish the extent of the breach and keep the market informed accordingly, an X (formerly known as Twitter) account purporting to belong to the attackers claimed to have access to root user credentials and sensitive client data.
At the time of writing, Iress has notified the ASX that personal information involved in the data breach was limited 20 individuals, all employees of OneVue (formerly owned by Iress) and its clients. Having completed an internal investigation into the incident, Iress is due to release a report to the market in late July.
Novigi saw firsthand the anxiety that the attack on Iress caused within the superannuation and wealth industry in Australia. Iress’s Acurity registry platform and its XPlan financial advice software are widely used by superannuation administrators and financial advice providers respectively. The existence of technology vendors within the superannuation and wealth industry with the scale and domain specificity to meet its unique needs is crucial. But it does create points of failure. This attack is a reminder that firms like Iress, and Novigi, need to hold themselves to the highest possible standard when it comes to cyber security.
The Crowdstrike incident: Y2K but for real this time
A buggy update provided by Crowdstrike, a maker of cyber security software, caused a massive global IT outage on 19 July. Flights were grounded, health services were disrupted, banks and supermarkets were closed — plunged into the very heart of darkness. Crowdstrike CEO, George Kurtz, clarified that the outage was “not a security incident or cyberattack.” One can only imagine the horror (the horror!) that Mr. Kurtz must have felt as the sheer scale of the disruption became clear. A fix has since been released, and at the time of writing, organisations all over the world were slowly getting back online.
In discussions about the root cause of the Crowdstrike incident, commentators have referenced the deep system access that Crowdstrike has to the Windows operating system. cyber security providers tend to have almost unfettered access to organisations’ systems and environments. Consider the permissions given to a third-party conducting a cyber audit or a penetration test. We often compare cyber security to healthcare, with hackers the disease-causing viruses and germs, governance s the equivalent of eating healthy and exercising, and the cyber security industry the healthcare professionals. Yet cyber security is still young, and it is difficult to tell the doctors from the snake-oil salesmen. This will be solved in time by government regulation, and perhaps the development of professional associations with exacting and self-policing standards, as in medicine.
In the meantime, organisations should apply the most rigorous standards to the selection of cyber security vendors. As with the XZ Utils backdoor, the Crowdstrike incident shows just how complex and interconnected the modern world has become. There isn’t much that enterprises can do about this, even if they wanted to. What they can control is the number of cyber security tools and vendors they use. Gartner, a technology consultancy, have called for organisations to adopt a “minimum effective toolset” for enterprise security. With Crowdstrike revealing that — in addition to being a potential cost and source of inefficiency — every cyber security tool poses a potential risk in itself, this increasingly looks to be a sensible approach.
Lessons learned
Despite our best efforts at prevention, technology incidents happen. Organisations need to ensure that they are ready to respond to them when they do, with practised response plans and backups. They need to understand the risks posed to them by their partners, vendors, and other third parties. Lastly, they need to take special care in who they choose to help them prevent and respond to incidents, given the level of access these parties inevitably need. And with each new incident, ensure that we learn what there is to be learnt, to be better able to face the next one.
This article was produced as part of The Quarterly – Data and Technology in Superannuation, Q4 FY24
For more information about anything you’ve read here, or if you have a more general inquiry, please contact us.
Key Contributors:
Kevin Fernandez is General Manager, Market Strategy and Propositions at Novigi, and is based in the Melbourne office.
Sophie Bowen-James is an analyst in the Market Strategy and Propositions team at Novigi, and is based in the Sydney office.